Clash Royale Simulator

Finally! A Clash Royale bot training environment that's playable, actively maintained and has a RL interface.

I like clash royale and want to train a RL agent to play the game. However, a simulator is needed to speed up training. I searched on GitHub and the only usable project I found was samdickson22's repository. I noticed that the code is almost completely written by AI, which is hard to read and impossible for humans to make improvements on the code.

In the end, I realized that the only way to make all of this work is to re-implement the whole game from scratch, without any vibe-coding.

Now, I present this functioning simulator that implemented 47 cards including troops, buildings and spells (complete list below) and can reach the simulation speed of 83 microseconds per tick. This means that the simulator can play more than 1000 games (with itself) within an hour.

I also designed a RL training environment compatible with gymnasium and stable-baselines3, supporting out-of-the-box training. I used a PPO algorithm to train a basic model for 3M timesteps and can reach a winrate of 83% against a non-trained version of the agent.

To quickly know more information about my project, see my paper.

Game Interface

Installation

Run the following command in your terminal to clone this repository and install the required python packages.

git clone https://github.com/Jason-XII/clash-royale-simulator.git
cd clash-royale-simulator
pip install pygame fastcore numpy==1.26.4 stable-baselines3 tensorboard --user --no-cache-dir

How to play

This simulator is already playable! You need two computers that connect to the same local network.

1. Run ipconfig on Windows or ifconfig | grep inet on macOS. Find your ip address in the local network. If you are at home, the address should look like 192.168.xx.xx, if you are connected to a school network, then that will probably start with a 10.xx.xx.xx.

2. Modify the parameter in server.py in src/clasher_new and run it. Code files all live in this directory.

3. Send the client_side folder to another device and run the client.py.

4. Follow the instructions in the GUI and enter the server ip address after choosing your deck.

5. When two clients are connected to the same server, the game starts automatically! You can see elixir, drag and drop to deploy cards, see entities on the screen, etc.

Supported features

I have implemented 47 cards in total, no evolutions, hero or champion cards are implemented yet. The simulator's behavior is not completely consistent with real gameplay and I'm continuously trying to improve this. For example, the path finding algorithm uses a non-search approach, which is faster but makes the characters slightly slower when walking. This might be solved in the future though.

Complete list of implemented cards:

• Knight
• Giant
• Archers
• Goblins
• Pekka
• MiniPekka
• Minions
• Skeletons
• SkeletonArmy
• Balloon
• Witch
• Barbarians
• Golem
• Valkyrie
• Bomber
• Musketeer
• BabyDragon
• Prince
• Wizard
• SpearGoblins
• GiantSkeleton
• HogRider
• MinionHorde
• RoyalGiant
• Princess
• ThreeMusketeers (Not the newest version though)
• BlowdartGoblin (Before nerf)
• AngryBarbarians (English name: Elite Barbarians)
• Bats
• DartBarrell (English name: Flying Machine)
• RoyalHogs
• Cannon
• Xbow
• IceWizard
• SkeletonWarriors
• DarkPrince
• LavaHound
• IceSpirits
• FireSpirits
• Miner
• Sparky
• Bowler
• Rage
• RageBarbarian (English name: Lumberjack)
• BattleRam
• Fireball
• Arrows

Read the code

If you are interested in this project, you can consider reading my code and figure out how the simulator work for yourself. I tried my best to write clear code. Here's a brief explanation of each python file in the repo, listed in suggested reading order:

The real code files are in src/clasher_new.

• __init__.py is an empty placeholder
• gamedata.json contains very necessary data extracted from the game, like hitpoints, damage, etc.
• cards_stats_xxx.json files are also game data files downloaded from royaleapi.com.
• card_utils.py reads gamedata.json and provides easy ways to access character attributes
• arena.py defines TileGrid which contains information on where each sides' King tower and princess towers are located
• player.py is a short file that stores a player's information in game, like current elixir.
• battle.py contains all the game logic, defining behavior for troops, buildings, projectiles and other mechanics.
• core.py and card_mechanics.py provides an interface for special card logic. Makes the system more flexible.
• server.py and client.py gives a simple pygame interface that allows two players to connect through local network and play in realtime.

To-do list

I trained the model for 8M steps and the win rate stops improving at around 70-80%. This is probably caused by the model trying to independently predict the card, position x and position y. Humans make decisions by choosing all three simultaneously, we first determine (vaguely) what we should do, and then do the actual placement. So I think the model can perform better if I let it produce the x and y position at once, or we use a different structure entirely by using the selection attention framework introduced in another paper. I might have to learn more about that paper.

Against a random agent, the PPO model should train relatively well and be consistently winning after 1-2M steps. So something's probably wrong with the model architecture.

I also thought of a way to determine the level of game playing agents: tier testing. Imagine we have a pool of agents and we let them fight for enough rounds. Good enough agents will consistently win and those with winrates over 70% can make it to the next tier. The logic might be a little flawed here but points to the right direction.

(might introduce elo score?)

Lots of new benchmarks can be added besides the winrate against a random agent. For example, we can test the agent's use of spells by placing swarm units at the bridge. By placing a mini pekka behind a giant we test the agent's ability of defending.

The environment can also be refined. The agent needs to know the phase of the game that it's currently in, because decks have different strategies in single, double and triple elixir. When it's near the end of the game and overtime, the agent may need to cycle spells to win.

The biggest flaw in the simulator is that the king tower activation seems a bit off. The king tower seems to have a shorter range than the musketeer, which is very strange. But this can be easily fixed.

I need help

This project is far from finishing. I already poured more than 100 hours into this project and many more still lies ahead. If you want to contribute, please submit issues or pull requests.

You are more than welcome to contact me via my email: 2243272839@qq.com

Or you can add my discord: jasoncoder_47308