Cosmic's PTCG Competition #1/?

About

As the Pokémon Kaggle Competition reaches the end of the evaluation stage, I wanted to share my experience competing in this project. This first post introduces the competition and a few important ML concepts, before going into the more sophisticated details in my follow up posts. This series is tailored for those with an interest in machine learning (applied or theoretical), trading card games, or competing against others who focused on LLM-generated pipelines ¹.

Full-disclosure: I did not do well. :D ²

The goal of the 2-month competition is to build an agent ("bot") that can autonomously play the Pokémon TCG against other competitors. During the submission phase of the competition, you could freely submit agents as you pleased, which would immediately enter the league. Other than playing a legal deck, there were no restrictions of the strategy or quality of your agent ³. Submissions autonomously played roughly 20-50 games every 24 hours, allowing you to evaluate your newest strategy and see where it landed compared to others. The top 20 GB of gameplay logs were made publicly available, allowing you to exhaustively track the top-200-or-so players: their exact deck choices and every single choice their agent made at every single game step.

The Competition

I will briefly summarize PTCG for those who aren't familiar. Before arriving, players must construct a 60-card deck from a pool of about 1300 cards that have recently been printed; "Standard Format". The goal of the game is to defeat the other player's Pokémon; once a player has no Pokémon left in play, they lose. To prevent the game from dragging on too long, there are also six "Prizes" that are randomly set aside from the 60-card deck during setup. Most basic Pokémon are worth 1 "Prize Card"; if you defeat your opponent's Pokémon, you choose and gain one of your Prize cards. Particularly powerful Pokémon can be worth 2 (EX) or even 3 (Mega-EX) Prize cards.

Each turn, a player plays cards from their hand in the order of their choice. This might involve evolving a Pokémon already in play, playing new Pokémon "to the bench" (not actively fighting but in play), or playing multiple "Trainer Cards" that each have individually weaker effect. Particularly powerful actions like attaching an Energy to a Pokémon or playing a "Supporter" (very powerful Trainer Cards) are limited to once per turn. Finally, if a player has an Active Pokémon with sufficient Energy attached to it, then they may declare an attack before ending their turn.

For anyone reading the news, you probably know that Pokémon TCG has a serious issue with scalpers; many people enjoy playing and collecting the game tremendously, both on paper and digitally. In competitive Standard play, Dragapult-EX makes up roughly 50% of the metagame at paper tournaments. When one powerful deck is such a significant proportion of the format, in order to be successful in competitive play you must be at least one of the following…

1. playing Dragapult,
2. have a reasonable plan to defeat Dragapult ⁴,
3. be vastly more skilled than the rest of the players ⁵.

You can imagine my surprise when in the first 3 weeks no one was being successful with Dragapult-EX. This piqued my curiosity…

Machine Learning

Machine Learning ("AI"/ML) describes a set of specific types of techniques that result in a digital system that is able to make predictions about the environment. I'll outline a few of them at a high level:

• Supervised Learning is classical ML; first you ingest a large labeled corpus ("this is a picture of a dog" // "this is a picture of a cat"). The resulting model receives similar-but-novel observations and must identify them with the most aligned label.
• Generative Learning transforms its input to novel-yet-somehow-related output. LLMs is the most well-known application of Generative AI, but also things like automatic transcription from text to speech or most modern foreign language translation are examples of this.
• Reinforcement Learning (RL) is when a model must learn to interact with an environment in order to achieve some pre-determined goal. Self-driving vehicles and robotics are common examples of RL.

The Pokémon TCG Kaggle Competition neatly fits in the Reinforcement Learning category. The LibCABT engine provides you with the current game state. Your agent then must choose one of the legal options, the engine executes it, and then presents the updated game state to whoever's-turn-it-now-is to take their next action. The natural goal is "to win", but you might break it into incremental goals such as taking Prize Cards. RL is particularly challenging since you must encode what incremental success even is.

Acting within the environment is done by returning a simple Python List of integers. Suppose you won the setup coinflip, LibCABT asks you if you'd like to go first (responding [0]) or second (responding [1]). An extremely simplified version of the JSON might look like this:

{
   "current": {"yourIndex": 0, "players": [{"hand": [1, 2, 3, 4, …]}, {"hand": [null, null, null]}]},
   "select": {"context": "GO_FIRST", "type": "YES_NO", "option": [0, 1]}
}

The game competition organizers gave 4 starter agents: Abomnasnow, Dragapult, Lucario, and Iono. Rather than any deep learning, these agents were Rules-Based. That is, they followed an elaborate decision process that ranked each action in a way similar to how an experienced human might make. "Can I attach energy to a Dragapult so that it can Phantom Dive? Well, I should do that. If I can't, what's my second best play…". It did this by scoring each option with a priority. It would then respond with as many options as was legal, sorting them from highest priority to lowest.

Transforming such observations into a consistent game-winning response is much more challenging than it might seem: an easy mistake is to assume that priority is synonymous with quality or value. Consider attacking: attacking is typically the highest-value action you perform every turn! After all, attacking is how you defeat your opponent's Pokémon. However, declaring an attack ends your turn. So, you want attacking to be a lower priority move, but never a negative priority. Your agent must somehow also come to this same realization.

Naive Rules-Based agents also lack a sense of planning: early in the competition, I watched a legendary mistake where an agent discarded 2 cards to get a Meowth, who was played to tutor for Boss's Orders… only to realize that Lillie's Determination was actually the preferred Supporter to play since the agent now only had 2 cards left in hand… promptly shuffling away the Supporter they just spent 4 cards fetching. Good grief. So, clearly, weighing each individual action from best-to-worst can result in critical mistakes when a robust agent needs to actually evaluate a trajectory or a horizon. Can you see The Play?