Agentic RL · Game AI · World Model

Weiyu Ma马纬彧

Former StarCraft II ProM.S., CASIA

Open to work Looking for research or engineering roles in Agentic RL, LLM/VLM agents and game AI. Email me →

My research asks one question: how can AI perceive, decide and keep learning from interaction the way people do? Using games and multi-step agent tasks as testbeds, I work on LLM/VLM agents, Agentic RL, world models and multi-agent decision-making.

Before research, I spent my undergraduate years as a professional StarCraft II player (Grandmaster, KR server). That background shapes how I build environments, data and benchmarks around real player interaction — moving game AI from winning to seeing, thinking, learning and cooperating. I did my M.S. at the Institute of Automation, Chinese Academy of Sciences (CASIA), then spent 2025.08–2026.09 as a Ph.D. student in Computer Science at KAUST before leaving the program.

Focus Agentic RL / Game AI M.S. CASIA Rank SC2 KR Grandmaster Lang Chinese / English
Google Scholar ↗ GitHub ↗ Email ↗
8
Papers
7
First / co-first author
450
Citations
GM
SC2 KR Grandmaster
01 / News

Latest

  • 2026.10
    ★ Released the Esports World Model Dataset — the first open multi-game dataset for long-horizon, multi-agent world modeling, with synchronized per-player views of complete matches (SC2 / Dota 2 / CS2; 2,673 matches, ~3,828 hours). ModelScope Code
  • 2026.09
    Released JEV-Star: real-time StarCraft II control with language-model planning and fast action selection — 9 of 10 wins vs. the Lv7 built-in AI. Paper Code
  • 2026.04
    ★ Released FreshPER, freshness-aware prioritized replay for LLM/VLM RL — +39% over baseline on Sokoban. arXiv Code
  • 2026
    EvoCurr accepted to EMNLP 2026 Findings (co-first author).
  • 2026
    AVA accepted to ACL 2026 Findings (first author).
Show earlier (6)
  • 2026
    LLM-SMAC accepted to AAMAS 2026 (co-first author).
  • 2026.04
    Started the CS2 World Model and a ~20 TB multi-agent eSports world-model dataset.
  • 2025
    TacticCraft accepted to the ICML 2025 MusIML Workshop.
  • 2024
    ★ TextStarCraft II (Chain of Summarization) accepted to NeurIPS 2024; the repo has 300+ GitHub stars. arXiv Code
  • 2024
    AVA won first prize in the Alibaba Cloud Qwen essay contest.
  • 2024
    Adaptive Command accepted to DAI 2024.

At conferences

NeurIPS 2024 Presenting the TextStarCraft II poster
DAI 2024 With Richard Sutton
02 / Research

Research Areas

01

Agentic RL Training

Multi-step RL for LLMs/VLMs — experience replay, asynchronous training systems and tool use, so models keep improving through interaction.

Representative work FreshPER · LLM-SMAC · Async Search-R1 ↗ · ROLL ↗
3 papers →
02

LLM/VLM Game Agents

Getting foundation models to understand and win real-time strategy games: environments, summarisation-based decision frameworks, language-driven tactics and human–AI collaboration.

Representative work JEV-Star · TextStarCraft II · AVA · TacticCraft · Adaptive Command
4 papers →
03

Multi-Agent Decision & Self-Evolution

Code-as-policy for coordinating many agents, and LLMs that design their own environments and curricula so tasks and policies co-evolve.

Representative work EvoCurr · LLM-SMAC · SMAC-Hard
3 papers →
04

World Models & eSports Data

Building an open long-horizon, multi-agent, multi-view eSports dataset from real players’ complete matches (SC2 / Dota 2 / CS2), and training interactive, geometry-aware multi-agent world models that respect game logic.

Representative work CS2 World Model · eSports World Model Dataset
03 / Projects

Selected Projects

10 projects
2023.06 – 2024.06 Open source ★ Flagship

TextStarCraft II

The first application of LLMs to real-time strategy decision-making

First author · NeurIPS 2024 · CASIA & UCL
  • Builds the TextStarCraft II environment: game state is rendered as language so models can read the situation, plan and issue commands.
  • Proposes Chain of Summarization, using single- and multi-frame summaries to track the game; benchmarks 10+ LLMs, fully open-sourced.
>50% vs. Lv5 built-in AI
arXiv ↗ Code ↗
2024.04 – 2025.03 Open source

AVA

The first VLM-based multi-agent real-time strategy decision system

First author · ACL 2026 Findings · CAMEL-AI × KAUST
  • A high-concurrency PySC2 micro environment that supports both VLM and RL agents for training and evaluation.
  • A multi-step reasoning VLM agent on CAMEL with RAG and skills; code-as-policy opponents generate diverse scenarios.
Alibaba Cloud Qwen Contest — 1st Prize
arXiv ↗ ACL ↗ Code ↗
2025.06 – 2026.05 Open source

EvoCurr

The first code-based recursive self-improvement (RSI) in complex decision-making

Co-first author · EMNLP 2026 Findings
  • An LLM designs, builds and tunes its own decision environments; curriculum difficulty evolves with the policy, with no human in the loop.
  • Ran autonomously for 72 hours, producing 10+ new environments and 100+ behavior-tree policies; commands 100+ units in late-game battles.
100% win rateBeyond human GM
arXiv ↗ Code ↗
2024.09 – 2025.08 Open source

LLM-SMAC

The first Agentic RL for multi-agent decision-making

Co-first author · AAMAS 2026 · with Polixir.ai
  • Generates executable Python decision-tree code to command multi-agent teams, closing the generate–execute–feedback–refine loop.
  • A 7B model trained with SFT, DPO and GRPO beats DeepSeek-V3; one policy generalises across 18 scenarios.
~100% win rate on 18 maps
arXiv ↗ Code ↗
2025.09 – 2026.05 Open source

FreshPER

The first general experience-replay algorithm for LLM post-training

First author · under review at ICLR
  • Brings classic PER to LLM/VLM RL, decaying priorities by sample freshness to fix priority staleness under fast-moving policies.
  • Works with PPO, GRPO and REINFORCE++; built on ROLL and slime; evaluated on math, reasoning, multimodal and agentic tasks.
NQ Search 68.75% (+20%)Sokoban 43% (+39%)MathVista 67% (+15%)
arXiv ↗ Code ↗
2026.09 Open source

JEV-Star

State of the art for direct LLM decision-making in StarCraft II

First author · 2026 preprint
  • Macro: real-time full Protoss games over 73 actions, with the Astra planner adding constraints or advice and JEV picking actions fast. Micro: all 35 SMAC-Hard maps.
  • Astra-constrained + JEV wins 9 of 10 games against the Lv7 built-in AI, where random and JEV-only win none; JEV’s median response is ~0.4 s.
9–1 vs. Lv7 AI~0.4 s median response
Paper ↗ Code ↗
2026.04 – 2026.10 Open source

eSports World Model Dataset

The first open-source, multi-game dataset for long-horizon, multi-agent world modeling in eSports

Project lead · released 2026.10 · CC BY-NC 4.0
  • Long-horizon: sampled as complete matches rather than short clips — whole StarCraft II and Dota 2 games (~11 / ~56 min on average) and full CS2 maps split by round — spanning RTS, MOBA and FPS: 2,673 matches, 748 hours.
  • Multi-agent: real players’ replays are decoded by the game engines into synchronized per-player views (up to 10 per Dota 2 match, ~3,828 hours in total), frame-aligned with actions, engine state and events — for action-conditioned world models, multi-view consistency and partial observability.
Full matches · 2,673Up to 10 synced views per match~3,828 player-view hours
Dataset ↗ Code ↗ Paper ↗
2024.05 – 2025.04

TacticCraft

The first large–small model collaborative decision system for real-time strategy

First author · ICML 2025 Workshop · CASIA & Tencent AI Lab
  • Freezes the DI-star policy and conditions it through a light adapter and a “tactic tensor”, so language can steer it.
  • Builds a language-to-tactic dataset from expert trajectories, supporting 9 switchable tactical dimensions.
arXiv ↗
2024.01 – 2024.07

Adaptive Command

The first human–AI interactive decision system for real-time strategy

First author · DAI 2024 · CASIA & UCL
  • Players talk to the AI by voice or text in real time: the LLM plans, behavior trees act, in a closed loop.
Novice win rate 0% → 42%
arXiv ↗
2026.04 – Ongoing

CS2 World Model

Geometry-aware multi-agent world model on Cosmos 3

  • A multi-agent framework for interactive world models, with scene reconstruction and trajectory cleaning on OpenCS2 and CS2 data.
  • Warm-up, multi-resolution and mixed-ratio training plus teacher forcing, self-forcing and DMD; multiple agents interact and fight in one world.
Demo ↗

World model in action

CS2 World Model — left: real match footage; right: generated by the model. Four maps, 8 s each.

04 / Publications

Publications

8 papers
  1. 2026

    Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learning

    Ma, W.Y., Zeng, Y., Song, Y., Cui, X., Zhao, J., Liu, X., Elhoseiny, M.

    arXiv 2026 First author Under review at ICLR arXiv Code
  2. 2026

    AVA: Attentive VLM Agent for Mastering StarCraft II

    Ma, W.Y., Fu, Y., Zhang, Z., Ghanem, B., Li, G.

    ACL 2026 Findings CCF-A First author arXiv ACL Code
  3. 2026

    EvoCurr: Self-evolving Curriculum with Behavior Code Generation for Complex Decision-making

    Cheng, Y.*, Wang, Z.*, Ma, W.*, Zhu, W., Deng, Y., Zhao, J.

    EMNLP 2026 Findings CCF-B Co-first author arXiv Code
  4. 2026

    LLM-SMAC: Solving Multi-Agent Decision-Making Tasks via LLM Decision Tree Code Generation

    Deng, Y.*, Ma, W.*, Fan, Y., Song, R., Zhang, Y., Zhang, H., Zhao, J.

    AAMAS 2026 CCF-B Co-first author arXiv Code
  5. 2025

    TacticCraft: Natural Language-Driven Tactical Adaptation for StarCraft II

    Ma, W.Y., Jiang, J., Fu, H., Zhang, H.

    ICML 2025 MusIML Workshop First author arXiv
  6. 2024

    Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach

    Ma, W.Y., Mi, Q., Zeng, Y., Yan, X., Wu, Y., Lin, R., Zhang, H., Wang, J.

    NeurIPS 2024 CCF-A First author arXiv Code
  7. 2024

    Adaptive Command: Real-Time Policy Adjustment via Language Models in StarCraft II

    Ma, W.Y., Xu, D., Lin, S., Zhang, H., Wang, J.

    DAI 2024 First author arXiv
  8. 2024

    SMAC-Hard: Enabling Mixed Opponent Strategy Script and Self-play on SMAC

    Deng, Y., Yu, Y., Ma, W., Wang, Z., Zhu, W., Zhao, J., Zhang, Y.

    arXiv 2024 arXiv Code

* Equal contribution · my name in bold

05 / Experience

Education & Internships

Education

  1. 2025.08 – 2026.09
    King Abdullah University of Science and Technology (KAUST)
    Ph.D. in Computer Science (left the program)
    Focus: game AI, multimodal LLMs, reinforcement learning, world models
  2. 2022.09 – 2025.06
    Institute of Automation, CAS / University of Chinese Academy of Sciences
    M.S. in Computer Technology
    Collective Decision Intelligence Lab · GPA 3.68/4
  3. 2016.09 – 2020.06
    Northwest A&F University, College of Life Sciences
    B.S. in Biotechnology
    Competed as a StarCraft II pro throughout

Internships

  1. 2025.02 – 2025.07
    Shanghai Jiao Tong University
    NLP Algorithm Engineer (Intern)
    • Built core modules of a study-companion agent: web retrieval, confidence scoring of results and decision flow.
    • Trained Qwen2.5-7B / QwQ-32B / Qwen2.5-72B with SFT, DPO, GRPO and REINFORCE++ for math reasoning and tool use.
    • Built an async Agentic RL pipeline on verl and ROLL, reproducing multi-node / async Search-R1 with ~40% faster training.
  2. 2024.06 – 2025.01
    CAMEL-AI × KAUST
    Joint Research Intern · Agents & Environments
    • Built StarCraft II agents and a micro environment on CAMEL, enabling VLMs to act on frames and text over many steps — led to the AVA paper.
06 / eSports

From Pro Gamer to Game AI

Player ProfileID://MEISAH
Meisah
马纬彧 · StarCraft II Professional Player
Race
Zerg
League
Grandmaster
Server
KR
Career
2016 – 2019
4 Titles
1 Runner-up
4 Deep runs
Incoming Transmission

Thousands of hours of high-level play gave me a first-hand feel for imperfect-information games, real-time adaptation and mind games. That experience shapes the questions I ask in research and how I evaluate agents.

— Meisah
Match History9 entries
  1. 2018 CSL College StarLeague Spring · Central-West 1st
  2. 2018 WESG World eSports Games · Xining qualifier 1st
  3. 2018 Lion King S1 Suning Global Invitational · city stage 1st
  4. 2017 NESO National eSports Open · Shaanxi division 1st
  5. 2018 NSL Pro Team League 2nd
  6. 2018 China’s Got StarCraft National tournament Top 16
  7. 2018 WESG China Finals Top 24
  8. 2016–2018 CSL College StarLeague · multiple deep runs Top 4
  9. 2017 NESO National eSports Open · nationals Top 24

Replays: photos & certificates

2017.10 NESO National eSports Open, Shaanxi — StarCraft 1st place
2018 WESG Xining qualifier — StarCraft II champion
Promoted to StarCraft II 1v1 Grandmaster
CSL College StarLeague, Season 5
2018 Suning Lion King Global Invitational S1 — city stage SC2 champion
2018 College StarLeague Spring — Central-West champion
2018 “China’s Got StarCraft” Top 16 — player card, Meisah (Zerg)
07 / Skills

Skills & Honors

LLM / Agentic RL PyTorch; LLaMA-Factory, verl, ROLL, slime; SFT, DPO, GRPO, REINFORCE++; async Agentic RL pipelines and environments.
Reinforcement Learning DQN, PPO, REINFORCE; multi-agent RL, experience replay, curriculum learning, code-as-policy.
Agents / Game AI LLM/VLM agents, RAG, tool use, skills, behavior trees; PySC2, DI-star, SMAC; world-model and multimodal game data pipelines.
Engineering Codex, Claude Code, Cursor; multi-machine data processing and training setups on Linux / Windows.
  • 2024 Alibaba Cloud Qwen Essay Contest — First Prize (AVA)
  • 2023 MathorCup Mathematical Modeling — First Prize
  • 2023 UCAS Innovation Practice — Second Prize
  • — IELTS 6.5 · fluent spoken English