About ten months ago, Stop Fighting with Polygon — Create and Test Problems Locally was written. It introduced Polyman, a CLI for problem setters. You write the validator, generators, checker, and solutions on your own machine. One command, polyman verify, generates every test, validates it, and checks that every solution gets the verdict it should. Then polyman remote push sends the whole thing to Polygon.

Thank you for the upvotes, the issues and the pull requests. A lot has changed since then, so here is what's new.

Upgrade:

npm install -g polyman-cli@latest
polyman --version   # 3.0.0

There is one breaking change in how generator scripts are written. See Migrating from v2 at the end; it takes about two minutes.

1. Polyman is now built for AI agents

This is the biggest change. More and more of us write code together with Claude Code, Codex, Cursor or Aider. A problem package is a good job for an agent. The pieces are small and well defined, and there is a single objective test of "done": polyman verify passes.

The catch is that an agent needs to know the rules of problem setting. For example, the validator must read the exact bytes, and a WA solution must actually fail. So every problem created with polyman new now includes that knowledge:

my-problem/
├── CLAUDE.md             ← short orientation: pipeline, layout, where things live
├── AGENTS.md             ← same thing, for Codex/Cursor/Aider/etc.
├── Config.json
├── Config.schema.json    ← JSON Schema; agents and editors get validation + autocomplete
├── instructions/
│   ├── working-rules.md  ← how to behave, what "done" means
│   ├── validator.md  generators.md  generator-script.md  checker.md
│   ├── solutions.md  manual-tests.md  statements.md  config.md
│   ├── cpp-performance.md
│   └── commands/         ← one file per polyman command
├── solutions/  generators/  validator/  checker/  statements/  manual/

The docs are split on purpose. The agent reads CLAUDE.md and the working rules once, then opens validator.md only when it touches the validator. That keeps its context small and the output focused.

The instructions cover the mistakes agents (and people) actually make:

• trailing spaces and missing final newlines that fail validation;
• CRLF confusion;
• Unicode dashes in LaTeX statements that Polygon warns about;
• 1000000 written without digit separators;
• "compiles fine" being mistaken for "done".

They also tell the agent not to rush ahead. If you paste a statement, it acknowledges it and asks a question. If you ask "what approach?", it analyses the problem without editing files. It writes code only when you ask it to.

Machine-readable results: --json

An agent shouldn't have to scrape box-drawing characters to find out whether a solution passed:

polyman verify --json

{
  "schemaVersion": 1,
  "ok": true,
  "command": "verify",
  "failedStep": null,
  "steps": [ { "name": "generate-tests", "ok": true, "errors": [] }, ... ],
  "solutions": [
    { "name": "main",   "tag": "MA", "matchesTag": true, "tests": [ ... ] },
    { "name": "slow",   "tag": "TL", "matchesTag": true, "tests": [ ... ] }
  ]
}

Standard output contains exactly one JSON document, and the human-readable log goes to stderr. If something fails before the solutions even run, such as a compile error in the validator, you still get valid JSON with failedStep set. polyman run <solution> --json works the same way.

Headless Polygon push

remote push used to stop and ask for confirmation before creating a new Polygon problem, which hangs an agent forever. Now:

polyman remote push . -y -n my-problem-slug

It creates the problem, uploads everything and saves the new problemId into Config.json. With no terminal and no flags, it exits with an error that names the flags instead of hanging.

What an agentic session looks like

Open the problem folder in your agent and say something like:

Here is the statement: given n numbers and S, find two indices i ≠ j with a_i + a_j = S. n ≤ 2·10^5. Write the validator with tests, a random generator plus one that makes many equal values, a main solution, a Python alternative, a brute force tagged TL, and two WA solutions that fail in different ways. Then make polyman verify pass.

The agent writes the files, runs polyman verify --json, reads which step failed, fixes it and runs it again until everything is green. You review the code, which is the part that needs a human, instead of doing the plumbing.

This isn't hypothetical. To test this release, I had an agent build exactly that problem from scratch, with 5 solutions, 2 generators, a custom checker and 12 tests. It then pushed the problem to Polygon, committed, built a package (READY, with verification), pulled it back into a fresh folder and verified it again. That run caught two bugs, and both were fixed before this release.

2. Generator scripts are real Polygon scripts now (breaking)

In v2 the generator script was a JSON commands array in Config.json, which Polyman translated to and from Polygon's format. The translation was fragile. Now the script is Polygon's format, in a plain text file:

<#-- generators/gen-script.txt -->

<#-- @group small -->
<#list 1..20 as i>
gen-random -n 10 -seed $$${i}  \gt  $$$
</#list>

<#-- @group large -->
gen-random -n 200000 > $
gen-equal  -n 200000 > $
gen-pair   7 > {40-41}

• > $ means the next free index, > 5 is an explicit index, and > {40-41} means one generator run writes several tests.
• <#-- @group X --> assigns groups, and <#list> loops work like they do on Polygon.
• remote push and remote pull send the script exactly as written, in both directions. What you run locally is what Polygon runs.

Manual tests are files with explicit indices:

"testsets": [{
  "name": "tests",
  "generatorScript": { "scriptFile": "./generators/gen-script.txt" },
  "manualTests": [
    { "input": "./manual/tests/m-01-sample.in", "index": 1, "group": "samples", "useInStatements": true }
  ],
  "groupsEnabled": true,
  "groups": [{ "name": "samples" }, { "name": "small" }, { "name": "large" }]
}]

3. Much faster: compile cache and testlib built once

Every verify used to recompile everything. testlib.h is a large header, so each validator, generator and checker took several seconds to compile, every time.

Compile cache. Each compiled binary is stored under .polyman/cache/. It is looked up by a hash of:

• the source;
• every local header it includes, testlib.h among them;
• the compiler flags;
• the compiler version.

If you change any of those, that one file is recompiled. Otherwise it is reused. Polyman can't serve a stale binary, because any change produces a different hash.

testlib compiled once. Polyman now compiles the body of testlib.h once per problem and links it into every testlib program. Your generators, validator and checker compile against a small header, and the heavy part is never compiled twice.

Here are timings on my laptop for the template problem from polyman new (3 solutions, validator, custom checker, generator):

• polyman verify --no-cache: 15.2 s
• polyman verify, first run: 9.5 s (testlib is built only once)
• polyman verify, next runs: 2.0 s
• polyman test checker after editing the checker: under 1 s (was about 4.5 s)

$ polyman cache status
  ℹ Prebuilt testlib 0.9.45: 490.6 KB, linked into every testlib program
  ℹ 4 cached binaries, 935.7 KB
   1. solutions/acc.cpp      16.6 KB  compiled in 1.7s, last used 2026-09-25 20:04
   2. checker/chk.cpp       306.6 KB  compiled in 709ms, ...

verify prints Compile cache: 4 hits, 0 misses · saved ~7.8s at the end. Use polyman cache clear to wipe the cache, or pass --no-cache to any command to skip it. A cache problem never fails a compile: Polyman just compiles the file normally.

4. C++23, and your sourceType is respected

• Local compiles now use -O2 -std=c++23 by default. Set "cppStandard": "c++20" (or another standard) in Config.json to change it.
• On push, each file's sourceType in Config.json is used, for example cpp.gcc14-64-msys2-g++23. Before, every C++ file was uploaded as g++17.
• .cc and .cxx files are accepted everywhere .cpp is.

5. Polygon sync fixes

• Push errors show the reason. "Failed to upload generator" now includes Polygon's actual error message.
• Generator names are mapped. A script line uses the name from Config.json, while Polygon resolves it against the uploaded file name. Polyman now rewrites the name on push, so pushed scripts run on Polygon.
• Test groups are set correctly on script-generated tests.
• Pushing from outside the problem folder is safe. It used to wipe the remote script and fail to upload a new one.
• Pulls are cleaner. A pulled custom checker no longer shows up as a generator.

6. Reliability

• Process spawning, including how time and memory limits are enforced, was rewritten on top of execa.
• Paths with spaces and shell characters (~/My Problems/it's (hard)) work, thanks to Gustavo-Harnisch.
• A fresh polyman new passes polyman verify straight away, so you start from a green state.
• The codebase has 967 tests, and CI runs on Node 20 and 22.

Migrating from v2

1. Move each testset's generatorScript.commands into a text file, one line per command, in the format above. Set "generatorScript": { "scriptFile": "./generators/gen-script.txt" }.
2. Move manual tests into manualTests[]. Name the files m-01.in, m-02-edge.in and so on, and give each one an explicit index.
3. Optionally add "$schema": "./Config.schema.json" to get editor validation. polyman new creates new problems with it already.
4. Run polyman verify.

If you have a problem on Polygon already, the easiest route is polyman remote pull <id> ./dir. You get the new layout directly.

Thanks

• Ahmad-Faraj built the compile cache.
• Gustavo-Harnisch fixed shell-safe paths.
• Everyone who opened issues: the C++23 request, the ulimit error and the generator upload failure all shaped this release.

What's next

These issues are open and PRs are welcome:

• a diff between local and Polygon before you push (#10);
• shell auto-completion (#9);
• parallel generation and validation (#7);
• time scaling for Java and Python (#5);
• auto-deriving names from file paths (#12).

Links:

• GitHub (a ⭐ helps)
• Tutorial
• Full guide
• npm

Tell me in the comments how you set problems today. And if you try the agent workflow, tell me where the agent got stuck. That is the most useful feedback for the next version.