Fixing the contest management mess

Revision en9, by purplesyringa, 2022-07-24 18:12:55

...or "How legacy systems fight obstacles no sane person would introduce in the first place".

Hello, Codeforces!

I am sure that those of you who tried to host their own programming competitions or write their own utilities for contest management are aware of the state of affairs regarding contest hosting. There are lots of incompatible formats, no one knows exactly how stuff ought to work, Polygon is full of nondescript and seemingly contradictory options, ej-polygon never seems to work quite correctly, modifications have to be applied to jury archives to make them work on your particular installation, non-trivial problem types require constant maintenance and hacks, ad infinitum.

Personally, I stumbled upon all these problems when I tried to get my own judge system working and suddenly discovered that the difficult part is not judging complex problems, but handling artificial complexity introduced by legacy software and standards. So here I am, attempting to solve these problems.


Firstly, I introduce a new format, which I shall call problem.xml format, which, as is perhaps obvious, is based on the Polygon format. I introduced one or two special cases here and there to ensure it's 99% compatible with the archives generated by Polygon presently. Unlike Polygon's format, it's completely documented, and as little leeway is allowed as possible while not compromising efficiency.

This format enables many, almost arbitrary problem types, in addition to the default types: input/output, interactive, run-twice, and problems with graders. For example:

  • Problems with custom scorers (better known as valuers by Ejudge adopters) are supported. This means that the points of a solution are not necessarily equal to the sum of points of each test; rather, any relations are possible, up to negative scores, efficiency-rated programs, and "write a program that prints your username".

  • What I shall call formulaic problems, when the user solution outputs a formula or a universal algorithm that is then executed by the judge.

  • Optional compilation on each test, which would come in handy for some practical development competitions.

  • Output-only problems, meaning that the user is asked to submit a ZIP archive that contains the answers to each test.

  • (Optional support) arbitrary strategies, which allow problemsetters to generalize the above as they see fit: run-thrice problems, compile-time testing, and CTF-like competitions are within reach with just a few lines of code,

  • Arbitration, allowing for marathon problems, which means that the points awarded to a submission may depend on the results of other submissions.

For existing platforms that support Polygon format or alike, few modifications will be necessary to get everything but strategy and arbitration working: I am hoping that major vendors are up to the task. Strategy support is somewhat tricky to implement, so that part is up in the air for the moment (not that this diminishes the rest of the work). Arbitration should theoretically be easy to get working, but might require some modifications to the software architecture.

The draft of the specification is available here: https://github.com/imachug/problem-xml-specs. While I think that the spec is mostly finished and I'm publishing it here for better visibility, I'd be glad to listen to your input and apply changes where necessary!


Tagging some people for better visibility—your input is highly appreciated: geranazavr555, MikeMirzayanov, grphil, andrewzta, dkirienko.

Tags polygon, ejudge, pcms2, codeforces

History

 
 
 
 
Revisions
 
 
  Rev. Lang. By When Δ Comment
ru9 Russian purplesyringa 2022-07-24 18:13:10 8
en9 English purplesyringa 2022-07-24 18:12:55 3 Tiny change: 'visibility--your input' -> 'visibility—your input'
en8 English purplesyringa 2022-07-24 18:11:10 0 (published)
ru8 Russian purplesyringa 2022-07-24 18:10:53 0 (опубликовано)
en7 English purplesyringa 2022-07-24 18:08:42 4 Tiny change: 'ssary!\n\n\n\n---\n\' -> 'ssary!\n\n---\n\'
ru7 Russian purplesyringa 2022-07-24 18:04:59 7 Мелкая правка: ' программы). Это зна' -> ' программы оценки). Это зна'
ru6 Russian purplesyringa 2022-07-24 18:04:17 275
en6 English purplesyringa 2022-07-24 18:02:23 253
ru5 Russian purplesyringa 2022-07-24 17:59:30 525
ru4 Russian purplesyringa 2022-07-24 17:52:21 921
ru3 Russian Death_on_2_Legs 2022-07-24 17:42:50 2 Мелкая правка: 'ьницами»\nПривет, ' -> 'ьницами»\n\nПривет, '
ru2 Russian Death_on_2_Legs 2022-07-24 17:42:00 2889
en5 English purplesyringa 2022-07-24 17:24:10 34 Tiny change: 'visibility: [user:ge' -> 'visibility--your input is highly appreciated: [user:ge'
ru1 Russian Death_on_2_Legs 2022-07-24 17:23:54 3084 Первая редакция перевода на Русский (сохранено в черновиках)
en4 English purplesyringa 2022-07-24 17:21:03 502
en3 English purplesyringa 2022-07-24 17:14:37 212
en2 English purplesyringa 2022-07-24 16:55:02 9 Tiny change: 'ems fight issues no sane' -> 'ems fight obstacles no sane'
en1 English purplesyringa 2022-07-24 16:43:35 2736 Initial revision (saved to drafts)