Recently, we’ve run into an unpleasant situation: some large companies have been using a huge number of accounts to submit AI-generated solutions purely for model training, which ended up causing serious “in queue” congestion for everyone else.
Maybe it’s time to consider restricting submissions from unrated users.
If those AI-driven accounts were required to participate in at least one contest before being allowed to submit normally, it would drastically cut down the number of accounts they could abuse. On top of that, we could improve AI-detection during contests and apply stricter monitoring to accounts that submit massive amounts of AI-generated code.
For regular users, the requirement is trivial — just join a single Codeforces round, and you’re good to go. Compared to the awful experience of having the queues jammed for hours, this small inconvenience might be a reasonable trade-off.
That’s the best idea I can think of at the moment. What do you think? How can we better protect the platform from such exploitative behavior?
Update：ig RainRecall's idea is much better
Perhaps assigning a separate evaluation machine specifically for Unrated Users would be a better approach.
If we group users by rating and let each group’s judges prioritize submissions from that group (only taking others when they’re idle), it should significantly reduce the load and help alleviate the issue.
Auto comment: topic has been updated by Hoks_ (previous revision, new revision, compare).
For many users codeforces rounds are always during their night or work hours, so for them just participating in one contest isn't trivial cuz they literally can't take part live. They would be forced to register, submit random stuffs, and go to sleep/work just to unlock practice privilege. That's not participation but just a formality.
If people start registering and sending random submissions only to pass this requirement, that itself creates extra useless load on cf judge machines. The same trick works for llm companies too, extremely easy to script auto-register for a contest, auto-submit some dummy code once, and the account is "unlocked" forever until admins ban it manually. So the barrier is annoying for honest users but almost zero for automated abuse.
Therefore, I'm afraid this rule would mainly punish normal users with bad time zones or busy schedules, while serious abusers can bypass it with a bit of scripting. maybe it's better to look at submission rate limits, better llm detection or limiting suspicious bulk submissions etc that target abusive behavior directly instead of tying practice access to contest participation.
broski is larping as System
That's a good idea, but it will limit unrated beginners (such as myself) who only do casual problem solving on Codeforces and who live in time zones where contests are at night.
I like this concept though and think that it can be improved by allowing one submission every, say, 5 minutes for unrated users. I believe unrated users don't need to submit code in quick succession (I almost never do so).
I don't think it's useful to limit the submission frequency because the AI companies could register a lot of accounts.
Then, what about making account creation invitation-only? Every one above a certain threshold of (participation, rating) get 1 invitation token per day.
If you really don't know anyone who does CP IRL, you can submit a request to a review queue, which volunteers (again, appointed from the community with a certain higher threshold of (participation, rating)) would review and grant on a case-by-case basis.
And then someone pays a LGM to invite a bot into the system, and everything breaks again.
This problem really seems unsolvable.
First, I want to ask: How do they register their accounts? By using temporary email? If RIGHT, can we start from this?
And how about using reCAPTCHA only for unrated users, especially Google reCAPTCHA? It can prevent bot efficiently, and won't be punitive for most users who have rating and human unrated users.
The thing is you can't reliably prevent automated account creation unless you introduce a strong mechanism that links accounts to individual persons. That would be infeasible for international websites like Codeforces.
Maybe phone verification? I am aware that codeforces uses emails.
Unfortunately, I do not think this is viable. For one, not everyone uses this site to partake in contests or has any desire to; it has a lot of other uses (for example, running mirrors of ICPC contests, practicing problems one is interested in, etc). Furthermore, this just moves the goalpost; what would happen next is that the bot accounts would flood contests, partake in one (which could make the experience of contests consistently worse), then go back to spam submitting on everything.
The cold, hard truth is that there is no "good" or "easy" solution to this. Most anything which would have a significant enough impact on the bots would have an arguably more significant impact on normal users. If such a solution existed, I'm sure Mike would have implemented it by now. I think that the best we can do as a community is wait patiently for if/when Mike finds a reasonable enough compromise instead of posting multiple blog posts every day complaining about the situation or suggesting draconian solutions that only sound good on the surface; I'm sure that Mike has enough to deal with as-is.
How about phone number verification for users to be able to participate in a rated contest and on top of that, only accounts with atleast one rated contest can submit solutions.
But I think that would be an issue if someone new were to register and find that there are no upcoming contests for a while, so maybe set a cap of something like 25-50 problems, unless you have participated in a rated contest, that's the maximum number of solutions you can submit. This and phone number verifications will drastically reduce the queue times (I believe).
(25-50 problems might seem a lot, especially since they can just create a new account after hitting the cap, but phone number verification is something that they cant just keep on doing, plus it will also help in cheaters' cases, if they get banned once, they would have to get a new phone number which is something they should eventually get tired of)
I wouldn't mind seeing a codeforces app that runs your submissions purely locally outside of contest as the credibility doesn't matter much which leaves a good room of servers for contest submissions My point stems from the fact that you can always copy a code and get the problem accepted
But there is a bit difference between different devices and this may cause TLE when your personal device is slow. Or if we can lengthen the time limit a bit?
An app that possibly mimics the performance of the servers somehow, I don't have knowledge in that field but it sounds doable
You can already run your code locally.
In order to run on actual test data, it'd need to have actual test data. Since the sum total of test data for all problems is surely large, it'd need to be transmitted from a central DB for a problem you select. We're back to the original problem of a long queue forming, only with deduplicating for multiple submissions, at the cost of making CF test data public to anyone dedicated enough to hack the app since when a submission is executed, it receives inputs in plaintext.
It looks like llm is being trained on cf problems. I haven't seen a bot account submit wronged submission again. Phone verification seems the best option as i remember in a previous contest the entire standing was filled with newly created bot account, so restricting submission idea will fail.
It looks like this issue might be hard to handle. The solution I suggested can be instantly solved by a large number of AI-driven accounts in any contest, and as for phone verification, big companies could easily get enough accounts simply by having a few employees register them.
This is a big challenge faced by almost all major OJs, including but not limited to Codeforces, Luogu, QOJ, and AtCoder.
What if we limit unrated accounts for like 10-30 Submissions a Day?
Honestly, out of contest, limit everyone to 50 (adjusted for teams) and unrated to 10. Spamming submissions is bad in general, not just for the testing system but also as a bad habit to develop instead of debugging locally, and as a sign of unhealthy obsession. A meatbag is physically best off trying next day after 10 submissions.
It won't solve the problem of account creation though.
I've had some intended solutions that required enough constant factor optimization to take me ~100 submissions to solve, so I personally wouldn't be in favor of that. I think there are other ways to filter bot accounts that work better (for instance, no more than 50 accepted submissions would work).
That's a situation in which you need to do the bulk of constant factor optimisation on your own machine. It's an absolute nightmare if you try to do it without knowledge of tests, so make your own tests — I suspect it took you that long because of the approach you took since outside of April Fools contests, it's not intended to make wrong submissions for a contest problem. Only use custom invocation to compare your machine with CF ones and submit what you're reasonably confident will pass, not as an optimisation strategy.
What's stopping the bots from joining a contest?
Perhaps assigning a separate evaluation machine specifically for Unrated Users would be a better approach.
wow wonderful idea
Auto comment: topic has been updated by Hoks_ (previous revision, new revision, compare).
Just paywall it, then distribute the revenues as dividends to long-time users of Codeforces. #TwentyOneHundredOrBustForPresident
I think the reason of the incrasing unrated AI submitters is there isn't any online judges (at least none of the online judges I know) has the judge service for AI training, so they can only use these accounts to test their AI's codes.
So why not codeforces start a paid service for AI companies? (for example, $1 for 20 submissions) The submission of AIs will have low priorities, that means, AIs' submission will only be judged if there isn't any normal accounts' code is still
in queue.This may create more earns for the codeforces community, and the money can be used to upgrade the judgers and other things benefits the whole community.
ig the companies won't pay for this.
For them, the current method offers higher efficiency and lower cost, doesn't it?
Maybe mike can update the privacy and use the power of laws to stop these AI companies from submitting their code in a non-AI channel?
like Luogu,Chen_Zhe have mentioned being troubled by ByteDance's AI tests. While it can be confirmed that ByteDance is involved, there is no conclusive legal evidence—this presents an enormous challenge.
For AI models, all they need is the problem statement, and they can independently perform tasks like data generation and comparison, making judge unnecessary
Is there any way we could find out which companies are doing it? That way at least we could bring the concern to a larger population, because I think a lot of people who aren't on Codeforces would see large companies ruining hobby platforms for profit as ethically unacceptable.
That being said I have no idea how, and assuming it's not possible, I do think that some action needs to be taken to reduce the inflow of AI submissions. An alternative I might propose is that any account that solves some large number of problems (maybe 25-50 as some other users suggest) within a single day gets banned from the platform.
Perhaps this method is not very user-friendly for beginners like me who have not been rated, as finding a time to play Codeforces is extremely difficult and affects my regular training. Suggest improvement.
I agree with you.
Give verified big companies offline access to test data under strict legal agreements—ban them from creating any accounts. This stops AI spam, reduces load, and protects contest integrity.
I think mobile number verification would help alot with it, so it would limit that part for companies. I see it as the best solution so far for this problem
Or to create another portal for those companies
In fact, I can't hand it in even after scoring, and I'm speechless = _ =
nan i cant use rmj anymore =(
I think this is not feasible. I originally registered an account just to do some exercises, not to participate in competitions, and it's been too late! It's very unfriendly to people in other time zones. If someone just joins a competition and submits random code, it will waste a lot of resources, which is almost no different from an AI attack! Therefore, I suggest implementing some sort of real-name verification, which could filter out many AIs (also, when will vjudge start allowing bot accounts online!).
im just commenting bc i cant submit my hw
these bots are slowing down codeforces
??What
this bots are slowing down codeforces and making a large queue of submissions :)
random comment
111
123
This is just stupid, banning Unrated wouldn't be useful, the bot can just take part in a contest (manually or by using scripts). And anyway vjudge is affected as well
yeah you're right.ban always a bad idea but we still have to face the problem.Fortunately,this issue does not occur frequently now.Good news,I guess
wtf it say on vjudge now that u need to be green
Hoks_ is cheaaaaaaaater, he reached grandmaster by cheating, I am his friend and I know that he cheated, hahahahahahahahahahhaa
But vjudge.net's users not is AI.And some users no time to participate in the competition.They won't be able to submit remotely through vjudge.net anymore.
Can't we just use better llm detecting tools than prohibiting unrated user submitions?
I can't submit codes now, since Chinese users have to participate at midnight...
Understand, but not very happy though...
OH NO
做事要讲良心
Please Speak Chinese
Perhaps Hoks_'s solution is not very friendly to people in some time zones, as most regions in China are already asleep at UTC 14:00 (UTC+8 22:00). But this is currently the least costly and most useful method. I have a question: can these robots actually pass Cloudflare's Turnstile (a CAPTCHA-like verification)? In fact, it is quite difficult even for real humans to pass it, especially when some users are using poor IPs provided by their ISPs (like me). I usually use methods such as using a VPN to get through the challenge.
Translate by DeepSeek.
We could optimize the system by allocating specific judges to different rating buckets. A judge should focus on its assigned rating range first, then help with the rest of the queue if it has spare capacity. ig this idea is the best
OK. That's good, but that could be more costly. Does Codeforces implements your plan? Could you answer my question: why can some robots pass the verification?
My English is not good, sorry.
Actually, shortly after posting, I realized that this situation is quite rare. It’s more of an edge case, so there’s probably no need to implement such a complex measure for it I never saw this happen again before my AFO.
It proves Codeforces is better and the most of AI companys are not terrible.
Preventing remote submissions based on rating points is basically meaningless, because many new users have low ratings, yet they still need practice problems. If these beginners are obstructed from remotely submitting problems, it could actually delay the time it takes for them to reach higher ratings.
At the same time, many countries and regions have inconvenient timings for CF contests (for example, China and Japan are in the nighttime during CF contests, and users from these two countries account for a large proportion), making it very inconvenient for participants to compete, which is also an important factor affecting ratings.
I suggest continuing to use submission frequency detection: if the same account submits code for the same problem repeatedly, it would temporarily prevent further submissions. While this sacrifices some accuracy (of course, the algorithm can be optimized to minimize this), it makes things easier for more users. Moreover, CF can block accounts that are obviously bots, like "Vjudge1" or "Vjudge2", without completely obstructing normal users.
Another method is to look at the duration since a CF account was registered and its previously submitted problems. Through some calculations, it can be determined whether it is a regular participant or a bot. This method is undoubtedly simpler.
Since your platform has developed bot detection and AI recognition technologies, you should put them into practical use by identifying bot accounts in the code and submission frequency, rather than simply implementing a "one-size-fits-all" block based on rating points.
Additionally, I have observed that even before this standard was introduced, there were no cases where submissions supposedly needed to be "In Queue" for several hours. On the contrary, now a lot of time is actually wasted on bot verification.
What's your opinion?