Блог пользователя cgmoreda

Автор cgmoreda, история, 19 месяцев назад, По-английски

Hello, Codeforces community!

Lately, AI-assisted submissions have gotten out of hand, making contests less fair and a lot less fun for those competing honestly.

My idea is simple: add a second Elo system, separate from the current one, just for AI users. When you register for a contest, you choose whether you’ll be using AI or not, and you’ll be ranked with others who made the same choice.

Of course, it’s not a perfect fix—some people might still break the rules—but I think it could keep about 70–80% of AI users out of the main competition. Plus, a lot of them might actually want to join the AI track to climb a separate leaderboard.

This was just a random idea that popped into my head. I’m not sure how easy it would be to pull off, but I thought it was worth sharing and hearing what you all think!

  • Проголосовать: нравится
  • +97
  • Проголосовать: не нравится

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +25 Проголосовать: не нравится

Like Meta. Full suppot sir.

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +53 Проголосовать: не нравится

Not a good idea IG. cause why cheater will let us know , they are cheater?

I think mobile number idea will work nicely! 1 mobile number 1 Codeforces account.!

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +35 Проголосовать: не нравится

So just "click here to certify that you are an AI cheater"? I don't think that would work

»
19 месяцев назад, скрыть # |
← Rev. 2  
Проголосовать: нравится +79 Проголосовать: не нравится

I think it could keep about 70–80% of AI users out of the main competition

This is so completely wrong. I'd be shocked if 10% of AI users chose the AI track. Even that is probably an overestimate. The whole reason they use AI is because they want to cheat. They want to perform well while claiming that they're doing so legitimately, by themself. They have no incentive to do an AI track. The only people who would choose this track are the very few people who might want to test their prompting skills or similar, I guess. But very few people fall under this category, and it seems mostly pointless.

As AksLolCoding pointed out, maybe we could use it as a kind of LLM benchmark. But then, there's not much reason for multiple users using the same AI. At that point, they could make an LLM benchmark track with just the AI, no (or very limited) human assistance. I think that'd be pretty interesting, for the sake of seeing which AI is best.

  • »
    »
    19 месяцев назад, скрыть # ^ |
    ← Rev. 3  
    Проголосовать: нравится 0 Проголосовать: не нравится

    I do think that an LLM benchmark is viable, but it should only be available to the people making LLMs (like most other benchmark websites) so that there is less bias

    • »
      »
      »
      19 месяцев назад, скрыть # ^ |
       
      Проголосовать: нравится 0 Проголосовать: не нравится

      Yep, I agree. (Isn't that what I said? lmao)

      I'm concerned that the people making the LLMs would be biased, though. We always see these claims of super high ratings, and I'm not 100% sure how legitimate they are (I'm not saying they're illegitimate, just that I'm not sure). It's possible that they gave away too many hints during prompting.

      One alternative approach is also to have the same person prompting several AIs simultaneously (or maybe virtualing the contest with each AI); ideally, a skilled and unbiased prompter. They could do two different types of benchmarks: one which is, input only the statement and submit only the first output, and see how good AI is without human assistance at all, and a second one which is, see how far the LLM can get with human assistance (where the degree of assistance is roughly invariant since the same person is prompting).

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +110 Проголосовать: не нравится

brilliant blog!!!

How to stop people cheating?

Just ask the cheaters to tick a box saying "I am a cheater". Problem solved.

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +6 Проголосовать: не нравится

I like this suggestion, in chess this is called "Centaur chess" but unfortunately the main chess servers nowadays don't have this service. I'd love to play some centaur chess using my engine shameless plug but unfortunately I think I'm a minority here. If the ratings would be completely separate then I'd participate in div2/div3/div4 rounds as a centaur just for the lulz.

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +1 Проголосовать: не нравится

Part of me likes the suggestion

»
19 месяцев назад, скрыть # |
← Rev. 2  
Проголосовать: нравится 0 Проголосовать: не нравится

I think codeforces should introduce asking permission from user for control over its hardware keyboard/clipboard

Basically,when user will start contest codeforces will ask permission over clipboard for duration of contest only.

It should disable copy paste option during codeforces over all of user's hardware and software

For that codeforces should develop some package and users have to install it on their hardware to give contests.

If user needs to urgently copy paste he may quit contest and then copy paste

I think this will reduce cheating by AI.(Its just a idea, I don't know if it will pass or not,just a prototype)

  • »
    »
    19 месяцев назад, скрыть # ^ |
     
    Проголосовать: нравится 0 Проголосовать: не нравится

    This idea has a lot of issues, first of all, you can easily not copy paste the code and type it, takes maybe 5 more minutes.
    second, in any os in the world you can probably spend 1,2hr to code a clipboard system that doesn't use the system clipboard, with tools like AI, learning how to code one or just coding it using an AI is the easiest thing possible.
    third, this would ruin data structure templates
    fourth, Imagine you're just a random guy doing cp for fun, why would you trust a website on the internet to have access over you keyboard?

  • »
    »
    19 месяцев назад, скрыть # ^ |
     
    Проголосовать: нравится 0 Проголосовать: не нравится

    "hardware package"? They are called drivers and are very restricted due to security reasons. It is extremely difficult to get one approved, and even if it was I doubt people would willingly install it (look at Vanguard for example)

  • »
    »
    19 месяцев назад, скрыть # ^ |
     
    Проголосовать: нравится 0 Проголосовать: не нравится

    bro but we have long ds templates

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +10 Проголосовать: не нравится

Cheaters cheat to gain rating on their own account, not to check how well can AI perform in a contest. So, i don't think so this will work

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится

You’re talking complete nonsense. Where did you get those percentages from? My dumb idea to convert all problem statements into images with added noise, alongside your suggestion, doesn’t sound so bad. At the very least, it would make life harder for cheaters.

  • »
    »
    19 месяцев назад, скрыть # ^ |
     
    Проголосовать: нравится 0 Проголосовать: не нравится

    Noisy images take a lot of space, and CF servers are slow enough already at the start of contests. It would probably do more harm than good for those with low bandwidth, and probably slow contests down even if you have a fast connection, without making it much harder to cheat

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +35 Проголосовать: не нравится

Sadly it won't work, because cheaters just want the title of "solving by themselves". Still they will participate in normal contests to "prove" themselves.

If they really want to test AI, they can choose to vp after the contest, instead of violating the rules.

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится

Then I think participation will be <2000 participants in honest division and >10000 participants in AI use division ;)

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится
but I think it could keep about 70–80% of AI users out of the main competition.

really? I doubt if the number would be even 1%

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +2 Проголосовать: не нравится

I had the same idea a while ago, but cheater won't admin they are cheating.

An alternative idea is to replace the "cheaters elo" with a "verified elo."

To compete in the verified elo, participants must screen record their entire contest session. After the contest, the top 10 finishers, along with a few randomly selected participants, will be required to upload their recordings for review.

If a participant fails to upload their recording, or if the recording shows evidence of cheating, their verified elo will be permanently revoked.

Additionally, if a participant's performance appears suspicious, others may request their recording for review within a few days after the contest.

If no request is made within a few days, participants are free to delete their recording.

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится +3 Проголосовать: не нравится

No way this gonna work.

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится

How this blog is even getting upvotes , for me this solution literally makes no sense at all.

»
19 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится

I think this is a brilliant idea. One step further, we can even allow users to choose for a specific problem whether they want to go with AI or not. The idea is that I don't believe every AI user is deliberately cheating, they may just want to practice their AI skills or try a different coding pattern (right??).

Another benefit is that the submissions in the AI track can be used to detect AI users in the normal track.