• Idea, blog content: 100% mine
• Blog title, blog tags, research: 100% AI-assisted

Problem: Nowadays, coding contests usually have two types of contestants -- genuine ones (referred to as GC from now on) and ones that simply use AI (referred to as LOSER from now on). This is significantly affecting a GC's rank and overall performance, since the evaluation algorithm primarily depends on rank.

The problem is even bigger for some sensitive users (like me :)), when they see an actually hard problem getting solved by thousands of contestants in seconds, which decreases there motive to even continue with the contest. Since, they’re not even 100 percent sure if the problem is actually easy or just a bunch of LOSERs using AI.

Solutions such as camera proctoring, browser blocking, etc., simply won't work because there are several ways to circumvent them. Offline contests are simply not feasible given the frequency of the contests.

My take: Lets rethink evaluation. I think it’s time to stop depending on rank and matter fact to even stop providing one. The algorithm should now prioritise one’s absolute performance rather than relative performance.

Metrics like problem rating, number of attempts, time-to-solve, runtime, and memory usage should carry more than 90% of the weight for performance evaluation.

Sub-problem: How to calculate problem ratings before contest? As far as my research goes (I mean Gemini’s research goes), Codeforces problem ratings are currently crowdsourced, data-driven, and dynamically calibrated based on how actual users perform in the contest. Simply put, LOSERs are directly affecting a problem’s rating. Hence if kept the same -> a GC’s rating is still indirectly getting affected by a LOSER.

Now the best solution that I can think of is to use some ML/AI algorithm(s) to generate a problem rating given a problem statement, corresponding tags/concepts, etc.. Now I’m not an ML/AI guy, but I think we have more than enough data to build such a model. It seems comparatively an easier task, given we can build something like “Attention is all you need” type ML architectures.

Adding to that, we can also take help from problem setters and sure GCs (like Benq, jiangly, tourist, Kevin114514, etc..) to evaluate problem ratings before the contest.

I think this should give a fair enough rating to a problem. Comment if you can add something more or think of something even better.


Evaluation: Now that we have a problem rating, we have everything we need to evaluate a contestant’s absolute performance. We can design an algorithm that generates a score of a user’s solution that will directly affect user rating, without comparing to anyone else.

The point is to appreciate GCs for their efforts and provide them with a fair evaluation. We can just stop giving a {f-word} about LOSERs, let them play with their profile and do whatever they want; GCs should not be bothered by them.
 Additionally:
Given this seperation, we can also think of an algorithm to identify GC profiles (I’m not sure, but I feel like it’s easy to identify GCs than LOSERs; also I’m running out of time, I need to get to the office, so we can discuss this later). This way, someone who wants to evaluate a user’s profile (like a recruiter) can easily identify if the user is a GC or a LOSER.

Let me know what you think of this.
Byeeeee