TLDR: Exact implementation of CF rating system described in original blog yields systematic bias. Mathematically impossible deflation detected in certain contests. Possible links to % of new accounts.

Originally uploaded onto another account, I feel like the blog was poorly written back so I rewritten certain parts and upload it here instead.

Background

A few days ago, I was messing around and trying to implement my own rating change calculator for Codeforces (basically my implementation of Carrot). All formulas and calculations are derived from this blog. When I compared my results with Codeforces actual rating changes, I observed a very significant positive bias.

Keep in mind that all calculations are done with True rating, Which accounted for ramp-up offset. The graph only includes data from 12 hand-picked contests (1000, 1100, 1200, 1300, 1400, 1500, 1585, 2000, 2050, 2130, 2200, 2234), but the trend generalize to other contests too (_more on this later_).

As for why Carrot tends to have a negative bias instead, I believe Carrot uses displayed rating for new players instead of true rating (?). Not too sure on this, but feel free to leave a comment if you have any info on this.

When I try to troubleshoot the issue, I discovered that lots of Codeforces rounds has mathematically impossible deflation.

As explained by this blog, to prevent rating inflation (top rated players from repeatedly gain rating from low rated players), Codeforces employs a layer of rating adjustment:

Let $$$s_{i}$$$, $$$r_{i}$$$ and $$$\Delta_{i}$$$ be the rating before contest, rating after preliminary calculation and the rating change of $$$i$$$-th player. Keep in mind $$$=$$$ is assignment.

• $$$r_{i} = r_{i}-\overline{\Delta}-1$$$, $$$\Delta_{i} = r_{i} - s_{i}$$$, ($$$\overline{\Delta}$$$ is arithmetic mean of $$$\Delta_{i}$$$)
• $$$r_{i} = r_{i} + min(max(-\frac{\sum_{i \in S}\Delta_{i}}{|S|},-10),0) $$$, ($$$S$$$ is the set of $$$min(n,4\sqrt{n})$$$ players with highest $$$s_{i}$$$)

Looking at the formula, we can see that total delta is at most $$$-n$$$ after the first round of adjustment, and at most $$$-11n$$$ after the second. Hence, it is impossible for a round to have $$$\overline{\Delta} \lt -11$$$ after all the adjustments, yet the data paints the opposite story.

Possible theories

Some pretty interesting relations. The deflation (and the positive bias) correlates really well with the number of "ramping-up" accounts in each contest.

Here, $$$S_{x}$$$ refers to the group of players who have only previously participated in $$$x-1$$$ contests, and $$$\text{old}$$$ refers to the group of players who have exhausted all of their ramp-up offset, or are from the old system. There are two things that stand out here:

• Errors for contests with close to $$$100\%$$$ players from the $$$\text{old}$$$ group are near $$$0$$$.
• Errors for $$$S_{1}$$$ are always close to $$$0$$$.

These 2 very clear trends lead me to believe that this peculiar behavior is related to how ramping-up accounts are handled. Although, I haven't figured out the exact formula yet.

Overtaking higher rated players by… performing worse than them

Another consequence of this is, since my rating calculations are referenced straight from the original blog, the inflation adjustment for all players in a certain contest is uniform. And by looking at the errors, I suspected there might be a case where the following condition mentioned in the original blog is violated:

“If the participant A had worse rating than the participant B before the contest and finished the contest on the worse place then after recalculations the the rating of A can’t be greater than the rating of B”

Turns out, there is a case like that.

Look at vernov.egor and unchainedos. If we work out their true rating, it would’ve been $$$1413 \to 1290$$$ for vernov.egor and $$$1400 \to 1299$$$ for unchainedos. Granted unchainedos didn’t rank strictly lower than vernov.egor, but we can all agree that this kind of overtake should not have happened.

Concluding remarks

The point I’m trying to make is, The official Codeforces blog on their rating system doesn’t seem to fully explain every single step in their calculations, unless there were some updates to the system since 2015 that I missed. If that is the case, feel free to point it out.

Codeforces team, if you see this, please confirm whether there are additional hidden mechanism related to rating inflation or not. I'd love to know either way.

Obviously, it is entirely possible that there are errors in my data pipeline and calculations, which is another reason why I wanted to post this blog. I will probably add a link to my calculations in a future update. But for now, that’s all I wanted to say.