TLDR: Exact implementation of CF rating system described in original blog yields systematic bias. Mathematically impossible deflation detected in certain contests. Possible links to % of new accounts.

Originally uploaded onto another account, I feel like the blog was poorly written back then and deserves some more refinements.

Background

A few days ago, I was messing around and trying to implement my own rating change calculator for Codeforces (basically my implementation of Carrot). All formulas and calculations are derived from this blog. When I compared my results with Codeforces actual rating changes, I observed a very significant positive bias.

Keep in mind that all calculations are done with True rating (i.e. hidden rating when adjusted for new account ramp-up), which is calculated by painstakingly counting the number of rated contests for each player up to that point (and differentiating old ramp-up system vs new ramp-up system). The graph only includes data from 12 hand-picked contests (1000, 1100, 1200, 1300, 1400, 1500, 1585, 2000, 2050, 2130, 2200, 2234), but the trend generalize to other contests too (more on this later).

This is

As for why Carrot tends to have a negative bias instead, I believe Carrot uses displayed rating for new players instead of true rating (?). Not too sure on this, but feel free to leave a comment if you have any info on this.

When I try to troubleshoot the issue, I discovered that lots of Codeforces rounds has massive deflation.

As explained by this blog, to prevent rating inflation (top rated players from repeatedly gain rating from low rated players), Codeforces employs two strategies: -

In short, after two rounds of inflation adjustments, the mean of all rating changes should be at least $$$~-11$$$. But, we can clearly see that this is not the case here.

Possible theories

Some pretty interesting relations. The deflation (and the positive bias) correlates really well with the number of "ramping-up" accounts in each contest.

Here, $$$S_{x}$$$ refers to the group of players who have only previously participated in $$$x-1$$$ contests, and $$$\text{old}$$$ refers to the group of players who have exhausted all of their ramp-up offset, or are from the old system. There are two things that stand out here:

• Errors for contests with close to $$$100\%$$$ players from the $$$\text{old}$$$ group are near $$$0$$$.
• Errors for $$$S_{1}$$$ are always close to $$$0$$$.

These 2 very clear trends lead me to believe that this peculiar behavior is related to how ramping-up accounts are handled. Although, I haven't figured out the exact formula yet.

Overtaking higher rated players by… performing worse than them

Another consequence of this is, since my rating calculations are referenced straight from the original blog, the inflation adjustment for all players in a certain contest is uniform. And by looking at the errors, I suspected there might be a case where the following condition mentioned in the original blog is violated:

“If the participant A had worse rating than the participant B before the contest and finished the contest on the worse place then after recalculations the the rating of A can’t be greater than the rating of B”

Turns out, there is a case like that.

Look at vernov.egor and unchainedos. If we work out their true rating, it would’ve been $$$1413 \to 1290$$$ for vernov.egor and $$$1400 \to 1299$$$ for unchainedos. Granted unchainedos didn’t rank strictly lower than vernov.egor, but we can all agree that this kind of overtake should not have happened.

Concluding remarks

The point I’m trying to make is, The official Codeforces blog on their rating system doesn’t seem to fully explain every single step in their calculations, unless there were some updates to the system since 2015 that I missed. If that is the case, feel free to point it out.

Codeforces team, if you see this, please confirm whether there are additional hidden mechanism related to rating inflation or not. I'd love to know either way.

Obviously, it is entirely possible that there are errors in my data pipeline and calculations, which is another reason why I wanted to post this blog. I will probably add a link to my calculations in a future update. But for now, that’s all I wanted to say.