Jarekczek's blog

Rating deflation in 2020 Div. 2
By Jarekczek, history, 3 years ago, In English

I came here after four years break. No matter for reasons, but I decided to participate to see how I can perform today. You know, years go by, covid eats our brains etc. From rating 1741 in 2018 now I dropped to 1597, performing on level of 1100. This was sad to me, but I was prepared too see I declined. However I was surprised how much I dropped. Started investigating a bit.

First I was trying to look at other players. I found one that is really interesting. Actually he's a hero for me, because Codeforces Round #777 (Div. 2) is his 777-th round! Best regards ruban :) If you take a look at his rating chart you'll notice a drop in rating somewhere in 2020. At first sight seems to be a 150 points drop.

There are some mysteries behind Elo rating system. I am not an expert, maybe some of you know it better. But there are ideas that huge growth in players caused by covid may cause rating deflation, like this discussion on a chess forum. (Basically deflation means that rating decreases while skill stays the same.) At Codeforces we also experienced huge income of players in 2019 and 2020. Just taking random contests from 2019 and 2021:

(By the way, such growth is not visible in problem C.)

It's important that I look at cyan segment. Things may look differently in other segments, especially in red group.

So I am writing this to share my thoughts and possibly make us all conscious of the fact. But also to ask:

Can you say more about rating deflation on Codeforces? Is it possible to calculate what number today resembles given number in 2018? Of course I'm interested in a number that reflects actual skills, although I know Elo is in general relative.

Update 2022-03-16: Here you'll find a tampermonkey script that shows who on the rating page has the greatest contests count (on F12 js console). With this you can find users close to you and watch their graphs.

  • Vote: I like it
  • +28
  • Vote: I do not like it

Comments Comments (18)
| Write comment?
»
3 years ago, # |
← Rev. 2   Vote: I like it +114 Vote: I do not like it

Drawing such conclusions based on a one user's rating graph is a very poor approach to me, there are millions of reasons why one particular user might have experienced a steady drop. Also, Codeforces is experiencing significant growth every year. Based on my feeling, if anything there was rather an inflation rather than deflation, because rating inflation was just always an issue here (however I have to admit I base that on observing >=red region). To be honest I would suspect you are drawing conclusions that you want to hear in order to justify your own drop, your methodology is super unreliable — I could probably "prove" absolutely any conclusion if I am allowed to pick data of one user and two problems. But don't worry, being a little rusty after significant break is totally understandable, I would hope you will come back to higher areas if you push for that

→ Reply
  • »
    »
    3 years ago, # ^ |
      Vote: I like it -10 Vote: I do not like it

    You could be right, if my goal was to prove my hypothesis. However I only wanted to assess its correctness. One chart was not my only hint, as I described in the blog. There are other explanations gathering here, and I expect more of them to come. Please comment again after seeing more voices.

    Please be careful, because being red you get upvoted even if you're not right :)

    → Reply
  • »
    »
    3 years ago, # ^ |
    ← Rev. 2   Vote: I like it +15 Vote: I do not like it

    Codeforces is experiencing significant growth every year.

    Do you mean the growth is regular? No, it is not. You can see it for example in data provided by another CF user, pani.natalia in this blog (thanks ssvb for giving this link). The graphs on her blog are showing the relations between gray color and others (nice job). But I was more interested in absolute numbers. And there they are:

    year | total users | gray
2018 |         52K |   8K
2019 |         68K |  12K
2020 |         96K |  57K
2021 |        108K |  74K

    So what do we see? Suddenly total number of users grew by 40%, while the number of grays grew almost 5 times! As you see the usual regularity in CF growth was disturbed in 2020.

    I am on a good way to gather more statistics, but I am not sure which direction would be most interesting. Seems like we already have quite much.

    → Reply
    • »
      »
      »
      3 years ago, # ^ |
        Vote: I like it +4 Vote: I do not like it

      Wow, if this is true, this is very unusual behaviour and indeed a solid argument for your thesis

      → Reply
  • »
    »
    3 years ago, # ^ |
      Vote: I like it +13 Vote: I do not like it

    Thanks for the reply. Because surely more data is needed to support any thesis, I did the following. Searched 4000 users around rating 1525. No one participated in 400 or more contests. But there were several guys who participated in more than 300. Here are their handles:

    All of them have their highest ranking at half of 2020 or much earlier. But we would expect that regular competitors would grow even after 2020. Wouldn't we? :)

    So to everyone who loses their rating and becomes cyan: cheer up :) It doesn't mean you get worse.

    → Reply
»
3 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I think that there's some official blog that says that the total rating of participants of a contest will slightly decrease to prevent inflation. Maybe this isn't enough because of the COVID account surge.

By the way, looking at your contest result, either you had a very unlucky run, or you might need to revise a bit :( So this probably isn't wholly the result of deflation tbh.

→ Reply
»
3 years ago, # |
  Vote: I like it +1 Vote: I do not like it

It would be very relevant to do the following research. Draw the distribution of ratings for the entire existence of codeforces, with an interval of, say, a year. It will be a very interesting picture.

→ Reply
  • »
    »
    3 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Yeah, I wanted to do it, but for me it would take several days probably. Thought it might be better to ask first. But if you say you would like to see that picture, that's encouraging. However for buddies like AaParsa (see Interesting Correlation between Rank and Rating Change), which are familiar with Codeforces API and making graphs — it could be a matter of minutes :)

    After starting to reply in this thread, I noticed who you are :) How wonderful. Could you comment about your graph? What's your opinion about your rating change?

    To make things plain: ruban's opinion is not going to be any proof. Just additional light on the subject.

    → Reply
  • »
    »
    3 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Ruban, probably that's what ssvb mentioned. Did you want this chart by pani.natalia?

    → Reply
    • »
      »
      »
      3 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      Thanks. I didn't notice this message. But it would also be nice to do it for a longer period — from the beginning of the appearance of the codeforces

      → Reply
»
3 years ago, # |
  Vote: I like it +17 Vote: I do not like it

https://codeforces.me/blog/entry/77890

The starting rating has been reduced from 1500 to 1400. In theory, assuming some consistent churn of Codeforces users, all ratings should decrease by 100 after a while.

Another change is that displayed ratings will begin from 0 rather than 1400 or 1500. This has been probably disincentivizing people from creating new accounts after having their ratings dropped from the first few contests.

→ Reply
  • »
    »
    3 years ago, # ^ |
      Vote: I like it +5 Vote: I do not like it

    Thinking more about this I have a conslusion. Please verify it, you seem to be the right person :)

    Elo system is not stable by itself. There are factors that may cause inflation or deflation. It's like keeping soup boiling — it needs attention and adjustments to current situation. And probably CF keepers have an eye on it.

    Deflating and inflating factors need to be kept in balance. Every action aiming to balance the situation may cause to imbalance it in the opposite direction.

    New accounts created only for 6 contests, during which their ranking falls down — that feeds points to other accounts. It causes inflation. In May 2020 Codeforces took measures to discourage such behaviour. This has deflating effect then, which may be affecting us now.

    Sudden huge income of new members is another factor to imbalance the system. That all makes sense.

    → Reply
»
3 years ago, # |
← Rev. 2   Vote: I like it +13 Vote: I do not like it

The same thing happened to me, I started giving contests after 2 and a half years, and my rating has dropped from 1800+ to 1400+, but I don't think the main reason for the drop is rating deflation rather I will say it is because I was out of practice and for some reason, I believe it is now relatively easy to gain more rating as compared to before.

→ Reply
  • »
    »
    3 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    However your example supports my rule: you still miss (on average) many points to the level you had in 2019. Despite taking part in 12 competitions. Wish you luck :)

    → Reply
  • »
    »
    3 years ago, # ^ |
    ← Rev. 4   Vote: I like it +3 Vote: I do not like it

    If you analyze your history, you can see what rating you achieve after solving 3 problems. The simplest way to judge performance is to add 3x delta to your rating after contest. I take 2 examples with smallest delta:

    • round #770, 2022-02-06, performance 1720
    • round #566, 2019-06-19, performance 1873

    Other rows seem to confirm this difference, at first glance.

    One should take more stats, but again: much effort which I cannot handle at the moment. The difference in performance (edit: was rating) after solving 3 problems can be explained only in two ways, I think:

    • problems tend to be easier after 2020
    • there was a deflation in rating
    → Reply
»
3 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

I was just checking some stats, in Codeforces Round 632, participants having a rank of around 1100 (among around 12800 participants) have a performance of 1900+, while if we check it for recent contests, with the same ranking range, the performance is around 1700. I don't know about any formula or maths behind the rating calculation, but it should be roughly the same if we have almost the same number of participants in the contest. I would be happy to know the logic behind it :)

→ Reply