I have noticed something interesting while practising on Codeforces.

Sometimes I open a 1600-rated problem and solve it in 10 minutes. On another day, I can spend an hour on a 1400-rated problem and still have no idea how to approach it.

I am sure many of you have experienced something similar.

It made me curious about one question:

How accurately does a problem's Codeforces rating represent its actual difficulty?

We generally assume that a 1700-rated problem is harder than a 1500-rated problem. That assumption is useful when choosing practice problems, but I don't think the whole story is that simple.

Two problems with the same rating can feel completely different depending on the topic, the observations required and our individual strengths.

So I decided to explore this question from a data perspective.

Instead of relying only on personal experiences, let's look at what the numbers can tell us.

1. What exactly are we trying to find?

I want to explore five questions:

1. Do higher-rated problems generally have fewer successful solvers?
2. How different can two problems with the same rating be?
3. Does the problem topic have any relationship with its solve count?
4. Can we identify problems whose solve counts look unusual for their rating?
5. Is problem rating enough to estimate difficulty for an individual programmer?

These questions might sound simple, but the answers are not necessarily straightforward.

2. Where will the data come from?

Fortunately, we don't need to collect everything manually.

Codeforces provides a public API:

https://codeforces.me/api/problemset.problems

It returns information about problems and their statistics.

For this experiment, we need:

We can combine the problem information and statistics using contestId and index.

One important clarification: solvedCount is a lifetime count of successful solvers, not the probability that a particular person can solve the problem.

That distinction will matter later.

3. Let's collect the data

I used Python for the analysis because it makes collecting and examining a large dataset relatively easy.

First, install these libraries:

pip install requests pandas matplotlib scipy numpy

Here is the data collection code:

import requests
import pandas as pd

URL = "https://codeforces.me/api/problemset.problems"

response = requests.get(URL, timeout=30)
response.raise_for_status()

payload = response.json()

if payload.get("status") != "OK":
    raise RuntimeError(
        payload.get("comment", "Codeforces API error")
    )

result = payload["result"]

problems = result["problems"]
statistics = result["problemStatistics"]

# Match problem statistics using contestId and index
solved_map = {
    (item.get("contestId"), item.get("index")):
        item.get("solvedCount")
    for item in statistics
}

rows = []

for problem in problems:

    contest_id = problem.get("contestId")
    index = problem.get("index")
    rating = problem.get("rating")

    solved = solved_map.get((contest_id, index))

    if (
        contest_id is None
        or index is None
        or rating is None
        or solved is None
    ):
        continue

    rows.append({
        "contestId": contest_id,
        "index": index,
        "name": problem.get("name", ""),
        "rating": int(rating),
        "solved": int(solved),
        "tags": problem.get("tags", [])
    })

df = pd.DataFrame(rows)

if df.empty:
    raise RuntimeError("No matching rated problems found.")

df = df[
    (df["rating"] > 0) &
    (df["solved"] >= 0)
].copy()

print("Total problems:", len(df))
print(df.head())

df.to_csv("codeforces_problems.csv", index=False)

This gives us a CSV file containing the rated problems and their successful solver counts.

We can now start analysing it.

4. First experiment: Does rating correlate with solve count?

My first guess is that higher-rated problems will generally have fewer successful solvers.

But let's not assume that. Let's calculate the correlation.

For this experiment, I am using Spearman's rank correlation coefficient.

Why Spearman?

Because the relationship between rating and solve count may not be linear. We are more interested in whether the two variables tend to move in opposite directions.

The coefficient ranges from -1 to +1.

• -1 means a perfect negative monotonic relationship.
• 0 means no monotonic relationship.
• +1 means a perfect positive monotonic relationship.

Our hypotheses are:

• Null hypothesis: There is no monotonic association between rating and solve count.
• Alternative hypothesis: There is a negative monotonic association.

Let's calculate it:

from scipy.stats import spearmanr

rho, p_value = spearmanr(
    df["rating"],
    df["solved"]
)

print(f"Spearman correlation: {rho:.4f}")
print(f"Two-sided p-value: {p_value:.6g}")

I am using a two-sided p-value here, so the statistical test checks for either positive or negative association.

The important thing is not just whether the p-value is small. We should also look at the actual correlation coefficient.

A statistically significant relationship does not necessarily mean a strong relationship.

5. Let's visualise what is happening

Numbers are useful, but a graph can sometimes reveal things that a summary statistic misses.

Since solve counts can vary enormously, I will use log(1 + solvedCount) for the vertical axis.

import matplotlib.pyplot as plt
import numpy as np

df["log_solved"] = np.log1p(df["solved"])

plt.figure(figsize=(11, 6))

plt.scatter(
    df["rating"],
    df["log_solved"],
    alpha=0.25,
    s=12
)

plt.xlabel("Codeforces Problem Rating")
plt.ylabel("log(1 + Successful Solvers)")

plt.title(
    "Problem Rating vs Successful Solvers"
)

plt.grid(alpha=0.25)
plt.tight_layout()

plt.savefig("rating_vs_solvers.png", dpi=180)
plt.show()

While looking at this graph, I would pay attention to three things:

• Is there a general downward trend?
• How much variation exists among problems with the same rating?
• Are there some high-rated problems with surprisingly high solve counts?

I find the second question particularly interesting.

Even if the overall trend is negative, individual problems might behave very differently.

6. Same rating, different experience?

Let's take a closer look at problems with the same rating.

Imagine two 1600-rated problems.

One has 15,000 successful solvers, while another has only 1,500.

Does that automatically mean the first problem is easier?

I don't think we can conclude that.

But it is worth investigating why the difference exists.

Let's calculate some statistics for each rating group:

df["rating_group"] = (
    (df["rating"] // 100) * 100
)

summary = df.groupby("rating_group").agg(
    problem_count=("rating", "size"),
    median_solved=("solved", "median"),
    mean_solved=("solved", "mean"),
    min_solved=("solved", "min"),
    max_solved=("solved", "max")
)

print(summary.to_string())

I am particularly interested in the median.

Consider these hypothetical solve counts:

100, 120, 150, 180, 10000.

The mean is 2110, but the median is 150.

One extremely popular problem changes the mean considerably.

That is why I think median solve count is a useful starting point for comparing groups of problems.

We can also visualise the distribution:

counts = df["rating"].value_counts()

selected_ratings = sorted(
    counts[counts >= 5].index
)

if selected_ratings:

    box_data = [
        df.loc[
            df["rating"] == rating,
            "log_solved"
        ].to_numpy()
        for rating in selected_ratings
    ]

    plt.figure(figsize=(12, 6))

    plt.boxplot(
        box_data,
        showfliers=False
    )

    plt.xticks(
        range(1, len(selected_ratings) + 1),
        selected_ratings
    )

    plt.xlabel("Problem Rating")
    plt.ylabel("log(1 + Successful Solvers)")

    plt.title(
        "Solve Count Distribution by Problem Rating"
    )

    plt.grid(axis="y", alpha=0.25)
    plt.tight_layout()

    plt.savefig("solve_count_by_rating.png", dpi=180)
    plt.show()

else:
    print("Not enough problems for the boxplot.")

I have included a minimum of five problems per rating so that we don't draw conclusions from extremely small groups.

7. Does the topic make a difference?

This is another question I would love to investigate.

Think about these topics:

• Implementation
• Greedy
• Mathematics
• Dynamic programming
• Graphs
• Binary search
• Data structures
• Constructive algorithms

A programmer who regularly solves graph problems might find a graph-based 1800 problem more approachable than a 1600 problem requiring an unfamiliar mathematical observation.

Let's examine the tags:

tag_rows = []

for _, row in df.iterrows():

    for tag in row["tags"]:

        tag_rows.append({
            "tag": tag,
            "rating": row["rating"],
            "solved": row["solved"]
        })

tag_df = pd.DataFrame(tag_rows)

tag_summary = tag_df.groupby("tag").agg(
    problem_count=("solved", "count"),
    median_rating=("rating", "median"),
    median_solved=("solved", "median")
)

tag_summary = tag_summary.sort_values(
    "problem_count",
    ascending=False
)

print(tag_summary.head(20).to_string())

This gives us a starting point for comparing topics.

However, I would be careful about drawing conclusions here.

For example, if graph problems have fewer successful solvers, it might simply be because the graph problems in our dataset have higher ratings.

We need to compare similar rating groups before claiming that one topic is harder than another.

8. Something we should not ignore: Problem age

There is a major limitation in using lifetime successful solver counts.

An older problem has had more time to accumulate successful submissions than a problem published last month.

Similarly, a problem from a very popular contest may receive more attention.

Other factors can influence the numbers too:

• Contest participation.
• Editorial availability.
• Popularity of the problem.
• Familiarity with the topic.
• Whether the problem appears in popular practice lists.

So, a problem with 20,000 successful solvers is not necessarily easier than one with 2,000.

This is an important distinction.

We are measuring accumulated successful solvers, not the probability of solving a problem under identical conditions.

9. What would a better difficulty model look like?

If I wanted to take this research further, I would try to account for additional variables.

For example, we could define:

• (R_i): Problem rating.
• (A_i): Problem age.
• (C_i): Contest popularity.
• (T_i): Topic information.
• (S_i): Successful solver count.

One possible model would be:

[ \log(1+S_i)= \beta_0+\beta_1R_i+ \beta_2A_i+\beta_3C_i+ \beta_4T_i+\epsilon_i ]

This is a simplified regression framework.

It would help us investigate whether rating still explains differences in solve counts after accounting for other factors.

Another approach would be to compare problems from the same contest.

That would reduce some of the differences caused by contest popularity and problem age.

An even more interesting direction would be to study contest-specific solve rates using contest standings.

That would be closer to measuring how participants actually perform when solving problems under similar conditions.

10. My takeaway

I don't think Codeforces ratings are useless or inaccurate just because two problems with the same rating can feel completely different.

In fact, I think ratings are extremely useful.

They give us a common reference point for choosing problems and measuring progress.

But we should not treat them as perfect measurements of individual difficulty.

A programmer comfortable with graph algorithms might find a graph problem relatively easy, while struggling with a lower-rated mathematical observation problem.

This is why I prefer combining rating-based practice with topic-based practice.

Rather than focusing only on whether I can solve 1800-rated problems, I would also like to understand which topics I consistently struggle with and why.

A problem's rating is a useful estimate, but it cannot capture every programmer's experience.

And perhaps that is what makes competitive programming interesting in the first place.

11. What do you think?

I would genuinely like to hear other people's experiences.

1. Have you ever found a 1800-rated problem easier than a 1500-rated problem?
2. Do you think problem ratings accurately represent difficulty?
3. Should Codeforces problem recommendations consider topic-specific strengths?
4. What other variables would you include in this experiment?
5. Would you use a personalised difficulty estimate based on your own submission history?

I am especially interested in hearing from people across different rating ranges.

If anyone has already explored this topic using data, I would love to see their approach.

Maybe we can find a better way to understand problem difficulty together.