I wonder how problem setters keep creating fresh and interesting problems for contests? With so many problems out there, how do they avoid repeats and come up with original stuff? And how do they make sure their problems are fair and challenging without being impossible? And how do they check or predict difficult or rating of problem?
If you’re a problemsetter, I’d love to hear about your creative process — where do you start? Thanks...
I too have this question
It's hard. The process also differs for everybody. Usually I just try to come up with some natural setup and try to solve it. Then I might modify it to have a better solution.
You can search if a problem already exists on yuantiji. If it already exists, discard it and come up with something else. Also, sometimes testers notice problem coincidence.
Testers.
Testers.
Thanks for the reply! That cleared up a lot.
Just one more thing I’m curious about — how do testers decide the difficulty/rating of a problem? Since a LGM might find something way easier than an Expert, do they average different opinions or follow some standard method?
Thanks again...
Well, some problems are just easier than others; sometimes everyone agrees. Even an LGM will usually take longer on a 1900 than a 1500.
If people disagree, it comes down to a combination of the majority opinion, the opinion of people at the level of the intended audience, the author's opinion, the coordinator's opinion, etc.
In general, if testing quality is good enough, we can get a reasonable guess to whether a problem is appropriate for a particular position or not.
Really interesting to know how the correct difficulty range is decided when even an LGM takes time to solve it. Makes a lot more sense now...
Thanks reirugan
I also feel the same
I guess after setting enough problems, I am qualified to share my approach. I have set over 40 problems in the past 4 years, including codeforces rounds and some international olympiads. I prefer to stay anonymous, as I am responsible for setting problems for my national olympiad this year.
I wonder how problem setters keep creating fresh and interesting problems for contests?
There are many approaches here. To me, the best problems are the ones that randomly show up in your life, you need to be nerdy enough to recognize that opportunity and write a cp problem based on that. Sometimes you don't have a full problem but a cool observation, and write a problem around it.
The most normal case when you are asked to write problems is to start with no ideas at all.
Some people like to read other past problems to look for inspiration and modify them enough so that the final problem has nothing to do with the original.
I like to split problems into three parts: the object you are working with (ex: an array, a graph were the degrees are no more than 3), the operations and constraints you are applying (ex: you update the array with some kind of query), and what you are looking for in this context. Generally, I first fix the object and the operations and constraints, and then look for something interesting to ask in the context I have set. The more random the object and the context, the more surprising the problem will be (or at least I believe it is highly correlated). This approach usually finds good ad hoc problems fast. Here are some examples (I am not the author of these problems):
https://codeforces.me/problemset/problem/4/A : Object: A single number Operation: Split the number into two even numbers Looking for: Is it possible?
https://codeforces.me/contest/2161/problem/H Object: Two arrays Operation: Swap two elements if the condition is met. Looking for: Final state of the array if the operation is done $$$k$$$ times.
When I don't have any idea of how to start a problem, I look for the most random possible object, set some operations, set what I am looking for, and try solving the problem. Sometimes I am unable to solve it, sometimes it is degenerated, and I usually keep changing details until I find something that is interesting. I like this approach since problems usually end up being creative and ad hoc enough if you start with an interesting object, and also, I believe it is much faster to set problems this way than just thinking about a full problem.
With so many problems out there, how do they avoid repeats and come up with original stuff?
Some years ago, in our national olympiad, we had some issues with this, luckily, now we have https://yuantiji.ac/en/, which works fairly well.
And how do they make sure their problems are fair and challenging without being impossible? And how do they check or predict difficult or rating of problem?
I don't think this is something you do. I simply create a problem, predict its difficulty (there might be several versions with different difficulties), and put it in a list of problems I have. When I need to set problems somewhere, I check this list and decide what to do.
A couple side notes: Your problemstting style might end up quite specific if yoy set every problem with the same approach, regardless of what approach you use, be aware that it is always better to set problems with other people so that the variety of styles is richer.
We live in 2025, it is always nice to ask chatgpt your problems, you might find solutions you were unaware of (chatgpt likes overkilling problems with standard techniques) and some other useful information. It is also quite useful for preparing the problems (can find heuristics that may pass your weak test cases, for example, can give you some advice over the statement, or tell you how to make your cases stronger), it is like having a not-so-intelligent but intelligent enough friend that is always with you giving advice.
Thanks a lot for your amazing reply! It was super thoughtful, and I really appreciate that you even made a separate account just to share your experience. You’ve contributed so much by setting 40+ problems, and your insights about creativity in problem-setting were genuinely inspiring.
I shared your message because I often feel discouraged when I come up with an idea, only to find out it already exists. Since school, I’ve always loved thinking about physics, math, and algorithms, but it happens a lot that my “new” solutions are already published somewhere. Even recently, I independently derived two Multi Criticality Systems(MCS) scheduling algorithms and later learned they were already well-known, TVT. It made me wonder how problem setters keep coming up with fresh ideas.
Your reply really helped. You showed that creativity isn’t about being the first — it’s about observing, experimenting, and shaping ideas in your own style. Genuinely, thank you once again!
Wishing you all the best in your future contests and olympiads — you're doing awesome work!
Maybe it's better to start from a "more random" setup. If you start from a scheduling setup with standard constraints, it's likely that you will find standard solutions that have already been found in the past. If you start from a brand new setup, you can find new ideas more easily.
Thanks, that actually makes a lot of sense. Starting from a completely new setup probably does give a better chance of finding something original. Actually, my CSE professor once gave me a topic to research just for fun, and without reading any literature first, I ended up independently inventing two MCS scheduling algorithms (something like LLF and another similar to EDF-VD).
It’s just funny because this kind of thing keeps happening to me. I get excited after exploring an idea, thinking it’s something new… and then I eventually discover that someone solved it years ago lol. When you mentioned a “more random” setup — it reminded me of another example. Back in 11th grade, when I learned about Heisenberg’s Uncertainty Principle — the idea that you can’t precisely know both position and momentum at the same time — I thought, “If uncertainty is built into nature, maybe I can use that to design a true randomness model.” I started working on that theory, only to later find out that the same approach already exists and is considered a very advanced method for random number generation.
Besides that, I’ve also explored ideas about gravity, time, black holes, and some math theory stuff purely out of curiosity from 5th grade. And again, every time I think I’m onto something unique… I discover the experts had already beaten me to it lol.
But honestly, that’s part of the fun. Even when I “rediscover” something, it just makes me more curious and motivates me to keep exploring.
Some things to add on this:
1) On the old IOI website (ioi.te.lv), there are, if I recall correctly, 3-5 papers on approaches to problemsetting, and if you broaden the scope to "schools of thought for problemsetting" there are quite a handful of them over there.
2) On problemsetting approaches:
Problems created from natural observation are bound to be fairly practical and are much more likely to possess degrees of reusability. Highly likely for cool stuffs to appear as well.
Problems created by "just cooking old problems, duh" might be unoriginal and all, but at least their difficulty are more or less controlled (i.e if an algorithm is used for the original, it is expected that the v2.0 only applies a slightly more difficult/different version of the thing"
.....unless it is UCS (IOI24 P4). Author intended to propose it for some small regional competition and it turned out to be an unfathomable monstrority that took 6 months to solve and scammed some 30 contestants into banging their head into the monster for 3 hours instead. It do be like that sometimes.
Yeah I wish every single one of my problems were impressive and natural problems, but I am problemsetting 20 problems this year and thinking about some random object (as random as I can), and working with it has been proven to be a nice strategy for me to set ad hoc problems.
Tbh proposing literal random bullshit to the IOI is not even that bad of an idea at this point. After Mosaic and the other incidents related to "easy" tasks as of late, proposing literally anything seems to work, as long as it's easy.
At least we have our glorious king FelixMP and the Yoneda brothers hard carrying the "simpler" tasks. Dear god.
Best of Luck Sir!
In one word — experience. By solving a lot of problems you develop a sense for what makes problems interesting, you just see a lot of problems, which allows you not to set the same problems if you have good enough memory.
In two words — experience and collaboration. reirugan put it there as testers, but either we are just using different terminology, or I'm starting to collaborate much earlier. I have teammates, people I have set problems with previously, people whose opinion I value. Once I come up with a new problem for which I think I have an interesting solution, I send it to one of them. Almost always I set a contest with a team of other setters, but even if that's not true, I will still ask some people to test the problems when they are prepared.
Now for your specific questions.
They come up with a lot of problems and discard the ones that are not fresh or interesting, sometimes by discussing the problems with other people.
Basically, they can't. It is impossible to be 100% certain that the problem you are using has not appeared in some local competition 20 years ago which didn't publish their problemset, unless someone who works on the contest has seen that particular problemset. Apparently, there is some tool that searches among the publicly available problems people are posting, but I haven't known about it before this blog. I have seen a significant percentage of good problems: I don't normally miss div1 rounds (either participate or test) on CF and AtCoder, I still participate in Ptz training camp, I don't normally miss ucup contests, I have no life. But it is still not all of them, and probably much less than 50%. Of course, my problems are seen by more people, but even if we assume they have the same wide coverage of problems they have seen (which is probably not true, I'd assume I'm in the highest 0.1% by my problem coverage) and it is independent from mine (which is also not true, most of us participate in the same contests), there still will be misses. Good problemsetters with support of platforms try to alleviate this by having a diverse set of testers that would cover lesser known / regional platforms, but it is still impossible to be completely safe. It is an unwinnable battle, but doing everything possible to achieve 99.9% certainty is much better than just saying "eh we can't win so why bother".
That's a weird question. If there is a correct main solution, it is not impossible. And if you don't have a correct solution, you don't have a problem. If by "impossible" you mean "too hard", the question is "too hard for whom?" I'm assuming div1E problems are "impossible" for you, but that doesn't mean that the problem isn't usable.
I don't know what you mean by "fair" here. Everyone is equally likely to solve it? Well that just can't be done, tourist is more likely than you to solve any problem.
"challenging" — again, challenging for whom? If I can't solve a problem in 5 minutes it is probably somewhat challenging. If none of my coathors/testers can solve a problem in 5 minutes it is very probably challenging.
When I test CF rounds, authors usually ask me to estimate problem ratings; I never do. I don't know what those numbers mean. But I do have a pretty good sense for how div1B should feel by difficulty because I solved hundreds of div1Bs in contests. Most of the time, I can compare two problems that I solved, for which one was harder for me. I have a feeling for how difficult overall the contest should be, how easy should be the easiest problems, how smooth is the difficulty gradient. It all comes from years of experience. It is not an exact science, different people can have different opinions. Setters then need to somehow congregate those opinions and make decisions.
Thanks a lot for the detailed reply — it really means a lot that you took the time to explain everything so clearly. I should have phrased my question better, but your points about experience and collaboration were super insightful and helped me understand what actually goes on behind creating fresh problems.
Also, just to clarify — when I said “impossible,” I meant something like NP-hard problems, in the sense that problem setters also consider the mathematical complexity, not literally unsolvable tasks.
I learned a ton from your answer, and it definitely motivated me to keep improving. Hearing this directly from one of the best in the world honestly feels unreal — thank you so much!
I'm still confused about the terminology.
There are NP-hard problems in contests, they just have reasonable limitations on $$$n$$$. Your problem must have limitations and a main correct solution that fits into TL and ML, all of that is a part of the problem.
Did you mean "do problemsetters come up with problems that they can't solve in polynomial time?" Yes, all the time. Sometimes even exponential solutions are interesting and the problem can be used. Sometimes there are different exponential solutions, some are better than bruteforce. At other times you just can't solve the problem and that's that. If you don't have a solution, you don't have a problem (usable in a contest before you come up with a solution).
Or did you mean "what if the solution the author came up with is incorrect?" Well, nobody is safe from making mistakes; that certainly happened, it happened on CF before, it happened with the problem I was setting (the problem turned out to be solvable in the same limitations, but our solution was incorrect).
I'm basically confused because "NP-hard" doesn't say anything about the usability of the problem in a CP contest. It seems that you are using it as a shorthand for something different, but I don't know what.
Thanks for the clarification — that really helped. I realize now I was mixing up the terminology, and your explanation made the whole thing much clearer. I understand why saying “NP-hard” doesn’t really describe whether a problem is usable in contests — what matters is the existence of a correct solution that fits within the limits.
So yeah, my third question is fully answered now. Really appreciate you taking the time to explain all this — learning directly from someone at your level means a lot! I hope that someday I’ll become capable enough to organize a contest and set my own problems as well.
Why are you talking like ChatGPT?
Hehe, yes, I took some help rephrasing my problem from ChatGPT because of the following two reasons:
English is not my native language, so I used it to remove any grammatical errors.
I got nervous, as anyone would when the world’s best doubt your question, so I used ChatGPT to check whether I was asking a silly question in the community. I also started doubting my understanding of the NP-hard concept, as it has not been taught to us yet since I am just starting my second year of undergraduate studies.
In summary, I used ChatGPT to clearly improve and correct my question. Also, I want to assure you that every single question I asked is the property of my own mind.
Just seeing the word “ChatGPT” triggered a whole chain of thoughts in my head.
Sorry for deviating a little bit from the blog but as you may know that AI is improving extremely fast. It recently solved all 12 ICPC WF problems faster than the human champion team ( i read about it in some article). Unless someone is Um_nik,tourist , Dominater069, or one of those mythical beings who read problems once and instantly solve them, most of us are already behind.
So what happens in like 5-10 years when AI can solve World Finals sets in minutes (just guessing). If platforms can’t fully prevent cheating, cheaters might end up with more success than honest participants, which would basically break the point of CP.
And regarding careers: will companies like Google or Microsoft still rely on DSA and CP skills for hiring, or will that shift once AI can already solve these tasks at a very high level.
What are your predictions about the future of CP, sir. I AM GENUINELY CURIOUS — and sorry if these questions have already been discussed somewhere.
That would depend on what is the point of CP for you. For me it is having fun solving interesting problems. Don't see how the existence of AI that can do it faster can ruin that for me.
I hope not, so people who are doing CP for careers will fuck off.
I hope not, so people who are doing CP for careers will fuck off.
It seems like I’ve accidentally suggested a great idea to eliminate cheaters. Lol.
Oh true, I also do this, I guess I wasn't thinking of that. There are a couple of people who I will send my problem ideas to right after I come up with them, to get their opinion. (I've also been on the other end of this, being asked for my opinion on problem ideas.)
I guess the idea is that you determine problem quality and difficulty by asking people, whether they're testers or otherwise.
Oh nice... I guess with this method, we can combine different people’s opinions to create some really interesting problems.
I think by fair or challenging he means good in their places in the contest, for example some Div2Bs are toooo hard to be Div2B like around 1600 or so and some Div2Cs are very ez like 1100 or so. This can be seen in the last Pinely round B and C (They were awesome problems by the way, but should be swapped). Of course when this swap can happen it is Ok, but sometimes u cannot swap like instead of Div2B then Div2C (but reversed), you find there are 2 Div2Cs or instead of C then D, u find 2 Div2Ds.
It's very fucking hard. I've been trying to author a div4, I started trying to write problems at the beginning of July, it's November now and I still only have ABCDEF.
but high quality problems(above average) take more time, right?
yes
You can get a lot of problem ideas by misreading existing problems, often times the solution to the modified problem is completely different
Hmmm, Great Point!
Agree! I know someone whose entire career in problem setting comes from misreading existing problems.
Edit: Nice
2222rating lol.Salute to their efforts!!
I made the round 1061 (div 2), all problems are my own idea, I can share how I come up with ABCDEF.
Problem A and C: they two are coming together, previous problems are too hard and location A and C are missing. So they based on experience that I take a large piece of meat from refrigerator and cut them into three pieces.
Problem B: A standard problem with n = 100000 and original location is C, coordinator says it will be more suitable to make n = 20 and put in the place of B.
Problem D: an experience of finding the lost phone number, I just count the prefix and found that it save me a lot of time.
Problem E: From the classical problem best time to buy and sell stock, I thought what if two players play a game on it ?
Problem F: Initially the operation is subtract the middle number by 2 then add the front and rear number by 1, and I was not able to solve it. Then I found my current problem can be solved with a greedy approach, and it took much more time to prove it. Initially we thought this is a div2E difficulty, then testers thought it is too hard, so I split it into two parts and place on the F.
Actually I came up with more than 20 problems, and only 7 is approved by coordinator. One problem is around 2600 difficulty and already prepared, even the test cases are made, but get removed at the last few days because we cannot have two div2F problems. If someone is preparing for div1 or div2 but lack a div1D/div2F level problem, you are very welcome to use my problem.
Thanks for sharing your experience hxu10 I AM CURIOUS... how much time did it take you to create all those problems? Coming up with 20 solid questions is seriously impressive.
About the 13 problems that weren’t accepted — was that mainly because they weren’t considered original enough? And as more and more problems get created over time, do you think it becomes harder to come up with ideas that haven’t already been used? With the growing collection of existing problems, the chances of accidentally repeating something seem to go up too. Does that mean creating high-quality contests in the future might become more challenging?
Good question… I think this blog is actually helping to understand and keep the Codeforces community self-sustaining and growing, which is great for the next generation of competitive programmers.