Hi!
CLIS (Competitive Learning Institute Symposium) is one of the programs at ICPC World Finals every year. As you know ICPC World Finals '20 is going to be held in ~2 weeks in Moscow. I had two proposals for the symposium that both accepted. Abstract of one of them goes as follow:
We're not good in our jobs?Peter Norvig: "One thing that I can share that was surprising to me is: being a winner of one of these programming contests was actually a negative factor for performing well on the job.",
What does it mean? We're performing badly on our jobs? If so, why?
In this presentation, I'm going to take a survey of competitive programmers on their job.
I'll do several interviews with old competitive programmers who are doing their job good and ask for their experience for better presentation.
- As a teacher, how to make CP as close as possible to the job?
- What happens in CP and what happens on the job, why they are different?
- What're the main abilities that CP makes them grow and what are the abilities needed on the job?
- Is the salary of competitive programmers more than others after several years?
- Which technologies (e. g. Networking, AI, IoT) are more close to CP?
- What are the most important negative factors of competitive programmers, in sight of their managers on their job?
As here is the biggest competitive programming community, can you please share your experience with me? For example,
- If you were a competitive programmer and you're now working in a company, which abilities you're using from CP?
- If you're a human resource specialist, can you please answer this question: Is the salary of competitive programmers more than others after several years?
- If you're a team lead and have someone in your team that is a competitive programmer, how do you evaluate him? Are you satisfied?
PS. If you're a WF attendee, I'll be very thankful if you attend my presentation.
Update. The presentation is over. You can download the slides and watch the presentation.
I think this is due to the Berkson's Paradox
Note that Peter Norvig said 9 months ago:
ability to write efficient code. Ability to analyze the scenario of a problem at it's best. Ability to thinks the fastest and efficient solution of any real life software problem.
Now Let's suppose, the technology we are using now,after 10 years it get backdated. Then what will you do with that employee who only knows that specific technology?? In this place competitive programmers can learn any technology so fast than others. The companies can use them in any field.You can mould them in any shape.
I really want to hear the experience of top coders (yellow, red) who are actually using their algorithm skill in their job daily. For me, as an average coder working in an average software engineering job, I don't really feel that my CP skill helps me to be better at my job than my other colleagues who never do CP. In fact, I think I should spend more time learning other skills if I want to get promoted.
I agree that pure CP skill has a low correlation with skills needed for an average software engineering job.
Code of real-life products has much different structure. For each new feature, you have to consider how the code may evolve in the future, how well does it combine with other features being implemented, how understandable is it for teammates. Communication skills are very important for this.
Also SE includes some separate important patterns to learn, like "how libraries are structured", "how tests work", "how are we making releases". It also requires deeper knowledge of language.
On the other hand, for some projects, like the one I joined this year (github), there is a requirement that code must be high-performant, and thus optimized as much as possible. And here knowledge of algorithms becomes an important skill, because it helps to understand which tasks are solvable in appropriate time, and which are not.
But when you joined a project already launched in production, it apparently means that all necessary algorithms are already implemented somewhere in deep, and mostly you work on regular tasks again as above :)
Why software engineer is our job?
Great. So let's see what are other options. Teaching? Researcher? University professor?
This activity is not related to jobs. It is definitely not related to software engineering, and it only uses a very old, outdated subset of algorithm research.
I think there is a confusion. It's great to use your CP experience to get good jobs in research, engineering, music, poetry, business, politics... And that proved to be a good strategy. But that doesn't mean you do CP in order to get a specific job. If you want a job in engineering, practice engineering.
This could be different for interviews since people practice them to get good jobs. But they are not competitive programming.
I think, almost all of the competitive programmers leave competitive programming when they are at most ~25. So you need to do something after that. It could be great if you know what could be your next step and when is the best time to switch from competitive programming to the job.
On the other hand, many people need data about competitive programmers and if they are good or not. For example HR specialists.
On the other hand, as a competitive programming teacher, one can teach students such that they become good competitive programmers and at the same time they learn something related to engineering or maybe research. For example, if one teaches them OOP instead of writing ugly codes, they'll become better software engineers, what do you think?
If you are trying to be a CP teacher that wants to also improve research or engineering skills, that's good. But I don't think the software engineering is "our job", and I disagree that the discrepancy of SWE skills and CP skills is a problem. It is a difference, and we have it because we want it to be like that.
I don't think I agree with your reasoning here. Most people leave competitive programming because they no longer participate in ICPC and/or start working full-time. So it's not that you need to do something because you stop competing but the other way around.
It's kinda annoying how people are pushing for the narrative that competitive programming is related to your career. This is also the root cause of much more cheating recently, because apparently some people think that a high rating would land them a good job. Competitive programming is as much a hobby as playing chess, going trekking or playing video games. Do you think about what you can apply to your future jobs when you're playing chess?
At least in the community that I'm inside it (Iran National Olympiad of Informatics (for high schoolers), Iran ICPC community, and also Iranian companies) when you're hiring, a competitive programmer is a lot different from a chess winner.
HR specialists completely believe (and I think it's true) that being a strong competitive programmer means that you can solve the problems in your job much better. There are a bunch of reasons:
Also when it comes to a job that needs designing algorithms the winning point of a competitive programmer gets much bolder. For example, I work in a company as an algorithm designer. So my daily job is making our products more efficient and I think that my CP experience is helping me so much. My daily job is not so much different from competitive programming, really!
It's great that you get to work as an algorithm designer but most of the time you need a very different skillset to be a good software engineer.
For the reasons you mentioned above, to whom you are comparing a competitive programmer with? When I went to university, I know some friends who didn't do competitive programming but spent their time doing open source or side projects instead. I would bet my money that on average, those would end up being a better software engineer compared to competitive programmers.
Competitive programmers write code faster. By not adding comments. By using short non-descriptive variable names. By having many nested loops in a single main/solve function instead of splitting it into several readable functions for each subtask. By always trusting that the input data is correct and within the constraints. And so on. Many of these things are harmful anti-patterns, they introduce hidden bugs and/or accumulate technical debt when used for real software development.
Competitive programmers are exposed to a mostly limited set of approved algorithms and are trained to use them. Similar to how chess players are exposed to a certain set of rules about how they can move their pieces. Will a strong chess player be automatically good at playing other board games with different rules?
Most of competitive programming contests have a short duration. Thinking about a problem for several days is not a typical contest experience and is not directly rewarded by contest rules.
Most of competitive programming contests forbid communication between contestants. And teamwork in such contests is a form of cheating.
what nonsense...you're comparing chess/trekking to literally one of the most useful skills(if not the most) you can have in the digital age?
I'm going to copy-paste my comment from an earlier blog
lots of people say "oh it's just a game, people do it just for fun". For that, I have to say, you've got to be kidding me. Sure there are tons of people who do it just for the thrill aspect, but this isn't like most games where the skills developed can only be used within the game. There are lots of people here who use cp as a way to get strong in algorithmic problem solving, so that they can use those skills to build awesome stuff. I can see lots of people in the future using it as a gym for their brain and I personally use it as such. Calling it "just" another hobby is failing to see the bigger picture or just trying to act cool. If you're creative enough, you'll figure out a way to use the skills developed here. It's only useless if you don't use it.
I don't deny that there are people who do it just for fun. My point is that one shouldn't pretend that fun is the only thing you get from doing cp, because you also gain skills which can be used to build awesome stuff as a side effect. Calling it "just" another hobby doesn't do it justice. At the end you're left with skills which can be used to do great things, especially when we're living in a world powered by computers. I can't think of any other hobby which leaves you with this much power.
I guess I went a bit too far with my analogy. Yes, I don't deny the benefits of doing competitive programming. But at the end of the day, they're just something nice to have. IMHO, using competitive programming as a means to prepare for jobs is far from optimal (unless your job is directly related to competitive programming itself).
Perhaps I'm a bit clueless so can you enlighten me with what great things you can do with skills obtained from competitive programming? You definitely can gain more problem solving skill by doing math problems, so that leaves writing code fast and efficiently to consider. I don't think such skill alone can help me do anything great.
So when I say cp is very useful, I really mean doing programming puzzles is very useful. For me personally, one of the useful skills I've picked up from doing cp is being able to do clever problem abstractions/reductions. Just to give you one example, I've done some graphics related stuff where I was able to reduce some important subroutines to graph theoretic problems which I don't think I could've even imagined doing pre-cp. You can build really cool stuff with programming in general, and cp is a way to improve your programming skills. It has definitely made me a more creative programmer/problem-solver, and I can clearly see the improvement in the quality of ideas that I get.
I beg to differ, 25 is/was just the age of majority. Age of majority will increase as the current majority approach that age (easy to prove this by induction/well-ordering).
Back then on twitch, most of the major gamers/speedrunners were 15+, even the YouTubers, now they're in their early 30s and they're still the majority. CP/internet is a fairly very young space, kind of like how the current ruling democrats/politicians/billionaires were young in the 70s and 80s.
Pioneering any space is fraught with peril: you may just become the "history" newcomers will learn from, Niccolo Machiavelli knew this many years ago in this quote. However, it isn't necessarilly all bad. The reward is also exponentially decreasing for non-pioneers.
And, I think Jaehyun Koo is right. CP skills doesn't necessarily converge near SWE job, massively-headhunted hackers/NSA analysts more likely. On the other hand, I can easily argue that there are 1000 less painful (and more rewarding) ways to add more gray matter than CP.
I agree with you that CP is a sport-like activity, and we should not assume anything about its correlation with the players and players' profession.
You described your opinion on what CP "is NOT". But I would like to hear your opinion, as a LGM, on what CP "is"?
I think most of people will say it is mainly for fun. I agree on that because I am doing it for fun too. However, I think CP might be more than just a fun hobby for people like you who invested so much time on it and become so good at it.
Hi!
Thank you for raising this question. I had discussed it with quite a lot of my friends in different time, and procrastinated to write something publicly, and now I have a good reason to do it.
For those who don't know who I am. As competitive programmer I'm 2014-th ICPC World Champion, IOI gold medalist, participant of more then ten CodeJams, TCO, Hacercup, Hashcode finals total. As software engineer I worked for VK as databases developer for 6,5 years, and now working on Kotlin project in JetBrains. (Just to be clear, everything written below is my personal opinion, and does not related to any organisation). So probably I can claim I understand at least something in both.
The main problem in discussions like this is statement "being good software engineer" makes totally no sense. I mean software engineering is very heterogeneous, and different parts require different skills and different mindset.
For example, do you need an engineer who works very fast, but with somewhat unmaintainable result, or the one who would create an ideal project, following all best practices, but this would took very long time? The answer is, if you are big enough, you need both. Because first one would be grate in prototyping and experimenting, which is important part of any big project, while the second one would be good in long-term support of experiments with good results. And having some people evenly distributed on this spectrum, would be even better.
In fact, different people are good in different tasks. And competitive programmers aren't somewhat unique at this question. The problem is, they often differ more, than by average dispersion. This requires some extra mentoring work for integrating in work process, especially on first jobs. But I really thinks result would worth this extra work in most cases.
Let's start with common points about what could be wrong with competitive programmer in job.
Stuff like "using one-letter non-meaningful variable", "bad code style", etc. This is kinda valid point, because habits can be hard to deal with it in the first time. But my experience shows, that this is fixed quite fast, although can require explanations of things which are obvious to people who started in other way. Because, when person starts to read much more code, than they writes, they understand why it's important. This can be wrong for some people, but that's not about competitive programming experience, that's about being too pigheaded, which can happen with anyone, and always a problem.
Overestimating importance of performance. This is again quite a valid point. But is it really a problem? Everyone have it's own priorities. And it's normal. When it's considered well while distributing tasks, that could be even converted to a pros, not a cons, because of fresh and different view on some issues. The problem is it would require some extra work on explaining why sometimes it could be out-of-priority or help with finding balance between performance and maintainability. But again, do you now anyone who never started to think something strange not-understood-by-anyone-else is very important?
Unmaintainable written code. This is often mentioned, but I think it's a big lie. The problem is, writing code, which would be easy to change in future in unknown conditions, is just very hard. Not only for competitive programmers, it's just hard. That's not about knowing patterns, good-code-style, even not about knowing language. It's about low code connectivity, good model of data ownership, predictability of result of code changes, It's really about experience, not knowledge. There only two reasons you can do it well, you are either lucky and choose good solution by random, or you have done it wrong hundreds of times before. So I don't really thinks, that competitive programmers differ in that point from any other beginner engineer. They both very bad in that, and that's normal.
Soft skills. This I think one of biggest problem. That is, top-level competitive programmer can think that they is most clever here, and don't need any advice, because he is so cool, and don't want to hear others, who are more experienced in other parts of engineering work. I think saying everyone will behave like this is too rough generalization. Finding compromises, hearing each other and understanding that other people, how are worse than you in some points, can better in other, is complex. People need time to study it, when they start work in team. But is it really connected with competitive programming, not with any big success in the past?
On the other hand, I think mainstream arguments of opposite side are quite strange too.
Knowing complex algorithms and data structures? Are you serious? On the whole work in VK most useful algorithm was binary search, while most useful data structure was doubly-connected-list. You don't need spending 20 hours a week for 5 years to be good in using them. Some graph algorithms are used in compiler, but again, most complex of them is finding SSC. The reality is, competitive programming algorithms are to deep to be used in general case, while too superficial to be used in special cases. That doesn't mean they are bad. Ideas you studied while understanding them are very good. Speed when you will understand some more complex special algorithms in future would be better. Understanding of limitations what can be principally done, and what can't would be better. That's useful. While the knowledge of exact algorithms in general is not. As always in studding.
Can write very performance code. Are you serious? Performance is not about "ha-ha I added three pragmas I don't understand and got ok". It's not about replacing vector by static array, and adding inline keyword on every function. High-level performance is mostly about searching of bottlenecks, with are almost always obvious in competitive programming. Low level is about accurate measuring, using profilers, reading assembly. Do you remember when you did this last during the contest? Contest experience could be useful in performance, because of a lot of known ideas, and some intuition what to test. But there are a lot of important things you need to study to do really well in it, which are totally out of scope of the competitive programming.
But I think there are several things, which are real strength of people with experience in competitive programming, which are not often said.
First of all, on top level you inevitably learn how to write locally not-error-prone code. I mean, your local code structure becomes such, that you have no ways to make small bugs. Because it's too long to fix them on contest. For example thinking in such way, that minimize number of corner cases, strange +-1, writing code such way, that errors catch the eye. Also, this includes much better then average skill of finding bugs and uncovered corner cases by reading, which is very useful during code reviews.
Second thing, that I want to mention, is better average debugging skills in case of low information. Debugging is hard. And it's often takes even more time, than writing code. And you are very lucky, if you have a good reproducer, which you can run many times, with debugger in IDE. Often you have just small amount logs or stacktrace, or just nothing like "Hey, I'm getting 502". Doesn't looks like anything well known? It's exactly the case when you get your wrong answer. Even a bit easier, you mostly have at least approximate information what went wrong.
And the third thing, which I already said a lot of times. Different mindset. Competitive programmers have a big another point of view on project. And this could be very useful in lots of cases, if find good balance. But finding it is hard. And I think it's one of main reasons of miscommunications and conflicts, Which leads to many experience like "I worked with competitive programmer, they wrote tones of strange code, no-one-else understand and didn't want to hear anyone, probably they all are not suitable for engineering".
So, summarizing my ideas. Competitive programmers are not bad engineers, are not good engineers, they are a bit different engineers, than community are used to. This definitely bring some problems, especially when mentor have no experience how to work with such people. But the weak sides can be mitigated with time and mentor's work, probably more work, then for others. While strong sides will not go away. So the much more correct question is not "Are competitive programmers good engineers?", but "For which tasks competitive programmers are good engineers, and how much of such tasks industry needs?".
P.S. Sorry for too many text, I hope it would be useful for someone.
I really enjoyed your very well written comment and I agree almost completely, it was certainly very useful!
The only little detail is that I would dare say that "Overestimating importance of performance.[...]" is actually a weakness BOTH in competitive programming and most industry jobs. When I coach teams and students, one of the critical instructions I give them is to "code the simplest, easiest, less-error-prone possible solution that is just fast enough".
Writing a complicated N lg N solution instead of say a trivial N^3 when problem has N <= 100 and normal time limits is a mistake that almost any strong competitive programmer will avoid. We actually made such a simple mistake in a hurry during an ICPC World Finals with my team: we correctly identified the easiest problem, and thinking it was a very standard "just convex hull and then pick largest", we quickly decided to copy convex-hull from our reference notebook, and then coded the little additional code. We got Accepted in like 15 minutes, but many other teams solved it much faster than us like just 5 minutes, because they just wrote "for for for" noticing N <= 100 allowed such a short simple solution, no need to copy, coordinate, think more, etc. Another classic example of this was Topcoder: when I started, I usually coded BFS or Dijkstra for some easy 250 problems, but then learnt by reading other's code that the uppder bound was N <= 50 in almost all problems, it was actually safer and much quicker to just always write Floyd-Warshall, even for the case of single-path, all edges same weight.
I think that this very strong quick "intuition" of what things will definitely be "really slow" and what things are probably going to be just fast enough given the current "goal", is a very strong advantage of competitive programmer's to avoid wasting time on premature optimizations, and is one of the things from competitive programming that I think has helped me most in my daily job.
Finally, my impression is that game programming in particular is an area where much more competitive programming skills transfer to the real job, compared to typical "business logic" in other domains.
Thanks for the comment!
It was like writing my own story better than me :) I was lucky with mentor who helped me a lot to avoid all these barriers and who was enough patient to explain me why something is not good.
I hope, one day I will have enough technical knowledge to learn newcomers in the industry.
Regarding the job opportunities for competitive programmer, I had hard time for finding the good opportunity in the industry in last few months (Luckily for me I end up in Google). I will share the opportunities which were interesting for me (and hope for many other competitive programmers):
DeepMind was like my dream (it was combination of Research + Google :)) , but for me it looks impossible to be there without some good research papers, or winning really big competitions (which means you are in top 10-20 competitive programmers). Personally, I find out really hard to get interview for any research role with Bachelor's and "I am good in the contests".
Trading companies (low-latency engineering) look like good fit for many competitive programmers, but I have find hard to join there if you need working visa and you do not have big experiences in other areas like networking/databases/OS...
So many thanks to you for writing this. This would help me a lot, thank you!
I work in a company (Mahsan Co.) where we really use CP algorithms, although maybe we are an exception.
I think we're not always replacing vectors with static arrays for better performance. Sometimes we find that "oh, the given pairs are ordered and can be found using binary search instead of linear search". I think many such cases happen in the industry and CP experience can help in these cases. For example, when I joined the company I'm currently working in it, I'm almost always reading others' codes and making performance optimizations.
Another note is, I think we can be really good at finding bottlenecks and making performance optimizations like this.
I learned how to use C++, debug it, build it, solely by solving codeforces problems, I never took online courses or anything like that. As for my job, it's not something unusual to face stuff like NTT or gaussian elimination or some bit tricks, so...
Wait, could you describe what is your job? I'm curious what's a job which uses NTT etc
The presentation is over. You can download the slides and watch the presentation.
I know a contest winner who is a doctor right now. If he is not good with the job, many people will all go toothless.