Puff_Puff_Pass's blog

Codeforces Round #650 Div 3 Unrated?
By Puff_Puff_Pass, 4 years ago, In English

I submitted the solution to 1367D - Task On The Board during the contest (**Pretest Passed**). Later on, someone hacked it, that's fine.

But after analyzing a bit I found my code wasn't even working correctly for TestCase 1, then how did it pass the pretests.

Here is my submission 83979354, kindly check my output on test1 of testcase1.

Input
Output

I don't have 2 'b's. This shouldn't have passed the checker was wrong itself during the contest.

It must have affected a lot of people as well.

Sorry to say but will this make the contest unrated?

UPD1: As expected my solution is hacked on Testcase 1 itself. The same checker first passes the solution, and later fails the solution on the same Testcase, does that make any sense?

  • Vote: I like it
  • +98
  • Vote: I do not like it

Comments Comments (72)
| Write comment?
»
4 years ago, # |
  Vote: I like it +1 Vote: I do not like it

True! that ruined the contest completely, it shouldn't happen.

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Is div 3 rated or unrated???

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +2 Vote: I do not like it

      Ratings have been updated so it is rated! :-)

      → Reply
      • »
        »
        »
        »
        4 years ago, # ^ |
          Vote: I like it +12 Vote: I do not like it

        Sometimes rounds will be rated but then the rating changes will be rolled back later if it's decided that it isn't fair for that contest to stay rated.

        → Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Wrong checker implies contest is unrated in most of the times, as far as i know. Obviously, Many people were affected by this

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    have there been such cases before? (where checker had some issues during the contest so contest was made unrated)

    → Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Yes ,it also effected me and later somebody hacked it with same testcase.

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it -12 Vote: I do not like it

    But why one will submit if his code is not working for sample input. So its fault from both the side

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +7 Vote: I do not like it

      Doesn't make any sense. That what checker do!

      Tell you when your code isn't correct.

      → Reply
      • »
        »
        »
        »
        4 years ago, # ^ |
          Vote: I like it -7 Vote: I do not like it

        But what was the point of submitting a code that you know will fail. Maybe you like wrong answer on test1?

        → Reply
        • »
          »
          »
          »
          »
          4 years ago, # ^ |
            Vote: I like it +10 Vote: I do not like it

          I submit ( sometimes ) my problems for div2A , or even div3A , div3B without even compiling my code locally ( to save penulty minutes ) when I am very confident about my code.

          Maybe ; just maybe someone might have done the same for div3D as well ?!

          → Reply
        • »
          »
          »
          »
          »
          4 years ago, # ^ |
            Vote: I like it +12 Vote: I do not like it

          There can be a lot of instances as to why that happens. One of'em which has happened to me, a lot of times, is when I submit a solution that passes the test case 1, but fails the pretests. So, when I go back to see the bug in my code, I often think up test cases on my own to check exactly where my code is failing. When I correct my code to fit the test case(the one that I thought of), I often submit the solution without checking for the official test case 1, assuming it might be right.Even if we ignore the tiny of chance of this happening to somebody, don't you think it's up to the person to submit a code without checking it(via official test cases)? Because, should that solution fail, (s)he and only (s)he will suffer the consequences of it.

          → Reply
          • »
            »
            »
            »
            »
            »
            4 years ago, # ^ |
              Vote: I like it +25 Vote: I do not like it

            Oh com'on you have only solved A and B. It is quite obvious why you are taking this stand

            → Reply
            • »
              »
              »
              »
              »
              »
              »
              4 years ago, # ^ |
                Vote: I like it +8 Vote: I do not like it

              I guess, you're right about that! But, having submitted just two solutions doesn't have anything to do with what I wrote. :)

              → Reply
        • »
          »
          »
          »
          »
          4 years ago, # ^ |
            Vote: I like it +1 Vote: I do not like it

          Wrong checker doesn't mean anything then?

          There can be cases, where sample testcase was running fine, but other pretests got accepted even the code was generating the wrong answer. Then?

          → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +16 Vote: I do not like it

      In the question they mentioned that any possible answer is accepted ,when i was getting different answer i checked if my answer was giving same array but i did not check it with the given string.

      → Reply
»
4 years ago, # |
  Vote: I like it +4 Vote: I do not like it

So are you guys saying the whole contest is unrated or that particular question alone for rating calculation?

→ Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 2   Vote: I like it 0 Vote: I do not like it

    You can't counter the effect caused by this by just discarding one problem. The contest should be unrated.

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +39 Vote: I do not like it

      In status I see 170 hacked solutions (out of them 111 come from official participants). Total number of accepted solutions for D were: 1765, 840 of which came from official participants. Seems to have had significant impact so I believie this should be unrated

      → Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I also faced the same issue. I didn't submitted because i could clearly see that it failed to pass even the first testcase on my own compiler!

→ Reply
»
4 years ago, # |
  Vote: I like it +2 Vote: I do not like it

This is a major issue if hidden test cases fail despite an AC verdict. If the public, sample test cases fail on your own local machine, then shouldn't the responsibility fall on the candidate to ensure correctness?

→ Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 2   Vote: I like it 0 Vote: I do not like it

    Wrong checker doesn't mean anything then?

    There are cases, where sample testcase was running fine, but other pretests got accepted even the code was generating the wrong answer. Then?

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +5 Vote: I do not like it

      This is a major issue if hidden test cases fail despite an AC verdict

      How many people were affected by this? 1000 people solved D correctly despite an incorrect checker. Moreover, the number of people affected by this issue seems small, less than 100 or so. Making the contest rated is unfair for them, but making the contest unrated is unfair to the other 11900 who participated.

      → Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Yes even there can be the situation that no such test cases are there in the pretests then I need to check for every possibility on my local machine I can't completely rely on the pretests verdict.

    → Reply
»
4 years ago, # |
  Vote: I like it -22 Vote: I do not like it

I believe round should still be rated....
If you know that your code is not passing given example test case and you still submitted....??

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Maybe, just a few minutes were left for the contest to finish, so he submitted without checking samples. But still you can't deny as to why his code was accepted.

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      In that case even if it has given wrong answer verdict then he/she can't debug it within the time.

      → Reply
      • »
        »
        »
        »
        4 years ago, # ^ |
          Vote: I like it +21 Vote: I do not like it

        If the answer is not unique you might not realize your answer is wrong on sample if you forget one constraint/assume "it looks reasonable, I don't want waste time to check precisely if it is correct, I'll just submit and maybe get WA on test 1 which doesn't give me penalty time"/simply mess something up while checking all requirements.

        → Reply
        • »
          »
          »
          »
          »
          4 years ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          I get your point but the thing is if there were no such tests in the pretests then it would have been a case of weak pretests which is very frequent in codeforces contests.

          → Reply
»
4 years ago, # |
  Vote: I like it +3 Vote: I do not like it

OH!I think it's unfair whether it's rated or not.

→ Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

It passed the checker I agree but then you got hacked which you therefore deserved so the net effect is 0 and so this round should NOT be unrated. Your code was wrong in the first place and so you got hacked as simple as that.

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +6 Vote: I do not like it

    Yes I agree with your words but if you would have got wrong verdict during contest then you would have rechecked and most likely submit correct one. But once you got ac you moved to next question and later it was hacked with sample input itself

    → Reply
»
4 years ago, # |
  Vote: I like it +6 Vote: I do not like it

Just curious, it seems your solution is hacked by exactly TestCase 1. If the checker is wrong, shouldn't the hacking attempt fail too?

→ Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

If someone submitted the wrong solution then he/she was not thinking of the case where it fails . Even if checker would have been correct but there were no such testcases then it would give OK response initially and later on solutions will be hacked but now since checker was not correct at that time making it a reason for hacked submission is not fair.

→ Reply
»
4 years ago, # |
  Vote: I like it +1 Vote: I do not like it

I mean, yes wrong solutions passed, but they later got apprehended in the hacking phase. Is that not the purpose of the hacking phase?

If anyhow a wrong solution is submitted, doesn't it make sense that it got hacked and then all is back to normal? If all solutions that cleared the pretests were complete correct by rule, a hacking phase would not make any sense imo

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +13 Vote: I do not like it

    There can be other testcases where your code may fail, that's the purpose of hacking phase.

    Doesn't make sense when your solution hacked on testcase 1 itself.

    The same checker first pass the solution, later on fails the solution on same testcase, does that make sense?

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      True, makes sense. This situation is too much in the grey of both the opinions

      → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      How is this any different from cases where pretests are weak?

      → Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +5 Vote: I do not like it

    The case here is that the solution was failing on pretests but got pretests passed verdict and then later got hacked with the pretest itself.

    If you say that's the point of hacking phase then i guess there would be no sense in having strong pretests itself or having pretests for that matter. but its like said here since number of people for which its unfair is small the round should/could still be rated. We just have to wait for an official announcement on this matter.

    → Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

When contest was over, I saw many solutions where they were not even checking if they have count of character which needed. I think there are many such sol.

→ Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 2   Vote: I like it 0 Vote: I do not like it

    is contest 650 unrated? because no change in ratings

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +1 Vote: I do not like it

      I think System Testing is not completed , so when system testing will be completed then after that ratings will be updated.

      → Reply
»
4 years ago, # |
  Vote: I like it +14 Vote: I do not like it

I think the round should be rated. This case is the same as when pretests are weak and your solution passes. Since the hacks work, the checker is actually fixed

→ Reply
»
4 years ago, # |
  Vote: I like it +6 Vote: I do not like it

I don't know how hard it's to do this, but making the round unrated for those who were wrongly judged in D makes more sense since it'll be equally unfair to make a mass decision that applies for all.

→ Reply
»
4 years ago, # |
  Vote: I like it +28 Vote: I do not like it

It seems I'll exclude all affected participants from the rating. I don't see any reason to exclude any who didn't catch this mistake during the round.

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +1 Vote: I do not like it

    Thanks, Mike i m gonna be Expert ...

    → Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +1 Vote: I do not like it

    You are the best Mike!

    → Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +5 Vote: I do not like it

    There are people who were affected by the checker in problem D but are still expecting a positive rating change. Will the round be unrated for them too?

    → Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 3   Vote: I like it 0 Vote: I do not like it

    I spent time and solved D correctly. Now it's removed. If I had not attempted D first, I would fair chances that I would have solved E. Just a suggestion that you should have removed only those who had a problem. Instead of removing it for everyone.

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +1 Vote: I do not like it

      Please read his comment in the contest anouncement blog, the question is removed for now, it will be back.

      → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      Yes, Exactly.

      → Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 2   Vote: I like it +14 Vote: I do not like it

    I think it'd be nice to have some guidelines on what happen when we find an error in problem/checker after the contest.

    → Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 2   Vote: I like it 0 Vote: I do not like it

    I was affected by the checker in problem D, but don't want to be excluded from the rating changes. Expecting a big chunk (yum-yum) of rating increment (under 80 official rank). It would be really nice if I can have my chunk (Please) !

    → Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    I have correct solution for D, but my first solutions failed. You have excluded me from rating list... Please, can you return me back?

    → Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    I found this error after submitting D, so I submitted another correct code. If it is not rated, is it unfair?

    → Reply
»
4 years ago, # |
  Vote: I like it +3 Vote: I do not like it

This contest was already unrated for u... XD

→ Reply
»
4 years ago, # |
  Vote: I like it +3 Vote: I do not like it

Can someone tell me whether people affected by the D question still have a chance to get rating？

→ Reply