Stress-testing - Codeforces

№	Пользователь	Рейтинг
1	jiangly	3810
2	Benq	3676
3	Kevin114514	3655
4	maroonrk	3463
5	strapple	3447
6	Um_nik	3387
7	heuristica	3322
8	turmax	3317
9	tourist	3307
10	jiangbowen	3291

№	Пользователь	Вклад
1	Qingyu	156
2	nik_exists	150
2	maspy	150
4	Um_nik	143
5	Errichto	139
6	adamant	137
7	AmShZ	136
8	BledDest	132
9	maroonrk	131
10	qwexd	129

Hello, Codeforces!!

Guys, how do you stress-test your solutions? I know at least three possible ways to do it.

1)Right in your solution file
Write second stress-solution ( definitly correct, but quite slow ), generator, checker, ect. right in your solution file as functions and call functions from main.
I personally use this variant, it is fast and convinient. But there is one problem using this method.. When you have your solution fixed, you have to comment all your stress-code.. I have a trick: just make copy of your file before adding stress-testing functions and add them in your second file, after debugging you will have to just fix jour solution in the first file.

2)Python script
Some people write Python scripts to stress their solutions. You have two programs: solution with bug and stress-solution. So you can just generate tests, lauch your solutions, check answers, etc. from Python script.

3)Bash or maybe Bat scripts
Method is idiologically similar to the 2)Python script, but you write bash/bat scripts.

Which method do you use? Or maybe you have your own techniques, share in comments.

Комментарии (36)

Показать архивные | Написать комментарий?

TsundereMagic

4 года назад, скрыть # |

+95

Too much stress can be detrimental to their performance. Codes need a caring and minimal stress environment to foster their growth.

→ Ответить

efimovpaul

4 года назад, скрыть # ^ |

aahahha, nice joke
But I think you are right, sometimes it is better to try to talk to it, look, search for bug just watching code

NaOH_Frog

+32

Big eye big brain method: Try to come up with nasty inputs and run them.

Yeah!! Of course you are right. It is supposed that you have already tried some tricky cases before you write stress tests.

← Rev. 2 →

Oh, I misread. In China, when participating in the provincial/national Olympiad, we do something that is called '对拍' which is basically writing a brute force solution and a generator, then using a bat script to connect those three and try to find a case that fail on the better solution. You have to be absolutely sure that the brute force solution is correct though. That's probably a mix of (1) and (3) mentioned above.

hm.. Is using Bat script to combine solutions and generator more convenient than just calling them from main program (of course I am talking about cases when all functions are written using one programming language) ?

Well that's probably because we don't use 'void solve()' or something like this, we write a ton of stuff in main(). Maybe I can try calling from main program later...

When I have tried writing all staff in other function and kerping main function minimalized, I found it really convenient

TWNWAKing

+11

I use FastOlympicCoding's stress in sublime text. And it's usage is like the second one. All we need is a correct solution, a wrong solution and a generator file.

hm, interesting, thanks

_Boody

Hi! Firstly I use ubuntu so the bash script may be diffrent if you use windows.I published the codes here.

in Windows it is Bat scripts, they are a bit different, but I think it Will be interesting to try to use your script on Linux, thank you!!

Borgov

Check this

Thanks!!

Penguin07

Here's the bash command that I use for stress testing.

Command

function run_stress_test() {
	# $1 : gen
	# $2 : brute
	# $3 : sol
	# $4 : starting test case id
	make $1
	make $2
	make $3
	for((i = $4; ; ++i)); do
		./$1 $i > debug_in
		./$3 < debug_in > my_out
		./$2 < debug_in > ans_out
		diff -Z my_out ans_out || break
		echo -e "\033[1;32mPassed test:" $i "\033[m"
	done
	echo -ne '\007'
	echo -e "\033[1;45mWA on the following test\033[m"
	echo -e "\033[0;46mInput\033[m"
	cat debug_in
	echo ""
	echo -e "\033[1;41mYour answer\033[m"
	cat my_out
	echo ""
	echo -e "\033[1;42mCorrect answer\033[m"
	cat ans_out
}

do you understand what is happening there? I will be glad if you give me some good resources to learn bash commands, I was quiet surprised when I found out that it is not easy to find full bash-learning courses and videos.

kristevalex

Using https://polygon.codeforces.com/ is also an option

how can I use CF Polygon to stress-test solutions?

Generate tests and ran an invocation, as you would normally do when preparing a problem.

a, yeah, I got it. Do you really use this method during contests?

I don't really use stress testing during contests at all) But I use this method during practice

prozpekt

All I do is write brute force and use this and this

you do not even write scripts connecting these utilities? is stress-testing fast enough using this method?

variety-jones

I've developed my own system for stress testing, which also allows me to do a ton of customizations, and run multiple tests in parallel.

cfstress.com

Although behind the scenes, it'd fall under category 3.

wow, nice job!! is only online version available? online versions are not allowed to use during some olympiads, maybe you have something like "portable version"?

in addition to the previous comment, it asks the number of CF problem, so is it important for problem to exist on CF? you get input format from there? is it possible to make your utility work using just problem's statement text?

No, it's not important for the problem to exist on Codeforces (for example, I also use it for Atcoder Beginner Contests). But that feature is only available on the portable version.

However, don't get the impression that it uses Machine Learning to parse problems and generate test automatically. It doesn't. But I have created various scripts, header files and macros that allow me to feed the data in a fixed format and it generates the test cases off of that. There's definitely a little coding required for each problem, but it's just 5 to 10 minutes of effort per problem. (If it takes more than 15 minutes, it's not worth stress testing IMO).

For example, for the last Edu Round, it took ms around 30 minutes to create generators for all 6 problems.

Yeah, I got it, so 30 minutes for 6 problems is a good result.

BTW: Did you try to use ML somewhere in Competitive Programming?

Dragonado

I use CodePal

it is only for VS code, isn't it?

Yep. You dont have to use all the featues of the extension. For example, I mainly use CodePal for stress testing and Atcoder Library Support.

Nice, thanks!!

KingMace

← Rev. 3 →

I can suggest IFDEF for C++, and if statement check for python. If you write ur code in python, you can create decorator which checks if code is used locally (for example using os.getlogin()), and place it right before ur stress tests and they will be skipped.

import os

def ignore(func):
    if os.getlogin() == "doreshnikov":
        func()
    else:
        pass

@ignore
def stress():
    print(123)

great idea!! thanks!!

nikgaevoy

Why not to always use first option + #ifdef? The only downside is that you should be a bit more careful with random and multitests, but if those problems arise, then you are probably doing something wrong.

yes, I think that it is a good solution to use ifdef some people offered it, I will try this one. Thanks!

Блог пользователя efimovpaul