Don't worry, your regularly scheduled useless information programming will resume soon.
Anyway, here is gtest, my attempt at writing a script to help with stresstesting.
Note that I still personally think stresstesting is slightly overrated. It is not a silver bullet, especially when some questions are unstresstestable, like some constructive problems, problems where the solution space grows way faster than even exponentially if you're trying to brute, other cases where your brute forcer has to make some core observation that could potentially be wrong, or if your brute forcer is bugged too and misses the same edge case your fast solution does (Cirno and Number with digit 0 be like). I also think it takes a lot of time so you decide whether you want to write everything up or not. With that being said though, it should be very helpful for debugging WA2s, hopefully.
The format is
gtest -s <path to solution 1> -t <path to solution 2> -g <path to generator> [-c <path to checker>] [-T <trials>]
It supports:
- Custom checker (the default just compares the tokens in the two outputted solutions). Also, the
-targument is optional, since for some constructive problems maybe you just want to code your solution and a checker that confirms your solution is correct. - Solutions, checkers, and generators can be written in Python or C++ (I didn't add support for any other languages sorry lol)
Example:
Solutions
Literally just drop your current solution (.cpp will be compiled automatically; .py will be run with python automatically; executables will be run by themselves) in. And code your brute force solution the same way since they'll be reading the same input.
In the example, we have two solutions for two sum on the same input format (n on one line, target on the next, then the array).
Using checkers
The two solutions' standard outputs are redirected to 1.out and 2.out so go ahead and open those files and see. You should return exit code 0 to give AC and exit code nonzero to give WA. For an example, see token_checker() in gtest.py.
Using generators
For C++ generators, just write your test data to stdout, it will be redirected. Seed your RNG as appropriate.
For Python generators, the script will load the module (using importlib) and extract every function you write that starts with gen_. So you can write multiple generators, like gen_small(), gen_medium(), gen_maximum() and it will run all of them. Again, just write to stdout (such as with print), it will be redirected.
Installation
Go here https://github.com/greatericontop/gtest, clone the repo, and run make install. Alternatively just copy the python script gtest.py and put it somewhere. It has no dependencies other than python STL.
I didn't see a lot of good stresstesting tools publicly available. Hopefully this is useful!
