I wrote a tool 'Datagen' using C++. It's a command-line tool, targeted at CPers.

Using a single-line command, you can generate testcase in batches.

For example, if you want to generate #11-#20 testcases for your problem, with a time limit of 1.5s, and compare the running time with brute force solution as well:

datagen <yourproblem> 11..20 -h -t 1500

You only need to provide: (just use standard input/output. do not include 'freopen's) - <yourproblem>_gen.exe or <yourproblem>_gen.cpp - <yourproblem>_sol.exe or <yourproblem>_sol.cpp - (Optional) <yourproblem>_hack.exe or <yourproblem>_hack.cpp

Features

Compare with a Hack solution -h

Set a Time Limit -t xxxx

Generate Outputs Only -n

Compile(C++) Before Running -c

Show Result Using Colors

Auto Wait Until 1s to Avoid time(0) issues

You can find more here.