Support 'wrong' as test data category about problemtools HOT 5 OPEN

It would be also nice to have a category for output validator checking.

from problemtools.

Agree that this would be nice. Do you have concrete suggestions/preferences for how to extend the problem format to include this?

For input files, one could consider adding something like input_format_validators/bad_inputs containing .in files, or data/bad_inputs.

For output files, maybe one wants to be able to add incorrect outputs to some of the actual test cases, rather than adding new cases? If so, maybe allowing adding data files with suffix .out.wrong-<id>?

@niemela do you want to weigh in on this?

from problemtools.

For separation of concerns input_format_validators/bad_inputs is probably nicer. Your suggestion for output files sounds good to me.

from problemtools.

Hey,

I implemented this feature based on your ideas. See pull request #86 for details.

from problemtools.

I've just implemented some version of this for BAPCtools: RagnarGrootKoerkamp/BAPCtools#35.

Happy to change that to whatever structure is decided upon eventually.

from problemtools.