geisler / touche Goto Github PK

Delete any ability to upload postscript file descriptions. They have outlived their usefulness.

The README is old and possibly incorrect. It needs to be updated to reflect the information on the current system. In fact, it may need to be auto-generated since some of the information is configurable by the contest administrator.

While fixing the README, be sure to put the Java information first since it is essential for Java program success and more submissions use that language than C or C++.

Generate a PDF for the final standings that can be distributed properly.

When logging execution of Java programs, each Java submission writes over the same log files, so there is no way to look past the most recently executed Java program to debug any problem that may occur.

There is a slight security issue with testing contests right now. Currently, testing is done by starting the contest, using a test user and submitting known correct (and incorrect) solutions, and verifying the judging system responds properly. If a team happens to know their username and password and can guess the URL to the contest, they are also able to log in during the testing period and access anything that is available during the running contest. That includes pointers to the contest documents, and a very bad cheating scenario ensues.

Therefore, it would be very good to be able to test the contest while not opening it up to anyone other than the administrators and judges running the contest.

The time limit may need to be configurable on a per problem basis, but certainly should be at the contest level. Currently it is semi-hard-coded in that each programming language has a fixed value in the code. One would need to know where those values are in the code to change them on a per-contest basis.

To start or restart a contest, some of the work needs to be done by the administrator and some needs to be done by a judge. This task needs be consolidated under a single role.

Does the system do the right thing when the administrator checks the box to ignore stderr? This needs investigation and a final answer of YES.

When an administrator tries to update a team, it doesn't always update like it should. I don't know how to produce this error to make it replicable, but believe the report to be valid.

When switching between roles and logging in as a team, judge, or administrator, one is automatically logged out of all the other roles. It would be nice to keep those logged in unless specifically logging out. That would make it much easier to switch tabs or tasks between these roles seemlessly.

It would be very nice if a judge or administrator (not sure which level, yet) could be able to access a team account without having to look up the account details. This would enable them to be able to see what a team is seeing in case there is some sort of error that needs to be fixed, but is not replicable from another account.

Add Python as a language for participants to submit programs with. At a minimum, this includes the proper language files, but really should also include a default list of forbidden words (exec?) and libraries that should/shouldn't be used in the Python environment. It also includes building a chroot jail to be used by all future contests.

Allow the contest administrator to set a different time penalty than is currently hard-coded into the standings. The default should still be 20 minutes as that is the current value that is hard-coded.

It would be awesome to have even more error types available for an administrator to be able to provide (on configuration) to judges for their responses to problem submissions. Can we make that happen?

Some examples of additional error types could be:

• failed all tests
• failed some tests
• too much output
• too little output
• etc.

I'm not asking for the software testing the programs to do anything more or ever be able to auto-suggest these outputs, but that they be made available for the human judge to override the default "incorrect output" with one of these new error types.

Contestants aren't used to using the Java naming conventions and they are always tripped up by them. Can't we do something better so that they can just submit whatever they want and we just run it properly?

At a minimum, can we get rid of the "no public classes" constraint?

When comparing the output of the submitted program with the expected output, the HTML viewer kills all newlines. They should be in there to make it easier to read the output. Keep the red tildes to signify spaces since they serve a good purpose.

Does the system properly accept files of PDF and HTML and provide links to them at the proper time? They should be hidden before the contest starts and only made visible when it does. I believe this is done through permissions in the proper public_html directory, but they may never be turned on by the system and only directly via the command line. Starting a contest should be sufficient. Disabling a contest should also re-hide them. Fix if necessary.

Can a team be deleted during a contest? I don't know and am too scared to try during a real contest for fear it would break everything. Find out and make sure the answer ends up being yes.

public_html/judge/lib/responses.inc and the RESPONSES tables hold duplicated information

Because Windows terminates text files with "\r\n" and UNIX terminates them with "\n", we have problems when the contest administrator submits data sets generated on Windows machines. The UNIX server expects the '\r' to be part of the output generated by the submissions, but they are run on UNIX and so will never do this unless the contestants are aware of the cross-OS portability and code super defensively (which happens to be a really bad strategy for winning these contests, so no one ever does).

Simple solution: run all data sets through dos2unix and make them UNIX files. Everything is moot after that.

Does multi-site timing work correctly? It should allow different sites to start at different times and score the submissions based on the start time of the site the submitting team is located at. Freeze time and finishing times also need to be per-site as well. This has never been tested thoroughly, but the developers that put the functionality in claimed it worked. We need better verification of this claim.

Currently, the standings pages always show the full name for a problem, but the problem title can be quite large at time. There should be a way to provide both a full name for browsing and submitting the problems, but a short name that makes the standings or other similar listings more digestible.

Weird things happen to numbering and ordering in standings when problems are added and deleted from the list of problems. Confusing, awful things. Fix them and make them make sense.

If the administrator uses certain names for their data sets, things break marvelously. Fix the code to accept any name including any suffix.

Some judging pages auto-refresh, but others do not. Make it more consistent and useful so that the human judges can know when they need to respond at even more locations. The one page that should NOT auto-refresh is when the judge is reviewing the response from a particular judged submission.

There is a button on the administrator's misc tab that implies that contest data (submissions, test input & output files, etc.) can be sent to contest participants. That functionality needs to be completed (i.e., started).

Contestants do not have access to the server compiler. It needs to be available so that they know that the code they've written actually works like it is supposed to.

We need a mechanism for confirming which participants actually were in the contest in case an alternate replaced a regular contestant.

When a contest is created, all the tabs on the administrator section are enabled, but shouldn't be since there are strict ordering constraints that must be met. For example, a problem must be uploaded before a data set for that problem can be uploaded. A more "wizard"-y way of guiding the administrator through the necessary steps to create the contest would ensure that the data is valid and complete. At a minimum, links that shouldn't be clickable should be disabled until they make sense to be clickable.