bigcode-project / bigcodebench-annotation Goto Github PK

See this build.

As we are moving to pytest for testing, there is no need to do if __name__ == "__main__":, run_tests() and doctest.testmod(). These parts will be automatically removed when collecting clean data.

The type of programming problems should be automatically classified. Neural-based methods like embeddings are preferred.

@2019ChenGong please create the PR when the data is ready.

The structure of the docstring in each clean sample should be validated using rule-based methods. The script will be uploaded soon. GitHub Actions may include this accordingly.

@huhanGitHub Please create a PR for adding refined data. Note that 427-431 have been refined already in data/raw.

Vulnerabilities raised by dependabots need to be resolved.