reem-alatrash / ccoha Goto Github PK

I currently have the following structure of the COHA which is available to me:

├── coha-db-001
│   ├── db_acad_1820.txt
│   ├── db_acad_1820.zip
|    .....
├── coha-lexicon.txt
├── coha-sources.txt
├── coha-text
│   ├── text_acad_1820.zip
│   ├── text_acad_1830.zip
|    ......
└── coha-wlp
    ├── wlp_acad_1820.zip
    ├── wlp_acad_1830.zip
    ├── wlp_acad_1840.zip

Will it work with your preprocessing scripts?

Hello,

I managed to get the code running on this repository and produce a copy of CCOHA. There was some technical stuff I had to sort out, which it would be useful to include in the README for future users. These include:

• Clarifying the code runs on Python 2.7
• Setting up a virtual environment for the packages (docopt, HTMLParser, nltk), and specifically mentioning nltk==3.4.5 to run this code.

There were also some basic code issues I had to fix, which might be helpful to note, or add another commit for. These include:

• Setting all tabs to spaces
• Renaming body to results in the function write_to_file(...)

Finally, there were a tiny number of files which the given processor failed to handle. The current code logs when this occurs and then sends a raise. Given the number of files in which this occurred (5? out of 100K), it didn't seem worth addressing this and just could be skipped over - but nonetheless the raise's needed to be commented out, which would have been helpful to've been noted.