This repository includes some recipes of natural language pre-processing.

The list of recipes are as follows:

• Data cleaner
• Word normalization
• Stopwords remover
• Tokenizer
• Word Vector

To install required modules, simply:

$ pip install -r requirements.txt

First, you should download livedoor news corpus and extract it. For downloading the corpus, please execute following command:

$ cd src/data
$ python make_dataset.py

Now, you can ready for classification!

Start jupyter notebook:

$ jupyter notebook

And you can execute notebooks/document_classification.ipynb.

Good NLP Life!

MIT

Hironsan