Repo for presenting online hate speech project to PyLadies Seattle, 4/28/2016

Primary project repo at https://github.com/eyspahn/OnlineHateSpeech

Link to google slides presentation

• xgboost: pip install xgboost
  • you may need to sudo yum install gcc before this works
• gensim: pip install gensim

    numpy, scipy, xgboost, gensim, matplotlib, scikit-learn, nltk, flask

Pulls comments from relevant subreddits out of Kaggle SQL reddit comments file (available here) & saves comments & labels into a pandas dataframe. Takes ~30 minutes to run on a macbook air. (Script is not optimized.) Generates pickled file "labeledhate_pyladies.p"

Run this to perform 5-fold cross validation, while outputting ROC curves, confusion matrices, feature importances, and AUC scores. Takes ~40 minutes to run on a macbook air.

Run this to build the model for future use/prediction. Creates xgboost model object hatepredictor_pyladies.model & tf-idf object vect_pyladies.p Takes ~10 minutes to run on a macbook air

Run this to run a comment through the model from the command line & get a prediction. Based on the small subset of data we worked with here.

Run to generate 2 word2vec models, one each of hate speech and not-hate speech. Takes <2 minutes to run on a macbook air.

The saved xgboost model. Load with

bst.load_model('../data/hatepredictor_pyladies.model')```

#### labeledhate_pyladies.p.zip
Unzip to get a pickle file which is a pandas dataframe of comments and labels.

#### vect_pyladies.p
The saved (trained) TF-IDF object.


### notebooks
```Example_Results.ipynb``` - A notebook to probe the classification & word2vec models.


### webapp_v1.zip
It's not hosted yet, but I have built a website to accept comments & return hate speech classification prediction. Unzip to get a folder containing the flask app. Run it locally by running ```python application.py``` in this ("webapp_v1") directory. Then navigate in your browser to the url shown in the terminal (for me, http://127.0.0.1:5000/). Ctrl+C in the terminal to quit the app.