This is a trial system for accurate related siggraph paper suggestion. A user can provide some keyword(select) or provide interested pdf file, and the system will parse it and search some other related, interesting paper for you.
-
Clone this repository
git clone https://github.com/karpathy/nipspreview.git
-
Download nips25offline from
http://books.nips.cc/nips25.html
and move it into the folder created in step 0 -
Make sure you have ImageMagick:
sudo apt-get install imagemagick
-
Run
pdftowordcloud.py
(to generate top words for each paper. Output saved in topwords.p as pickle) -
Run
pdftothumbs.py
(to generate tiny thumbnails for all papers. Outputs saved in thumbs/ folder) -
Run
scrape.py
(to generate paperid, title, authors list by scraping NIPS .html page) -
Run
makecorpus.py
(to create allpapers.txt file that has all papers one per row) -
Run
python lda.py -f allpapers.txt -k 7 --alpha=0.5 --beta=0.5 -i 100
. This will generate a pickle file calledldaphi.p
that contains the LDA word distribution matrix. Thanks to this nice LDA code by shuyo! It requires nltk library and numpy. In this example we are using 7 categories. You would need to change thenipsnice_template.html
file a bit if you wanted to try different number of categories. -
Generate the abstract files inside abstracts/ folder using
getabstracts.py
. Some user interaction may be necessary because of poorly formatted papers that make abstract extraction a pain. See script. -
Finally, run
generatenicelda.py
(to create the nipsnice.html page)
WTFPL licence