subiet / silcc Goto Github PK
View Code? Open in Web Editor NEWThis project forked from ushahidi/silcc
RESTful semantic tag extraction application for SMS, Twitter and Text
Home Page: http://swift.ushahidi.com/extend/silcc/
This project forked from ushahidi/silcc
RESTful semantic tag extraction application for SMS, Twitter and Text
Home Page: http://swift.ushahidi.com/extend/silcc/
SiLCC =========== Synopsis =========== SiLCC aims to provide a learning NLP based relevancy filter/tagger/classifier that is applied to the various information sources that Swift aggregates (RSS, twitter, SMS). ============ Installation ============ To set up a silcc server instance: 1) Create a Python virtualenv 2) In the this distribution run: $ python setup.py develop This should install all needed libraries in your virtual environment. 3) You also need to install nltk. Download it from the following location: $ wget http://nltk.googlecode.com/files/nltk-2.0b8.zip $ unzip nltk-2.0b8.zip $ cd nltk-2.0b8.zip Now make sure you have activated the virtualenv created in step 1. If you are sure its activated, issue the following to install nltk into your virtualenv. $ python setup.py install 4) After installing nltk you need to download one of the Treebank Corpus for the part-of-speech tagger to work: $ python >>> import nltk >>> nltk.download() This will take you into an interactive download environment. The corpus you want to download is: maxent_treebank_pos_tagger After downloading it quit python. 5) Create the database. If using MySQL: $ mysql -u root -p Once in the mysql environment: >>> create database silcc default charset utf8; >>> grant all on silcc.* to silcc@localhost identified by 'password'; Make sure the .ini file contains a DB URI to the above database. (Please see instructions below on using SQLAlchemy migrate scripts to generate the db schema) 6) To start the server: $ paster serve --daemon prod.ini Look for any errors in paster.log ======== Placing your SiLCC Instance DB under version controll. ======== You may want to place your database under version control so that you can easily upgrade the schema, should it evolve over time. To do this you need to first create an empty database, and then place it under version control. 1) First install sqlalchemy-migrate: $ easy_install sqlalchemy-migrate 2) now place your database under version control with: $ python db_repository/manage.py version_control mysql://silcc:silcc@localhost:3306/silcc This will create the migrate_version table in your database and set the initial version to 0. After that you may run any schema upgrade scripts including the first one which will create the necessary tables. Most data in the SiLLC distribution can be recreated using scripts and data dumps provided. 3) To make upgrading your local instance easier its best to create a manage.py scripts as follows: $ migrate manage manage.py --repository=db_repository --url=mysql://silcc:silcc@localhost:3306/silcc Since the manage.py will be specific to your local instance it is not included in the SiLCC repo. 4) Now run a db update to generate the initial schema: $ python manage.py update 5) The Web Service API needs to have one or more valid keys in the apikey table. For testing purposes add a key with a valid_domains value of '*' (valid from all domains)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.