brigriffin / spider Goto Github PK
View Code? Open in Web Editor NEWThis project forked from johnnagro/spider
Spider is a Web spidering library for Ruby. It handles the robots.txt, scraping, collecting, and looping so that you can just handle the data.
License: MIT License