A simple scraper to retrieve movie scripts by genre or title from IMSDB.
npm install -S movie-script-scraperMovie Script Scraper exposes a function; simply pass this function the options (see below), and it will return a promise with an array of the file paths of the scripts it saved.
const mss = require('movie-script-scraper');
const options = {
genre: 'Action',
total: 10,
};
mss(options)
.then(filePaths => {
console.log(filePaths);
})
.catch(err => {
console.error('There was a problem');
});genre[string] - Any valid film genre, a complete list can be found here.- Defaults to "Action".
total[number] - the total number of scripts you want from a given genre.- Defaults to 10.
title[string] - The name of the film's script you want.dest[string] - Location that you want to save your scripts.- Defaults to ./scripts in the root directory.
You can run the Movie Script Scraper directly from the CLI (if it's globally available in your PATH, e.g. by npm install -g movie-script-scraper) with variety of useful options.
movie-script-scraper --total 10 --genre Comedyor enter a title:
movie-script-scraper --title 'american sniper'Conveniently IMSDB provides RSS feeds based on movie genre (ex. http://www.imsdb.com/feeds/genre.php?genre=Comedy). Using the awesome fetch module we are then able to grab that page and use a regular expression to generate an array of movie script URLs. We then visit each URL with Fetch and use another awesome module called Cheerio to select just the movie script from each page and finally output each script to a file.
- Install dependencies with:
npm install- Run Tests
npm testDon't hesitate to create a pull request. Every contribution is appreciated. In development you can start the tests by calling npm test. Checkout our contribution README for more info.
TLDR;
- Fork it!
- Create your feature branch:
git checkout -b my-new-feature - Commit your changes:
git commit -am 'Add some feature' - Push to the branch:
git push origin my-new-feature - Submit a pull request :D
Joe Karlsson |
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent