This supplements a more efficient Rust-based static Web scraper (smart_scraper) for pages that require a headless browser that can accept cookies and render content via Javascript.
neilg63 / mini-puppeteer Goto Github PK
View Code? Open in Web Editor NEWBasic puppeteer scraping tool to supplement "smart scraper" (textsurfer) to capture rendered HTML from pages that require cookies and javascript to be enabled.