So happy you have found me, and hopefully found us in the future!

This demo and code base is just to illustrate how to approach learning any new tool or technology in general. We show case how to scrape a website that I had approval to do so on, and more importantly are here to show how Puppeteer can be used in tandem with Node.js to scrape data from websites. With a few added features just for fun.

Difference Between Original Demo VS Refactor Demo

- All of the code between the two directories produce the same output.

- The large benefit of the refactor is abstracting some of the cleanup functions into utilities, and also isolating the scrape call itself for use in the express demo's.

- In the current version, so much code has been cut out by doing the labor of modularization.

Why

• Why Learn Web Scraping?

  Basically personal data and large datasets are big money. This can be everything from tracking the different websites you visit, what kind of credit card you use most online, and several other factors.

What

• What is Web Scraping?

  It is just simply a tool to collect information based upon a specific url to scrape. The idea is, load a page, take a look at all of it's rendered HTML, decide what you would like to do with that info, and return something useful from all of your hard work.

How

• How does Web Scraping work?

  The fundementals of Puppeteer are:

  • start a Chrome browser instance
  • after, start a page instance in the browser
  • decide and implement a url to target
  • do your amazing logic
  • do something with your fruit

Node

• 12.12.0

NPM

• 7.5.2

Puppeteer

• 8.0.0

Operating System

• macOS Big Sur
  • 11.2.2

• Puppeteer Docs https://devdocs.io/puppeteer/

• NPM home for Puppeteer https://www.npmjs.com/package/puppeteer

• NPM Docs https://www.npmjs.com/

• Huge Shout to this person on stackoverflow! This saved my life, Many Thanks...

• Code Fellows Home Page https://www.codefellows.org/

• Find Me on Linkedin Connect with Me!