Discord web scraping bot used to scrape websites with dynamic content and send notifications when there is a new item.

Simply click the link: https://discord.com/api/oauth2/authorize?client_id=1007981622005596171&permissions=2147485696&scope=bot%20applications.commands

Use /create-job command and fill in the form:

• name - name of the job
• url - url that the bot will scrape
• selector - selector for an a tag with link to element (examples below)
• interval - interval of the job in minutes (min 1 minute)
• active? - default true, if the job should be active right away (you can always enable/disable job with [enable/disable]-job commands)
• channel? - defualt channel where the command was run which channel should be messaged when the new item appear
• clean? - default true, if the query params should be ignored (essential for some sites like Ebay)

After that you can do basic CRUD opeartions on jobs with commands like /list-jobs /update-job /delete-job...

After that you simply wait for the job run and it will send message when new items are found:

You can also run the job manually with /run-job. Note that the first run will get all the elements that will be on the webiste.

The selector is a querySelectorAll string that the bot uses to get unique a tags linking to items you want to scrape. To verify if your selector is working you can run document.querySelectorAll(<your selector>) in browser console and check if the function returns items you want to scrape.

• Ebay: .srp-river-results .s-item__image a
• Otodom: [data-cy=listing-item-link]
• Olx: [data-cy=l-card] > a