Bible Crawler that crawls data out of www.bible.com, using Selenium and Python
I was in need of high quality Bible content (reference, indentations, and headings included) for my Bible Android app, but I could not find one except through third-party API. After some research, I found that www.bible.com has everything that I wanted so all I had to do was to crawl through the website and gather the data.
I am using Selenium to crawl through their React-built website. All I had to do was to set the proper waits and keep clicking next chapter until it does not find one anymore.