philshem / gmaps_popular_times_scraper Goto Github PK
View Code? Open in Web Editor NEWScraper for Google Maps "Popular Times" for place entries
Home Page: https://twitter.com/philshem/status/1232395590144753664?s=20
License: The Unlicense
Scraper for Google Maps "Popular Times" for place entries
Home Page: https://twitter.com/philshem/status/1232395590144753664?s=20
License: The Unlicense
Hi I tried to work with the program but its getting errors. plz help.
20231115_223829.log
currently the code requires a path to a CSV file, with the first column containing a gmaps URL
Would be cool to smarten it up a bit, and let the user point to EITHER a CSV path, OR a single gmaps URL. This way you could loop over variables in a batch script, which could be fed dynamically.
I'm using Linux Mint. When I run the script using a single URL I get a datatype error:
`/home/elena/Scrivania/places scraper/gmaps_popular_times_scraper/scrape_gm.py:35: DtypeWarning: Columns (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38) have mixed types. Specify dtype option on import or set low_memory=False.
urls = pd.read_csv(sys.argv[1])
ERROR: (a=window.gtbExternal.pageT()) 20231028_195921
ERROR: if (window['wtf']&& window['wtf']['trace']&& 20231028_195921
ERROR: window['wtf']['trace']['timeStamp']){window['wtf']['trace']['timeStamp']('application.' + t);} 20231028_195921
ERROR: } 20231028_195921
.. etc...
`
When I run the script using the CSV file, my output gives a generic error:
ERROR: https://maps.app.goo.gl/X7J2w3QXUXssXXzb9 20231028_200733
I've tried several solutions but I can't seem to fix the problem. Any suggestions?
I have currently a 30 second sleep period to make sure the page and data is rendered. This is to avoid missing data. Better code would be starting with 5 seconds and then doing a while loop with increasing sleep, until 60 seconds or so.
This would help the overall performance of the code over many places. (It’s important to collect data from each place at least 1x per hour)
Because it’s not trivial to get running, would be cool to put this out as a docket image, or at least a docket build script. I know there are some python-selenium images that may be built on.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.