dawe35 / facebook-event-crawler Goto Github PK
View Code? Open in Web Editor NEWThis is a python crawler, it saves the given facebook page's event details to mysql
This is a python crawler, it saves the given facebook page's event details to mysql
In the reame you state
- Create a new mysql table callled 'event_crawler'
when it should read "Create a new mysql database"
thanks for the amazing script :)
Example event: 415740479048949
Here are two use cases I was wondering if you maybe could take under consideration?
On Thursday I have 12 events between 17:00 and 22:00 (I live in a city) all over the city. It would be great if they could be displayed on a map so I could "plan my route" easier. In other words, if this crawler could extract the address of all my upcoming events and display them on a map? Or maybe even easier, just extract the location and format it in a way that Google Map can import? (A shortcut would be to just concatenate strings to build the URL string for Google maps, like this example https://www.google.com/maps/dir/Amazon+Go,+7th+Avenue,+Seattle,+Washington,+USA/Seattle,+Washington,+USA/Boeing+Everett+Factory,+West+Casino+Road,+Everett,+Washington,+USA/Microsoft+Building+92,+Northeast+36th+Street,+Redmond,+Washington,+USA/@47.7679854,-122.3785765,11z - as you can see, the format is pretty straightforward)
There are also those events listed under Discover. Would be nice if these could be displayed on a map in a similar way to (1), including filters (art, sport, music etc) so you easily can find a nearby event if you are looking for something to do
In many events description are html characters (like a href, strong, italic, etc).
Is possible scrapping descriptions with 'a href' html tags?
Crawler does not download event descriptions...
The crawler is working fine apparentaly, but 'events" table still empty...
python3 Crawlr.py
Logging in to facebook...
Getting page: tomorrowland
Getting page: Ultra
Getting page: djhardwell
Getting page: timmytrumpet
Getting page: Willcoxofficial
Getting page: OpenUpYourEars
0 old row deleted
Pages are already updated less than an hour ago, no new events queried
Script end at 13:28
How can i debug this ?
It seems Facebook changed their page's upcoming events structure severely if my observations are representative.
Example:
https://www.facebook.com/changa.dance/events/?ref=page_internal
If you are not logged in, you only see past events now + even if you are logged in - the data is no longer delivered via html if my analysis is correct so far.
Getevent error
ERROR:root:message
Traceback (most recent call last):
File "C:\Crawlr.py", line 113, in getevent
datefrom = timestring.Date(splitted[0]).date
File "C:\Users\abc\AppData\Local\Programs\Python\Python37-32\lib\site-packages\timestring\Date.py", line 176, in init
new_date = new_date.replace(hour=int(max(hour)))
ValueError: hour must be in 0..23
Error, writing del.html...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.