Git Product home page Git Product logo

python-web-scraping-lab-demo-inperson-ds-000's Introduction

Web Scraping Lab

Questions

Resident Advisor is an events listing website for electronic music.

Go to www.residentadvisor.net/events. This is the url we'll be starting with for this lab. For question 1, just use this url. In the next two, you'll use country and region in the format: http://www.residentadvisor.net/country/region/ i.e. us/losangeles/. Be sure to explore the web pages in both the browswer and the HTML file. You'll need both to really understand what's going on.

  1. Which venues are hosting events this week?
  2. Make a function which returns the events this week given region and country (this will take two arguments)
    • return the event name, link, and list of artists
    • function returns list of ['event name', 'www.linkaddress.com', ['artist1','artist2','artist3']]
  3. Create a function which returns the users attending
  4. Bonus

Question 1 - Which venues are hosting events this week?

import requests
from bs4 import BeautifulSoup

r = requests.get('https://www.residentadvisor.net/events')
c = r.content
soup = BeautifulSoup(c, 'html.parser')

Your solution output should look like: '101bklyn', '291 Hooper St', '99 Scott Ave','Alphaville', 'Analog Bkny'...

Question 2 - Write a function to which returns the events this week given region and country.

def find_events(country, region):

    
    
    
    
    
    
    
    
    
# you should be able to output something like this
find_events('us','sanfrancisco')[0]
['Housepitality:Della, Homero Espinosa, Jason Peters at F8 1192 Folsom',
 'http://residentadvisor.net/events/1173172',
 ['Della', 'Homero Espinosa', 'Jason Peters']]

Question 3 - Create a function which returns the numbers of users attending each event this week, given country and region. Then plot a histogram

def users_attending(country, region):

    
    
    
    
    
# you should be able to output something like this
users_attending('us','newyork')[:10]
[8, 5, 4, 3, 49, 18, 10, 3, 2, 11]
#now use the function to make a histogram
import plotly.offline as offline
import plotly.graph_objs as go

offline.init_notebook_mode()

offline.iplot([go.Histogram(x = users_attending('jp','tokyo'))])

Bonus: Build object relations between artists, venues, and events with sqlalchemy!

Think about what each table should include - URLs, dates, etc.

python-web-scraping-lab-demo-inperson-ds-000's People

Contributors

taylorhawks avatar

Watchers

Kevin McAlear avatar  avatar Victoria Thevenot avatar Belinda Black avatar Soundarya avatar Bernard Mordan avatar  avatar Joe Cardarelli avatar Sara Tibbetts avatar The Learn Team avatar Sophie DeBenedetto avatar  avatar Jaichitra (JC) Balakrishnan avatar Antoin avatar Alex Griffith avatar  avatar Amanda D'Avria avatar  avatar Scott Ungchusri avatar Nicole Kroese  avatar Lore Dirick avatar Lisa Jiang avatar Vicki Aubin avatar Maxwell Benton avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.