deephotel's Introduction

Only Scraping Hotel's Data @TripAdvisor.jp, Booking.com

Python Function To Scrape Hotel's data is hotel's name, address, review text, score, and so on, "Except pricing data"

Based on 'https://github.com/monkeylearn/hotel-review-analysis'

Special thanks to @monkeylearn

crawl to review-text of hotels in kyoto @TripAdvisor.jp

scrapy crawl tripadvisor_review -a start_url='https://www.tripadvisor.jp/Hotels-g298564-Kyoto_Kyoto_Prefecture_Kinki-Hotels.html' -o filename.csv

crawl to review-score of hotels in kyoto @TripAdvisor.jp

scrapy crawl tripadvisor_score -a start_url='https://www.tripadvisor.jp/Hotels-g298564-Kyoto_Kyoto_Prefecture_Kinki-Hotels.html' -o filename.csv

Thank you for @monkeylearn!!

crawl to review-text of hotels in osaka @Booking.com

scrapy crawl booking_review -a start_url='https://www.booking.com/searchresults.ja.html?city=-240905;ss=%E5%A4%A7%E9%98%AA%E5%B8%82' -o filename.csv

crawl to review-score of hotels in osaka @Booking.com

scrapy crawl booking_score -a start_url='https://www.booking.com/searchresults.ja.html?city=-240905;ss=%E5%A4%A7%E9%98%AA%E5%B8%82' -o filename.csv

As you know, I'm Python Beginner. I'm glad you to give feedback.

deephotel's People

Contributors

Stargazers

Watchers

deephotel's Issues

UnboundLocalError: local variable 'url' referenced before assignment

def parse_hotel(self, response):
        try:
            list_url = response.xpath('//a[@class="show_all_reviews_btn"]/@href')
            url = response.urljoin(list_url[0].extract())
            return scrapy.Request(url, callback=self.parse_score_scrape)
        except:
            return scrapy.Request(url, callback=self.parse)

I think this point is problematic: if the spider except, it will have to use url, which is undefined... at least that's the error I get, every time a hotel has no reviews:

UnboundLocalError: local variable 'url' referenced before assignment

Recommend Projects