organicirradiation / ratebeer Goto Github PK
View Code? Open in Web Editor NEWGet information out of RateBeer.com without losing your mind.
ratebeer's People
Contributors
Stargazers
Watchers
Forkers
philadams-zz parryc surye ivandjuricic phelian rcrown1 shamrt gevurtz afowles jwrubel ingryan sversage arfenarf oddbloke gtietjen ianpcook catullus gmellison benheubl plotxy tomppasalmi rohan-thakur myrc kcmillie zoeylau infvie brendon-node1 wboccard timchildersratebeer's Issues
Can't grab style list
I am getting this error when I try to grab the style list:
rb.beer_style_list()
Traceback (most recent call last):
File "<pyshell#0>", line 1, in
rb.beer_style_list()
File "C:\Python27\lib\site-packages\ratebeer\ratebeer.py", line 159, in beer_style_list
columns = soup.find_all('table')[2].find_all('td')
IndexError: list index out of range
Cannot Fetch Beer
Hi, submitting a new issue, as I'm not sure poking the closed similar one does anyone any good.
I can search beers but I cannot retrieve beer data.
Here is typical output from test.py. TestBrewery all passed, as did TestMisc and TestSearch. TestAlpha failed and of TestBeer, only the 404 scripts passed. Can you help?
`Error
Traceback (most recent call last):
File "/Users/kath/anaconda/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File "/Users/kath/Documents/datacourse/udacity/04Exploratory_Analysis/project4/test.py", line 20, in test_beer
results = RateBeer().beer('/beer/new-belgium-tour-de-fall/279122/')
File "/Users/kath/anaconda/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 143, in beer
return self.get_beer(url, True).__dict__
File "/Users/kath/anaconda/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 139, in get_beer
return models.Beer(url, fetch)
File "/Users/kath/anaconda/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/kath/anaconda/lib/python2.7/site-packages/ratebeer/models.py", line 134, in _populate
self.overall_rating = int(soup.find('span', text='overall').next_sibling.next_sibling)
TypeError: int() argument must be a string or a number, not 'Tag'
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): www.ratebeer.com
requests.packages.urllib3.connectionpool: DEBUG: "GET /beer/new-belgium-tour-de-fall/279122/ HTTP/1.1" 200 37369
--------------------- >> end captured logging << ---------------------`
Search broken in v2.1
RateBeer recently updated their search page to use bootstrap, which has broken RateBeer.search().
Do you have time to fix this? If necessary, I can do so and submit a pull request.
Thanks!
Encoding appears to have issues
It appears some point in the past few weeks (July 2017), the encodings for unicode data coming from RateBeer got all messed up. There are two tests test_beer_unicode and test_beer which are failing because of it.
test_beer fails because the copy has switched back from using ' to using ’, which can have erroneous encodings.
test_beer_unicode fails because the beer name comes through as Steðji Októberbjór.
Both appear to be due to BS4 thinking the encodings are iso-8859-1, but the raw source of the page is being displayed as unicode, and the charset in the tags says utf-8 as well. Not sure what's up.
Country for beer() function
I realized after I made the pull request today, that brewery country had been removed in v2.0. Is it possible to add it back in, or did you find issues with the way it was derived?
The necessary code would be (around line 143):
if ',' in brewery_info[5]:
# Non-USA addresses
self.brewery_country = brewery_info[5].split(',')[1].strip()
else:
# USA addresses
self.brewery_country = brewery_info[8].strip()
Users
Next up: let's get per-user information and user information as a whole. I want information like top-rated beers and most-consumed beers. More research required.
meta doesn't get all information from beer page
s_contents[1].find_all('td')[1].div.small.find_all('big') isn't grabbing all the elements on a beer page. It gets to IBU and then gives up.
Failure to fetch beer object
I downloaded the package a week or so ago, and it originally was working beautifully. When I tried it a couple of days ago, it started throwing this error:
>>> rb.beer('/beer/new-belgium-tour-de-fall/279122/')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/site-packages/ratebeer/ratebeer.py", line 143, in beer
return self.get_beer(url, True).__dict__
File "/usr/local/lib/python3.4/site-packages/ratebeer/ratebeer.py", line 139, in get_beer
return models.Beer(url, fetch)
File "/usr/local/lib/python3.4/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/usr/local/lib/python3.4/site-packages/ratebeer/models.py", line 134, in _populate
self.overall_rating = int(soup.find('span', text='overall').next_sibling.next_sibling)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'Tag'
I've looked into the code on the lines it's referencing, and can't find anything that should be throwing this error. Any ideas? Other functions (rb.search) seem to be working fine. I tried updating the package, but to no avail.
Beer scraper broken
If I run the following line from v2.3.0 of the package README…
$ rb.beer("/beer/new-belgium-tour-de-fall/279122/")
…I get the following error:
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): www.ratebeer.com
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/smartin/dev/workspaces/myliqbo/env/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 130, in beer
return self.get_beer(url, True).__dict__
File "/Users/smartin/dev/workspaces/myliqbo/env/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 126, in get_beer
return models.Beer(url, fetch)
File "/Users/smartin/dev/workspaces/myliqbo/env/lib/python2.7/site-packages/ratebeer/models.py", line 71, in __init__
self._populate()
File "/Users/smartin/dev/workspaces/myliqbo/env/lib/python2.7/site-packages/ratebeer/models.py", line 180, in _populate
if overall_rating and overall_rating[1].text != 'n/a':
IndexError: list index out of range
RateBeer search does not support ! in beer names
If you search for the (delicious) Mikkeller George!, RateBeer chokes and searches instead for Mikkeller George!, which returns nothing. Interestingly just searching for George! returns Tarn 51 By George!, so it's probably an issue with how they are storing/searching for the beer names.
It might be worthwhile to have an option for "searching-with-fixes" (toggle-able, of course). That option would help construct searches that are a little bit more in line with what the user was hoping to search for.
Thoughts?
Broken beer search
Similar to issue #26, the searching for beers function is also broken.
Running:
import ratebeer
rb = ratebeer.RateBeer()
rb.search('Summit')
Will fail with:
File "/usr/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 115, in search beer.name = row('td')[0].a.string.strip() AttributeError: 'NoneType' object has no attribute 'strip'
Thanks!
Returns incorrect weighted average for a given beer
For example, the following returns a weighted average of 5.0 for Heineken:
from ratebeer import RateBeer
rb = RateBeer()
hs = rb.search('heineken')
h1 = hs['beers'][0]
hb = rb.beer(h1.url)
print hb
The true weighted average, as shown on the website is 2.09.
Does it need another float() ?
I have a similar problem like in Issue #35. When trying to _populate() the get_beers() from the brewery <Brewery('/brewers/paulaner-brauerei-schorghuber/61/')> I get following output:
File "/usr/lib/python3.6/site-packages/ratebeer/models.py", line 388, in get_beers
beer.abv = float(abv)
ValueError: could not convert string to float: '-'
Because there is already an exception handling for beer.abv in line 175-178 I just changed line 388 to beer.abv = abv and it works for me.
Has there been a change in format?
RateBeer has switched to React
As a result, the search isn't working. In particular, it dynamically searches by querying https://beta.ratebeer.com/v1/api/graphql/. Unfortunately, I'm not quite sure how to authenticate, as it would be idea to use an actual API to get search data, rather than scraping the page.
Search Not Working
Can't get search to work, only returns empty lists. Is search currently working for others?
Thanks.
New UI, but a new API!
It looks like the UI is now generated based on a new API call to the beta.ratebeer.com endpoint. It will require some reworking of the _populate function. Hopefully this means the scraping will be a little bit more stable.
An example request:
[{"operationName":"beer","variables":{"beerId":"5977"},"query":"query beer($beerId: ID!) {\n info: beer(id: $beerId) {\n id\n name\n description\n style {\n id\n name\n glasses {\n id\n name\n __typename\n }\n __typename\n }\n styleScore\n overallScore\n averageRating\n abv\n ibu\n calories\n brewer {\n id\n name\n __typename\n }\n ratingCount\n isRetired\n isUnrateable\n seasonal\n labels\n availability {\n bottle\n tap\n distribution\n __typename\n }\n __typename\n }\n}\n"},{"operationName":"beerReviews","variables":{"beerId":"5977","order":"RECENT","first":10},"query":"query beerReviews($beerId: ID!, $authorId: ID, $order: ReviewOrder, $after: ID) {\n beerReviewsArr: beerReviews(beerId: $beerId, authorId: $authorId, order: $order, after: $after) {\n items {\n id\n comment\n score\n scores {\n appearance\n aroma\n flavor\n mouthfeel\n overall\n __typename\n }\n author {\n id\n username\n reviewCount\n __typename\n }\n checkin {\n id\n place {\n name\n city\n state {\n name\n __typename\n }\n country {\n name\n __typename\n }\n __typename\n }\n __typename\n }\n createdAt\n updatedAt\n __typename\n }\n totalCount\n last\n __typename\n }\n}\n"},{"operationName":"tagDisplay","variables":{"beerId":"5977"},"query":"query tagDisplay($beerId: ID!, $first: Int) {\n tagDisplayArr: beerTags(beerId: $beerId, first: $first) {\n items {\n id\n urlName: plain\n __typename\n }\n __typename\n }\n}\n"},{"operationName":"beer","variables":{"beerId":"5977"},"query":"query beer($beerId: ID!) {\n beer(id: $beerId) {\n id\n style {\n name\n id\n __typename\n }\n __typename\n }\n}\n"},{"operationName":"beer","variables":{"beerId":"5977"},"query":"query beer($beerId: ID!) {\n info: beer(id: $beerId) {\n id\n name\n isAlias\n isUnrateable\n brewer {\n id\n name\n __typename\n }\n __typename\n }\n}\n"}]
Test fails: PageNotFound
I installed ratebeer on my Mac using python 2.7 by running:
virtualenv testA
source testA/bin/activate
pip install git+https://github.com/alilja/ratebeer.git
wget https://raw.githubusercontent.com/alilja/ratebeer/master/test.py
Running python test.py returns:
======================================================================
ERROR: test_beer (__main__.TestBeer)
Make sure the results for a beer contain the expected data
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 20, in test_beer
results = RateBeer().beer('/beer/new-belgium-tour-de-fall/279122/')
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 136, in beer
return self.get_beer(url, True).__dict__
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 132, in get_beer
return models.Beer(url, fetch)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/new-belgium-tour-de-fall/279122/
======================================================================
ERROR: test_beer_aliased (__main__.TestBeer)
Check that AliasedBeer exception is raised properly
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 111, in test_beer_aliased
self.assertRaises(rb_exceptions.AliasedBeer, rb.beer, "/beer/new-belgium-biere-de-mars/113241/")
File "/Users/mariel/anaconda/lib/python2.7/unittest/case.py", line 473, in assertRaises
callableObj(*args, **kwargs)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 136, in beer
return self.get_beer(url, True).__dict__
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 132, in get_beer
return models.Beer(url, fetch)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/new-belgium-biere-de-mars/113241/
======================================================================
ERROR: test_beer_closed_brewery (__main__.TestBeer)
Handling beers from closed brewers
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 42, in test_beer_closed_brewery
results = RateBeer().beer('/beer/hantens-hildener-landbrau/140207/')
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 136, in beer
return self.get_beer(url, True).__dict__
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 132, in get_beer
return models.Beer(url, fetch)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/hantens-hildener-landbrau/140207/
======================================================================
ERROR: test_beer_closed_contract_brewery (__main__.TestBeer)
Handling beers from closed contract brewers
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 47, in test_beer_closed_contract_brewery
results = RateBeer().beer('/beer/crew-republic-x-11-wet-hop/298026/')
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 136, in beer
return self.get_beer(url, True).__dict__
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 132, in get_beer
return models.Beer(url, fetch)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/crew-republic-x-11-wet-hop/298026/
======================================================================
ERROR: test_beer_contract_brewed (__main__.TestBeer)
Handling contract brewed beers
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 53, in test_beer_contract_brewed
results = RateBeer().beer('/beer/benediktiner-weissbier/157144/')
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 136, in beer
return self.get_beer(url, True).__dict__
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 132, in get_beer
return models.Beer(url, fetch)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/benediktiner-weissbier/157144/
======================================================================
ERROR: test_beer_get_reviews (__main__.TestBeer)
Check to make multi-page review searches work properly
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 61, in test_beer_get_reviews
self.assertIsNotNone(next(reviews))
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 213, in get_reviews
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/deschutes-inversion-ipa/55610/
======================================================================
ERROR: test_beer_no_abv (__main__.TestBeer)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 70, in test_beer_no_abv
results = RateBeer().beer('/beer/deschutes-altitude-amber/92102/')
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 136, in beer
return self.get_beer(url, True).__dict__
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 132, in get_beer
return models.Beer(url, fetch)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/deschutes-altitude-amber/92102/
======================================================================
ERROR: test_beer_no_ratings (__main__.TestBeer)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 84, in test_beer_no_ratings
results = RateBeer().beer('/beer/deschutes-abyssident/194792/')
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 136, in beer
return self.get_beer(url, True).__dict__
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 132, in get_beer
return models.Beer(url, fetch)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/deschutes-abyssident/194792/
======================================================================
ERROR: test_beer_retired_beer (__main__.TestBeer)
Attributes for retired beers display properly
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 105, in test_beer_retired_beer
results = RateBeer().beer('/beer/shorts-funkin-punkin/79468/')
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 136, in beer
return self.get_beer(url, True).__dict__
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 132, in get_beer
return models.Beer(url, fetch)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/shorts-funkin-punkin/79468/
======================================================================
ERROR: test_beer_unicode (__main__.TestBeer)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 97, in test_beer_unicode
results = RateBeer().beer('/beer/stedji-oktoberbjor/292390/')
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 136, in beer
return self.get_beer(url, True).__dict__
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 132, in get_beer
return models.Beer(url, fetch)
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 72, in __init__
self._populate()
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/models.py", line 112, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /beer/stedji-oktoberbjor/292390/
======================================================================
ERROR: test_str_nonascii_search (__main__.TestSearch)
Test out the search function with a str with more than ASCII characters
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 184, in test_str_nonascii_search
results = RateBeer().search("to øl jule mælk")
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 120, in search
url = '/beer/{0}/{1}/'.format(beer_data['name'].replace(' ', '-').lower(), beer_data['id'])
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf8' in position 3: ordinal not in range(128)
======================================================================
ERROR: test_unicode_nonascii_search (__main__.TestSearch)
Test out the search function with a unicode string with more than ASCII characters
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 202, in test_unicode_nonascii_search
results = RateBeer().search(u"to øl jule mælk")
File "/Users/mariel/testA/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 120, in search
url = '/beer/{0}/{1}/'.format(beer_data['name'].replace(' ', '-').lower(), beer_data['id'])
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf8' in position 3: ordinal not in range(128)
======================================================================
FAIL: test_fetch_by_letter (__main__.TestAlpha)
Make sure the results for a brewery list by index contain the expected data
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 216, in test_fetch_by_letter
self.assertTrue(beer.url == u'/brewers/a-duus-and-co/1668/')
AssertionError: False is not true
----------------------------------------------------------------------
Ran 24 tests in 9.073s
FAILED (failures=1, errors=12)
Any ideas on how to fix this?
Tests seems to be failing
Hello!
I think Ratebeer changed their layout, so the tests are no longer passing. Could you take a look?
Thanks!
$ python test.py
E.EEE.E..........
======================================================================
ERROR: test_beer (__main__.TestBeer)
Make sure the results for a beer contain the expected data
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 12, in test_beer
results = RateBeer().beer("/beer/new-belgium-tour-de-fall/279122/")
File "/Users/aeyoll/Code/ratebeer/ratebeer/ratebeer.py", line 130, in beer
return self.get_beer(url, True).__dict__
File "/Users/aeyoll/Code/ratebeer/ratebeer/ratebeer.py", line 126, in get_beer
return models.Beer(url, fetch)
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 71, in __init__
self._populate()
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 180, in _populate
if overall_rating and overall_rating[1].text != 'n/a':
IndexError: list index out of range
======================================================================
ERROR: test_beer_closed_brewery (__main__.TestBeer)
Handling beers from closed brewers
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 29, in test_beer_closed_brewery
results = RateBeer().beer('/beer/hantens-hildener-landbrau/140207/')
File "/Users/aeyoll/Code/ratebeer/ratebeer/ratebeer.py", line 130, in beer
return self.get_beer(url, True).__dict__
File "/Users/aeyoll/Code/ratebeer/ratebeer/ratebeer.py", line 126, in get_beer
return models.Beer(url, fetch)
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 71, in __init__
self._populate()
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 176, in _populate
style_rating = ratings[3].findAll('span')
IndexError: list index out of range
======================================================================
ERROR: test_beer_contract_brewed (__main__.TestBeer)
Handling contract brewed beers
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 37, in test_beer_contract_brewed
results = RateBeer().beer('/beer/benediktiner-weissbier/157144/')
File "/Users/aeyoll/Code/ratebeer/ratebeer/ratebeer.py", line 130, in beer
return self.get_beer(url, True).__dict__
File "/Users/aeyoll/Code/ratebeer/ratebeer/ratebeer.py", line 126, in get_beer
return models.Beer(url, fetch)
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 71, in __init__
self._populate()
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 180, in _populate
if overall_rating and overall_rating[1].text != 'n/a':
IndexError: list index out of range
======================================================================
ERROR: test_beer_get_reviews (__main__.TestBeer)
Check to make multi-page review searches work properly
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 49, in test_beer_get_reviews
self.assertIsNotNone(next(reviews))
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 233, in get_reviews
self._populate()
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 180, in _populate
if overall_rating and overall_rating[1].text != 'n/a':
IndexError: list index out of range
======================================================================
ERROR: test_beer_unicode (__main__.TestBeer)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 58, in test_beer_unicode
results = RateBeer().beer("/beer/steoji-oktoberbjor/292390/")
File "/Users/aeyoll/Code/ratebeer/ratebeer/ratebeer.py", line 130, in beer
return self.get_beer(url, True).__dict__
File "/Users/aeyoll/Code/ratebeer/ratebeer/ratebeer.py", line 126, in get_beer
return models.Beer(url, fetch)
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 71, in __init__
self._populate()
File "/Users/aeyoll/Code/ratebeer/ratebeer/models.py", line 176, in _populate
style_rating = ratings[3].findAll('span')
IndexError: list index out of range
----------------------------------------------------------------------
Ran 17 tests in 24.879s
FAILED (errors=5)
get_reviews throwing an error
When trying to iterate through reviews getting AttributeError.
for r in rev:
...\site-packages\ratebeer\models.py", line 235, in get_reviews
yield Review(review_soup)
...\site-packages\ratebeer\models.py", line 261, in init
raw_ratings = zip(*[iter(review_soup.find('strong').find_all(["big", "small"]))] * 2)
AttributeError: 'NoneType' object has no attribute 'find_all' `
Was working a couple of weeks ago.
Any insight how this can be fixed?
Thanks
Beginner's problem
Thanks to Andrew and everyone for making this excellent Ratebeer python module available in the public domain. This was just what I needed for a small hobby project of mine.
I realise you won't have time to help everybody who wants to use this library but I was wondering if the error I get when trying to access reviews is indicative of something having broken in the HTML interface of RateBeer relative to the assumptions being made in your python library of the same name or if (more likely) it's my inaptitude at getting it to work.
Specifically, I get an unexpected error message that seems to have to do with HTML parsing using BeautifulSoup.
Finish beta.ratebeer.com API work
Although work has been done to support core beer searching and information gathering, some of the other modules have not yet been updated to use the API calls.
The following (non-exhaustive list, most likely) work needs to be done to update the entire code base:
- (models.py) Update Review model to use graphql call to populate data if possible, otherwise, use BeautifulSoup scraper
- (models.py) Update Brewery model to use graphql call to populate data, otherwise, use BeautifulSoup scraper
- (ratebeer.py) Fix test case for browsing brewers by alpha or create NoLongerSupported error if not possible
- (ratebeer.py) Update beer styles list function
- (ratebeer.py) Update beer style function
- (test.py) Validate all tests now pass
- (optional, test.py) Add additional tests for updated ratebeer.py functions
The graphql queries can be gotten by viewing the network tab in the browser's dev tools and looking for /v1/api/graphql calls and looking at what data is posted. If I recall correctly, reviews should be accessible, but I don't recall what brewery information is.
API not working?
Is this project still active and working? I have installed ratebeer via pip but cannot get any of the examples to work.
import ratebeer rb = ratebeer.RateBeer() rb.beer("/beer/new-belgium-tour-de-fall/279122/")
`Traceback (most recent call last):
File "/home/thomas/.local/lib/python3.8/site-packages/ratebeer/models.py", line 109, in _populate
soup_rows = soup.find('div', id='container').find('table').find_all('tr')
AttributeError: 'NoneType' object has no attribute 'find'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 1, in
File "/home/thomas/.local/lib/python3.8/site-packages/ratebeer/ratebeer.py", line 143, in beer
return self.get_beer(url, True).dict
File "/home/thomas/.local/lib/python3.8/site-packages/ratebeer/ratebeer.py", line 139, in get_beer
return models.Beer(url, fetch)
File "/home/thomas/.local/lib/python3.8/site-packages/ratebeer/models.py", line 72, in init
self._populate()
File "/home/thomas/.local/lib/python3.8/site-packages/ratebeer/models.py", line 111, in _populate
raise rb_exceptions.PageNotFound(self.url)
ratebeer.rb_exceptions.PageNotFound: /beer/new-belgium-tour-de-fall/279122/
rb.brewery("/brewers/deschutes-brewery/233/")
Traceback (most recent call last):
File "/home/thomas/.local/lib/python3.8/site-packages/ratebeer/models.py", line 338, in _populate
s_contents = soup.find('div', id='container').find('table').find_all('tr')[0].find_all('td')
AttributeError: 'NoneType' object has no attribute 'find'`
Any ideas?
Empty Results
Hi,
I pip installed ratebeer (tried on both Python 2.7 and 3.4), and am getting empty results when running the sample query. Is there a step I've missed in setup? Please let me know if you guys have any insight. Thanks!
Create new pip repo?
Since it appears @alilja may be permanently AFK, do we want to make a new pip repo (either on my account or someone else's)? We could bump the version up to 3 and call the package ratebeer3. Making sure to update all relevant documentation, of course.
What do you think, @OrganicIrradiation?
Add exception for rate limiting
If you try searching too much (looks like maybe... 10 times in as many minutes?), Clourflare jumps in an responds with a 500 status.
Add reviews
The markup for beer reviews is such a mess I can't even begin to think about how to solve it.
Broken Brewery
Thanks so much for your earlier help! I've stumbled again.
If I understand things correctly, rb has changed their Brewery page so the brewery information is no longer wrapped in a div id='container' tag; and it's not in a table, either. Calls to _populate(self) in models.py fail with a 'PageNotFound' error.
test_brewery, test_brewery_get_beers, and test_brewery_unicode fail, everything else (other than TestAlpha, which didn't work before on this machine) is OK.
I'm loath to tinker with this too much. Would you mind confirming whether this error is more than just weirdness on my end?
Error
Traceback (most recent call last):
File "/Users/kath/anaconda/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File "/Users/kath/Documents/datacourse/udacity/04Exploratory_Analysis/project4/test.py", line 78, in test_brewery
results = RateBeer().brewery("/brewers/deschutes-brewery/233/")
File "/Users/kath/anaconda/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 154, in brewery
return self.get_brewery(url, True).__dict__
File "/Users/kath/anaconda/lib/python2.7/site-packages/ratebeer/ratebeer.py", line 150, in get_brewery
return models.Brewery(url, fetch)
File "/Users/kath/anaconda/lib/python2.7/site-packages/ratebeer/models.py", line 293, in __init__
self._populate()
File "/Users/kath/anaconda/lib/python2.7/site-packages/ratebeer/models.py", line 340, in _populate
raise rb_exceptions.PageNotFound(self.url)
PageNotFound: /brewers/deschutes-brewery/233/
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): www.ratebeer.com
requests.packages.urllib3.connectionpool: DEBUG: "GET /brewers/deschutes-brewery/233/ HTTP/1.1" 200 24001
--------------------- >> end captured logging << ---------------------
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.