petrpatek / airbnb-scraper Goto Github PK

Hello!

Firstly, thank you for making this script available for free.

I'm having trouble running the program with the "location query option".

Here are my settings:

{
  "locationQuery": "London",
  "currency": "USD",
  "includeReviews": true,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

And I get a bunch of errors like the following:

2020-02-04T22:58:04.202Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://api.airbnb.com/v2/search_results?location=London&price_min=3500&price_max=4000&_limit=10","retryCount":2,"id":"iASSDmkQQcFPTIU"}
2020-02-04T22:58:04.203Z   TypeError: Cannot read property 'body' of undefined
2020-02-04T22:58:04.204Z     at getData (/home/myuser/src/index.js:68:29)
2020-02-04T22:58:04.205Z     at process._tickCallback (internal/process/next_tick.js:68:7)
2020-02-04T22:58:04.207Z ERROR: GetData error
2020-02-04T22:58:04.208Z   Response code 404 (Not Found)

It would be great if you could help me out!

Thank you.

Hi Petr
I cannot find the accommodation price for each listing in the results , only the 'priceDetails' 'attributetype' and these appear to be 'extras'. Is there a column / element with the listing's price?

1. Does not work with default input
   When you run it as anonymous user without changing anything it fails, because currency is not set.

2. Currencies in input schema are not sorted (or at least pseudo sorted)
   Would be even better if you had currency codes and currency names as labels

3. The log messages could be improved
   Currently most visible log messages are only "Saving home detail" which says absolutely nothing to the user, only that it does something.
   There should either be some ID (price range + listing number?) or title loaded from the page, so that user can see that it's loading different pages.

4. Check out date label in input schema has a type "Check ut date"

5. The default input has too many results, it shoudl idealy run for max 5 minutes, so that the user can see the results without having to abort it (or wasting all his free CU).

6. The number inputs in input schema has no validation (at least they shoudl have minValue set to 0)

7. Check in and check out dates should have pattern validation and expected date format in help text.

8. Why is includeReviews required in the schema? You can just expect it to be false if it's not set.

9. Why are you using Apify 13.3?

10. This is just an option, maybe you can require the input schema and read the currencies from there, so that you have just once place with the large array.

11. MIN_PRICE and MAX_PRICE could be renamed to DEFAULT_MIN_PRICE and DEFAULT_MAX_PRICE since they are not an actual limit, just default value. Default limit and offset could also be in constants.js

12. You should validate that URLs provided to startUrls are actualy for airbnb.

13. The readme should mention that startUrls and locationQuery are mutualy exclusive

And that is all from me, the code looks good.