Comments (2)
Thank you for your question. nod()
is intended to "modify the path" not define it anew. In this case you need to bow()
to the new url:
library(polite)
url1 = 'https://seeker.copernicus.org/'
url2 = 'https://seeker.copernicus.org/search.php?abstract=atmospheric+chemistry&startYear=2008&endYear=2020&paperVersion=final&journal=386&page=1'
bow(url1) %>% nod(url2)
#><polite session> https://seeker.copernicus.org/search.php
#> User-agent: polite R package
#> robots.txt: 1 rules are defined for 1 bots
#> Crawl delay: 5 sec
#> The path is scrapable for this user-agent
Having said that, I agree that the error message you are getting is not informative and I should improve it. Will close the issue once the error message is improved.
from polite.
Fair enough. I will adjust my code accordingly.
from polite.
Related Issues (20)
- Release polite 0.1.0 HOT 1
- Future ideas
- Error in polite_download_file HOT 1
- discrepancy between rvest and bow + scrape HOT 8
- polite isn't polite enough? HOT 2
- attempt_get() and attempt_post()
- replace `httr::GET` with `httr::RETRY`
- no encoding supplied message HOT 4
- randomized scrape delay HOT 2
- bow() missing default error in robotstxt HOT 3
- politely should use on.exit to restore HTTPUserAgent HOT 4
- More control over handling non-200 responses when scraping
- bow() timing out on several sites? HOT 1
- Polite doesn't seem to work HOT 1
- Set number of retry attempts
- Cannot install package HOT 1
- Issue w/ Encoding while using Fedora HOT 4
- Release polite 0.1.2
- is_url function does not recognise urls with localhost
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from polite.