Comments (4)
Try this:
library(crul)
url = "https://github.com/ycphs/openxlsx/blob/master/inst/extdata/namedRegions.xlsx?raw=true"
cc <- Async$new(url = url)
res <- cc$get()
class(res[[1]]$content)
writeBin(res[[1]]$content, (f=tempfile(fileext=".xlsx")))
openxlsx::read.xlsx(f)
the data in $content
is of type raw
, you can write it to a file first, then read with openxlsx
let me know if that works for yoyu
from crul.
Thank you very much.
Ultimately I'm trying to adapt this from Async
-> AsyncQueue
and the url example you provided works when I try it for AsyncQueue
like this:
> reqlist <- list(
HttpRequest$new(url = "https://github.com/ycphs/openxlsx/blob/master/inst/extdata/namedRegions.xlsx?raw=true")$get(),
HttpRequest$new(url = "https://github.com/ycphs/openxlsx/blob/master/inst/extdata/namedRegions.xlsx?raw=true")$get(),
HttpRequest$new(url = "https://github.com/ycphs/openxlsx/blob/master/inst/extdata/namedRegions.xlsx?raw=true")$get()
)
> out <- AsyncQueue$new(.list = reqlist, req_per_min=3)
> out$request() # make requests
> out$responses()[[1]]
<crul response>
url: https://raw.githubusercontent.com/ycphs/openxlsx/master/inst/extdata/namedRegions.xlsx
request_headers:
response_headers:
status: HTTP/2 200
cache-control: max-age=300
content-security-policy: default-src 'none'; style-src 'unsafe-inline'; sandbox
content-type: application/octet-stream
etag: W/"485760db4f84748f4d4cfa4ea6fa6b5711ca08f7adf6424667f42f171301ee35"
strict-transport-security: max-age=31536000
x-content-type-options: nosniff
x-frame-options: deny
x-xss-protection: 1; mode=block
x-github-request-id: 7CA6:7A92:39526:ACBFC:60F05459
accept-ranges: bytes
date: Thu, 15 Jul 2021 16:12:46 GMT
via: 1.1 varnish
x-served-by: cache-sjc10030-SJC
x-cache: HIT
x-cache-hits: 1
x-timer: S1626365567.904596,VS0,VE0
vary: Authorization,Accept-Encoding
access-control-allow-origin: *
x-fastly-request-id: 0e888ce59af5c93de05cbe1bc3dbdd42cf2c1df3
expires: Thu, 15 Jul 2021 16:17:46 GMT
source-age: 0
content-length: 8209
status: 200
> class(out$content()[[1]])
[1] "raw"
> writeBin(out$content()[[1]], (f=tempfile(fileext=".xlsx")))
> openxlsx::read.xlsx(f)
This.is.C2 X2
1 Col1 Col2
2 Data11 Data12
3 Data21 Data22
4 Data31 Data32
However, when I try it on my URL(s) only the Async
method works but not the AsyncQueue
method? (I get a 504?)
Works
> url <- "https://api.targetsafety.info/api/alertcenter/dataset/param?from_date=2021-04-03&to_date=2021-06-02&version=63973&token=[TOKEN]"
> cc <- Async$new(url = url)
> res <- cc$get()
> class(res[[1]]$content)
[1] "raw"
> writeBin(res[[1]]$content, (f=tempfile(fileext=".xlsx")))
> openxlsx::read.xlsx(f)
Target.ID Target Action Genetic.Studies/Variants Genetic.Studies/Variants.(literature.term) Uniprot.ID Gene
1 5 ALK Inhibitors <NA> <NA> Q9UM73 ALK
2 5 ALK Inhibitors <NA> <NA> Q9UM73 ALK
...
Doesn't Work
> reqlist <- list(
HttpRequest$new(url = "https://api.targetsafety.info/api/alertcenter/dataset/param?from_date=2021-04-03&to_date=2021-06-02&version=63973&token=[TOKEN]")$get(),
HttpRequest$new(url = "https://api.targetsafety.info/api/alertcenter/dataset/param?from_date=2021-02-20&to_date=2021-04-03&version=63973&token=[TOKEN]")$get(),
HttpRequest$new(url = "https://api.targetsafety.info/api/alertcenter/dataset/param?from_date=2021-01-01&to_date=2021-02-20&version=63973&token=[TOKEN]")$get()
)
> out <- AsyncQueue$new(.list = reqlist, req_per_min=3)
> out$request() # make requests
> out$responses()[[1]]
<crul response>
url: https://api.targetsafety.info/api/alertcenter/dataset/param?from_date=2021-04-03&to_date=2021-06-02&version=63973&token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjAwMDE2IiwibmFtZSI6IkdlbmVudGVjaCIsImRlcGFydG1lbnQiOiJHZW5lbnRlY2ggQVBJIn0.p01jxx7PekOUQLOGjvlhR3YWP2GdcD7_r6MM515bYmM
request_headers:
response_headers:
status: HTTP/2 504
server: awselb/2.0
date: Thu, 15 Jul 2021 16:25:36 GMT
content-type: text/html
content-length: 132
params:
from_date: 2021-04-03
to_date: 2021-06-02
version: 63973
token: [TOKEN]
status: 504
> class(out$content()[[1]])
[1] "raw"
> writeBin(out$content()[[1]], (f=tempfile(fileext=".xlsx")))
> openxlsx::read.xlsx(f)
Error in file(con, "r") : invalid 'description' argument
In addition: Warning message:
In unzip(xlsxFile, exdir = xmlDir) : error 1 in extracting from zip file
Please let me know if you'd like DM privately to debug so I could share the token 💁🏼
from crul.
Sorry for the long delay. Yeah, can you send me the token so I can try to debug it? You can use the email associated with this pkg
from crul.
I get those 504 errors, but I do with every request. I think what's happening is intermittent server downtime, resulting in the 504s. I doubt this has anything to do with Async vs. AsyncQueue. Unless you can make a reproducible example.
By the way, you're token is above in #161 (comment), you should get a new one
from crul.
Related Issues (20)
- funky times headers should warn instead of error HOT 6
- Add HttpClient tests for new number formatting
- Update docs telling users about number formatting for query params
- POSTing GraphQL Mutations? HOT 4
- How do you avoid exceeding the rate limit when making API calls? HOT 11
- S3 method for async ``$responses` method
- consider making `AsyncQueue` non-blocking
- How to retry requests for AsyncQueue? HOT 4
- Add support for pagination for Async calls HOT 3
- `Error in nchar(hh) : invalid multibyte string` when response has Latin1 headers HOT 1
- Async$get fails with non default disc param HOT 3
- server certificate verification failed. CAfile: none CRLfile: none HOT 2
- Integrate progress bars for Async requests HOT 3
- Number of parallel requests with AsyncVaried HOT 9
- use cookies from browser HOT 1
- tweak tests/egs to make cran happy HOT 1
- rgbif users reporting that "HTTP/2 stream was not closed cleanly before end of the underlying stream" HOT 4
- Fix errors in cran checks
- Add _PACKAGE thingy for pkg level doc
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crul.