arawles / bmrsr Goto Github PK

When service_type = "csv", a tibble is returned with the first column having two levels, even though only one value is present.

There are currently no type checks or converstion on input parameters for the build_x_call functions.
Many of the parameters will be characters, but some (like perhaps "settlement_period") should really accept a numeric or character. Similarly, there are many temperamental datetime/date/time parameters that should have some form of check/conversion to ensure they are in the correct format for the request.

Instead of returning response codes in the header of the response, they are in the body, with the header always returning 200 (as long as the request is received). Checking of the response should therefore accomodate for this; checking the content for the response code rather than the header.

Rather than relying on paste0 calls, the URLs should be built using httr as a URL object

Any column returned with date, time, or datetime values is being treated as a factor rather than as a date. It's not necessarily the factor aspect that's the issue, but rather just that they are not being returned as the correct value.

This makes graphing and automated analysis difficult.

Currently, BMRSr has a fairly long list of dependencies, made even bigger with the latest CRAN release.

I want to keep an eye on the dependencies to make sure they don't spiral out of control. This issue acts as a persistent reminder.

When attempting to build calls for data items that are provided in lower case (e.g. "temp"), the function returns "data item not valid" error rather than auto-converting to upper case.

Develop the get_parameters function, currently located in the utility.R file to return a list of the required/allowed parameters for a particular data item. This will likely require going through the documentation and creating a named list with the data items and parameters.

There are multiple places where data item checks take place (if (data_item %in% etc.).

This should be a centralised function/set so that a typo or issue doesn't need to be updated in multiple places.

Not all .csv files are returned in the same format. Currently, the parser assumes that there the file still contains the annoying "//EOF" tag and starts with 4 lines of rubbish.

As someone has pointed out though, with data items like "B1610", the file is returned with a single line with the data item and then the data starting from line 2.

To compensate, the parser will require a more flexible structure.

Currently, all the allowed parameters are prescribed for each build call. But it might be better to allow arbitrary parameters to be added using ... to protect against the case that an additional parameter gets added to the API and this package needs to updated - the build calls can still be used until the package is updated this way.

@ARawles
Just a thought ...

NOW: Inside each function, each parameter is checked one-by-one for being NULL or properly formatted.

Any thoughts on using switch or dplyr::case_when() to make the code a little cleaner?

Thx.

DISBSAD is mispelt DISBAD.

This means that the request always fails, because it can't get the data item type or rename the columns.

Some of the columns returned are in slightly different formats, the full request function then fails when actually it has a return tibble, it just doesn't know what to do with a certain and so fails.

Two solutions:

1. Implement tryFormats to try multiple formats for the columns.
2. Implement a try() statement so that the tibble is still returned (in an albeit unformatted format), rather than not returned at all.

The documentation for the functions does not follow a consistent format. All parameters should be detailed as "(parameter) (type); (description)".

Currently, because the build_call function uses an elipsis (...) to pass a list of requirements to the appropriate build_x_call function, the parameters required do not autocomplete when using RStudio.

Hi, i am using the BMRSr package to query many things, I last used it a few months ago (June 2021?) however those same queries do not work, the principle error appears to be the time format but i am not sure;

(i'm using the development version devtools::install_github("ARawles/BMRSr")' )

full_request(data_item = "FUELINST",
             api_key = my.bmrs.key,
             from_datetime = "01 07 2019 00:00:00",
             to_datetime = "03 07 2019 00:00:00",
             service_type = "csv",
             parse=T,
             clean_dates = TRUE)

gives

$response
$response$responseMetadata
$response$responseMetadata$httpCode
$response$responseMetadata$httpCode[[1]]
[1] "400"


$response$responseMetadata$errorType
$response$responseMetadata$errorType[[1]]
[1] "Bad request"


$response$responseMetadata$description
$response$responseMetadata$description[[1]]
[1] "Either FromDateTime Parameter is not in proper format (yyyy-MM-dd HH:mm:ss) or is invalid"


$response$responseMetadata$queryString
$response$responseMetadata$queryString[[1]]
[1] "FromDateTime=2019-07-01%2000:00:00,ToDateTime=2019-07-03%2000:00:00"


Warning message:
csv requested, xml returned. Error code within response = 400 

the only thing i have tried is to change the datetime to;

from_datetime = "2019-07-01 00:00:00",
to_datetime = "2019-07-03 00:00:00",

as per the API guide pdf, which gives exactly the same error message. Could be something at BMRS's end