ropensci / datefixr Goto Github PK

All versions of datefixR (CRAN, ROpenSci, and Github) do not seem to be working on Windows machines running R 4.3.0 or R 4.3.1. I have tried on both Windows 10 (build 19044.3086) and a clean Windows 11 (build 22000.2057). Any function from the package freezes the R session.

At the same time, the Linux version works fine.

Windows 10 sessionInfo

R version 4.3.0 (2023-04-21 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=English_New Zealand.utf8  LC_CTYPE=English_New Zealand.utf8
[3] LC_MONETARY=English_New Zealand.utf8 LC_NUMERIC=C
[5] LC_TIME=English_New Zealand.utf8

time zone: Pacific/Auckland
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] datefixR_1.4.1.9000

loaded via a namespace (and not attached):
[1] compiler_4.3.0  cli_3.6.1       tools_4.3.0     rstudioapi_0.14 Rcpp_1.0.10
[6] lifecycle_1.0.3 rlang_1.1.1

Ubuntu 20.04 sessionInfo

 > sessionInfo()
R version 4.3.0 (2023-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3.10.3
LAPACK: /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3.10.3;  LAPACK version 3.9.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C

time zone: UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] datefixR_1.4.1.9000

loaded via a namespace (and not attached):
 [1] compiler_4.3.0  magrittr_2.0.3  cli_3.6.1       tools_4.3.0
 [5] glue_1.6.2      rstudioapi_0.14 Rcpp_1.0.10     vctrs_0.6.3
 [9] stringi_1.7.12  stringr_1.5.0   lifecycle_1.0.3 rlang_1.1.1

fix_date_char() throws an error on ddmmyy dates.

library(datefixR)
fix_date_char("140922")
#> Error in .separate_date(date): object 'date_vec' not found

# compare

fix_date_char("14-09-22")
#> [1] "2022-09-14"

datefixR installed today from default branch

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23 ucrt)
#>  os       Windows 10 x64 (build 19044)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language en
#>  collate  German_Germany.utf8
#>  ctype    German_Germany.utf8
#>  tz       Europe/Berlin
#>  date     2022-09-14
#>  pandoc   2.19.2 @ C:/PROGRA~1/Pandoc/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date (UTC) lib source
#>  cli           3.3.0      2022-04-25 [1] CRAN (R 4.2.0)
#>  datefixR    * 1.1.0.9000 2022-09-14 [1] local
#>  digest        0.6.29     2021-12-01 [1] CRAN (R 4.2.0)
#>  evaluate      0.16       2022-08-09 [1] CRAN (R 4.2.1)
#>  fansi         1.0.3      2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap       1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>  fs            1.5.2      2021-12-08 [1] CRAN (R 4.2.0)
#>  glue          1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  highr         0.9        2021-04-16 [1] CRAN (R 4.2.0)
#>  htmltools     0.5.3      2022-07-18 [1] CRAN (R 4.2.1)
#>  knitr         1.40       2022-08-24 [1] CRAN (R 4.2.1)
#>  lifecycle     1.0.1      2021-09-24 [1] CRAN (R 4.2.0)
#>  magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>  pillar        1.8.1      2022-08-19 [1] CRAN (R 4.2.1)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>  purrr         0.3.4      2020-04-17 [1] CRAN (R 4.2.0)
#>  R.cache       0.16.0     2022-07-21 [1] CRAN (R 4.2.1)
#>  R.methodsS3   1.8.2      2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo          1.25.0     2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils       2.12.0     2022-06-28 [1] CRAN (R 4.2.1)
#>  reprex        2.0.2      2022-08-17 [1] CRAN (R 4.2.1)
#>  rlang         1.0.5      2022-08-31 [1] CRAN (R 4.2.1)
#>  rmarkdown     2.16       2022-08-24 [1] CRAN (R 4.2.1)
#>  rstudioapi    0.14       2022-08-22 [1] CRAN (R 4.2.1)
#>  sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>  stringi       1.7.8      2022-07-11 [1] CRAN (R 4.2.1)
#>  stringr       1.4.1      2022-08-20 [1] CRAN (R 4.2.1)
#>  styler        1.7.0      2022-03-13 [1] CRAN (R 4.2.0)
#>  tibble        3.1.8      2022-07-22 [1] CRAN (R 4.2.1)
#>  utf8          1.2.2      2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs         0.4.1      2022-04-13 [1] CRAN (R 4.2.0)
#>  withr         2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>  xfun          0.32       2022-08-10 [1] CRAN (R 4.2.1)
#>  yaml          2.3.5      2022-02-21 [1] CRAN (R 4.2.0)
#> 
#>  [1] C:/Users/Daniel.AK-HAMBURG/AppData/Local/R/win-library/4.2
#>  [2] C:/Program Files/R/R-4.2.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Lines 28-36,

  as.Date(sapply(dates,
    .fix_date_char,
    day.impute = day.impute,
    month.impute = month.impute,
    format = format,
    USE.NAMES = FALSE
  ),
  origin = "1970-01-01"
  )

Could be rewritten as,

  as.Date(
    vapply(
      dates,
      .fix_date_char,
      day.impute = day.impute,
      month.impute = month.impute,
      format = format,
      FUN.VALUE = numeric(length(dates)),
      USE.NAMES = FALSE
    ),
    origin = "1970-01-01"
  )

Because using vapply() is safer and faster than using sapply().

See #49

Currently it reads:

"datetime"). However datetime formats are not supported by datefixR. The current

Elsehwere you use datefixR to format the package's name in the README.

Interested in adding translations for warnings, errors and messages in your language? Please register your interest on this issue! I will then get your language set up on GitLocalize and you can easily add your translations (no knowledge of translating software is required). I will also attribute you as a translator in the package DESCRIPTION file.

The warnings and errors can be given in German by making edits to the the German translation file. Translations (msgstr) need to be provided for each English line (msgid). Running the below R code should then update the package to use these translations.

tools::update_pkg_po(".")

The warnings and errors can be given in Portuguese by making edits to the the Portuguese translation file. Translations (msgstr) need to be provided for each English line (msgid). Running the below R code should then update the package to use these translations.

tools::update_pkg_po(".")

Recently, we have been focusing on localising the package by allowing it recognise months and date formats in other languages. At present we have focused on French, German, Spanish and Portuguese, but can add more languages if there is interest (and probably offers of assistance).

Unfortunately, I only actually know English so the package would likely benefit greatly from help from native speakers to make sure the package works as it should. Here’s a couple of ways you could help:

1. Just download the most up-to-date version of datefixR from R-universe/GitHub and try it out with dates you would expect to come across in your language of choice! If there's any common types of dates it can't recognise, please create an issue for it.
2. Create some tests! If you could add some tests to test_languages.R if you feel dates in your language aren't properly tested that would be even better!
3. I am also hoping to have translations for the warnings and errors the package gives. If this is something you are interested in, please take a look at the po directory in the devel branch, find the file for your language, and contribute changes. msgid gives the English version for each warning/error and msgstr is where you can contribute a translation. These translations have their own issues: #41 #43

If you help in any of these ways, I would be very happy to add you to the package description (role = c(“ctb”, “trl”); contributor and translator)

fix_date_char("03/10/90") works but fix_date_char("3/10/90") does not

Would it be possible to include dates where the month is given as a roman numeral, please?

E.g. 13/vi/2021 would be 2021-06-13

This format is used in some database systems (https://database.guide/how-to-format-the-month-in-roman-numerals-in-oracle/) and by some biological recorders.

Since the introduction of multi-byte characters, a warning has been issued to the user when datefixR is used in a locale which does not support multi-byte characters. Unfortunately, this warning was introduced after previous translations, so it is not currently translated.

GitLocalize has now been set up which makes it much easier to do translations for datefixR. If you are interested in making a translation contribution, check out the GitLocalize page for this package (requires GitHub login)
https://gitlocalize.com/repo/8364

As before, I am happy to acknowledge contributors as translators in DESCRIPTION (role = "trl")

Status by language

• DE
• ES
• FR

Just a random thought ... The {gt} package has a Twitter account https://twitter.com/gt_package, which is really useful to keep up to date with new releases. Would it be something to consider for datefixR ?

You mention lubridate::parse_date_time() and anytime::anydate() (and the apparently abandoned {linelist})in the README, but there is more prior art in date parsing than those two.

parsedate::parse_date()

library(datefixR)
library(parsedate)

parse_date(exampledates$some.dates)
#> [1] "1992-05-02 UTC" "2020-01-04 UTC" "1996-05-01 UTC" "2020-05-01 UTC"
#> [5] "1996-02-04 UTC"

parse_date() can handle all some.more.dates individually, but somehow trips up with the vector.

parse_date(c("2015", "02/05/00", "05/1990", "jan 2020"))
#> [1] "2015-01-01 UTC" "2000-02-05 UTC" "1990-01-05 UTC" "2020-01-01 UTC"
parse_date("2012-08")
#> [1] "2012-08-01 UTC"

I expect it to be faster than datefixR, because it uses C code from the git date parser. It offers fewer customisation options compared to datefixR.

readr::parse_date()

library(readr)
#> 
#> Attaching package: 'readr'
#> The following object is masked from 'package:parsedate':
#> 
#>     parse_date

parse_date(exampledates$some.dates)
#> Warning: 4 parsing failures.
#> row col   expected      actual
#>   1  -- date like  02 05 92   
#>   2  -- date like  01-04-2020 
#>   4  -- date like  2020-may-01
#>   5  -- date like  02-04-96
#> [1] NA           NA           "1996-05-01" NA           NA
parse_date(exampledates$some.more.dates)
#> Warning: 5 parsing failures.
#> row col   expected   actual
#>   1  -- date like  2015    
#>   2  -- date like  02/05/00
#>   3  -- date like  05/1990 
#>   4  -- date like  2012-08 
#>   5  -- date like  jan 2020
#> [1] NA NA NA NA NA

This is a bit unfair, because one should define locale and a format.

parse_date("01/02/2010", "%d/%m/%Y")
#> [1] "2010-02-01"
parse_date("01/02/2010", "%m/%d/%Y")
#> [1] "2010-01-02"
parse_datetime("1 enero 2015", "%d %B %Y", locale = locale("es"))
#> [1] "2015-01-01 UTC"
parse_datetime("14 Maart 1962", "%d %B %Y", locale = locale("nl"))
#> [1] "1962-03-14 UTC"

clock::date_parse()

clock::date_parse() is very similar to readr::parse_date().

library(clock)

date_parse("January 5, 2020", format = "%B %d, %Y")
#> [1] "2020-01-05"
# but
date_parse("July 4th, 2020", format = "%B %d, %Y")
#> Warning: Failed to parse 1 string at location 1. Returning `NA` at that
#> location.
#> [1] NA

date_parse("3. Februar 2020", format = "%d. %B %Y", locale = clock_locale("de"))
#> [1] "2020-02-03"

stringi::stri_datetime_parse()

library(stringi)

Again, one needs to define the format

x <- c('2015-02-28', '2015-02-29')
stri_datetime_parse(x, 'yyyy-MM-dd')
#> [1] "2015-02-28 19:17:35 CET" NA
stri_datetime_parse(x, 'yyyy-MM-dd', lenient=TRUE)
#> [1] "2015-02-28 19:17:35 CET" "2015-03-01 19:17:35 CET"
stri_datetime_parse("14 Maart 1962", 'date_long', locale = 'nl')
#> [1] "1962-03-14 19:17:36 CET"
# alternatively
stri_datetime_parse("14 Maart 1962", 'dd MMMM yyyy', locale = 'nl')
#> [1] "1962-03-14 19:17:36 CET"

# apparently it uses the current date to fill-up incomplete dates
stri_datetime_now()
#> [1] "2022-10-17 19:17:35 CEST"
stri_datetime_parse("Mei 62", 'LLLL yy', locale = 'nl')
#> [1] "1962-05-17 19:17:36 CET"

The requirement to supply a format make readr::parse_date(), clock::date_parse(), and stri_datetime_parse() obviously unsuitable for vectors with mixed dates.

{stringi} builds on the ICU library, as you may know, and contains weekday and months symbols (names) in ~211 locales, see stringi::stri_datetime_symbols(). Both {readr} and {clock} use these internally to be able to parse international dates (which also explains, why they all behave similarly when parsing dates).

See #49

Update exampledates to include date formats which support has been added for since exampledates was created

The warnings and errors can be given in Spanish by making edits to the the Spanish translation file. Translations (msgstr) need to be provided for each English line (msgid). Running the below R code should then update the package to use these translations.

tools::update_pkg_po(".")

As I'm calculating date ranges, I have been looking at how to use fix_date_df() to impute the last day of the month. Originally I was planning to impute 31 and then altering this for the other months:

df <- datefixR::fix_date_df(df, "date_end", day.impute = 31, month.impute = 12, excel = TRUE, roman.numeral = TRUE) %>%
dplyr::mutate(month = as.numeric(gsub("^.....", "", gsub("...$", "", date_end))),
date_end = dplyr::case_when(!grepl("31$", date_end) ~ date_end, # ignore non-imputed dates
month %in% c(9,4,6,11) ~ gsub("-31", "-30", date_end, fixed = TRUE), = months with 30 days
month == 2 & as.numeric(gsub("\-......$", "", date_end))%% 4 == 0 ~ gsub("-31", "-29", date_end, fixed = TRUE), # leap februaries
month == 2 ~ gsub("-31", "-28", date_end, fixed = TRUE), # other februaries
TRUE ~ date_end))

But this is producing "Error: day.impute should be an integer between 1 and 28"

Could day.impute be expanded to allow 31?

Thank you for offering to help with Russian translations @atsyplenkov (#77)!

The easiest way for you to help with translations is likely via Gitlocalize. If you use the following link, you should be able to provide translations for the two files used to translate warnings/errors into Russian (there's one file for messages raised within R and another for messages raised within C++ code) https://gitlocalize.com/repo/8364/ru

Would you be also interested in helping to add support for Russian dates? I must admit, I am not at all familiar with Russian dates. Would it just be a case of needing to add translations for names of months, or are dates given in different formats to most other languages? For example with French, I needed to add support for "1er" and Spanish required "de" to be supported as a separator for dates (like ".","-","/" etc. are).

See #49

Via GitHub actions (see https://github.com/r-lib/actions/blob/master/examples/pkgdown.yaml)

Update readme and package vignette to cover features added since they were created. Linked to #81.

Stop imputation of day or month by allowing the user to pass NA as an argument for day.impute or month.impute

Parse numeric Unix time or Excel time. Numeric Excel dates are particularly difficult.

Thank you for offering to help with Czech and Slovak translations @MichalLauer (#77)!

The easiest way for you to help with translations is likely via Gitlocalize. If you use the following link, you should be able to provide translations for the files used to translate warnings/errors (there's one file for messages raised within R and another for messages raised within C++ code for each language) https://gitlocalize.com/repo/8364.

Would you be also interested in helping to add support for Czech and Slovak dates? I must admit, I am not at all familiar with dates in these languages. Would it just be a case of needing to add translations for names of months, or are dates given in different formats to most other languages? For example with French, I needed to add support for "1er" and Spanish required "de" to be supported as a separator for dates (like ".","-","/" etc. are)

See progressr

Add support for if date is given as [day] de [month] de [year]. Easiest approach seems to be to treat " de " as a separator like "/", "-" or whitespace.

The following packages are required for the shiny app:

DT,
shiny,
readxl,
htmltools

Ideally we should be able to install datefixR without these dependencies for users who are using datefixR from a secure research environment (e.g a Data Safe Haven) where these packages may not be available. We can then ask users to install the above packages if they are not installed when the shiny app is called for the first time.

Maybe it would be helpful if we extend the statement on the Value returned by fix_date to the following:

An object belonging to R's built in Date class with the following format yyyy-mm-dd

Just to easily understand the format of the output, although this can be seen if we manually map the input to output and see where the date, month and year fall based on the input format argument of fix_date, but I think it would be easy to understand if we have it stated already in the value returned in the docs.

fix_date_char() throws an error if a date with letters contains a . or a ,.

library(datefixR)
# dot with numbers only is fine
fix_date_char("03.10.1990")
#> [1] "1990-10-03"

# but with the month spelled out it throws an error
fix_date_char("3. Oktober 1990")
#> Error in FUN(X[[i]], ...): unable to tidy a date

# with a comma
fix_date_char("July 4th, 1776")
#> Warning in .checkoutput(day, month): NAs introduced by coercion

#> Warning in .checkoutput(day, month): NAs introduced by coercion
#> Error in if (as.numeric(month) > 12 || as.numeric(month) < 1) {: missing value where TRUE/FALSE needed

NB It’s customary to write a date in German with a dot on the line

From that wikipedia article:

the century was sometimes seen being replaced by an apostrophe: “31.12.’91”

IMO that is not very common, but not completely unusual either.

fix_date_char("3.10.'90")
#> Error in charToDate(x): character string is not in a standard unambiguous format
fix_date_char("3. Oktober '90")
#> Error in FUN(X[[i]], ...): unable to tidy a date

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23 ucrt)
#>  os       Windows 10 x64 (build 19044)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language en
#>  collate  German_Germany.utf8
#>  ctype    German_Germany.utf8
#>  tz       Europe/Berlin
#>  date     2022-09-14
#>  pandoc   2.19.2 @ C:/PROGRA~1/Pandoc/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date (UTC) lib source
#>  cli           3.3.0      2022-04-25 [1] CRAN (R 4.2.0)
#>  datefixR    * 1.1.0.9000 2022-09-14 [1] local
#>  digest        0.6.29     2021-12-01 [1] CRAN (R 4.2.0)
#>  evaluate      0.16       2022-08-09 [1] CRAN (R 4.2.1)
#>  fansi         1.0.3      2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap       1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>  fs            1.5.2      2021-12-08 [1] CRAN (R 4.2.0)
#>  glue          1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  highr         0.9        2021-04-16 [1] CRAN (R 4.2.0)
#>  htmltools     0.5.3      2022-07-18 [1] CRAN (R 4.2.1)
#>  knitr         1.40       2022-08-24 [1] CRAN (R 4.2.1)
#>  lifecycle     1.0.1      2021-09-24 [1] CRAN (R 4.2.0)
#>  magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>  pillar        1.8.1      2022-08-19 [1] CRAN (R 4.2.1)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>  purrr         0.3.4      2020-04-17 [1] CRAN (R 4.2.0)
#>  R.cache       0.16.0     2022-07-21 [1] CRAN (R 4.2.1)
#>  R.methodsS3   1.8.2      2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo          1.25.0     2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils       2.12.0     2022-06-28 [1] CRAN (R 4.2.1)
#>  reprex        2.0.2      2022-08-17 [1] CRAN (R 4.2.1)
#>  rlang         1.0.5      2022-08-31 [1] CRAN (R 4.2.1)
#>  rmarkdown     2.16       2022-08-24 [1] CRAN (R 4.2.1)
#>  rstudioapi    0.14       2022-08-22 [1] CRAN (R 4.2.1)
#>  sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>  stringi       1.7.8      2022-07-11 [1] CRAN (R 4.2.1)
#>  stringr       1.4.1      2022-08-20 [1] CRAN (R 4.2.1)
#>  styler        1.7.0      2022-03-13 [1] CRAN (R 4.2.0)
#>  tibble        3.1.8      2022-07-22 [1] CRAN (R 4.2.1)
#>  utf8          1.2.2      2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs         0.4.1      2022-04-13 [1] CRAN (R 4.2.0)
#>  withr         2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>  xfun          0.32       2022-08-10 [1] CRAN (R 4.2.1)
#>  yaml          2.3.5      2022-02-21 [1] CRAN (R 4.2.0)
#> 
#>  [1] C:/Users/Daniel.AK-HAMBURG/AppData/Local/R/win-library/4.2
#>  [2] C:/Program Files/R/R-4.2.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

The warnings and errors can be given in French by making edits to the the French translation file. Translations (msgstr) need to be provided for each English line (msgid). Running the below R code should then update the package to use these translations.

tools::update_pkg_po(".")

At the moment, datefixR uses R's built in month.abb (with "sept" also included) and month.name which I believe always uses English names regardless of locale. It would be great to add months (and their corresponding abbreviations) from other languages. If implemented smartly, it could be made easy to add an arbitrary number of additional languages.

https://github.com/r-lib/asciicast

Refactor tests to use the same format as in #57

This is not strictly necessary but it is documented in section 2.8 here: https://colinfay.me/writing-r-extensions/writing-r-documentation-files.html

Use \R for the R system itself. Use \dots for the dots in function argument lists ‘…’, and \ldots for ellipsis dots in ordinary text.98 These can be followed by {}, and should be unless followed by whitespace.

Dates is often tied up with time, and I often assume that date libraries also caters datetime input with it, which is out of the scope of this package (upon testing it), wouldn't it be useful therefore to add this to the limitations of the package that datefixR does not include date time inputs, e.g. "2021-05-01 08:20:30"? Although, it is clear from the name already and the statement on the purpose of the package, I still assumed it actually, and maybe that's just me, so you can also ignore this if you think it is well stated.

Checks on the ropensci branch indicate "datetime", found in the README, is not in the WORDLIST.