about this function: https://github.com/oggioniale/ReLTER/blob/07b2c9c2a7760adc6310f5b06fd6ffb8c411ce54/R_newFunctions/todo_getDataset.R

In the case of LTER Zöbelboden, Austria, Air chemistry, 2012 the xsl is comprensive of more than one tabs.
By B2Share API is not possible to reach which is the one with the data.

I'd like to suggest a new function to acquire MODIS layers (relying on the MODIStsp packatge) and clip DEIMS site boundaries. Can I do a PR to the dev branch for this?

about this function: https://github.com/oggioniale/ReLTER/blob/07b2c9c2a7760adc6310f5b06fd6ffb8c411ce54/R_newFunctions/todo_getDataset.R

In the case of Zöbelboden site (e.g. LTER Zöbelboden, Austria, Foliage chemistry, 2019) the landing page link in the "B2Share Landing Page" element in the API of record is the DOI provided by B2Share and, following this, is not possible to reach the URI of the landing page of B2share where downloading the file.

These are no longer maintained so we recommend not using it (you should be able to replace their use with httr(2) and xml2). https://devguide.ropensci.org/building.html#recommended-scaffolding

Some sites throw a warning in the getSiteBoundaries() function:

deimsid = "https://deims.org/79d6c1df-570f-455f-a929-6cfe5c4ca1e9"   # Zone Atelier Alps
 b <- getSiteBoundaries(deimsid=deimsid) 
 Imported 1 records. Simplifying...
Warning messages:
1: In biomeColor$geoBonBiome == geoBonBiome :
  longer object length is not a multiple of shorter object length
2: In biomeColor$geoBonBiome == geoBonBiome :
  longer object length is not a multiple of shorter object length

I think this is due to multiple colors in the site attributes:

geoBonBiome <- jsonlite::fromJSON(
    paste0(
      "https://deims.org/",
      "api/sites/",
      sub("^.+/", "", deimsid)
    )
  )$attributes$environmentalCharacteristics$geoBonBiome

geoBonBiome
[1] "Fresh water lakes" "Terrestrial" 
 

Two suggestions:

1. Can we define biomeColor$geoBonBiome Capitalized and with spaces as in DEIMS, so that the strings match?
2. Perhaps add a [] subset in the color definition lines 52, 53:

  color <- biomeColor$fill[biomeColor$geoBonBiome == geoBonBiome[1]]
  colorBorder <- biomeColor$border[biomeColor$geoBonBiome == geoBonBiome[1]]

• use the new logo
• change also the thumbnails
• use the function described here https://usethis.r-lib.org/reference/use_logo.html

We currently have a repo of almost 300MB. (!)
About 210MB are images in the both the ./man and .docs directories. Some of these images are 15-25 MB each. Do we really need such high quality images?

I can run a quick convert -resize (from ImageMagick) on those to make them smaller.

THese functions should run with no parameter and return a tibble for all DEIMS sites. However the sitenum default value is set to NULL instead of NA, then the test for sitenum fails:

> getILTERParameters()
Error in if (is.na(sitesNum)) { : argument is of length zero
> getILTERGeneralInfo()
Error in if (is.na(sitesNum)) { : argument is of length zero

The default sitenum value in function definition should be NA (instead of NULL)

> getILTERGeneralInfo(NA)
 Imported 1 records. Simplifying...
 Imported 1 records. Simplifying...
 Imported 1 records. Simplifying...
 Imported 1 records. Simplifying...
 .....

For info @oggioniale, these might be useful when reducing the number of Imports.

• https://recology.info/2018/10/limiting-dependencies/
• https://www.jimhester.com/talk/2019-rsc-deps/

about this function: https://github.com/oggioniale/ReLTER/blob/07b2c9c2a7760adc6310f5b06fd6ffb8c411ce54/R_newFunctions/todo_getDataset.R

If the file shared by B2Share is a CSV (e.g. LTER Northern Adriatic Sea (Italy) marine data from 1965 to 2015).
There is no information as to which character separates the different columns.
Once downloaded into R this cannot be parsed or trasformed in a Dataset or Tibble.

Currently we use two different packages for displaying maps: leaflet and tmap. The leaflet package is used in 7 functions, and tmap in only two. In an effort to reduce package dependencies, we should focus on only one of these.

Last year I added tmap since I needed to display raster images (i.e. in the get_site_ODS() function), and leaflet is a bit troublesome and tricky with raster data. However now there is a new function in terra (which we need installed anyway) called plet (yes, that's the spelling 😃 ) which displays a terra::SpatRaster with leaflet. This function requires leaflet >= 2.1.1. That minimum version is already specified in the DESCRIPTION file.

So I suggest to change the functions that currently use tmap over to leaflet. And remove the dependency on tmap.

In produce_site_map(), neither ISO countryCode, nor listOfSites should be required.
if no ISO country code is supplied, the function should just plot the site boundary, without country boundary. If no listOfSites is supplied, again plot the site boundary without points for other sites.

This package http://docs.ropensci.org/deposits/ by @mpadge interfaces services like Zenodo. Now I'm not sure whether it covers exactly your use case for Zenodo as I don't know neither ReLTER nor deposits well enough!

@maelle @micha-silver
I solved most of the problems in the packages and a new version of this is deployed in the DEV branch.
The test-coverage (https://github.com/ropensci/ReLTER/actions/runs/7460327719/job/20298178183) fails because the dependency ‘MODIStsp’ is not available for package ‘ReLTER’.
I try to solve this by installing the MODISTSP package from remote

- name: Install remotes
        run: |
          Rscript -e "install.packages('remotes', repos='https://ftp.belnet.be/mirror/CRAN')"

      - name: Install MODIStsp
        run: |
          Rscript -e "remotes::install_url('https://cran.r-project.org/src/contrib/Archive/MODIStsp/MODIStsp_2.1.0.tar.gz')"

because from 2023-12-29 the package is not included in the CRAN (https://cran.r-project.org/web/packages/MODIStsp/index.html).

Any ideas?

this is the code:

insects <- data.frame(
   taxonID = c(1, 2, 3, 4, 5, 6),
   family = c(
     "Alexiidae", "Anthicidae",
     "Anthribidae", "Anthribidae",
     "Biphyllidae", "Brentidae"
   ),
   scientificName = c(
     "Sphaerosoma seidlitzi", "Endomia tenuicollis tenuicollis",
     "Anthribus fasciatus", "Phaenotherion fasciculatum fasciculatum",
     "Diplocoelus fagi", "Holotrichapion (Apiops) pisi"
   )
)
output <- ReLTER::taxon_id_pesi(
  table = insects,
  taxaColumn = 3
)
output

the output return also the check of first row of the input table

taxonID<dbl> | family<chr> | originalNameUsage<chr> | scientificName<chr> |  
-- | -- | -- | -- | --
1 | Alexiidae | Sphaerosoma seidlitzi | Sphaerosoma seidlitzi |  
2 | Anthicidae | Endomia tenuicollis tenuicollis | Sphaerosoma seidlitzi |  
3 | Anthribidae | Anthribus fasciatus | Sphaerosoma seidlitzi |  
4 | Anthribidae | Phaenotherion fasciculatum fasciculatum | Sphaerosoma seidlitzi |  
5 | Biphyllidae | Diplocoelus fagi | Sphaerosoma seidlitzi |  
6 | Brentidae | Holotrichapion (Apiops) pisi | Sphaerosoma seidlitzi

Following this rspatial blog post we should get organized to remove dependencies on sp. raster, and rgdal. Functions using these should be changed to use instead sf and terra.

since the docs are built by rOpenSci (well #62). Happy to answer questions!

When running getSiteBoundaries I am getting this error:

> deimsid
[1] "https://deims.org/d0a8da18-0881-4ebe-bccf-bc4cb4e25701"
> getSiteBoundaries(deimsid)   
 mapview::mapshot(map, file = paste0("sites_", gsub(" ", 
+                                                        "_", boundaries$title), ".png"), append = FALSE)
Auto configuration failed
139863174203328:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:185:filename(libssl_conf.so): libssl_conf.so: cannot open shared object file: No such file or directory
139863174203328:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
139863174203328:error:0E07506E:configuration file routines:MODULE_LOAD_DSO:error loading dso:conf_mod.c:285:module=ssl_conf, path=ssl_conf
139863174203328:error:0E076071:configuration file routines:MODULE_RUN:unknown module name:conf_mod.c:222:module=ssl_conf
Error in (function (url = NULL, file = "webshot.png", vwidth = 992, vheight = 744,  : 
  webshot.js returned failure value: 1

If I step thru the function, everything goes OK until the mapview::mapshot line
Regards, Micha

This new function is a nice addition!
Two comments:

• Can we return an sf object instead of a list? (provided the species occurrences have longitude/latitude). This is to keep the spatial character of ReLTER. Throughout the package we have a "show_map" flag for demonstration of the various functions capabilities. But in a research context, I think users will always want to do further analysis - spatial relationships, etc. So returning a spatial object would make sense here.
• Maybe this is "nitpicking" but could the function name be more understandable: i.e. get_site_speciesOccurances ??

as recommended in https://devguide.ropensci.org/building.html#recommended-scaffolding

Maybe you could use curl::has_internet() with an argument instead.

If you run desc::desc_normalize(), the dependencies in particular will be ordered alphabetically, it will be quite neat when reading the list of Imports.

Loading of ReLTER depends on the github version of waffle. The CRAN version is at 0.7.0 whereas ReLTER requires 1.0.1:

** byte-compile and prepare package for lazy loading
Error in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]) : 
  namespace ‘waffle’ 0.7.0 is being loaded, but >= 1.0.1 is required
Calls: <Anonymous> ... namespaceImportFrom -> asNamespace -> loadNamespace
Execution halted
ERROR: lazy loading failed for package ‘ReLTER’

Is the more recent github version of waffle necessary? If yes, then this needs to be documented in installation section.

For instance

would be

withr::local_envvar("LOCAL_DEIMS" = FALSE)

and

would be deleted as the withr change is only for within the test.

from @allisonhorst
After entering a number to specify the record “most similar to the taxa name” (note: it currently says “mostsimilar”), it isn’t clear that anything happens (but the table does exist as a stored object). Possible to have a message returned after choosing that says “Object created” or something similar, just so the user knows it worked more obviously?

Running get_ilter_generalinfo() outputs:
---- The requested page could not be found.
Please check the DEIMS ID ----

even though it runs successfully.

hohe = get_ilter_generalinfo(country_name = "Austria", site_name = "Hohe Tauern")

---- The requested page could not be found.
            Please check the DEIMS ID ----

class(hohe)
[1] "sf"         "tbl_df"     "tbl"        "data.frame"

str(hohe)
sf [1 × 10] (S3: sf/tbl_df/tbl/data.frame)
 $ title               : chr "Hohe Tauern National Park - Austria"
 $ uri                 : chr "https://deims.org/d6936d5d-e036-4b3d-bf3c-4a8702e82f1b"
 $ geoCoord            :sfc_POINT of length 1; first list element:  'XY' num [1:2] 12.5 47
 $ country             :List of 1
  ..$ : chr "Austria"
 $ geoElev.avg         : int NA
 $ geoElev.min         : int 1000
 $ geoElev.max         : int 3798
 $ geoElev.unit        : chr "msl"
 $ affiliation.networks:List of 1
  ..$ :'data.frame':	2 obs. of  3 variables:
  .. ..$ network :'data.frame':	2 obs. of  2 variables:
  .. .. ..$ name: chr [1:2] "EUROPARC" "LTER Austria"
  .. .. ..$ id  :'data.frame':	2 obs. of  2 variables:
  .. .. .. ..$ prefix: chr [1:2] "https://deims.org/network/" "https://deims.org/network/"
  .. .. .. ..$ suffix: chr [1:2] "ce875203-854b-414b-a4d5-0719aff25622" "d45c2690-dbef-4dbc-a742-26ea846edf28"
  .. ..$ siteCode: logi [1:2] NA NA
  .. ..$ verified: logi [1:2] FALSE FALSE
 $ affiliation.projects:List of 1
  ..$ :'data.frame':	3 obs. of  2 variables:
  .. ..$ label: chr [1:3] "Birds Directive (2009/147/EC)" "Natura 2000" "eLTER (H2020)"
  .. ..$ uri  : chr [1:3] NA NA "https://cordis.europa.eu/project/id/654359"
 - attr(*, "sf_column")= chr "geoCoord"
 - attr(*, "agr")= Factor w/ 3 levels "constant","aggregate",..: NA NA NA NA NA NA NA NA NA
  ..- attr(*, "names")= chr [1:9] "title" "uri" "country" "geoElev.avg" ...

about this function: https://github.com/oggioniale/ReLTER/blob/07b2c9c2a7760adc6310f5b06fd6ffb8c411ce54/R_newFunctions/todo_getDataset.R

In some datasets (e.g. LTER Northern Adriatic Sea (Italy) marine data from 1965 to 2015) the "ePIC_PID" element in the API of record in B2Share has 2 URI. Only one point to the dataset.

How to distinguish between the two (or more) link the one pointing to the dataset?

as it's loaded in tests/testthat.R

Tried to set credentials:

Sys.setenv(earthdata_user = 'jeancaffou',
           earthdata_pass = ',,,')

They seem to be set correctly

> Sys.getenv("earthdata_user")
[1] "jeancaffou"

But when fetching data, I get 401 unauthorized, wrong pass, even though the same credentials work for the login on the website of usgs

> site_modis_lst <- get_site_MODIS(
+   site_id, product = "LST",
+   from_date = "2022.10.01", to_date = "2023.01.30",
+   output_dir = output_dir,
+   output_proj = crs_fr,
+   plot_ts = TRUE)
Acquiring: LST_3band_emissivity_8day_1km (M*D21A2)
2023-02-09 12:00:10 - Download starting.
Request failed [401]. Retrying in 1 seconds...
Request failed [401]. Retrying in 1 seconds...
Request failed [401]. Retrying in 1 seconds...
Request failed [401]. Retrying in 1 seconds...
Error in MODIStsp_download(modislist, proc_opts$out_folder_mod, download_server,  : 
  Username and/or password are not valid. Please provide
             valid ones!

New API are coming. We must be able to adapt our logics to changes.
We can test on pre-production site with different url.
Proposal (partially done):

• create global but adjustable package setting with base deims url (configurable in one's system variable) - done
• use package onLoad function to possibly change the base url variable reading from system. - done
• use a function to read from baseurl/api the deims version - done
• set a global (changeable) package setting to the read version (it's a string). - TODO
• put JQ queries in a variable (list) containing slots for API versions (e.g. jq_queries$'1.0' will be a list).
• adapt each function that contains a jq query to the new pattern: move the jq code into the JQ_queries variable. - STARTED for api version "1.0"
• add jq queries for version "1.1" and test the package against the pre-prod new version of deims - TODO

consider to create a caching mechanism for queries against external services in order to reduce the amount of requests.
This could be done by incapsulating the logics of do_Q and subsequent statements (e.g functions like "get_activity_info") in one function accepting e.g. the id and type of an entity: the function could implement the caching mechanism.
We could also consider packages like rcache to implement the mechanism.

As I'm still examining the package to find what in it is different for pkgdown, I see the dependency on Rdpack. What does it do?

WHen installing, the required version of ggplot2 is
ggplot2 (>= 3.3.5.9000), # In Imports section of DESCRIPTION
but the current version on CRAN is 3.3.5. This causes an error:

> remotes::install_github("oggioniale/ReLTER")
Downloading GitHub repo oggioniale/ReLTER@HEAD
.....
Error in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]) : 
  namespace ‘ggplot2’ 3.3.5 is already loaded, but >= 3.3.5.9000 is required

Is this newer version of ggplot2 necessary?

(Thanks for addressing all issues so quickly! )

@micha-silver
I don't understand the example currently in the get_site_MODIS function

#'  \dontrun{
#'  deimsid = "https://deims.org/86e3a1ca-b2ba-4b06-b096-71447df52841"
#'  product_bands <- MODIStsp_get_prodlayers("M*D13Q1")
#'  print(product_bands[,6:7])
#' }

Please can you specify better the example?

First release:

• usethis::use_cran_comments()
• Update (aspirational) install instructions in README
• Proofread Title: and Description:
• Check that all exported functions have @return and @examples
• Check that Authors@R: includes a copyright holder (role 'cph')
• Check licensing of included files
• Review https://github.com/DavisVaughan/extrachecks

Prepare for release:

• devtools::build_readme()
• urlchecker::url_check()
• devtools::check(remote = TRUE, manual = TRUE)
• devtools::check_win_devel()
• rhub::check_for_cran()
• Review pkgdown reference index for, e.g., missing topics
• Draft blog post

Submit to CRAN:

• usethis::use_version('minor')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• usethis::use_github_release()
• usethis::use_dev_version()
• Finish blog post
• Tweet
• Add link to blog post in pkgdown news menu

In the function getSiteAffiliations there is use of the pipe operator %>%. However dplyr is not loaded, so the operator is not available.

> library(ReLTER)
Registered S3 methods overwritten by 'stars':
  method             from
  st_bbox.SpatRaster sf  
  st_crs.SpatRaster  sf  
ReLTER is specially drafted for the LTER community.

Contribute to the improvement of it, join the group of developers
    (https://github.com/oggioniale/ReLTER).

If you use this package, please cite it.
for references type 'citation('ReLTER')'.

> getSiteAffiliations("https://deims.org/d0a8da18-0881-4ebe-bccf-bc4cb4e25701")
Error in jj %>% jqr::jq(as.character("{status: .errors.status}")) %>%  : 
  could not find function "%>%"
> 

Perhaps just put: @import dplyr into the header section?

Please, remove the rgeos package from the DESCRIPTION file, it was taken down from the CRAN and it seems it is not used anymore since it was deprecated in favour of sf and terra packages.

@oggioniale

Quitting from lines 20-43 [listContacts] (sites_contacts.Rmd)
                                                                                                         
Error in `map()`:
ℹ In index: 1.
Caused by error in `dplyr::select()`:
! Can't subset columns that don't exist.
✖ Column `generalInfo.siteManager` doesn't exist.
Backtrace:
  1. ... %>% unique()
 14. dplyr:::select.data.frame(., title, uri, generalInfo.siteManager, geoCoord)
Execution halted

about this function: https://github.com/oggioniale/ReLTER/blob/07b2c9c2a7760adc6310f5b06fd6ffb8c411ce54/R_newFunctions/todo_getDataset.R

In the case of all datasets provided by Mar Piccolo di Taranto site (e.g. Abundance of E. coli, coliforms and Vibrio in water, sediments and mussels from the Mar Piccolo of Taranto 2007-2009) the metada cointain B2share DOI but is not in the right or same field. It is shared in the "onlineDistribution" element in the API of record but are written like this:

onlineDistribution": {
     "dataPolicyUrl": null,
     "doi": "http://doi.org/10.23728/b2share.d0f6225e78124be5b14562f90323e064",
     "onlineLocation": null
}

However, #4