It might be a good idea to split them in sections, probably as a few different top-level sections?

Formula is on pp.6-7 of my current notebook

clip currently uses the stride, which might return approximation errors as in #144.

@tpoisot had an idea to avoid using stride completely. Possibly using indices instead of coordinates?

This results in big layers, so it might be a good idea to create one from their datafiles:

https://os.zhdk.cloud.switch.ch/envicloud/chelsa/chelsa_V2/GLOBAL/input/landseamask.mgrd
https://os.zhdk.cloud.switch.ch/envicloud/chelsa/chelsa_V2/GLOBAL/input/landseamask.nc
https://os.zhdk.cloud.switch.ch/envicloud/chelsa/chelsa_V2/GLOBAL/input/landseamask.prj
https://os.zhdk.cloud.switch.ch/envicloud/chelsa/chelsa_V2/GLOBAL/input/landseamask.sdat.aux.xml
https://os.zhdk.cloud.switch.ch/envicloud/chelsa/chelsa_V2/GLOBAL/input/landseamask.sdat
https://os.zhdk.cloud.switch.ch/envicloud/chelsa/chelsa_V2/GLOBAL/input/landseamask.sgrd

This might require to try some different packages to read it...

The core point is to get the bipartite and tripartite code working, which isn't super difficult.

Specifically, need to deal with the colormap code, which isn't super elegant.

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

The two systems use slightly different approaches, and the Fauxcurrence version (using an analytical solution to the first geodetic problem) is faster.

For now, every component is tested independently, so the code coverage is all messed up. This should probably be fixed.

Some GBIF docstrings still refer to next! and complete!, this needs to be corrected.

The weighted interspecific one should be symmetrical

Am I missing a new change to the API or are the files in ./src/io not exported? Is there another way to read/write from disk?

Things like layer1 & (!layer2) would make the creation of masks a lot easier.

Add a layerdescriptionsto SimpleSDMDatasets, which by default would return some sort of mapping from a layer index, code, and human readable description. This function can be exported, and the fallback would be that the usual layer is its own description (this is true for everything except bioclim, for the moment).

The revamped meta-package will need a new logo. Presumably something we can make a sticker out of.

This could be cool to have, alongside a more general way to rasterize shapefiles over a layer

https://ecoregions.appspot.com/

Now that the packages have reached their new equilbrium, the "docstrings" section of the documentation needs to be put back in.

https://journals.ku.edu/jbi/article/view/15483

(this will require the NetCDF driver which we also need for #29)

There is a lot in the tests that is actually relying on SimpleSMDatasest, which needs to be removed. These tests can move in the main repo, and be replaced by mock examples.

Description of the to-do item

Add a new data provider

More information about the item

https://www.nature.com/articles/s41597-020-00669-x#Tab1

The model cards in the model catalogue are a little bit of a mess, it would be a good idea to list the layer names, the years or time intervals, and have the info presented in a less overwhelming way.

Add a test for the fix in #144. This should also test for non-natural values with 0.5 or 0.25 increments

The WorldClim server is less and less reliable, so it would be a good idea to allow this entire test suite to fail, and make a mention of this in the documentation. For the moment, this is not a big issue as none of the examples use the WorldClim data, but this might become one later.

This is another field to allow in the GBIF queryset, and should not be too difficult to do (should default to only presences)

See the doc example, which gives out

SDM response → 450×1088 grid with 489600 Int64-valued cells
  Latitudes	43.18333333333334 ⇢ 43.65208333333334
  Longitudes	-80.00000000000001 ⇢ -70.93333333333335

where it should be

SDM response → 450×1088 grid with 489600 UInt8-valued cells
  Latitudes	43.18333333333334 ⇢ 46.93333333333335
  Longitudes	-80.00000000000001 ⇢ -70.93333333333335

@tpoisot I ran into the following issue with clip while trying to subset a layer for Canada into smaller subregions. The boundary returned for some exact coordinates (e.g. 56.0 as a top boundary) sometimes leaves out one pixel row/column. I'm running into this issue with SpeciesDistributionToolkit v0.0.2 in a temporary environment, but I also had it with SimpleSDMLayers v0.8.3.

Here is an example:

using SpeciesDistributionToolkit

spatialrange = (left = -145.0, right = -50.0, bottom = 40.0, top = 89.0);
layer = SimpleSDMPredictor(RasterData(WorldClim2, BioClim); spatialrange...); # also happens with 2.5 arcmin

# Wrong coordinates while clipping
clip(layer; top=56.0).top # 55.833333333333336
clip(layer; top=56.0 - 0.0000001).top # 56.0
clip(layer; top=56.0 + 0.0000001).top # 56.16666666666667

I would expect the first two clip calls to return 56.0 exactly, otherwise it leaves out one cell row. I tried with other exact coordinates and the problem seems to happen with no specific pattern:

julia> exact_lats = (spatialrange.bottom+1.0):1.0:(spatialrange.top - 1.0)
41.0:1.0:88.0

julia> show([clip(layer; top=l).top for l in exact_lats]) # patterns seems random...
[40.833333333333336, 41.833333333333336, 43.0, 43.833333333333336, 45.0, 46.0, 47.0, 47.833333333333336, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.0, 55.833333333333336, 57.0, 58.0, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 66.83333333333333, 68.0, 69.0, 70.0, 71.0, 71.83333333333333, 73.0, 74.0, 75.0, 76.0, 76.83333333333333, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0]

The problem also happens with the bottom bound and with longitudes. On the other hand, it does not happen with geotiff reading calls.

julia> exact_lats = (spatialrange.bottom+1.0):1.0:(spatialrange.top - 1.0)
41.0:1.0:88.0

julia> exact_lons = (spatialrange.left+1.0):1.0:(spatialrange.right - 1.0)
-144.0:1.0:-51.0

julia> all(isinteger.([clip(layer; top=l).top for l in exact_lats])) # patterns seems random...
false

julia> all(isinteger.([clip(layer; bottom=l).bottom for l in exact_lats])) # only 1 case
false

julia> all(isinteger.([clip(layer; right=l).right for l in exact_lons])) # happening with longitudes too
false

julia> all(isinteger.([clip(layer; left=l).left for l in exact_lons]))
false

julia> all(isinteger.([SimpleSDMPredictor(RasterData(WorldClim2, BioClim); left = -145.0, right = -50.0, bottom = 40.0, top = t).top for t in exact_lats]))
true

I tracked the problem down to the _match_latitude function. It seems to be a rounding issue. I have a fix I'll make in a PR and let you review.

Closed by #92

This is mostly going to involve scouring the repos for the components packages and moving the relevant issues here, then assigning them a priority based on ongoing projects.

@tpoisot The links to the documentation in the original repositories are broken, but I can't fix them since they have been archived. Raising this here since it would be a good thing to fix them until the monorepo is ready.

The links that need to be updated (in the README and the About section):

• SimpleSDMLayers.jl: https://poisotlab.github.io/SimpleSDMLayers.jl/stable/
• GBIF.jl: https://poisotlab.github.io/GBIF.jl/stable/
• Fauxcurrences.jl: https://poisotlab.github.io/Fauxcurrences.jl/

While working on this, GEO-BON/bon-in-a-box-pipelines#10, we realized that some types of missing values throw an error when trying to load

What happened?

Reading layers in with geotiff, we run into several issues with missing/NaN values in geotiffs.

As an example

layerpath = "chelsa_clim_bio11981.tif"
geotiff(SimpleSDMPredictor, layerpath)

fails with the below error where the chelsa tiff is available here

Stacktrace

ERROR: MethodError: no method matching iterate(::Nothing)
Closest candidates are:
  iterate(::Union{LinRange, StepRangeLen}) at range.jl:872
  iterate(::Union{LinRange, StepRangeLen}, ::Integer) at range.jl:872
  iterate(::T) where T<:Union{Base.KeySet{<:Any, <:Dict}, Base.ValueIterator{<:Dict}} at dict.jl:712
  ...
Stacktrace:
 [1] indexed_iterate(I::Nothing, i::Int64)
   @ Base ./tuple.jl:91
 [2] (::SimpleSDMLayers.var"#53#54"{SimpleSDMPredictor, Int64})(dataset::ArchGDAL.Dataset)
   @ SimpleSDMLayers ~/.julia/packages/SimpleSDMLayers/lYDIT/src/datasets/geotiff.jl:81
 [3] read(f::SimpleSDMLayers.var"#53#54"{SimpleSDMPredictor, Int64}, args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ ArchGDAL ~/.julia/packages/ArchGDAL/zkx2f/src/context.jl:267
 [4] read
   @ ~/.julia/packages/ArchGDAL/zkx2f/src/context.jl:264 [inlined]
 [5] #geotiff#52
   @ ~/.julia/packages/SimpleSDMLayers/lYDIT/src/datasets/geotiff.jl:46 [inlined]
 [6] geotiff (repeats 2 times)
   @ ~/.julia/packages/SimpleSDMLayers/lYDIT/src/datasets/geotiff.jl:32 [inlined]
 [7] top-level scope
   @ ~/Code/Tmp/foo.jl:9

Description of the to-do item

support for different drivers when writing geotiff

More information about the item

The current geotiff function for writing files uses the GTiff driver in ArchGDAL by default.

https://github.com/PoisotLab/SimpleSDMLayers.jl/blob/dcc49145c2c844bfe63fff99b96cf86720076512/src/datasets/geotiff.jl#L191

It would be nice to have this either be a keyword argument or something to enable selection of different drivers, e.g. I need to use the "COG" driver to write a cloud-optimized geotiff

Building the MLJ integration should not be too hard since there is already an interfaces to tables. What would help is a small wrapper around multiple layers with the same coordinates to build the tables quickly. The materializer will take care of re-projecting everything.

In #30 (comment) we discussed having an option to save with different geotiff drivers (needed for cloud-optimizer-geotiff (COG), see GEO-BON/bon-in-a-box-pipelines#40 (comment)).

I don't think this is currently implemented, but would be as simple as moving the argument to this call

They were dropped from #104 / #105 but they need to be in the next release

Add options like leftexpand = true to clip to specify if the function should include outside pixels when the clipping coordinate is exactly between two pixels

As per #30 -- the arguments to read/write geotiffs should follow the convention of e.g. CSV.write.

When building download links, this bit of code

When I import the SDT in a module, e.g. MyModule.jl:

module MyModule
   using SpeciesDistributionToolkit
  
  function foo()
        SimpleSDMPredictor(
            RasterData(CHELSA2, BioClim),
            Projection(SSP370, GFDL_ESM4);
            layer="BIO1" ) 
  end   
   export foo
end

and run

include("MyModule.jl")
MyModule.foo()

fails because it trys to download a url with slug 2011-2040/SIMPLESDMDATASETS.GFDL-ESM4/simplesdmdatasets.ssp370/bio/CHELSA_bio1_2011-2040_simplesdmdatasets.gfdl-esm4_simplesdmdatasets.ssp370_V.2.1.tif.

The example from Zurell (https://damariszurell.github.io/SDM-Intro/) is really well written, and something that could be a good vignette. The spatial extent is also relatively small, and could be a good opportunity to show how polygon clipping works.

This will likely require MLJ integration, so this would work as a "capstone" vignette.

Using "hasCoordinate" => true as a query will result in this being interpeted as "hasCoordinate" => 1. The fix will probably involve making sure that everything is a string before being passed to HTTP for the request.

Move to top-level package with the rest of the table interface for types

Phylopic has a public RESTful API @ http://phylopic.org/api

It would definitely be a good idea to establish a way to get pictures from a GBIF taxonomic name. This should probably turn into a new component package, and one additional issue would be to see how we can turn the phylopic into something that Makie can use.

This is a simple tweak to occurrences, taking an array of GBIF valid taxa, and duplicating them in the query.

This is trivial and would greatly improve UX

Add a function to split a layer into tiles

http://www.climatologylab.org/terraclimate.html?s=09

This should involve the current methods, but also possibly methods based on background information (i.e. other number-type layers).

Reading the CHELSA2 data is going to require a little bit of trickery since they put their NODATA as a number that cannot be represented as the number type

https://www.ecoclimate.org/ - this will be very easy to wrap, essentially the same template as CHELSA1

This can be done by essentially replicating the former vignette on the BIOCLIM model - this would mostly show the various ways to do stats, including quantiles, on layers.

Currently, there is a giant try/catch around it, to handle cases (like CHELSA1) where the CRS is not properly defined / readable by ArchGDAL.

Can default to nothing, but would also return a URL to the project -- users can pick up the information from there.