ali-johnston / ebird_sdms_dd_paper Goto Github PK

Using eBird data to estimate species distributions. Code to reproduce the analyses and figures in published paper.

R 100.00%

ebird_sdms_dd_paper's Introduction

eBird Analysis WoodThrush

This repository contains the code and data to reproduce the analyses in a paper published in Diversity and Distributions: Johnston, Hochachka, Strimas-Mackey, Ruiz Gutierrez, Miller, Auer, Kelling, Fink. (2021) Analytical guidelines to increase the value of community science data: An example using eBird data to estimate species distributions. Diversity and Distributions.

For a more general and comprehensive guide to analysing eBird data, including more code, comments, and explanations, we recommend this alternative source of code: Strimas-Mackey, Hochachka, Ruiz Gutierrez, Robinson, Miller, Auer, Kelling, Fink, Johnston. 2020. Best Practices for Using eBird Data. Version 1.0. https://cornelllabofornithology.github.io/ebird-best-practices/. Cornell Lab of Ornithology, Ithaca, New York. https://doi.org/10.5281/zenodo.3620739

Here is a brief description of the scripts in this repo, that only produce the analyses and figures in the published paper.

Processing the data

The first few scripts contain data to process the code. The processed datasets have been saved within the data_proc/ folder. So if you want to just run the models, you can proceed directly to scripts 05, 06, and 07.

00_gis-data.R Read in and prepare the BCR boundaries and mapping layers

01_ebird-data.R Filter the eBird data from a local version of the eBird Basic Dataset (EBD) select certain species, region, season.

In order to run this, you will need to download your own local version of the EBD from the eBird website and use auk_set_ebd_path to define the folder where this is located. BUT, the full dataset requires hundreds of GB of space and once you have downloaded this it takes 3-4 hours to run 01_ebird-data.R. So proceed with caution!

02_identify_bbs.R Identify the BBS routes and stops that are within the eBird dataset

03_validation_data.R Split the data into training and validation datasets, based on year, BBS status, etc.

04_habitat-covariates.R Extract and process the MODIS landcover information for each checklist

Some complex setup can be required for the MODIS data processing. We recommend that if you want to run this for your own data, you follow the instructions in https://cornelllabofornithology.github.io/ebird-best-practices/

Running the models

05_a_25_encounter_models.R Run 25 sets, each with 7 encounter rate random forest models.

05_b_encounter_models_plot.R Run a single set of 7 encounter rate models and create maps

06_a_occupancy_models_plot.R Run a single set of 6 occupancy models

07_sample_size_encounter_models.R Run 25 sets, each with 2 encounter rate random forest models and 5 different sample sizes

Functions

The R/ folder contains several functions used to help run the models. They most important/relevant functions are:

fit_model_enc Runs a single encounter rate model, taking as parameters instructions about how to subsample the data, what type of model to run (maxent or random forest) and whether to fit covariates

predict_model_enc Takes the output from a fit_model_enc run and predicts to a given dataset

fit_model_occu Runs a single occupancy model, taking as parameters instructions about how to subsample the data and whether to fit covariates

validate Takes the output from a fit_model_enc run, predicts to a new dataset, then calculates performance metrics.

Directory Structure

The code is setup to read most 'data' files for modelling from data_proc Results will be put within subfolders into the folders figures/ and output/ R contains functions used in the modelling

ebird_sdms_dd_paper's People

Contributors

Stargazers

Watchers

Forkers

mstrimas diptonil-nandi

ebird_sdms_dd_paper's Issues

Errors happen for 05_a_25_encounter_models.R

Hi, thanks for sharing the code! As you mentioned, "So if you want to just run the models, you can proceed directly to scripts 05, 06, and 07", I tried that, but errors happened when I ran line 39. The error message is as follows.

Rows: 42539 Columns: 17                                                                                                    
── Column specification ────────────────────────────────────────────────────────────────
Delimiter: ","
chr  (10): checklist_id, observer_id, sampling_event_identifier, state_code, localit...
dbl   (5): latitude, longitude, effort_distance_km, chwwid, woothr
lgl   (1): all_species_reported
date  (1): observation_date

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
[1] ""                                                                                                                     
[1] "======================================="
[1] "RUNNING FOR SIM 1"
[1] "PREPARING DATA"
Resolution: 13, Area (km^2): 31.9926151554038, Spacing (km): 5.58632116604267, CLS (km): 6.38233997895802
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2100)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2100)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2100)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2100)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 1974)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 1986)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2079)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 1971)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 1971)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 1971)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2100)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2076)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2001)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 1971)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2079)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2079)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 1986)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2016)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 1971)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2016)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2079)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2100)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2079)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2079)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2079)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2100)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 1974)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2076)
Joining with `by = join_by(checklist_id, sampling_event_identifier, type)`
Rows: 3395 Columns: 2                                                                                                      
── Column specification ────────────────────────────────────────────────────────────────
Delimiter: ","
chr (1): checklist_id
dbl (1): route_id

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Resolution: 13, Area (km^2): 31.9926151554038, Spacing (km): 5.58632116604267, CLS (km): 6.38233997895802
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2100)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2010)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2001)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2031, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2187, 2004)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2028, 2187)
FATAL ERROR: DgQ2DDtoIConverter::convertTypedAddress():  coordinate out of range: (2025, 2187)
Joining with `by = join_by(species_observed, checklist_id)`
[1] "RUNNING MODELS"
[1] "------- Running encounter rate model -------"
[1] "raw data n = 16204"
[1] "Keeping only presences"
[1] "50000 background points"
Error in `pmap()`:
ℹ In index: 1.
Caused by error in `sample_n()`:
! Can't compute indices.
Caused by error:
! `size` must be less than or equal to 440 (size of data).
ℹ set `replace = TRUE` to use sampling with replacement.
Run `rlang::last_trace()` to see where the error occurred.
Warning messages:
1: In dir.create(figure_folder, recursive = TRUE) :
  'figures/encounter/RF_sample_together_woothr_2024-01-10' already exists
2: In dir.create(output_folder, recursive = TRUE) :
  'output/encounter/RF_sample_together_woothr_2024-01-10' already exists
3: One or more parsing issues, call `problems()` on your data frame for details, e.g.:
  dat <- vroom(...)
  problems(dat) 
4: There was 1 warning in `mutate()`.
ℹ In argument: `time_observations_started =
  as.numeric(as.character(time_observations_started))`.
Caused by warning:
! NAs introduced by coercion 
5: There was 1 warning in `mutate()`.
ℹ In argument: `number_observers = as.numeric(as.character(number_observers))`.
Caused by warning:
! NAs introduced by coercion 
Error in exists(cacheKey, where = .rs.WorkingDataEnv, inherits = FALSE) : 
  invalid first argument
Error in assign(cacheKey, frame, .rs.CachedDataEnv) : 
  attempt to use zero-length variable name

Line 39 is the beginning of a loop, so I looked into the loop line by line. Before line 199, everything seems to be fine. But after running line 199, an error happened.

ppm_bbs_stop <- mutate(mod_set, ppms = map(models, validate, data = ebird_test_bbs, bbs_combine = FALSE)) %>%
        select(-models) %>% unnest(cols = c(ppms)) %>%
        mutate(val_type = "bbs")
# Error in `mutate()`:
#   ℹ In argument: `ppms = map(models, validate, data = ebird_test_bbs, bbs_combine
#                              = FALSE)`.
#   Caused by error:
#     ! object 'models' not found
#   Run `rlang::last_trace()` to see where the error occurred.

Could you help me with that? Thanks.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.