thets / palmsplusr Goto Github PK

View Code? Open in Web Editor NEW

5.0 5.0 1.0 6.31 MB

palmsplusr: PALMS post-processing

Home Page: http:/thets.github.io/palmsplusr/

License: GNU Lesser General Public License v3.0

R 95.59% CSS 0.18% TeX 4.23%

palmsplusr's People

Contributors

Stargazers

Watchers

Forkers

vincentvanhees

palmsplusr's Issues

Create a proper vignette with a detailed example workflow

Reading PALMS data
Reading other data (e.g., csv files, shapefiles)
Setting up the 4 configuration tables
- palmsplus fields
- palmsplus domains
- trajectory fields
- trajectory locations
Building palmsplus
Building days
Building trajectories
Plotting results

Create a brief README

This should be brief as most information will be located in the vignette.

Image of project structure
Logo

tibble_error_assign_incompatible_size when calling palms_build_trajectories()

(Running on R version 4.1.0)

Was following the steps in the A Complete Example using our own palm-processed cohort data.
No issues with reading data, creating field tables with , and was able to execute these two steps:

# Build palmsplus
palmsplus <- palms_build_palmsplus(palms)

# Build days
days <- palms_build_days(palmsplus)

However, encountered <error/tibble_error_assign_incompatible_size> after calling:

# Build trajectories
trajectories <- palms_build_trajectories(palmsplus)

Here is the output from rlang::last_error()

> rlang::last_error()
<error/tibble_error_assign_incompatible_size>
Assigned data `value` must be compatible with existing data.
x Existing data has 100 rows.
x Assigned data has 67 rows.
i Only vectors of size 1 are recycled.
Backtrace:
Run `rlang::last_trace()` to see the full context.

Following is the output from rlang::last_trace()

> rlang::last_trace()
<error/tibble_error_assign_incompatible_size>
Assigned data `value` must be compatible with existing data.
x Existing data has 100 rows.
x Assigned data has 67 rows.
i Only vectors of size 1 are recycled.
Backtrace:
     x
  1. +-palmsplusr::palms_build_trajectories(palmsplus)
  2. | \-`%>%`(...)
  3. +-dplyr::mutate_if(., is.logical, as.integer)
  4. | \-dplyr:::check_grouped(.tbl, "mutate", "if")
  5. |   \-dplyr::is_grouped_df(tbl)
  6. +-dplyr::mutate(., !!!args_after)
  7. +-sf::st_cast(., "LINESTRING")
  8. +-dplyr::summarise(., !!!args, do_union = FALSE)
  9. +-sf:::summarise.sf(., !!!args, do_union = FALSE)
 10. | +-base::`[[<-`(...)
 11. | +-dplyr:::`[[<-.grouped_df`(...)
 12. | +-base::NextMethod()
 13. | \-tibble:::`[[<-.tbl_df`(...)
 14. |   \-tibble:::tbl_subassign(...)
 15. |     \-tibble:::vectbl_recycle_rhs(...)
 16. |       +-base::withCallingHandlers(...)
 17. |       \-vctrs::vec_recycle(value[[j]], nrow)
 18. +-vctrs:::stop_recycle_incompatible_size(...)
 19. | \-vctrs:::stop_vctrs(...)
 20. |   \-rlang::abort(message, class = c(class, "vctrs_error"), ...)
 21. |     \-rlang:::signal_abort(cnd)
 22. |       \-base::signalCondition(cnd)
 23. \-(function (cnd) ...

I went on and test run the participant data one by one, and turned out this error happened only to some participants, at the same step when calling palms_build_trajectories(palmsplus).

Is this a known issue? Do you suggest any workaround? Thank you.

Implementing continuous integration

Hi @TheTS, as discussed I would like to help implement continuous integration for palmsplusr. This will, once implemented, automatically check whether changes to the code break existing functionality. In other words it will make contributing to palmsplusr more attractive and less stressful.

I have just installed palmsplusr and tried to run the package tests and R package checks, and noticed some issues. To get started with the continuous integration it would be good to first have a version of the code where all tests and checks pass. Here is a list of the issues I ran into:

In DESCRIPTION file: License: LGPL-3 needs to be License: LGPL-3 | file LICENSE
Many of the functions in functions.R have undeclared variables, which R check doesn't like.

For example:

palms_add_domain: no visible binding for '<<-' assignment to
     ‘palmsplus_domains’
   palms_add_domain: no visible binding for global variable
     ‘palmsplus_domains’
   palms_add_field: no visible binding for '<<-' assignment to
     ‘palmsplus_fields’
   palms_add_field: no visible binding for global variable
     ‘palmsplus_fields’
   palms_add_multimodal_field: no visible binding for '<<-' assignment to
     ‘multimodal_fields’
   palms_add_multimodal_field: no visible binding for global variable
     ‘multimodal_fields’
   palms_add_trajectory_field: no visible binding for '<<-' assignment to
     ‘trajectory_fields’
   palms_add_trajectory_field: no visible binding for global variable
     ‘trajectory_fields’
   palms_add_trajectory_location: no visible binding for '<<-' assignment
     to ‘trajectory_locations’
   palms_add_trajectory_location: no visible binding for global variable
     ‘trajectory_locations’
   palms_build_days: no visible binding for global variable
     ‘palmsplus_domains’
   palms_build_days: no visible binding for global variable
     ‘palmsplus_fields’
   palms_build_days: no visible binding for global variable ‘domain_field’
   palms_build_days: no visible binding for global variable ‘name’
   palms_build_days: no visible binding for global variable ‘identifier’
   palms_build_days: no visible binding for global variable ‘datetime’
   palms_build_days: no visible binding for global variable ‘.’
   palms_build_days: no visible global function definition for ‘UQ’
   palms_build_days: no visible binding for global variable ‘duration’
   palms_build_multimodal: no visible binding for global variable
     ‘identifier’
   palms_build_multimodal: no visible binding for global variable
     ‘tripnumber’
   palms_build_multimodal: no visible binding for global variable
     ‘geometry’
   palms_build_multimodal: no visible binding for global variable
     ‘end_point’
   palms_build_multimodal: no visible binding for global variable
     ‘start_point’
   palms_build_multimodal: no visible binding for global variable
     ‘end_prev’
   palms_build_multimodal: no visible binding for global variable
     ‘distance_diff’
   palms_build_multimodal: no visible binding for global variable
     ‘time_diff’
   palms_build_multimodal: no visible binding for global variable
     ‘mmt_number’
   palms_build_multimodal: no visible binding for global variable
     ‘mmt_criteria’
   palms_build_multimodal: no visible binding for global variable
     ‘multimodal_fields’
   palms_build_multimodal: no visible binding for global variable ‘mot’
   palms_build_multimodal: no visible binding for global variable
     ‘variable’
   palms_build_multimodal: no visible binding for global variable ‘value’
   palms_build_multimodal: no visible binding for global variable
     ‘trajectory_locations’
   palms_build_multimodal: no visible binding for global variable
     ‘palmsplus’
   palms_build_multimodal: no visible binding for global variable
     ‘triptype’
   palms_build_multimodal: no visible binding for global variable
     ‘start_trip’
   palms_build_multimodal: no visible binding for global variable
     ‘end_trip’
   palms_build_palmsplus: no visible binding for global variable
     ‘palmsplus_fields’
   palms_build_palmsplus: no visible binding for global variable
     ‘identifier’
   palms_build_trajectories: no visible binding for global variable
     ‘trajectory_fields’
   palms_build_trajectories: no visible binding for global variable
     ‘after_conversion’
   palms_build_trajectories: no visible binding for global variable
     ‘trajectory_locations’
   palms_build_trajectories: no visible binding for global variable
     ‘tripnumber’
   palms_build_trajectories: no visible binding for global variable
     ‘identifier’
   palms_epoch: no visible binding for global variable ‘datetime’
   palms_in_polygon: no visible binding for global variable ‘.’
   palms_in_time: no visible binding for global variable ‘school_id’
   palms_in_time: no visible binding for global variable ‘class_id’
   palms_remove_tables: no visible binding for global variable
     ‘palmsplus_domains’
   palms_remove_tables: no visible binding for global variable
     ‘palmsplus_fields’
   palms_remove_tables: no visible binding for global variable
     ‘trajectory_fields’
   palms_remove_tables: no visible binding for global variable
     ‘trajectory_locations’
   palms_remove_tables: no visible binding for global variable
     ‘multimodal_fields’
   Undefined global functions or variables:
     . UQ after_conversion class_id datetime distance_diff domain_field
     duration end_point end_prev end_trip geometry identifier mmt_criteria
     mmt_number mot multimodal_fields name palmsplus palmsplus_domains
     palmsplus_fields school_id start_point start_trip time_diff
     trajectory_fields trajectory_locations tripnumber triptype value
     variable

The use of parse_expr is not flagged by R checks but can become a security vulnerability when using PALMSplus in the cloud, see also this and this post. Ideally users should not be allowed to insert code snippets. If this is unavoidable then my proposal would be that we add a function that checks the code string for potential malicious content before it is evaluated with parse_expr.
Vignettes do not build, probably because they depend on local files. To get around this I have set all R blocks to eval=FALSE. Long term solution would be to let vignettes run based on data that is included in the repository.
Function files do not reflect name of function inside them, and functions.R seems a container for various smaller functions. The combination of both complicates finding functions by name, e.g. when errors refer to functions names. For now this is not super urgent, but would be good to revisit at a later stage.
funs in dplyr has been depricated:

Warning message:

Warning (test_all.R:73:3): Testing normal workflow
`funs()` was deprecated in dplyr 0.8.0.
Please use a list of either functions or lambdas: 

  # Simple named list: 
  list(mean = mean, median = median)

  # Auto named with `tibble::lst()`: 
  tibble::lst(mean, median)

  # Using lambdas
  list(~ mean(., trim = .2), ~ median(., na.rm = TRUE))
This warning is displayed once every 8 hours.
Call `lifecycle::last_lifecycle_warnings()` to see where this warning was generated.
Backtrace:
 1. palmsplusr::palms_build_days(palmsplus) test_all.R:73:2
 9. dplyr::funs(. * palms_epoch(data)/60)

Test1.R and test2.R rely on your local files. This is not ideal as tests should be written such that they can run anywhere to enable contributors to check that their proposed changes do not break functionality. Also things like setwd() should be avoided. As a temporary fix I have added both tests to the .RBuildignore. However, it would be good if tests could be rewritten such that functionality is tested without this dependency. For example with a tiny dummy dataset or real anonymized dataset.

Next steps:

I suppose 2, 3, and 7 are most critical for Habitus. So, I can start with 2 and 3.

Create a read_palms function with sanity checks

Basically combine these steps, with column name checks:

palms <- read_csv(system.file("extdata", "one_participant.csv", package = "palmsplusr"))
palms <- st_as_sf(palms, coords = c("lon", "lat"), crs = 4326)

Create test suite

Create a testthat test suite that covers all functions.

See if it's feasible to implement parallel processing for calc_palmsplus

Try and foreach this loop. If significant speed improvement, parameterise parallel_processing

  for (i in unique(data$identifier)) {
    x[[i]] <- data %>%
      filter(identifier == i) %>%
      mutate(!!! field_args) %>%
      mutate_if(is.logical, as.integer)

    if (verbose) {
      cat("[", j, "/", len, "] Computed palmsplus for: ", i, "\n", sep = "")
      j <- j + 1
    }
  }

Add trajectory_locations to multimodal

Base is there, need to figure out the best way to calculate from the trajectory_locations table.

  if (exists("trajectory_locations")) {

  }

Also:

palms_add_multimodal_field(c("sedentary", "light", "mvpa"), "sum")

Maybe better than messing with trajectory_fields...

Fix all roxygen2 documentation

Including spelling etc.

Make a palms_load_defaults() function

This function should populate all of the field and domain tables with the basic fields that will probably be used in all cases. This will limit the number of fields that users have to create manually.

palms_load_defaults <- function() {
  palms_add_field(...)
  palms_add_field(...)
  #etc...
}

Make multimodal summary fields dynamic

Currently:

# TODO, make these summary columns dynamic - e.g. from trajectory fields
  cols <- c("mvpa", "speed", "length", "duration", "nonwear", "wear",
            "sedentary", "light", "moderate", "vigorous",
            "mot", "mmt_number", "identifier", "geometry")

Perhaps add an additional mmt_field parameter to the trajectory_fields table:

palms_add_trajectory_field(name, formula, after_conversion, mmt_field)

Create a fake data set for including in the package examples

Easiest way will be to manipulate some existing data in QGIS.

Check if fields already exist in table before appending

In all of these functions

palms_add_field <- function(name, formula, domain_field = FALSE) {
  if (!exists("palmsplus_fields"))
    palmsplus_fields <<- tibble(name = name, formula = formula, domain_field = domain_field)
  else
    palmsplus_fields <<- rbind(palmsplus_fields, c(name, formula, domain_field))
}

updating palmsplusr to align with change in dplyr

Hi Tom, with the latest version of dplyr and running palmsplusr I get the error:

Returning more (or less) than 1 row per `summarise()` group was deprecated in dplyr 1.1.0.
ℹ Please use `reframe()` instead.
ℹ When switching from `summarise()` to `reframe()`, remember that `reframe()` always returns an ungrouped
  data frame and adjust accordingly.

I think this means that we need to replace summarise() for reframe(). You are probably more familiar with the tidyverse jargon. Do you know what we should do with the last part of the warning? Does this mean we need to group our data and if so how do I do that?

Add multimodal trips

Add a function to reprocess trajectories into multimodal trips. It should take built trajectories as input, along with spatial/temporal threshold:

trajectories <- palms_multimodal(trajectories, 5, 200)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.