thets / palmsplusr Goto Github PK
View Code? Open in Web Editor NEWpalmsplusr: PALMS post-processing
Home Page: http:/thets.github.io/palmsplusr/
License: GNU Lesser General Public License v3.0
palmsplusr: PALMS post-processing
Home Page: http:/thets.github.io/palmsplusr/
License: GNU Lesser General Public License v3.0
Currently:
# TODO, make these summary columns dynamic - e.g. from trajectory fields
cols <- c("mvpa", "speed", "length", "duration", "nonwear", "wear",
"sedentary", "light", "moderate", "vigorous",
"mot", "mmt_number", "identifier", "geometry")
Perhaps add an additional mmt_field parameter to the trajectory_fields table:
palms_add_trajectory_field(name, formula, after_conversion, mmt_field)
In all of these functions
palms_add_field <- function(name, formula, domain_field = FALSE) {
if (!exists("palmsplus_fields"))
palmsplus_fields <<- tibble(name = name, formula = formula, domain_field = domain_field)
else
palmsplus_fields <<- rbind(palmsplus_fields, c(name, formula, domain_field))
}
This function should populate all of the field and domain tables with the basic fields that will probably be used in all cases. This will limit the number of fields that users have to create manually.
palms_load_defaults <- function() {
palms_add_field(...)
palms_add_field(...)
#etc...
}
Hi @TheTS, as discussed I would like to help implement continuous integration for palmsplusr. This will, once implemented, automatically check whether changes to the code break existing functionality. In other words it will make contributing to palmsplusr more attractive and less stressful.
I have just installed palmsplusr and tried to run the package tests and R package checks, and noticed some issues. To get started with the continuous integration it would be good to first have a version of the code where all tests and checks pass. Here is a list of the issues I ran into:
License: LGPL-3
needs to be License: LGPL-3 | file LICENSE
palms_add_domain: no visible binding for '<<-' assignment to
‘palmsplus_domains’
palms_add_domain: no visible binding for global variable
‘palmsplus_domains’
palms_add_field: no visible binding for '<<-' assignment to
‘palmsplus_fields’
palms_add_field: no visible binding for global variable
‘palmsplus_fields’
palms_add_multimodal_field: no visible binding for '<<-' assignment to
‘multimodal_fields’
palms_add_multimodal_field: no visible binding for global variable
‘multimodal_fields’
palms_add_trajectory_field: no visible binding for '<<-' assignment to
‘trajectory_fields’
palms_add_trajectory_field: no visible binding for global variable
‘trajectory_fields’
palms_add_trajectory_location: no visible binding for '<<-' assignment
to ‘trajectory_locations’
palms_add_trajectory_location: no visible binding for global variable
‘trajectory_locations’
palms_build_days: no visible binding for global variable
‘palmsplus_domains’
palms_build_days: no visible binding for global variable
‘palmsplus_fields’
palms_build_days: no visible binding for global variable ‘domain_field’
palms_build_days: no visible binding for global variable ‘name’
palms_build_days: no visible binding for global variable ‘identifier’
palms_build_days: no visible binding for global variable ‘datetime’
palms_build_days: no visible binding for global variable ‘.’
palms_build_days: no visible global function definition for ‘UQ’
palms_build_days: no visible binding for global variable ‘duration’
palms_build_multimodal: no visible binding for global variable
‘identifier’
palms_build_multimodal: no visible binding for global variable
‘tripnumber’
palms_build_multimodal: no visible binding for global variable
‘geometry’
palms_build_multimodal: no visible binding for global variable
‘end_point’
palms_build_multimodal: no visible binding for global variable
‘start_point’
palms_build_multimodal: no visible binding for global variable
‘end_prev’
palms_build_multimodal: no visible binding for global variable
‘distance_diff’
palms_build_multimodal: no visible binding for global variable
‘time_diff’
palms_build_multimodal: no visible binding for global variable
‘mmt_number’
palms_build_multimodal: no visible binding for global variable
‘mmt_criteria’
palms_build_multimodal: no visible binding for global variable
‘multimodal_fields’
palms_build_multimodal: no visible binding for global variable ‘mot’
palms_build_multimodal: no visible binding for global variable
‘variable’
palms_build_multimodal: no visible binding for global variable ‘value’
palms_build_multimodal: no visible binding for global variable
‘trajectory_locations’
palms_build_multimodal: no visible binding for global variable
‘palmsplus’
palms_build_multimodal: no visible binding for global variable
‘triptype’
palms_build_multimodal: no visible binding for global variable
‘start_trip’
palms_build_multimodal: no visible binding for global variable
‘end_trip’
palms_build_palmsplus: no visible binding for global variable
‘palmsplus_fields’
palms_build_palmsplus: no visible binding for global variable
‘identifier’
palms_build_trajectories: no visible binding for global variable
‘trajectory_fields’
palms_build_trajectories: no visible binding for global variable
‘after_conversion’
palms_build_trajectories: no visible binding for global variable
‘trajectory_locations’
palms_build_trajectories: no visible binding for global variable
‘tripnumber’
palms_build_trajectories: no visible binding for global variable
‘identifier’
palms_epoch: no visible binding for global variable ‘datetime’
palms_in_polygon: no visible binding for global variable ‘.’
palms_in_time: no visible binding for global variable ‘school_id’
palms_in_time: no visible binding for global variable ‘class_id’
palms_remove_tables: no visible binding for global variable
‘palmsplus_domains’
palms_remove_tables: no visible binding for global variable
‘palmsplus_fields’
palms_remove_tables: no visible binding for global variable
‘trajectory_fields’
palms_remove_tables: no visible binding for global variable
‘trajectory_locations’
palms_remove_tables: no visible binding for global variable
‘multimodal_fields’
Undefined global functions or variables:
. UQ after_conversion class_id datetime distance_diff domain_field
duration end_point end_prev end_trip geometry identifier mmt_criteria
mmt_number mot multimodal_fields name palmsplus palmsplus_domains
palmsplus_fields school_id start_point start_trip time_diff
trajectory_fields trajectory_locations tripnumber triptype value
variable
parse_expr
is not flagged by R checks but can become a security vulnerability when using PALMSplus in the cloud, see also this and this post. Ideally users should not be allowed to insert code snippets. If this is unavoidable then my proposal would be that we add a function that checks the code string for potential malicious content before it is evaluated with parse_expr
.Warning (test_all.R:73:3): Testing normal workflow
`funs()` was deprecated in dplyr 0.8.0.
Please use a list of either functions or lambdas:
# Simple named list:
list(mean = mean, median = median)
# Auto named with `tibble::lst()`:
tibble::lst(mean, median)
# Using lambdas
list(~ mean(., trim = .2), ~ median(., na.rm = TRUE))
This warning is displayed once every 8 hours.
Call `lifecycle::last_lifecycle_warnings()` to see where this warning was generated.
Backtrace:
1. palmsplusr::palms_build_days(palmsplus) test_all.R:73:2
9. dplyr::funs(. * palms_epoch(data)/60)
setwd()
should be avoided. As a temporary fix I have added both tests to the .RBuildignore. However, it would be good if tests could be rewritten such that functionality is tested without this dependency. For example with a tiny dummy dataset or real anonymized dataset.Next steps:
I suppose 2, 3, and 7 are most critical for Habitus. So, I can start with 2 and 3.
Including spelling etc.
Basically combine these steps, with column name checks:
palms <- read_csv(system.file("extdata", "one_participant.csv", package = "palmsplusr"))
palms <- st_as_sf(palms, coords = c("lon", "lat"), crs = 4326)
Base is there, need to figure out the best way to calculate from the trajectory_locations table.
if (exists("trajectory_locations")) {
}
Also:
palms_add_multimodal_field(c("sedentary", "light", "mvpa"), "sum")
Maybe better than messing with trajectory_fields...
Hi Tom, with the latest version of dplyr and running palmsplusr I get the error:
Returning more (or less) than 1 row per `summarise()` group was deprecated in dplyr 1.1.0.
ℹ Please use `reframe()` instead.
ℹ When switching from `summarise()` to `reframe()`, remember that `reframe()` always returns an ungrouped
data frame and adjust accordingly.
I think this means that we need to replace summarise() for reframe(). You are probably more familiar with the tidyverse jargon. Do you know what we should do with the last part of the warning? Does this mean we need to group our data and if so how do I do that?
Try and foreach this loop. If significant speed improvement, parameterise parallel_processing
for (i in unique(data$identifier)) {
x[[i]] <- data %>%
filter(identifier == i) %>%
mutate(!!! field_args) %>%
mutate_if(is.logical, as.integer)
if (verbose) {
cat("[", j, "/", len, "] Computed palmsplus for: ", i, "\n", sep = "")
j <- j + 1
}
}
Add a function to reprocess trajectories into multimodal trips. It should take built trajectories as input, along with spatial/temporal threshold:
trajectories <- palms_multimodal(trajectories, 5, 200)
Create a testthat test suite that covers all functions.
This should be brief as most information will be located in the vignette.
(Running on R version 4.1.0)
Was following the steps in the A Complete Example using our own palm-processed cohort data.
No issues with reading data, creating field tables with , and was able to execute these two steps:
# Build palmsplus
palmsplus <- palms_build_palmsplus(palms)
# Build days
days <- palms_build_days(palmsplus)
However, encountered <error/tibble_error_assign_incompatible_size> after calling:
# Build trajectories
trajectories <- palms_build_trajectories(palmsplus)
Here is the output from rlang::last_error()
> rlang::last_error()
<error/tibble_error_assign_incompatible_size>
Assigned data `value` must be compatible with existing data.
x Existing data has 100 rows.
x Assigned data has 67 rows.
i Only vectors of size 1 are recycled.
Backtrace:
Run `rlang::last_trace()` to see the full context.
Following is the output from rlang::last_trace()
> rlang::last_trace()
<error/tibble_error_assign_incompatible_size>
Assigned data `value` must be compatible with existing data.
x Existing data has 100 rows.
x Assigned data has 67 rows.
i Only vectors of size 1 are recycled.
Backtrace:
x
1. +-palmsplusr::palms_build_trajectories(palmsplus)
2. | \-`%>%`(...)
3. +-dplyr::mutate_if(., is.logical, as.integer)
4. | \-dplyr:::check_grouped(.tbl, "mutate", "if")
5. | \-dplyr::is_grouped_df(tbl)
6. +-dplyr::mutate(., !!!args_after)
7. +-sf::st_cast(., "LINESTRING")
8. +-dplyr::summarise(., !!!args, do_union = FALSE)
9. +-sf:::summarise.sf(., !!!args, do_union = FALSE)
10. | +-base::`[[<-`(...)
11. | +-dplyr:::`[[<-.grouped_df`(...)
12. | +-base::NextMethod()
13. | \-tibble:::`[[<-.tbl_df`(...)
14. | \-tibble:::tbl_subassign(...)
15. | \-tibble:::vectbl_recycle_rhs(...)
16. | +-base::withCallingHandlers(...)
17. | \-vctrs::vec_recycle(value[[j]], nrow)
18. +-vctrs:::stop_recycle_incompatible_size(...)
19. | \-vctrs:::stop_vctrs(...)
20. | \-rlang::abort(message, class = c(class, "vctrs_error"), ...)
21. | \-rlang:::signal_abort(cnd)
22. | \-base::signalCondition(cnd)
23. \-(function (cnd) ...
I went on and test run the participant data one by one, and turned out this error happened only to some participants, at the same step when calling palms_build_trajectories(palmsplus)
.
Is this a known issue? Do you suggest any workaround? Thank you.
Easiest way will be to manipulate some existing data in QGIS.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.