Git Product home page Git Product logo

capsule's Introduction

capsule

capsule status badge

Installation

R Universe

install.packages(
   "capsule", 
   repos = c(mm = "https://milesmcbain.r-universe.dev", getOption("repos")))

Overview

A capsule is a stable project-specific R package library that you consciously choose to execute code within. Think of it as representing 'production', while your normal interactive R session represents 'development'.

You develop interactively in a dynamic package environment. You run code for real in a well-defined static capsule. Periodically you'll want to have your development environment reflected in the capsule as new stuff is integrated.

When sharing with others, a capsule is the fallback that ensures your code can always be run, no matter what issues appear in your collaborator's development environment.

Usage

There are 3 functions you need to know about to use capsule: capsule::create(), capsule::run(), capsule::recreate().

create() a capsule for my pipeline

> capsule::create("./packages.R")
Finding R package dependencies ... Done!
* Discovering package dependencies ... Done!
* Copying packages into the cache ... [132/132] Done!
The following package(s) will be updated in the lockfile:

# CRAN ===============================
- anytime          [* -> 0.3.6]
- askpass          [* -> 1.1]
...TRUNCATED...
- BH               [* -> 1.69.0-1]
- spatial          [* -> 7.3-11]
- survival         [* -> 2.44-1.1]

# GitHub =============================
- geojsonsf        [* -> SymbolixAU/geojsonsf]
- h3jsr            [* -> obrl-soil/h3jsr]
- jsonify          [* -> SymbolixAU/jsonify]
- qfes             [* -> milesmcbain/qfes]
- renv             [* -> rstudio/renv]

* Lockfile written to 'c:/repos/capsule/renv.lock'.

You supply a vector of file paths to extract dependencies from. The default is "./packages.R". These dependencies are copied from your regular (dev) library to your local capsule.

Notice how this is easier when you keep your library calls all in one place? πŸ˜‰

You'll notice some things created in your project folder. Assuming you have the project under version control... you definitely want to commit the ./renv.lock file. This will allow someone else to run() code in the capsule context.

run() code in the capsule

Render a document in the capsule:

capsule::run(rmarkdown::render("doc/analysis.Rmd"))

Or run your {targets} plan in the capsule:

capsule::run(targets::tar_make())

So what about code that you've just been handed? It has a renv.lock but no local library? How do you build the library to run the code? You don't! run() will check to see if a local library exists, and build it if required. (You can do this manually with reproduce_lib(), if that feels better before calling run()).

recreate() the capsule

You've done some development work, updated a few dependencies, and the output has tested successfully. You can make the capsule reflect the project dependencies installed in your dev environment using recreate().

Other Useful Stuff

Debugging in the capsule with a REPL

Try capsule::repl() to attach a REPL for a new R process in the context of the capsule. This is handy for interactive work like debugging. The tradeoff here is that depending what editor you use strange behaviour may be induced by the outer REPL being overtaken. In ESS I lose my autocompletions.

Automating lockfile creation with capshot()

capshot() is designed to create a lockfile for your project very quickly so that it can be integrated into your build or rendering pipeline. On my {tflow} projects laden with dependencies it typically takes 1-2 seconds to detect the dependencies and write the lockfile.

Unlike create() it does not populate a local library automatically. See:

  • capsule::capshot() - fast lockfile creation
  • capsule::capshot_str() - quickly generate the lockfile json for further processing, e.g. embedding in a document.

Helpers

  • capsule::delete() - remove the capsule (local library and lockfile).
  • capsule::delete_local_lib() - remove the local library.
  • capsule::delete_lockfile() - remove the lockfile.
  • capsule::reproduce_lib() - build the local library from the lockfile.

It's an renv in the end

A capsule is an renv. The full power of renv can always be used to manipulate the lockfile and library if you wish.

capsule's People

Contributors

milesmcbain avatar gadenbuie avatar njtierney avatar

Stargazers

Stuart Lee avatar Chitra M Saraswati avatar Zach Nelson avatar Carlo Bottai avatar  avatar Cynthia Huang avatar Nicholas Knoblauch avatar Harry Eslick avatar David Mateos avatar cswaters avatar ZENβ™‘ avatar  avatar  avatar Bryce Mecum avatar Matt Pettis avatar RadosΕ‚aw JaΕΊwiec avatar  avatar Tom Tensfeldt avatar Paulina Marczak avatar Felipe Carvalho avatar Sam Abbott avatar Mike K Smith avatar Felipe avatar Brendan Clarke avatar GAURAV avatar Guilherme C Brito avatar Freeman Wang avatar Adrien Taudiere avatar Markus Lang avatar  avatar Zane Billings avatar Zac Knitter avatar John avatar Caleb Jenkins avatar Andrew Skelton avatar Selin Jessa avatar Larefly avatar Lachlan Deer avatar atanas avatar Kenneth Blake Vernon avatar Val. Lucet avatar Matt Kerlogue avatar Allan Irvine avatar Mohammed Hamdy avatar Sono Shah avatar Nischal Shrestha avatar Kyle Mitchell avatar Matthew Begun avatar Michael Sumner avatar zane avatar  avatar Kun Ren avatar James Goldie avatar Lisa Levinson avatar  avatar Quang Nguyen avatar  avatar JoΓ£o Santiago avatar  avatar Mark Murphy avatar Marian Eberl avatar Hugo Gruson avatar AurΓ©lien Ginolhac avatar Darek Kedra avatar Tatako Felici avatar Thomas Sandmann avatar Juraj SzitΓ‘s avatar Luke Zappia avatar John MacKintosh avatar Michelle Jamieson avatar Hugh Graham avatar Srikanth K S avatar Nikita avatar Danielle Navarro avatar Nicholas Williams avatar Lukas Burk avatar  avatar Nana Mensah avatar AM avatar nima hejazi avatar Zhian N. Kamvar avatar Ryan Chitwood avatar Patrick Howard avatar Alexander Keth avatar Will Bonnell avatar Blas Benito avatar Chris Kennedy avatar  avatar  avatar Francisco Bischoff avatar Petr Bouchal avatar Michael DeWitt avatar Jimmy Briggs avatar Adam H. Sparks avatar Emil Hvitfeldt avatar Henrik Bengtsson avatar Eivind Gard Lund avatar M. Foos avatar Thomas Neitmann avatar stephhuynh avatar

Watchers

Anthony North avatar James Cloos avatar Adam H. Sparks avatar Ross Gayler avatar  avatar  avatar  avatar  avatar

capsule's Issues

workflow/option to install packages locally

Basically I would like something like capsule::reproduce_lib() but that installs the packages locally for interactive use outside of the capsule::run workflow.

Perhaps the function could be called:

install_lockfile_pkgs() or something?

Create a capsule from a DESCRIPTION file

I'd love to see an alternative capsule::create() workflow that uses the DESCRIPTION file to set up the capsule rather than an R script with library() calls.

This is typically handled via renv::install() and in my experience it works really well as a light-weight and user-friendly entry point for dependency declaration. Basically, DESCRIPTION is to renv.lock as package.json is to package.lock.json.

Add some tools for rendering Rmardown document in a capsule ?

Hello @MilesMcBain !

Happy new year ! Hope you're fine !

I open this issue to open a discussion on an interesting use-case.
Today, there was a question on RStudio community about using a renv lockfile per Rmd document to be able to render a document in its own locked environment with specific package.
https://community.rstudio.com/t/renv-lockfile-per-rmarkdown-doc/

I found this idea very interesting and it made me think about your {capsule} package. I willing to try protyping some functions to help do that, but I wanted first to have your thoughts regarding the work you've done on this package, what you encounter on a daily basis, and if capsule already brings a solution to this or not. It seems to me you could have only one lockfile per project for all document in a project, where libraries are all in the same packages.R files. I may be missing something though...

If you think it is interesting, do you capsule is a good place for such functions ? Or should it live elsewhere ?

As you could see in the small try I made, I think I am very close to the challenge you try to deal with.

I have some other ideas, and I think this would be very powerful to be able to have some companion lockfile (or even a way to embed it inside a Rmd - let's be audacious) and use it to render a Rmd Analysis in in its own temporary capsule without to have one project with a renv lib per Rmd document.

When you have some spare time, I am very interested to read you πŸ’­ on this!

Thank you !

Detect using::pkg() dependencies

Adding support for detecting dependencies from using::pkg() removes the need for redundant library() calls in the ./packages.R script.

# packages.R

using::pkg(dplyr, min_version = "1.0")
library(dplyr)

drop purrr dependency to avoid dependency on rlang

Unfortunately, rlang is a very inconvenient recursive dependency to have for windows users.

It could mean dev_mirror_lockfile() fails on Windows due to rlang being in the session and not able to be updated.

error using `capsule::create` - Error: callr subprocess failed: group length is 0 but data length > 0

capsule::create(here::here("packages.R"))
#> Error: callr subprocess failed: group length is 0 but data length > 0

Created on 2021-06-09 by the reprex package (v2.0.0)

Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.1.0 (2021-05-18)
#>  os       macOS Big Sur 10.16         
#>  system   x86_64, darwin17.0          
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_AU.UTF-8                 
#>  ctype    en_AU.UTF-8                 
#>  tz       Australia/Perth             
#>  date     2021-06-09                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date       lib source                              
#>  backports     1.2.1      2020-12-09 [1] CRAN (R 4.1.0)                      
#>  callr         3.7.0      2021-04-20 [1] CRAN (R 4.1.0)                      
#>  capsule       0.1.1.9000 2021-06-09 [1] Github (MilesMcBain/capsule@e27771d)
#>  cli           2.5.0      2021-04-26 [1] CRAN (R 4.1.0)                      
#>  crayon        1.4.1      2021-02-08 [1] CRAN (R 4.1.0)                      
#>  digest        0.6.27     2020-10-24 [1] CRAN (R 4.1.0)                      
#>  ellipsis      0.3.2      2021-04-29 [1] CRAN (R 4.1.0)                      
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 4.1.0)                      
#>  fansi         0.5.0      2021-05-25 [1] CRAN (R 4.1.0)                      
#>  fs            1.5.0      2020-07-31 [1] CRAN (R 4.1.0)                      
#>  glue          1.4.2      2020-08-27 [1] CRAN (R 4.1.0)                      
#>  highr         0.9        2021-04-16 [1] CRAN (R 4.1.0)                      
#>  htmltools     0.5.1.1    2021-01-22 [1] CRAN (R 4.1.0)                      
#>  knitr         1.33       2021-04-24 [1] CRAN (R 4.1.0)                      
#>  lifecycle     1.0.0      2021-02-15 [1] CRAN (R 4.1.0)                      
#>  magrittr      2.0.1      2020-11-17 [1] CRAN (R 4.1.0)                      
#>  pillar        1.6.1      2021-05-16 [1] CRAN (R 4.1.0)                      
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.1.0)                      
#>  processx      3.5.2      2021-04-30 [1] CRAN (R 4.1.0)                      
#>  ps            1.6.0      2021-02-28 [1] CRAN (R 4.1.0)                      
#>  purrr         0.3.4      2020-04-17 [1] CRAN (R 4.1.0)                      
#>  R6            2.5.0      2020-10-28 [1] CRAN (R 4.1.0)                      
#>  reprex        2.0.0      2021-04-02 [1] CRAN (R 4.1.0)                      
#>  rlang         0.4.11     2021-04-30 [1] CRAN (R 4.1.0)                      
#>  rmarkdown     2.8        2021-05-07 [1] CRAN (R 4.1.0)                      
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 4.1.0)                      
#>  stringi       1.6.2      2021-05-17 [1] CRAN (R 4.1.0)                      
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 4.1.0)                      
#>  styler        1.4.1      2021-03-30 [1] CRAN (R 4.1.0)                      
#>  tibble        3.1.2      2021-05-16 [1] CRAN (R 4.1.0)                      
#>  utf8          1.2.1      2021-03-12 [1] CRAN (R 4.1.0)                      
#>  vctrs         0.3.8      2021-04-29 [1] CRAN (R 4.1.0)                      
#>  withr         2.4.2      2021-04-18 [1] CRAN (R 4.1.0)                      
#>  xfun          0.23       2021-05-15 [1] CRAN (R 4.1.0)                      
#>  yaml          2.2.1      2020-02-01 [1] CRAN (R 4.1.0)                      
#> 
#> [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library

I get this when trying to run this on https://github.com/njtierney/njtcv, if that helps

Should capsule transform RSPM repo urls

When writing a lockfile, currently {capsule} diverges from {renv} with respect to RSPM repo URLS.

On a system with:

> options('repos')
$repos
                                                           RSPM 
"https://packagemanager.rstudio.com/all/__linux__/focal/latest" 
                                                           CRAN 
                                     "https://cran.rstudio.com"

{capsule} will write:

    "Repositories": [
      {
        "Name": "RSPM",
        "URL": "https://packagemanager.rstudio.com/all/__linux__/focal/latest"
      },
      {
        "Name": "CRAN",
        "URL": "https://cran.rstudio.com"
      }
    ]

while {renv} writes:

    "Repositories": [
      {
        "Name": "RSPM",
        "URL": "https://packagemanager.rstudio.com/all/latest"
      },
      {
        "Name": "CRAN",
        "URL": "https://cran.rstudio.com"
      }
    ]

Here {renv} has transformed the platform specific binary url for RSPM into a general source package url. However when restoring from the lockfile {renv} will attempt to transform the source url into a platform specific binary one.

I am surprised this hasn't caused issues at work with collaborators working on windows and linux with different repository URLs, so it seems like {renv} falls back to the local system's repo urls if it can't use the lockfile ones. Need to confirm this, but if so, I see no reason to mirror this complexity.

export detect_dependencies

I find myself doing this sometimes:

capsule:::detect_dependencies("packages.R") |> install.packages()

Support `callr::rscript()` or `callr:r()`

I've been using (and enjoying!) capsule to write scripts to run in GHA workflows. One minor point of friction is missing the ability to call a capsule-managed callr::rscript().

One option would be a new function, e.g. run_callr_rscript that calls callr:rscript() instead of callr::r().

An alternative that's either clever or more ergonomic would be to dispatch to callr::rscript() if run_callr() receives a character string input:

run_callr <- function(x, ...) {
  reproduce_lib_if_not_present()
  UseMethod("run_callr", x)
}

run_callr.character <- function(x, ...) {
  callr::rscript(x, libpath = c(renv::paths$library()), show = TRUE, ...)
}

run_callr.function <- function(x, ...) {
  callr::r(x, libpath = c(renv::paths$library()), show = TRUE, ...)
}

Happy to submit a PR for either approach πŸ˜„

repl printing plots in RStudio?

Loving capsule::repl() - however it doesn't seem to produce plots when I run them in the console - just wondering if this is something you have a workaround for?

If the package SHA is not a SHA, treat as NA

Due to some unfortunate legacy decisions, multiple RStudio packages (remotes, pak) will populate the SHA metadata with the version number if the package is downloaded from CRAN.

In capsule this can trigger a warning when comparing packages to the lockfile that the version numbers are the same but the SHAs are different. It might be possible to avoid this by just treating anything that is a version number as an NA for the SHA.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.