kentonwhite / projecttemplate Goto Github PK

Also add GitHub issues URL.

The person who releases to CRAN should be primary maintainer.

Rationale: Load order of packages may influence visibility of symbols defined in more than one package.

Perhaps a new function like restart.project()? Or even only if necessary?

Autodetection of course would be best, but this probably only works with the libraries section in config/*.dcf. Trouble is that this is not recognized by packrat (third item in #46).

Thoughts?

Relevant material:

http://stackoverflow.com/questions/7505547/detach-all-packages-while-working-in-r
http://stackoverflow.com/questions/6979917/how-to-unload-a-package-without-restarting-r

Will need to deprecate current syntax in case someone is using it.

test_dir() works only for file name with "test-*.r" when default option.

I expect, "stub.test()" needs to be modified to produce file name like this "test-autogenerated.r"

Add parameter envir = .GlobalEnv to each reader, use it.

Requires #62.

All readers currently perform two things: reading the data, and assigning at least one variable to an environment. It would be better if the assignment took place in a central location, and the readers just returned a named list. (What about performance?)

to avoid code duplication.

There should be a way to add comments to config files

Need to figure out a neat way. Perhaps make bump-x where x is the position in the version to increment, with the help of muxspace/crant.

• Suggests
• Project defaults

• Missing else
• Only three cases: bad extension, with temp file, without temp file
• Test cases, perhaps with file:/// URL

config directory contains only one file, overkill. Suggestions (in order of preference):

• .ProjectTemplate
• ProjectTemplate.cfg
• ProjectTemplate.dcf

?

Related: #79.

Currently, file has to be removed manually from cache subdirectory. Can we do better?

invalidate.cache <- function(names = NULL) { ... }

Or even a config option?

https://github.com/johnmyleswhite/ProjectTemplate/blob/master/R/load.project.R#L255

This if is technically wrong, and will warn if the object has more than one class. For instance:

> class(data.table::data.table())
[1] "data.table" "data.frame"

Should everything that is a data frame be converted to a data table (any(...)), or should only "pure" data frames be converted to data tables (all(...))?

Implementing the second option as part of #76, still open for discussion.

For the lazy among us...

Missing values might correspond to "legacy" behavior that we better turn off for new projects but still want to support for backward-compatibility reasons.

• Provide an option to initialize a project with packrat support
• Call packrat::status() during load.project()
• Write dummy .R file for all libraries that are loaded from the configuration file (unless it's possible to turn off code scanning in packrat)
• Wait for packrat to appear on CRAN

• Provide an option to initialize a project with RBundler support
• Call bundle() during load.project()

Most internal code uses library() to load packages when required. Using require() inside functions is preferred. Created a helper function require.package() to load packages as needed and display a more helpful error message, including which function is trying to load the package, which package was being loaded, and instructions for installing the missing package.

Internal code should be moved from using library() to using require.package().

To allow changing the layout later.

nuff sed

As requested. The pages are now at

http://johnmyleswhite.github.io/ProjectTemplate/

and will be updated when they change.

How do I convert from Markdown to HTML?

Also relevant perhaps:

• https://help.github.com/articles/setting-up-a-custom-domain-with-pages
• https://help.github.com/articles/custom-404-pages

Use the test_that function as supported by the testthat package; store tests in tests/testthat.

Too easy to forget.

Options:

• Add DUMMY files (instead of .gitignore which get stripped off during building)
• Store templates as a tar

Closes #48.

As in plyr.

Should be enough, otherwise the index gets cluttered. Should link to the corresponding "worker" function.

The idea is to create a package repository and library exclusively for the project, to support true reproducible research.

rbundler is on CRAN, packrat supports GitHub. Tricky decision...

Perhaps allow setting up a Git repository and/or auto-commit to Git

• Just the current stuff
• New maintainers

In load.project(), datasets are not added to the cache and data members of project.info if they already exist in the global environment. I wonder if this is by design.

Steps to reproduce:

1. Create empty project using create.project("test").

2. Add test.csv with the following contents to the data directory:

   answer
   42
   

3. Run supplied script src/eda.R -- the test dataset is printed

4. Re-run the same script in the same session -- the test dataset is not printed

@johnmyleswhite: Could you please comment? From the logs, it looks like this behavior always has been like this, however it seems counterintuitive to me.

Some of the readers, like the SQL readers, require password information. Currently this is stored directly in the data file. Storing passwords in files is a security risk -- they can be checked into repositories or made accessible to the public.

Other communities use environment variables to store sensitive information. Can you do this in ProjectTemplate?

#25 (comment)

This might mean to transform ChangeLog to NEWS.md. Logic to transform the latter to an R NEWS file can be stolen from e.g. the knitr package.

R CMD check in R-devel does not like that: https://travis-ci.org/johnmyleswhite/ProjectTemplate/jobs/20100039, line 6295+. Should be removed entirely, the search path for functions inside packages should be fine already. From the manual:

The namespace controls the search strategy for variables used by functions in the package. If not found locally, R searches the package namespace first, then the imports, then the base namespace and then the normal search path.

@KentonWhite: Are you ready to switch to roxygen 4.0.0?

Use R yaml to read configuration.

https://github.com/viking/r-yaml

See tikzDevice.

The parameter minimal should be replaced by a parameter template.name = c("full", "minimal") to allow adding other templates.

Website documentation has to be updated.

Test, possibly fix, document.

See e-mail below:

Dear maintainers,

This concerns the packages

AWS.tools BAS COMPoissonReg CPHshape DOBAD GSM LinearizedSVR
Mobilize NPsimex OIdata OLScurve Ohmage PhysicalActivity
ProjectTemplate RSQLite.extfuns RUnit Rknots SGCS
SortableHTMLTables StreamingLm TGUICore UScensus2010 arf3DS4 bcool
bigdata blender cacheSweave cggd cudaBayesreg cumplyr disp2D emg
events fCertificates fastGHQuad fractaldim fts gWidgetsWWW genridge
govdat hof ktspair labeledLoop liso log4r mixreg mutatr
objectProperties plumbr primer profr qtbase qtpaint rggobi s4vd
sROC simSummary spatialsegregation traitr treebase turboEM webvis
zipcode

maintained by one of you, for which R CMD check finds problems in the
'for hidden files and directories' check, see below for details.

Can we pls have updates fixing these within the next 2 weeks?

Note that other R CMD check problems may need to be addressed as well.

Best
-k

Package: AWS.tools Version: 0.0.5
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: BAS Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.Rbuildindex.1064
.Rbuildindex.1800
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: COMPoissonReg Version: 0.3.4
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: CPHshape Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.Rapp.history
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: DOBAD Version: 1.0.1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
inst/doc/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: GSM Version: 1.3
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: LinearizedSVR Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: Mobilize Version: 2.11-2
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: NPsimex Version: 0.2-1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.Rapp.history
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: OIdata Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.Rapp.history
data/.Rapp.history
data/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: OLScurve Version: 0.1.3
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: Ohmage Version: 2.11-2
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: PhysicalActivity Version: 0.1-1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
data/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: ProjectTemplate Version: 0.4-2
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
inst/defaults/full/cache/.gitignore
inst/defaults/full/config/.gitignore
inst/defaults/full/data/.gitignore
inst/defaults/full/diagnostics/.gitignore
inst/defaults/full/doc/.gitignore
inst/defaults/full/graphs/.gitignore
inst/defaults/full/logs/.gitignore
inst/defaults/full/munge/.gitignore
inst/defaults/full/reports/.gitignore
inst/defaults/full/src/.gitignore
inst/defaults/minimal/cache/.gitignore
inst/defaults/minimal/config/.gitignore
inst/defaults/minimal/data/.gitignore
inst/defaults/minimal/munge/.gitignore
inst/defaults/minimal/src/.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: RSQLite.extfuns Version: 0.0.1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: RUnit Version: 0.4.26
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.cvsignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: Rknots Version: 1.2.1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: SGCS Version: 1.7
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.cproject
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: SortableHTMLTables Version: 0.1-3
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: StreamingLm Version: 0.1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: TGUICore Version: 0.9.15
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
.settings
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: UScensus2010 Version: 0.11
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
data/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: arf3DS4 Version: 2.5-4
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: bcool Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
inst/doc/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: bigdata Version: 0.1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.Rapp.history
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: blender Version: 0.1.1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
data/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: cacheSweave Version: 0.6-1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: cggd Version: 0.8
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: cudaBayesreg Version: 0.3-13
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.directory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: cumplyr Version: 0.1-1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: disp2D Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.directory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: emg Version: 1.0.5
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
data/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: events Version: 0.5
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: fCertificates Version: 0.5-2
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: fastGHQuad Version: 0.1-1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: fractaldim Version: 0.8-1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
tests/.Rapp.history
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: fts Version: 0.7.7
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
.gitmodules
src/RAbstraction/.gitignore
src/tslib/.gitignore
src/tslib/test/.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: gWidgetsWWW Version: 0.0-23
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
inst/javascript/js/.htaccess
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: genridge Version: 0.6-3
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: govdat Version: 0.1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: hof Version: 0.1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
.settings
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: ktspair Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.Rbuildindex.10034
.Rbuildindex.31551
.Rbuildindex.31678
.Rbuildindex.9995
inst/doc/.fr-hy2VZf
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: labeledLoop Version: 0.1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.Rapp.history
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: liso Version: 0.2
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.DESCRIPTION.swo
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: log4r Version: 0.1-4
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: mixreg Version: 0.0-4
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
data/.Random.seed.save
data/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: mutatr Version: 0.1.2
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: objectProperties Version: 0.6.5
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: plumbr Version: 0.6.6
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
inst/tests/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: primer Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
data/.Rapp.history
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: profr Version: 0.2
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: qtbase Version: 1.0.4
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
inst/doc/.Rhistory
vignettes/.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: qtpaint Version: 0.9.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
inst/profiling/.Rhistory
vignettes/.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: rggobi Version: 2.1.19
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: s4vd Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: sROC Version: 0.1-2
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.Rapp.history
inst/script/.Rapp.history
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: simSummary Version: 0.1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
vignettes/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: spatialsegregation Version: 2.38
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.cproject
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: traitr Version: 0.13
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
testing/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: treebase Version: 0.0-6
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.gitignore
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: turboEM Version: 2012.2-1
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
inst/doc/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: webvis Version: 0.0.2
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
.project
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Package: zipcode Version: 1.0
Check: for hidden files and directories ... NOTE
Found the following hidden files and directories:
data/.Rhistory
These were most likely included in error. See section ‘Package
structure’ in the ‘Writing R Extensions’ manual.

Files in config/ are .dcf, but files in other directories are as well. This is not good. For example, according to the documentation,

.sql: A DCF file that contains database connection information for a MySQL database.

but the OS and other programmes will see this as a file containing SQL code. A project maintainer may want to have SQL code in .sql files to keep it out of R code and also share it with other programmes. It doesn't make sense to redefine all these file types just for the convenience of the framework.

A better solution would be foo.sql.dcf. That way the format of the file is clearly the same as that in config/, but it's also clear to humans and the framework that it deals with the configuration of loading databases.

The same would go for .url and .file.

https://github.com/johnmyleswhite/ProjectTemplate/network

• New parameter to load.project and reload.project, interpreted as basename without extension in the config directory
• Cascading configuration

Closes #82.

Should be a function instead, there seems to be no obvious way to fix the NOTEs fired during CRAN checks.

To make sure transition is as painless as possible, we could say something like

makeActiveBinding(
  "project.info",
  function(x) { warning("project.info is deprecated, use xxx instead."); ... },
  .GlobalEnv)

and later replace the warning by a stop.

@johnmyleswhite: Do you still have the original uncompressed images for the website?

So that Git does not ignore the empty directories. This would also fix warnings during R CMD build.

#25 (comment)

Currently it's a static HTML list. Now possible with Jekyll.

For better reproducibility. See sort_c in roxygen2.

So that projecttemplate.net can be redirected to GitHub pages.