Few spots this has come up. Metadata is one word...

I just wanted to have a place where people can put ideas if they want for styling of the site.

@karthik suggested a theme from http://www.blacktie.co/ and they do look really nice AND it would be easy to incorporate.

I think I will deal with this after the official Hackathon so I can get a super clear picture of the content we need to display, but any thoughts are welcome.

None of the links are hyperlinked . You may need to do

http://recomputation.org/

instead of simply putting http://recomputation.org/

Would be easier for folks to navigate I think if workflows were more obviously separated from one another, and perhaps some sort of navigation bar for the different workflows on that page, esp needed as there are more workflows put there.

Would be nice to have some introductory materials/pre-amble to the guide that explains what the guide is and isn't.

Along those lines a working definition of reproducibility might be nice. Looking back at hackathon issue #22 suggest that there is at least a gradient of reproducibility. Much of our discussion (or at least my recollection of it) has centered around producing reproducible documents, but as many have indicated, reproducibility is a much larger issue than that.

At a simple level as long as one has access to code/data/paper it could meet some minimum criteria of reproducibility. The discussions on data archiving #21 and sustainability/versioning #19 bring up bigger issues that, if dealt with, would make a given piece of work a greater degree of reproducibility. Lastly, @mfenner made the good point that to be fully reproducible automation needs to be considered.

I have done a (very) little work with Linked Open Data and as I have laid it out here it reminds me of the 5 star system of Linked Open Data. Would it be useful to think of reproducibility in the same terms? Is that out of scope for this guide (quite possibly).

In any event, I would find it useful if we could develop an introduction and specific definition about what type/level/etc of reproducibility we are referring to. Will help focus the rest of the guide, I think.

👋 @iamciera! Thanks a lot for your work on this. As the guide isn't updated we'll archive it... unless you decide otherwise. 😁 We can also transfer it to your personal account. If you don't answer within one week we'll archive it but you could still email us to request we unarchive it. Thank you!

I just want to mention I am aware that the grey background really isn't working for readability. I am thinking of making the title of the section in a grey container and the body will be white. Any other suggestions?

This guide is getting pretty rusty. We should really think about what the function of it is. It has a lot of useful information, but I feel like the information is hard absorb. There are too many sections, uneven content, and outdated information.

The first question is, in the condition the site is in now, does it serve a valuable function? And/Or does it do any harm staying up in the condition it is in?

If we decide to keep it going, we should figure out how to make the site better. What do we need to change? What can stay? Organization? Ect?

One idea: Reproducibility is such a malleable topic and there are no real hard rules, making the creation of a good guide extremely difficult. I really like the questions section. Maybe we could put that on the front page and link content/references to learn more along those specific questions. Reproducibility is such a dry topic, I like the idea the front page having useful information that can be quickly digestible. Then give the audience the ability to look deeper with a few highlighted references.

Also, I really like the introduction.

Any thoughts?

The workflow section needs work, but take the most time to contribute to. It is also in my opinion one of the coolest sections! How to contribute:

1. If you have a workflow that you would like to add and you are familiar with jekyll/css, you can add a button on the main workflows page that connects to a page in a subfolder in the workflow directory that goes into more detail of the workflow.
2. Give a link to a site that contains a appropriate workflow. Collecting them at the moment, unsure how to incorporate. Suggestions welcome.
3. Email Ciera (ccmartinezATucdavis.edu). I will put it together for you.
   a. picture
   b. accompanying text explaining your workflow

Just saw http://www.gitbook.io/

Looks super simple, and you can make it work with just a few changes I think. ALthough this would be a departure from Jekyll I guess, not sure if you'd want to do that.

After the instruction 'You'll now have a local copy of your version of that repository called "origin"', do you think we should add 'you'll need to change directory in git bash to be working in the new directory that is the local copy'. I think I've been stuck with cryptic error messages at this point a few times. What do you reckon?

http://ropensci.github.io/reproducibility-guide/sections/tools/
para 1: lose the extraneous 'to'
“All the tools listed to here are free,” should read:
“All the tools listed here are free. “

is it 'Markdown' or 'markdown'?  should be consistent.
    “a list of applications that provide support for authoring in Markdown”
    “including a basic tutorial on markdown”

is it 'html' or 'HTML'? should be consistent.
    “html in current release, additional formats in preview release”
    “An R Markdown document can also be converted to standalone HTML file”

Another resource worth listing: https://github.com/kbroman/Tools4RR

@kbroman does fantastic things in this space and you guys should ask him to do a little review at some point.

After I moved the 2014...Introduction-to-Tools.md content from _posts to to introduction folder, I was sent an email that jekyll is not able to build the site. It works on my computer though. Not sure what is going on.

It would be nice to have the workflows easily generated so in the future people can add their own, ideally with all of them matching in format and color coding. I can generate them fast in Illustrator, but this would not help long term.

Another will be collecting the workflows.

"... greater efficiency that simply ..." should be "... greater efficiency than simply ..."

After Github switched to a different Jekyll we get page build problems:

You are currently using the 'redcarpet' Markdown engine, which is no longer supported by GitHub Pages and may cease working at any time. To ensure your site continues to build, remove the 'markdown' setting in your site's '_config.yml' file and confirm your site renders as expected. For more information, see https://help.github.com/articles/updating-your-markdown-processor-to-kramdown.

This needs to be fixed.

I like the idea that this site is a curation of information where one function is the uniting of existing material. If you guys have excellent references pertaining to a particular section, even if they are in the References section, we should highlight them in the sections. Maybe in a resources section on each page. We should keep them to only the best though, so not to overwhelm. Feel free to add as you please.

Links to Gentzkow and Shapiro' writing code manuals are broken. The proper one is http://web.stanford.edu/~gentzkow/research/CodeAndData.pdf . As Gentzkow wrote, '[it] includes material previously circulated as "Notes for RAs on Writing Code"'

As the first thing anyone reads, the "About" text on the index page should consist of well-edited paragraphs.

Some issues:

1. "Unfortunately, reproducibility of results is actually becoming increasingly more difficult, owing to the variety of ways of approaching analysis and incapability of data structures and file types." Is it really becoming MORE difficult? I doubt that. Edit this sentence to avoid making a potential untrue statement, or provide references. Then, "incapability of data structures", is this supposed to say "incompatibility"? Otherwise I'm not sure I understand how file types and data structures can be incapable.
2. "While technology increasingly evolves": "increasingly" or "increase" appears in 3/4 of the previous sentences, delete or replace with something more meaningful
3. "creating entirely new section": creating an entirely new section
4. "Contribution to the guide is highly encouraged by collaboration through Github.": add link to the github site, ideally to a page with instructions on how to contribute

Currently the site is building the links with the blog style permalinks and is also ordering those in a manner that I can't seem to figure out.

Any thoughts on how to change the ordering (i.e [here (http://ropensci.github.io/reproducibility-guide/)) of the links? Should we override the permalinks with shorter, less bloggy links? I don't have a lot of experience with Jekyll, but I think we can specifiy the specific link with the YAML on each page.

Few spots this has come up. Metadata is one word...

I do not like how the sections align on top of each other in response to smaller windows. I haven't had time to figure out how to change this. If someone is very familiar with Bootstrap/CSS/HTML this could be a good way to help. I was thinking just have the section "buttons" get smaller, but still remaining in two rows. This will become a bigger issue if more topics are added.

For some reason I could not get the baseurl for the posts to work correctly. So I cheated and added the projects name at all the links with the liquid tags {{ base.url }} should be. Can anyone help?

What do you guys think about the collaborators section by the made possible by header on main page? I always enjoy learning about the people behind projects, even if it is just a some names and hyperlinks. If we encourage people to add their name when they contribute, it would likely promote collaboration. But then again, seems a little self promoting?

What do you guys think?

First Cut at an outline for orgnaizing/navigating the site. I am not wed to any of this. I just like lists!

• Introduction
  • Definitiion (Gradient of and Focus for the Guide)
  • Tools
• Workflows
  • @Edild
  • Kieran
  • Web
  • RStudio (0.98.745)
• Expanding Reproducibility (see #5)
• Related Topics
  • Data Archiving
  • Metadata Best Practices
  • Dependencies
• Further Reading

Currently there are no widely accepted methods of conveniently telling people that your work is reproducible, or where it is on the spectrum of reproducibility. We might consider supporting one of the proposed standards and promoting use of tags, labels or badges to simplify communication of reproducibility and promote the practice in general.

Here are a few standards that have emerged so far:

• Schwab et al.: ER (Easily reproducible), CR (Conditionally reproducible), NR (Not reproducible)
• Biostatistics kite-marking of articles (Peng 2009): D (data), C (code), R (both)
• Reproducible Research Standard (Stodden 2009): we should release
  • The full compendium on the internet
  • Media such as text, figures, tables with Creative Commons Attribution license (CC-BY)
  • Code with one of Apache 2.0, MIT, LGPL, BSD, etc.
  • Original "selection and arrangement" of data with CC0 or CC-BY

Victoria Stodden also suggests these categories:

• 'Reproducible': compendium of text-code-data online
• 'Reproduced': compendium available and independently reproduced
• 'Semi-Reproducible': when the full compendium is not released
• 'Semi-Reproduced': independent reproduction with other data
• 'Perpetually Reproducible': streaming data

Randy Leveque http://icerm.brown.edu/html/programs/topical/tw12_5_rcem/icerm_report.pdf

• Reviewable Research. The descriptions of the research methods can be independently assessed and the results judged credible. (This includes both traditional peer review and community review, and does not necessarily imply reproducibility.)
• Replicable Research. Tools are made available that would allow one to duplicate the results of the research, for example by running the authors’ code to produce the plots shown in the publication. (Here tools might be limited in scope, e.g., only essential data or executables, and might only be made available to referees or only upon request.)
• Confirmable Research. The main conclusions of the research can be attained independently without the use of software provided by the author. (But using the complete description of algorithms and methodology provided in the publication and any supplementary materials.)
• Auditable Research. Sufficient records (including data and software) have been archived so that the research can be defended later if necessary or differences between independent confirmations resolved. The archive might be private, as with traditional laboratory notebooks.
• Open or Reproducible Research. Auditable research made openly available. This comprised well-documented and fully open code and data that are publicly available that would allow one to (a) fully audit the computational procedure, (b) replicate and also independently reproduce the results of the research, and (c) extend the results or apply the method to new problems.

Open Science Framework & the journal Psychological Science have attractive badges that could be useful: http://centerforopenscience.org/journals/ http://www.psychologicalscience.org/index.php/publications/observer/2013/november-13/whats-new-at-psychological-science.html

"How can software tools can make ..." should be "How can software tools make ..."

Are the files in the _site/ directory of any use at all? From what I can see, the website text content, for example, is generated from sections/. _site/ is in .gitignore. Perhaps the directory should be just deleted.