Git Product home page Git Product logo

rcloud's Introduction

RCloud: Integrated Exploratory Analysis, Visualization, and Deployment on the Web

RCloud is an environment for collaboratively creating and sharing data analysis scripts. RCloud lets you mix analysis code in R, HTML5, Markdown, Python, and others. Much like Jupyter notebooks, Beaker notebook, Apache Zeppelin, Sage, and Mathematica, RCloud provides a notebook interface that lets you easily record a session and annotate it with text, equations, and supporting images.

Unlike these other systems, RCloud:

  • lets you easily browse and search other users's notebooks. You can comment on notebooks, fork them, star them, and use them as function calls in your own notebooks.

  • lets you interpret notebooks as web services: your exploratory data analysis are one step away from an automated dashboard.

  • provides an environment in which R packages can create rich HTML content (using, for example, d3 and dc.js).

  • provides a transparent, integrated version control system. In essence, RCloud never forgets what you did. If you need low-level access to RCloud notebooks, you can simply clone the associated git repository. This is because RCloud notebooks are Github gists

Interested? Try RCloud on the public instance on rcloud.social, or install the Docker image.

rcloud's People

Contributors

agrawal-mohit avatar amol-jore avatar anatoliyg avatar bashlee avatar benmarwick avatar dougmet avatar ejen1 avatar gaborcsardi avatar ganapatibhat avatar gordonwoodhull avatar holyguster avatar jameesy avatar nandakishorkoka avatar paulinshek avatar prateek05 avatar prayagverma avatar s-u avatar shaneporter avatar shyamram avatar smschauhan avatar sneha-bharti avatar sujitbehera27 avatar tlarchukatatt avatar useless5771 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rcloud's Issues

Keys for submitting part of the input as a command

Now that we have the ace command entry, shift+enter (or shift or ctrl) allows entering multiple lines, enter submits. Should we add ALT+enter to submit a line at a time?

Perhaps if something is selected it sends that, otherwise it sends the current line? And we could use the same convention for the scratch area when we have that.

Unimplemented 23

RCloud can't recover from syntax errors. I'd like to fix this myself, unless there's some reason why this is more difficult than it seems?

Error running rcloud

I am trying to run rcloud. I followed the install instructions, but after running scripts/fresh_start.sh I get the error ERROR: dependency ‘uuid’ is not available for package ‘rcloud.support’.

I see that the uuid.r file has a bunch of functions, but I don't see why this error is occurring.

notebook tree sort is not stable

if there are a bunch of notebooks with the same name, editing one of them will cause it to go to the end of the set.

of course, this isn't a fun place to be anyway, but we shouldn't make it worse on people.

Unable to Install rcloud

This is what I got

> install.packages("rcloud.support",, 
+     c("http://RForge.net", "http://R.research.att.com"), type="source")
Warning in install.packages :
  packagercloud.supportis not available (for R version 3.0.1)
Error in install.packages : argument is missing, with no default
> sessionInfo()
R version 3.0.1 (2013-05-16)
Platform: x86_64-apple-darwin10.8.0 (64-bit)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] tools_3.0.1

wild scrolling

the edit control with the focus frequently isn't visible because rcloud is scrolling to some other edit box or something. seems to be triggered by in ACE.

Well-defined API for interaction between R and a notebook

We currently have a few examples of function calls and libraries that do more than simply display results via knitr (wplot, wgeoplot, and the fuller-featured wdcplot that Gordon wrote). All of the interaction happens via rcloud.deferred.result and a hard-coded dispatch. This is a gigantic kludge.

We should have a more defined API for an R library to manipulate the client-side state in a notebook.

I don't think we will ever hit a point in the design space that is easier for a user to use than Shiny has, so that's not where we should aim. On the other hand, I think there's a nice opportunity to create an API that lets people write libraries: I'd love for RCloud to foster an ecosystem (however small) of libraries that offer rich R/Javascript interaction. So how do we make this work? At the highest level, there should be a place for libraries to register pieces of Javascript for Rcloud to serve. This piece of Javascript would be called via an R function, and passed all parameters given to the R invocation. This is more or less how the current infrastructure works, but without the extensiblility.

Here's a strawman proposal for bare-bones functionality. From the RCloud library point of view, the R code would look something like this:

javascript.caller <- rcloud.register("my_fancy_library.js")

my_fancy_library.js is a file containing a single Javascript function definition. javascript.caller would be an R callable object which, when invoked, will send the values down to the javascript function. This Javascript function would have access to the rest of the Javascript-side RCloud API, and would be able to create divs, do fancy things with graphics, etc.

With a more fleshed-out API, we could have calls to create entries in the notebook, to issue warnings, errors, etc.

(In addition, as RServe moves to the object-capability idea we discussed, the Javascript object could be initialized with the right set of R capabilities to enable calling back from Javascript-land to R-land without resorting to full-blown eval.)

Rcloud requires Rserve 1.7

Building Rserve fails

It should be noted that Rcloud depends on Rserve 1.7 for at least two reasons: the option --RS-source and the .conf option http.port where older versions have port.

I could not get Rserve 1.7 to build on OSX (neither could Rforge).

malecki$ gcc --version
i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)

The first set of errors (the ones reported on Rforge) are about -fnested-functions; even setting CFLAGS= -fnested-functions in the R Makeconf, I get new errors:

Rserv.c:2913: error: invalid storage class for function ‘setup_signal_handlers’
Rserv.c:2923: error: invalid storage class for function ‘restore_signal_handlers’
Rserv.c:3298: error: expected declaration or statement at end of input

Hack

I downloaded the binary package that Simon managed to build (how?! how?? just old gcc? I may try rebuilding R with clang to see if that works).

I hacked the meta/package.rds to pretend that it had been built on x86_64-apple-darwin12.0.0 instead of universal-apple-darwin9.8.0, and moved libs/x86_64/* to libs/ and it works fine.

R command prompt greedy with the keystrokes

A few hotkeys don't work within the command prompt (at least on OSX Chrome):

  • command-R reload
  • command-` switch windows

Oddly, command-T new tab does work (usually?)

Can click on other elements to get the hotkeys to work. They do work from within the RMarkdown cell.

run button

would be nice to have a run button next to already-run cells: when a cell above has changed it's natural to just run the dependent cells

cursor issues in edit control

Reported by a user:

Editing a long script – cursor issues
a. If I am working with a long script and run the script and then opened it for editing.
b. Edit a line but cursor is not near the top (for example cursor at line 27),
c. Run the script and open for editing – cursor is displayed at line 27
d. Ttype a character and the cursor moves to the first character of the script and then puts the character that I typed there (line 1)
e. This causes me to screw up the script J - I think I’ve noticed variations on this where the movement of the cursor is quite right – sometimes I have to scroll first and sometimes I have to hit return to get the cursor on the file and the cursor on the screen to “synch” – I’m not sure its always repeatable but steps a- d seem to be repeatable.

Duplicates in the notebook list

As new commands are run in the currently edited notebook, new entries show up in the notebook list with the same name but different timestamps, e.g.:

My Interests
[New Notebook]
Notebook 1   6/17
Notebook 1   9:52
Notebook 1   9:53
»All Notebooks
Notebook 1   6/17
Notebook 1   9:52
Notebook 1

To replicate: open a notebook, close it (e.g. by clicking on another one or closing RCloud), open it and enter a new command

(Note: gitsts branch)

Scratch area / pane / file

It'd be nice to have a scratch area on the side where one can keep snippets of code to paste into the active section. Even better, assign a key combo to copy text and execute it in the notebook proper.

This will require some changes to the layout.

It could also be used for modifying the (not yet used) notebook CSS, which will help e.g. with dc.js. We may need some sort of picker or tabs to decide what "extra file" to edit in the scratch area.

Create proper UI for help

The help (e.g., triggered by typing ?ls) is currently placed in a div on the side which is a) not visible by default [the user must expand it explicitly] and b) is too small (narrow) to read the help properly.

zero-prefix notebook part numbers

A notebook will look more coherent in the github gist UI if its parts are in order, but presently we get part1, part10, part11... part2 because it's lexicographical order.

Instead name them part001, part002, etc. I don't see any reason to reject manually-generated (or old) gists that are named part1, part2 -- but this might require upgrade logic because rcloud doesn't keep track of the actual filename internally.

floating right-hand-side menu (notebooks ,search, help )

When working on something two -three pages deep, users are forced to stop work and scroll up in order to a) refer a different notebook b) search for something of interest .

one solution :
adding following styling to div element allows the right-side menu to keep floating with the user's work. (However, this hard-fixes the div at 60:10 ...)

main.html , line 87 :
<div class="span3" style="position:fixed;top:60;right:10">

programmatically invoking notebooks

Right now each rcloud notebook is (to oversimplify a bit) a sequence of R calls. If there were a way to associate formal parameters and results to this sequence, then we'd have a procedure.

Since we're accessing R scripts via HTTP already, it seems natural to want to have support for turning these same R scripts into web services. If we had some function call in rcloud that interpreted an rcloud notebook (a gist) as a lambda, then this would have two consequences:

  1. web services could simply be implemented as calling said gists with appropriate parameters (url-encoded or something?)
  2. we could expose this API in rcloud.support, and then rcloud notebooks could invoke other notebooks as function calls. This would pave the way for notebooks as libraries.

Git submodules permission problems

Here is what I end up with when I do
git submodule update

Cloning into 'htdocs/lib/js/rserve-js'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Clone of '[email protected]:cscheid/rserve-js.git' into submodule path 'htdocs/lib/js/rserve-js' failed

Gist errors are unhandled

When get.gist() fails with an error, the JS side attempts to parse the result as JSON and fails. For example, when a notebook gist is deleted on github, clicking on it in RCloud will exhibit this behavior.

Deleting multiple cells doesn't always succeed

When deleting multiple cells in succession, some of them may still remain.

To reproduce: create a notebook with let's say 5 cells and then delete them all by quickly clicking on the trash icon one after the other. The cells disappear, but typically one or more cells will remain when the notebook is reopened.

autocomplete

ACE has autocomplete support, and so does R. We should look into doing this. Making it low-latency will be.. interesting.

Bad gist schema for creating a notebook

Creating a notebook fails due to bad gist schema. The github API expects a schema like -

{
  "description": "the description for this gist",
  "public": true,
  "files": {
    "file1.txt": {
      "content": "String file contents"
    }
  }
}

But the gist schema used to create a notebook is -

{
   "description":"Notebook 1",
   "public":false,
   "files":{
      "scratch.R":"
   }
}

This leads to a 500 Internal Server Error. This is probably a bug in the github API but sending a schema as expected -

{
   "description":"Notebook 1",
   "public":false,
   "files":{
      "scratch.R":{
         "content":"Lorem Ipsum"
      }
   }
}

solves the problem.

Editing non-writable gist should fork

Currently, RCloud allows users to edit other people's notebooks, but the change is not persistent. Ideally, it should fork the notebook in that case and continue work on the forked copy.

private notebooks

presently we use only private gists because we figure at some point people may not want to make all their notebooks public - but there is no way not to actually share them.

the idea is to add a flag to the config that tells rcloud whether to let other users know about a notebook - they would still be private in the gist sense (with a big unguessable hash name) but they would visible in the tree, as all notebooks currently are.

there is no reason notebooks couldn't also be public in the gist sense, except i don't know where it would go in the ui.

do we want an "open in github" link?

as a developer, this is something i do - is this something power users will want too?

if so, is there an inobtrusive place to stow the link? seems too advanced to go up on the main menu.

Redirect users not logged in to the login page

Sharing a notebook URL with a user not logged into RCloud leaves the user with no option to login. This page should preserve the request, redirect to login, and then bring back to the original URL if successfully authenticated..

not-logged-in

Configuration Madness

We need a single configuration file for rcloud, wherever that ends up being. Refer to readme.md for evidence that the current configuration dance is crazy :)

add `hash` package to R package depends

In doc/Readme.md, change to
install.packages(c("hash", "Rserve", "FastRWeb", "knitr", "markdown"),, c("http://rforge.net", "http://r.research.att.com"))

More generally I think if Rcloud were done up as an R package itself you'd gain the benefit of formal dependencies (and questionable benefit of a namespace) … install-failure where I had to find debugmode and read pretty closely to find this failure.

Error after logging in

I get the following message after I try to log in

Error in content(result)$access_token : 
  $ operator is invalid for atomic vectors

Any thoughts on what might be causing this?

R_TempDir is fixed across fork

The tempdir() in R cannot change after initialization so forked instances share it. This is particularly bad when using user switching. Can be addressed either in Rserve or in iotools (or both) by direct R_TempDir access after fork.

Run Rcloud as guest

We need to eventually allow people to view and run Rcloud notebooks as guests, without requiring them to have github accounts.

I think the technically correct solution here is to provide limited API access via rgithub, since, for example, gists are visible (but not editable) without OAuth.

notebook tree updates are slow

we are seeing maybe 0.5s just updating the tree

partly this may be because rcloud doesn't currently use the updateNode function of jqTree because we only recently updated to the latest.

also we are making a whole lot of small changes when it may make more sense to make be one big change.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.