cesium-ml / cesium_web Goto Github PK

Add new table for in progress tasks, only create entry for featuresets/models/predictions once that job is successfully completed. For now we just add the full entry to the DB and if the job fails it's orphaned.

Make sure deleting datasets doesn't delete files from other datasets...

Could just be internally. Mostly I just don't like that I don't know how to do it 😬 .

Would it be crazy to try to automate this via Travis...?

Like the ones for features and predictions.

• For datasets, show the list of files (+ any metadata)?
• For models, show type, hyperparameters, etc. (accuracy?)

Instructions still mention setting up the RethinkDB.

NaNs, etc.

Currently activity on forks also triggers Slack notifications

When a PR is made against cesium (the library), the dependent build server starts a cesium_web build, and sets two variables: TRIGGERED_FROM_REPO and TRIGGERED_FROM_SHA.

We should modify the cesium_web build to pull its version of cesium from the commit specified in these variables (if available), otherwise from PyPi.

As in Ben's reskin

Currently the error provided is Unknown label type: array([[nan], ..., [nan]]).

via Stormpath or maybe something else?

Since reorganizing celery tasks to be more easily chained/used in the web front end, performance has gotten horrible: the EEG example takes ~20s to featurize with celery as opposed to <1s without.

Could be:

1. Bug in celery involving chains/chords
2. Something inefficient about my implementation (but what really changed...?)
3. celery is bad and it should feel bad

Right now we hard-code a single 1-thread worker; should use the # of cores or something along those lines.

• Tooltips / documentation / info
• Verify data management / upload
  • Empty file
  • Improperly formatted header
  • Invalid zipfile
  • Mismatched header/zipfile
• Project / login info in sidebar (move from top)
• Data tab
  • Transformations (train/test split)
• Features tab
  • List in-progress tasks
  • Organize features tab
    • Separate out Lomb Scargle features
  • Custom features upload
• Models tab
  • Models tab hyperparameter optimization
• Predict tab
  • Display prediction results
  • Export as CSV
• Demo Docker container
• Plots
• Display notification for successful / failed tasks
• Look into similar open source projects
• Miscellaneous UI fixes
  • Update project should create when no projects exist (@stefanv)
  • Remove Debug columns
  • Add Dataset Delete, fix Featureset Delete
  • Add Finished column (already stored as model.finished), or some kind of In Progress label
  • Allow prediction from featureset (w/o recomputing)

Right now, fixture data is created before each test, but it isn't necessarily selected on the front end; I think we just keep whatever the default (first) value is.

Actual example of why this is bad: a test was added that checks that a 0-feature featureset can't be created; #71 broke this, and so when running those tests that featureset gets created and then also used for model-building tests. So every single test after that one fails, even though actually everything is working properly. Kind of defeats the purpose of using fixture data at all.

I assume this is an easy fix but have no idea how form selections work in React...cc @acrellin .

Could be related to @stefanv's changes to param validation in the Tornado port?

Error: Parameter alphas in model RidgeClassifierCV has wrong type

Users can deselect all features

When trying to use the docker image, I'm getting the following:

ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@rabbit:5672//: [Errno 111] Connection refused.

The log file for Celery looks like this:

[2016-07-15 14:55:52,141: WARNING/MainProcess] /usr/local/lib/python3.4/site-packages/celery/apps/worker.py:161: CDeprecationWarning: 
Starting from version 3.2 Celery will refuse to accept pickle by default.
2016-07-15T14:55:52.141976247Z 
The pickle serializer is a security concern as it may give attackers
the ability to execute any command.  It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.
2016-07-15T14:55:52.142135760Z 
If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::
2016-07-15T14:55:52.142188897Z 
    CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
2016-07-15T14:55:52.142217264Z 
You must only enable the serializers that you will actually use.
2016-07-15T14:55:52.142251624Z 
2016-07-15T14:55:52.142265507Z 
  warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
[2016-07-15 14:55:52,289: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@rabbit:5672//: [Errno 111] Connection refused.
Trying again in 2.00 seconds...
2016-07-15T14:55:52.290243145Z 
[2016-07-15 14:55:54,302: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@rabbit:5672//: [Errno 111] Connection refused.
Trying again in 4.00 seconds...
2016-07-15T14:55:54.302881711Z 
[2016-07-15 14:55:58,318: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@rabbit:5672//: [Errno 111] Connection refused.
Trying again in 6.00 seconds...
2016-07-15T14:55:58.318454238Z 
[2016-07-15 14:56:05,622: WARNING/MainProcess] celery@18946890287d ready.

It almost looks like the initial connection errors occur while the rabbit image is booting, and that error is carried through to when I try to generate features.

Hi,

i have a feature request regarding the plotting.
at the moment all features are plotted within the page:
https://s31.postimg.org/huxwtniiz/Screen_Shot_2016_07_20_at_5_47_08_PM_1.png

would it be possible to select the feature to be plotted and plot it in a separated window in order to see the data with a higher resolution and possible zoom and move?

Currently the tables are only added by tools/db_drop.py. It would make a lot more sense to have them also be added in db_create.sh so that that's a self-contained operation.

Function and module docstrings

Should include

• What is the project: short description of purpose
• License notice (actual license is in COPYING)
• Installation instructions from packages
• Installation instructions from source

Features too, maybe...?

Otherwise nothing works, and users don't know why

Display probabilistic prediction results in order of decreasing class probability. Perhaps even label first as "Top Class"

Presumably still using Docker.

latest would be an optional pass? It's annoying to not know if the build is actually broken or if a release just needs to be made; this would automatically clarify that.

Hi,

Im not sure how to start the app on osx. Could you please give some feedback/infos?
http://ideone.com/oDCNje

Add a mechanism to grow the number of workers.

Currently we call other Celery tasks from within celery_tasks.featurize_task; this is a bad idea and cause an exception starting in Celery 3.2.

Option 1) Microservice that monitors launches new tasks in subprocesses + monitors them
Option 2) Chained Celery tasks (chord to accumulate?)

The newly created project is not selected, and this message (with a square?) is displayed when it shouldn't be.

Console shows Uncaught TypeError: Cannot convert undefined or null to object which is raised during throw err; in

      var nextState = state;
      var nextError = void 0;
      try {
        nextState = reducer(state, action);
      } catch (err) {
        nextError = err.toString();
        if ((typeof window === 'undefined' ? 'undefined' : _typeof(window)) === 'object' && typeof window.chrome !== 'undefined') {
          // In Chrome, rethrowing provides better source map support
          setTimeout(function () {
            throw err;
          });
        } else {
          console.error(err);
        }
      }

Works fine in Firefox.

Especially when an error occurs, having the full (potentially detailed) description of what failed is important for debugging on the front end.