cloud-cv / fabrik Goto Github PK

I don't think TensorFlow should be part of the requirements.txt. Users could install this separately (or a build script can be written for the same - e.g. a Makefile) since TensorFlow wasn't completely registered under pip (pip installable) until their initial version release. This can also help new users of Fabrik to opt for GPU-tensorflow (which requires NVIDIA CUDA and CuDNN installed).

We want to make the Fabrik ModelZoo a rich collection of models from all streams of deep learning research (vision, speech, text) and want to add models from both frameworks that are currently well supported by this project (Caffe and Keras).

For people looking to get started with contributing to this project, this is a very good starting point. To add a model to the model zoo, please do the following:

• Add the model prototxt/json to the example/<framwork> folder depending on the model being from caffe or keras
• Add the entry to the models drop-down list on the front-end
• Test if loading the model and exporting it to both or at least one framework is working fine

We want to have a feature to generate a shareable link to a network built by a user that another user can view, edit. We will need to store the link and prototxt in our database when the user clicks share, and look up and regenerate the network from DB when another user tries to access the link.

Eventually we want to also allow collaborative editing of networks with real time updates.

@deshraj @dexter1691.

task runner which runs our various tasks of compiling JS, bundling it and reloading browser on change.

I suggest that when the user clicks the "export" button, the system will show a dialog
or some other UI components which is easy to seize the user's attention. Since the position of "progress circle" is assigned to the top-right which is not very intuitive for the users, I think this design is not very well. Also, if the "progress circle" can show the progress to the user, it will be better.

Thanks for this tool!

Could you put a section outlining what is in-scope and out-of-scope in the README?

It's unclear to me for example whether I can use this to create LSTMs, seq2seq, or in general handle text.

Also what the input to the models are, i.e. what plumbing is required when I have the output from the tool in order to get data in and out from the model would be good to outline in the README.

Keras supports recurrent layers like LSTM / RNN / GRU etc. Right now model zoo for Fabrik doesn't contain any examples of network that uses these layers.

We should add a few examples to demonstrate the capability of using recurrent layers in a network and to test how import / export works for these layers.

We want to setup Sphinx for the documentation of Fabrik. Feel free to look at the configuration of sphinx that we have for EvalAI (https://github.com/Cloud-CV/EvalAI/tree/master/docs).

Current Scenario

Currently setting up Fabrik on a local machine for development requires running a lot of commands. We want to setup the docker containers so that setting up Fabrik is not pain.

Deliverables

A basic setup that fulfills the following requirements:

• Different containers for Postgres, NodeJS and Django
• Add instructions in README

Please comment for if you have questions.

The Video2Text network isn't working.
I believe the missing layers are the same as the layers needed for #36
I'm going to see if the fix for #36 will support it too.

@gauravgupta22 can you update the README with the project description and steps for installation?

The canvas should be refreshed and the layer block removed as soon as the user deletes a layer.

Fabrik can build / visualize very complex multi-modal architectures using Keras. One of the popular tasks for which we should add a model in model zoo is VQA. Here are some examples:

• Keras JSON files that can directly be imported: https://github.com/avisingh599/visual-qa/tree/master/models
• https://github.com/anantzoid/VQA-Keras-Visual-Question-Answering
• https://github.com/iamaaditya/VQA_Demo

We should add a title pane with the model name – this should be an editable field which:

• defaults to untitled if the user is building a model from scratch
• takes the name of whatever the model file is called if loading from zoo / importing from any framework
• Should go into DB – when a shared link is opened, the same name should show up.

When importing the json file of a keras model, the API throws a ValueError. There seems to be some trouble in loading the JSON data in the correct format. Following is a screen-shot of the error log:

The Squence to Sequence - Video to Text Model doesn't work.
The layers that cause this failure are:

• LSTM
• Embed
• HDF5Data
• Reshape

We should try to successfully import all the models from Caffe ModelZoo to see if some layers are not working.
Consequently fix or add the missing the layers into the IDE.

https://github.com/BVLC/caffe/wiki/Model-Zoo

For GSOC participants:
This seems like a very good starting point to see how IDE works. Here is what you have to do

• Choose one of the models from the link above and create an issue for it. Cross reference it here for others to see.
• We will assign the issue to you
• If the model is working. Add a unit test for it so that any later changes to the projects ensure compatibility with those models.
• If the model isn't working, try to discuss with the mentors and fix the issue!

We are maintaining the list of models tested by contributors here:
https://github.com/Cloud-CV/IDE/wiki/Models

This issue is dependent on #136.

Things to take care of while writing the documentation:

• Write detailed documentation
• Provide small snippets
• Add a simple example of adding ReLU

Image Captioning is another popular task for which we should have an example in Model Zoo

When I drag and drop any layer, the parameter is shown well.
But if new layer is dragged and dropped, that it doesn't display parameter.
Except for the first layer, the other layers don't show the parameters.

I tried importing the prototxt file from here:
https://github.com/terrychenism/ResNeXt

And it doesn't seem to work because a few layers are not supported
1.) BatchNorm
2.) Element Wise
3.) Scale

We should add the support for these layers specially because ResNet is now a standard model that people use a lot.

@virajprabhu @dexter1691 I've used this repo(https://github.com/GilLevi/AgeGenderDeepLearning) for testing the age & gender classification model from their project page (http://www.openu.ac.il/home/hassner/projects/cnn_agegender/) should I try this one?

The Inception Network is not working with Fabrik at the moment.
Layers that fail:

• Scale Layer

• BatchNorm Layer

Prototxt files exported from Fabrik (to Caffe) do not load parameters from conv layers.

I imported the example GoogleNet caffe model and when I tried to export it, the following error was thrown.

Error encountered: Layer not found: blob140
Internal Server Error: /tensorflow/export
Traceback (most recent call last):
  File "/home/sudar/anaconda2/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/home/sudar/anaconda2/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/home/sudar/anaconda2/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/sudar/anaconda2/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/sudar/anaconda2/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/sudar/Desktop/cloudcv/Fabrik/tensorflow_app/views/export_graphdef.py", line 40, in exportToTensorflow
    net = __import__ (str(randomId))
ImportError: No module named 20170403111133ncyxt
[03/Apr/2017 11:11:37] "POST /tensorflow/export HTTP/1.1" 500 49731

Alternatively, we could move the "choose file" functionality into the "import"? Having two steps doesn't seem necessary.

Whenever a layer is added in workspace and later deleted, I find this exception being thrown through the Chrome Console.

First traceback is while rendering the tooltip of hoveredLayer which would be undefined since it was deleted (I guess?):

Uncaught TypeError: Cannot read property 'info' of undefined
    at Tooltip.render (bundle.js:44344)
    at bundle.js:16503
    at measureLifeCyclePerf (bundle.js:15783)
    at ReactCompositeComponentWrapper._renderValidatedComponentWithoutOwnerOrContext (bundle.js:16502)
    at ReactCompositeComponentWrapper._renderValidatedComponent (bundle.js:16529)
    at ReactCompositeComponentWrapper._updateRenderedComponent (bundle.js:16453)
    at ReactCompositeComponentWrapper._performComponentUpdate (bundle.js:16431)
    at ReactCompositeComponentWrapper.updateComponent (bundle.js:16352)
    at ReactCompositeComponentWrapper.receiveComponent (bundle.js:16254)
    at Object.receiveComponent (bundle.js:8423)

Next, when I hover on another layer, it throws a new exception:

bundle.js:28035 Uncaught TypeError: Cannot read property 'info' of undefined
    at Content.changeHoveredLayer (bundle.js:28035)
    at Canvas.hoverLayerEvent (bundle.js:28862)
    at onMouseEnter (bundle.js:33113)
    at Object.ReactErrorUtils.invokeGuardedCallback (bundle.js:6822)
    at executeDispatch (bundle.js:6607)
    at Object.executeDispatchesInOrder (bundle.js:6630)
    at executeDispatchesAndRelease (bundle.js:6028)
    at executeDispatchesAndReleaseTopLevel (bundle.js:6039)
    at Array.forEach (<anonymous>)
    at forEachAccumulated (bundle.js:6921)

Cascaded Fully Convolutional Networks for Biomedical Image Segmentation does not load.

The missing layers are:

• Crop
• Deconvolution

while installing through conda tensorflow version required is >=0.12.0 so update readme for that detail

Caffe has all the layers and it's layer parameters defined in a prototxt definition. Instead of adding support for layers one by one inside IDE, it will be nice to automatically parse the definition from the master branch of caffe and add support for those layers.

Requirements: Understanding caffe's codebase written in C++, Understanding the prototxt structure and familiarily with automated tasks in Javascript to automatically build the layer support into IDE.

• We can first start by parsing the protobuf definitions to identify all the supported layer parameters.
• Identify the mapping between layer and layer parameters
• Identify number of input / output for each layer
• Add them into IDE

Benefits: Doing this will ensure that we don't need to actively maintain the IDE repo to in sync with the caffe branch.

I have found a bug while working on a fix for #38
If your network has RELUs before Dropout you will not have any trouble (Ex: AlexNet)
However in S2VT they use Dropout without RELUs, now this causes an error at jsonToPrototxt.py at line: 322
The reason for this is that the variable "inplace" is only set inside the RELU's context, and in this network
there are no RELUs so this causes a:
local variable 'inplace' referenced before assignment

I'm going to fix this as part of #38

Hello. I have some questions. When Python pane drags and drops, it doesn't work and occurs error. Maybe is it not implemented yet?

It would be nice to show layer details on hover (instead of the user having to click the layer box) - this is a nice example.

No border and border-radius for boxes also seems to look nicer.

I use a computer with 1366x768 resolution monitor. The drop-down menu for inserting a new layer overflows out of the screen. I am not able to add layers like embed, lstm etc. This has to be made scrollable at least for the moment.

When you try to export, it just throws internal server error.

What algorithmic frameworks are used on the project?

We want to support Keras as an additional backend.

Export: Since our intermediate backend representation is prototxt, we can use this to export.

Import: Keras exposes a model.to_json() function, we will have to write logic to parse the json to prototxt.

Minimum: We need to write documentation how you can plug the model config file generated from Fabrik into your code.

Stretch: We should write a wrapper function that does this, and add it to our documentation.

We would like to add different markdown files for setting up IDE on different Platforms such as

• Linux
• OS X
• Windows

The scroll to zoom is jerky and not very useful. Disabling it and adding a vertical scrollbar to the canvas will provide a better user experience.

I found a bug.

-Step to reproduce the problem

1. The model is loaded.

2. Export the model once.

3. Export the model again.

Then. An error occurs.

The error message is:
bundle.js:44086 Uncaught TypeError: Cannot read property '0' of undefined
at bundle.js:44086
at Array.forEach ()
at Tooltip.render (bundle.js:44081)
at bundle.js:16306
at measureLifeCyclePerf (bundle.js:15586)
at ReactCompositeComponentWrapper._renderValidatedComponentWithoutOwnerOrContext (bundle.js:16305)
at ReactCompositeComponentWrapper._renderValidatedComponent (bundle.js:16332)
at ReactCompositeComponentWrapper._updateRenderedComponent (bundle.js:16256)
at ReactCompositeComponentWrapper._performComponentUpdate (bundle.js:16234)
at ReactCompositeComponentWrapper.updateComponent (bundle.js:16155)

Trying to export GoogLeNet from the "Model Zoo" dropdown at the URL below:

http://fabrik.cloudcv.org/#/?_k=ngzknm

And Im getting the error, "Cannot convert LRN to Keras" [see attached]. Im guessing this is because the LRN regularization layer doesnt have an associated entity in Keras.

Test Model

Train Model

We should display the number of parameters(weights of the model) the model contains somewhere on the screen. These parameter values should change whenever a new layer is added or some existing layer is changed.