thinkingforms / vlml Goto Github PK

View Code? Open in Web Editor NEW

23.0 10.0 3.0 10.93 MB

Machine learning in a visual language

License: Other

C# 100.00%

vl machine-learning library flow-based-programming cntk ai dotnet

vlml's Introduction

VLML

Machine learning in the visual language VL, part of vvvv Gamma and vvvv Beta

VLML is a machine learning framework that aims to take some of the hassle out getting your hands dirty with machine learning, making it a relatively easy to get started. VLML uses CNTK as its back end, one of the most performant machine learning libraries out there. There is a short guide here on how to install VLML.

For now I recommend installing and testing VLML with vvvv alpha f7d5bf1879 because later versions currently have some compile errors, I'll update this when they stabilize.
There are now some demos available to download from here that go through some basic usecases for machine learning. A git repository can be found here.
Finally there is also a toy to play with: pix2pix face painting!. This is an evaluator of a pix2pix (ConditionalGan) model was trained on photographs and segmentation data. The neural network will be released soon so you can train your own. See more about this subject here.

Setting up a basic network

Create and two flattened collections of data.

The first will be X (the kind of data you want to process) and second will be Y (the kind of data you expect the process to result in).

DataInput: Create DataIn nodes for each of your datasets

The sample size refers to the number of elements in each sample of your dataset. In this example X has 3 elements, Y has 1.

Design your network

Put in some layers. Here we have just one and without an activation function.

Pass your network and target data into a loss function.

There several to choose from that are best suited to different ML tasks. You can also write your own.

Set up the trainer

Pass your network and loss function into a trainer:

Set your Minibatch Size This the size of a chunk of you data set to train consecutively each step. In this case it is set to zero so the whole batch of data (every sample of the data set) is used in each training step.
Set the learning rate for your trainer. This defines the size of the learning steps the trainer can adjust the network with.
Press train to run the training session

Evaluate your model

Method 1

Create an DataIn node with a sample size that matches the one used for the network's 'x' input (3). This will have test data.
Make an Evaluate(Debuggable) node Pass in the Network Pass in the test DataIn
Use a DataOut node to output the result in a format of your needs, in this case float.

Method 2

Use 'SetInput' node, taking your model and a DataIn node with a size matching the input of the your model.
pass this into Evaluate node
Use a DataOut node to output the result in a format of your needs.

The main difference between these two approaches is that the Debuggable Evaluator can use the latest model while it updates. The 'SetInput' node only updates the model when a new model is passed into in to it. It is best used with finished models.

vlml's People

Contributors

Stargazers

Watchers

Forkers

rogalag lkstudio simaoppcastro

vlml's Issues

Cleanup the tooltip patch

There was a lot of hacking around to get it working, but now that it is, it could do with some serious refactoring to make it easy to update and borrow from in later projects. Its like a jungle in there...

High level image classification module

Need to finish the high-level module you have for training image classification.

Is Validator seeing all of the examples in minibatch data set.

Might be an issue with double calls on 'get_next_minibatch' for multiple batch sources. More checks needed

Onnx

Its a slightly tricky subject but there has been some progress I'll report back on. Microsoft should have some too.

Documentation

This is a toughy. The subject is pretty dense but there is a lot of literature out there that I can link people to (as suggested by @tebjan) but they will need some introduction to the concepts without going needing to go too deep. Deep enough to know how to get something done, or experiment with the possibilities, but not so much that they think they will be making Skynet by the weekend.

I made a quite an effort getting quite a lot of the nodes documented, but It would be cool if I could have a central repository where all of those summaries and remarks could be seen on a wiki page in order of category and name.

I'm also working on some examples that break down how the framework generally works as a system whilst still trying to avoid getting my hands too dirty with theory.

Thoughts welcome

Need some demos

This project needs demos o people can understand how to use this library.

Doesn't currently work with the exporting version of vvvv Gamma

There are some known breaking changes I will need to clear out, namely Script regions used on occasion, a some unknown issues that will relate to the compiler. Its currently unknown how this behave will be changed but I hope it won't be too difficult.

Replace LTSM Layer

The LSTM was first produced using scripts because it is complicated to create with nodes. The removal of scripts in Gamma means they will have to be replaced with nodes again or a plugin needs to be written to re-implement it.

Separate Repo for demos and useful modules?

If all the demos and modules are kept here and ship with the library, it might mean that this repo gets filled up with stuff that isn't really suitable. So a demo repo, like a Model Zoo might be the way to go. It may have also be where helpful utilities go too? Or should they go in another tools repo (for dataset downloading, preparing datasets, special deep learning networks and functions, general extensions repo?

CNTK 2.7.0 GPU outtage

GPU is unseen by CNTK 2.7.0. Fix may be in the later builds, but for now, will roll back to 2.6.0.

VVVV gamma 2019.2 0169 - ConvNet Layer Issues

Hi YanYas,
first thanks a lot for creating VLML! That´s exactly what I was dreaming of!

Unfortunately on Win10 with VVVV gamma 2019.2 0169 with the current VLML on nuget the Convnet Layers are having 2 errors, Creating and Updating is having an issue. Is there a way to solve it or is there any current VVVV version VLML works best?

Thx in advance!
All the best,
Emanuel