nathaniel-rodriguez / alectrnn Goto Github PK

Add a function like execute_async_batch that executes all batches in a list before closing the scheduler.

The Eigen-based RNN integrator/layer combination has a bug where previous state is not carried on and the wrong equation is calculated.

Currently the motor layer is created automatically, preventing it from adopting a different activation function (in the case of spiking networks for instance) and preventing the adoption of say laterally-inhibiting motor layers.

Options: Add an override to motor layer in python NeuralSystem class.
Options: Currently all layers use same activation function, add override to select activation functions on a layer-by-layer basis.

also a support new weight initialization method

This object would injected in a batch and its call back executed to store the network. The object can then just be added to the storage section of the batch object.

We would need to add a soft-max function and RNG for the soft-max. This means also passing in a seed on agent creation.
Soft-max could be an option in agent-parameters
Soft-max would be applied to Neural net output and then the RNG would use the soft-max output to select an action for the agent.

The experiment, its results, and batch are loaded fully. This can be costly in terms if time and memory. If we implement the load to return a generator then experimental results and experiments can be pulled as they are used.

Support running NN agents with numpy arrays from python and maintaining a log of the NN history. The whole time-series array of inputs would be provided to the agent and then ran. This may involve adding support in experiments and experiment managers for this capability for a given agent.

During spike weight normalization the weights are shrunk rather than multiplied.

See newest asyncga additions.

Use Python macros to release the GIL in order to allow other threads to use the interpreter while the objective function runs.

Add a class that supports building and running neural network without ALE.

This can be added by expanding conv + recurrent layers, where a graph is made by default connecting neurons that are adjacent (via a grid) within a specific channel.

The current IAF equation doesn't account for step size in the inputs.

Attempt to work around the numpy memory leak

Make sure seeds are properly working.
See if multi-episode runs properly support summing scores and adjusting random starts works.

A little bug where the index of the first layer corresponds with 1 rather than 0 because the inputs are counted and as a result we need one initial buffer value to start. Can fix by just doing -1 on the layer indices so layer indices match the bound indices.

update the async_execute function to use arguments from the batch to configure the scheduler so that loopback and nanny's can be used.

Add new 3-dim binary motor format. Make new agent that follows new format.
Re-evaluate this. It requires -1 to 1 activations which means non-spike and it means weights leading up to motor layer should be positive and negative (alternative might be 0-2.0 with 0.5-1.5 as center).

Add a member that allows for parameter rescaling with the asyncga

Add a new agent, layer, integrator, bindings, and nn code for an agent which gets motor inputs and reward feedback.

Updated load_experiment function that uses consolidated results.
New load_experiment function that loads just a single batch result.
Update async batch function to consolidate following run.

Make the default IAF neurons have not refractory period. This can be added at a later date to a new neuron.
Adjust the IAF equations to properly handle time-scaling.

Add support for other CSA GA by adding a new member

current single runs are not agnostic to the type of optimizer, it tries to load from the optimizer. This is bad, just make one that takes parameters.