project-rig / pynn_spinnaker Goto Github PK

Aside from plasticity, all that is really required for feature parity with SpYnNaKeR:

• Finish SDRAM-bound C version of SSA and port to CPP
• Create SpikeSourceArray region that turns whatever the hell PyNN produces into SDRAM bitmaps

• Delay rows can mean that row estimate is way off
• Partitioning of poisson sources is particularly thick - 1000 15Hz poisson sources with 0.1ms timestep don't need splitting into 25s
• If initial partitioning doesn't fit on a chip should reduce width and try again

• Port profiler to CPP so it uses scope-based tags
• Integrate extraction of profiler data nicely with PyNN

This is something to do with buffer offsets

• Add functionality to determine if a Projection can be generated on-chip - annotate Connector
• weight_fixed_point estimation
• Add SynapticMatrix.partition_on_chip_matrix to build SubMatrix structures based on estimated upper bound on row-length
• Implement NativeRNG for pynn_spinnaker - next should PROBABLY throw
• Implement parameter translation for different distributions supported by NativeRNG
• Attempt to use existing parameter translation system to write out Connector parameters
• Implement ConnectionBuilder region to write out parameters etc
• First pass C++ implementation - AllToAll and FixedProbability connectors; and Uniform and Constant parameters for static synaptic matrices
• Logic to load connection_builder executable first for synapse processor vertices which require them
• Move number of rows up next to key in connection structure rather than in matrix
• Move constructors into cpp
• Generally a bit too much PyNN creeping down through the stack - attempt to beat back up!
• Matrix type rather than synapse type should be passed to connection builder
• Rowlets
• Plastic synaptic matrices
• Extended synaptic matrices (extra parameters)
• OneToOneConnector
• Seeding of NativeRNG
• Multiple RNGs
• 'Fallback' RNG for NativeRNG

• C++ version of plasticity framework
• Extend parameter map system to write LUTs
• Different row formats for SynapticMatrix region
• Figure out keys etc for post-spike back propogation
• GetWeights implementation
• Take STDP weight constraints into account in fixed point determination
• Flush packet sending
• Separate last update time from last pre-spike time in STDP algorithm so flushing can be a bit more efficient and possible with nearest-neighbour spike pairing
• Provide template specialism of PostEventHistory for PostEventHistory<void, NumEntries>
• Some C++ variant of the runtime STDP logging i.e. limiting logging to a particular pre+post neuron
• Think about power law weight dependence (Hermite spline LUT?) to reproduce MAD results

This could be done using lazy_param_map functions similar to the refractory period for neurons

Using SpikeSourcePoisson to provide input to all neurons in network is bad:

• Although SpikeSourcePoisson C++ code can run at 1024 neurons a core, architectural limit is in the order of 10Hz - well below typical cortical background
• One-to-one connected spike sources are also a worst case for synapse processors

As PR neural populations already receive input via DMAd currents, adding current source populations to implement http://data.andrewdavison.info/docs/PyNN/reference/electrodes.html API would be relatively trivial.

• When population and projection parameters are changed, determine if these changes require re-replacement
• If re-placement is required, download mutable parameters, run normal flow
• Otherwise, mark vertices and regions as dirty and re-upload inplace

Cell types should produce a shift of 1024 i.e 1, 0.5, 0.25, 0.125

• Constraining that doesn't rely on slices matching between input and output vertices
• Neuron populations should partition based on firing rate, timestep and cost
• Current input populations should partition based on timestep and cost

The hypergeometric variate generator, used for the FixedTotalNumberConnector(with_replacement=False) connector, fails for large parameter values.

The hypergeometric generator generates a sample from the hypergeometric distribution with parameters ngood, nbad, and nsample. This is the distribution over the number of red balls we find when we sample nsample times without replacement from an urn containing ngood red balls and nbad white balls.

The generator works only if ngood, nbad, and nsample are less than 2^16.
In our case nbad is the size of the submatrix of the synaptic matrix we are dealing with, minus the number of synapses within the submatrix. This can take values at least as high as 1024^2.

Note that this sampler is only used for the FixedTotalNumberConnector with with_replacement=False. This is not a commonly used connector; it is currently unimplemented in PyNN, which ignores the with_replacement flag.

Because the actual rows can become so short, potentially max synaptic event rate might be a poor metric. SynapseCluster could partition pre-synaptically based on CPU cycles, a more complex constant + N * synapse model of row cost and estimates for number of extension rows based on delay distribution

• Integrate basic model
• Update connection tests
• Profile STDP in more depth and split model into weight and timing dependence
• Update synapse types in extra models and BCPNN
• Performance model for neuron processing
• New partitioning algorithm
• Estimation of CPU load required to process synaptic matrices with delay extensions - https://gist.github.com/neworderofjamie/903ce0d4735f20c8fedd3ca2ace04098
• IF_cond_exp performance - 167 cycles per neuron
• IF_curr_ca2_adaptive performance
• IF_curr_dual_exp performance
• Estimation of SDRAM required form synaptic matrices with delay extensions - https://gist.github.com/neworderofjamie/f7897f1f58d0e44a87410c07f476201b

• Max cols is used to allocate delay buffers - should use non-delay partitioned row length
• Because the actual rows can become so short, potentially max synaptic event rate might be a poor metric. SynapseCluster could partition pre-synaptically based on CPU cycles, a more complex constant + N * synapse model of row cost and estimates for number of extension rows based on delay distribution

Right hand side is different

There is not enough space to duplicate IF_curr_exp neuron and synapse parameters for 1024 neurons, solution is probably to store immutable parameters for unique parameter configurations and have num neurons long array of uint16_t indices pointing to each neuron's immutable parameter set or neuron long array of pointers:

• Figure out if the difference in generated assembly is worth the extra 2 bytes per neuron
• Generalize the python side into lazy_param_map

These should be converted back to milliseconds

Connectors are PyNN objects so it should be clear that these are internal additions

As discussed with @grey-area, negative weight or delay parameters can come about due to:

• Inhibitory weights - inhibitory weights are specified negative in PyNN
• The tail of a distribution - these should be zeroed

Proposal:

• Absolute value of 'constant' parameters and appropriate distribution parameters (specified by lazy_param_map) should be taken on host
• In on-chip connection builder, delays < 1 timestep should be set to 1
• In on-chip connection builder, unsigned weights < 0 should be set to 0

enum DMATag
{
  DMATagSpikeDataRead,
  DMATagMax,
};

For:

enum class DMATag
{
  SpikeDataRead,
  Max,
}

Possible also type to uint32_t to ensure good ARM code results
http://blog.smartbear.com/c-plus-plus/closer-to-perfection-get-to-know-c11-scoped-and-based-enum-types/

They kinda should as spike sources are quite big

Setting parameters within a view definitely doesn't work

It would be very useful if the connection builder had the ability to return stats e.g. number of overflowing rows

These are hooked to the connection building so look crap - at minimum unhook!

Fixed probability connector should have this flag

max_spike_blocks should be number of non-empty bins in sum of bincounts for each spiketrain in vertex. In constructor, spike times could be quantized so they can be used both in sizeof and write_subregion_to_file

• setup.py boilerplate to install python script in path which uses file to locate python model and then get path to base make file
• Makefile use script to find Makefile.depend
• model_binaries always live in folder beneath python models so inspect.getfile(C.class) should find it
• Expose models through native models interface

This means network tester can be used without special code

This is starting to break stuff

Will also need to check matrix doesn't exceed allocated size

Back propagation of spikes via router for learning seems to result in a lot of dropped packets

• Allocate double-buffered bit field large enough for number of neurons
• Neuron cores should write bitfield of output every timestep
• Synapse cores should read bitfield and apply spikes to post history

e.g. RandomDistribution("normal_clipped", [1.5, 0.75, 0.1, np.inf]) - Which is pretty typical usage

• Seeding of NativeRNG
• Multiple RNGs

This requires passing through of vertex slices etc

• Synapse 1 max j constraint = 1024
• Synapse 2 max j constraint = 256
• Neuron j constraint = 256
• Current input j constraint = 512

Synapse constraint clamped down by population size (1000) to 1000 therefore it is no longer rounded to 4 * self.neuron_j_constraint

Simulator._constrain_cluster could return a chip count estimate. Perhaps algorithm could be something like:

# List of spare cores in ascending order
spare_cores = []
num_chips = 0

...
# Where constrains are built
num_cluster_cores = 1 + len(n.input_verts)

# Loop through list of chips with spare cores
# **TODO** could we use bisect here
spare_cores_found = False
for i in len(spare_cores):
    # If there are enough space on this chip
    if spare_cores[i] >= num_cluster_cores:
        spare_cores[i] -= num_cluster_cores    
        spare_cores_found = True
        break

# **TODO** mechanism to remove zeroes entries from spare_cores list

# If no chip was found with sufficient resources to contain this cluster add a chip and
# a new spare_cores entry containing the cores that remain on it after adding this cluster
if not spare_cores_found:
    num_chips += 1
    spare_cores.append(16 - num_cluster_cores)

• Setting multiple populations parameters using an assembly
• Creating multiple population-to-population connections using an assembly

• PEP8
• Automated tests for stuff within spinnaker_folder i.e. region level stuff
• Investigate what higher-level testing PyNN has and integrate with that
• Python code review
• C++ code review

Figure out where time is being spent in Python-side of P.R. attempt to optimise.

This is generally not an issue, but when neuron processor runs out of CPU, time interrupts get skipped and data doesn't get written leading to crap being returned.

• NeuronVertex -> OutputVertex
• SynapseVertex-> InputVertex (encompassing SDRAM current-based inputs)
• not happy with the Population in SynapsePopulation, NeuralPopulation and CurrentInputPopulation - Populations are PyNN constructs which consist of a number of synapse and neural populations each of which consists of a number of homogeneous input and output vertices!
• Pinus Rigida may not actually be the best name :-(

1. Set a flag in DTCM
2. Copy it to SDRAM
3. Read it back from host and if flag is set download io_buf

Fast and slow sources seperation MAY be optimal but:

• Changing rate can change size of struct etc etc
• All the lazy_param_map.apply_indices crud could be removed
• Special purpose code in regions.SpikeSourcePoisson could be removed

DMAs should be started before rows are processed so they can occur when CPU is busy

However it's a pain - A+ and A- are parameters of SpikePairRule but need to be scaled based on the weight dependence