nengo / nengo-fpga Goto Github PK

Just putting this here so I don't forget.

Probably put quick start info in the README (I'm looking at nengo-ocl).

Parameters are transfered to the FPGA using an npz file. We should implement a system to cleanup old npz files so they don't buildup indefinitely.

I followed the installation steps for both DE1 images (i.e. 20190521_DE1_0.2.0b.img and 20200124_DE1_v0.2.1.img) and I cannot ssh into the FPGA. In fact, I cannot even log into the board via the serial port (i.e. ttyUSB0 because it never gets into the login stage).
The DE1-SoC boots well using the Terasic LXDE image.

Output via the serial port

[  OK  ] Listening on D-Bus System Message Bus Socket.
         Starting Restore Sound Card State...
         Starting sshd.socket.
[  OK  ] Listening on RPCbind Server Activation Socket.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Reached target Timers.
[  OK  ] Started Restore Sound Card State.
[  OK  ] Listening on sshd.socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
         Starting Login Service...
[  OK  ] Started Timestamping service.
[  OK  ] Started D-Bus System Message Bus.
         Starting Network Service...
         Starting Avahi mDNS/DNS-SD Stack...
         Starting Connection service...
[  OK  ] Started Kernel Logging Service.
         Starting (null)...
         Starting Network Time Service (one-shot ntpdate mode)...
[  OK  ] Started System Logging Service.
[  OK  ] Started Network Service.
[  OK  ] Started Network Time Service (one-shot ntpdate mode).

output via ssh

$ ssh [email protected] -c aes256-ctr
ssh: connect to host 10.162.177.236 port 22: No route to host

output of the ifconfig

$ ifconfig eth4
eth4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.162.177.10  netmask 255.255.255.0  broadcast 10.162.177.255
        inet6 fe80::ae74:a799:3ace:17ab  prefixlen 64  scopeid 0x20<link>
        ether b4:96:91:0f:54:ef  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2239  bytes 148620 (148.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0x90200000-902fffff

Hello, I just learned about the Nengo project trying to figure out a way to build something like the Loihi-based VOC sensor artificial nose from Imam & Cleland (2020).

I was wondering if it was possible to port the nengo-fpga backend to the Lattice iCE40UP5K, or even better an array of such devices?

Thanks !

Add a "quick start" set of instructions for various board usage scenarios (e.g., multiple boards on the same network, etc) in the docs. Works in concert with the quick start sections for each of the board docs.

nengo.SpikingRectifiedLinear and nengo.RectifiedLinear both have an amplitude parameter that scales the height of the tuning curve. The amplitude parameter currently appears to be ignored by nengo-fpga. For ReLU curves this is the same as scaling both the gain and the bias, and so the workaround is simple enough. But still a surprise to see the amplitude ignored.

The fpga_pes_ensemble_network code and the DNA_extractor code contain a lot of duplicate code. They should be refactored (make an SSH class or something) to reduce code copy/pasting. Refactoring the code would also make it easier to user the same SSH communication methods for future fpga ensemble / networks.

Nengo has modified the make_step function structure so we need to update our definitions as well. (nengo/nengo#1387)

Please add support for DE0 Nano CycloneV SOC, it's cheap and popular.

Spec:

FPGA Device
Altera Cyclone® V SE 5CSEMA4U23C6N device
Serial configuration device – EPCS
USB-Blaster II onboard for programming; JTAG Mode
2 push-buttons
4 slide switches
8 green user LEDs
Three 50MHz clock sources from the clock generator
Two 40-pin expansion header
One Arduino expansion header (Uno R3 compatibility), can connection with Arduino shields.
One 10-pin Analog input expansion header. (shared with Arduino Analog input)
A/D converter, 4-pin SPI interface with FPGA
HPS (Hard Processor System)
925MHz Dual-core ARM Cortex-A9 processor
1GB DDR3 SDRAM (32-bit data bus)
1 Gigabit Ethernet PHY with RJ45 connector
USB OTG Port, USB Micro-AB connector
Micro SD card socket
Accelerometer (I2C interface + interrupt)
UART to USB, USB Mini-B connector
Warm reset button and cold reset button
One user button and one user LED
LTC 2x7 expansion header

We should rename references to "DNA" (filename, scripts, etc) to Device ID.

When you use an FPGA name string not found in fpga_config, the current error message is something like:

Traceback (most recent call last):
  File "/home/xchoo/.conda/envs/nengo-fpga/lib/python3.6/configparser.py", line 1138, in _unify_values
    sectiondict = self._sections[section]
KeyError: 'de112'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "nengo_fpga/dna_extractor.py", line 246, in <module>
    fpga.connect()
  File "nengo_fpga/dna_extractor.py", line 164, in connect
    fpga_config.get(self.fpga_name, 'ip'), flush=True)
  File "/home/xchoo/.conda/envs/nengo-fpga/lib/python3.6/configparser.py", line 781, in get
    d = self._unify_values(section, vars)
  File "/home/xchoo/.conda/envs/nengo-fpga/lib/python3.6/configparser.py", line 1141, in _unify_values
    raise NoSectionError(section)
configparser.NoSectionError: No section: 'de112'

This can be better handled with better error messaging.

I'd replace the current input to the error with:
nengo.Connection(input_node, error, function=lambda x: x**2, transform=-1)

or with a previous declared function so it's easy to use this code to learn any nonlinearity as well (as written only linear functions are easy).

Working on a demo I added nengo-fpga as a backend to the GUI (in nengo_gui/exec_env.py).

I can switch between nengo and nengo-ocl backends freely, and from either of these to nengo-fpga. However, I cannot switch back from nengo-fpga to either nengo or nengo-ocl.

I haven't looked into this since it doesn't seem like a frequent use case, but my random guess:
I use the FpgaPesEnsembleNetwork with the other backends, and this simple uses the dummy ensemble inside the FPGA network object. Once I switch to the nengo-fpga backend, the FPGA is built and properly connected, but switching back to nengo doesn't disconnect or cause the model to rebuild the FPGA network with the dummy ensemble, so it hangs.

We should make an examples section in the docs to make them more obvious/easier to find.

Maybe also put the videos of the lego robot somewhere.

I want to be able to run networks on the FPGA from an external host. SSH works with port forwarding but I haven't been able to get the UDP working through port forwarding yet. Not sure if I need to tweak the network settings or tweak the socket implementation.

import numpy as np
import matplotlib.pyplot as plt

import nengo
from nengo_fpga.networks import FpgaPesEnsembleNetwork

n = 1
neuron_type = nengo.SpikingRectifiedLinear()
rate = 100
sim_t = 35

# 1) Nengo Ensemble implementation
with nengo.Network() as model1:
    a1 = nengo.Ensemble(
        n_neurons=n,
        dimensions=1,
        neuron_type=neuron_type,
        gain=np.zeros(n),
        bias=rate*np.ones(n),
    )
    
    p1 = nengo.Probe(a1.neurons, synapse=None)
    
# 2) Equivalent FpgaPesEnsembleNetwork implementation
with nengo.Network() as model2:
    a2 = FpgaPesEnsembleNetwork(
        fpga_name='de1',  # <-- change to pynq to get 1 time-step difference
        n_neurons=n,
        dimensions=n,
        learning_rate=0,
    )
    a2.ensemble.gain = np.zeros(n)
    a2.ensemble.bias = rate*np.ones(n)
    a2.ensemble.neuron_type = neuron_type
    a2.connection.solver = nengo.solvers.NoSolver(np.eye(n))
    a2.connection.synapse = None
    
    p2 = nengo.Probe(a2.output, synapse=None)

with nengo.Simulator(model1) as sim1:
    sim1.run_steps(sim_t)

with nengo_fpga.Simulator(model2) as sim2:
    sim2.run_steps(sim_t)

plt.figure()
plt.plot(sim1.data[p1], label="nengo.Simulator")
plt.plot(sim2.data[p2], label="nengo_fpga.Simulator")
plt.xlabel("Timesteps")
plt.ylabel("Output")
plt.legend()
plt.show()

The difference is one time-step for pynq (not shown) and two time-steps for de1 (shown above).

However if we change the second simulator from nengo_fpga.Simulator to nengo.Simulator we get the expected result:

The built docs at nengo.ai/nengo-fpga have the autoclass documentation rendering incorrectly for the API page. It is missing a space between the parameter/attribute name and the type.

Not sure if this is an issue with the sphinx or theme version we are using? the source code appears to be correctly formatted as far as I can tell.

Unfortunately the DE1 still runs py2.7 and in numpy 1.17 they changed the pickle format from 2 to 3 for np.savez_compressed. This means our compressed params file is no longer readable on the DE1 side.

For now we can just impose numpy<1.17, but we should find a solution going forward.

The fpga_config file has the password for the device written out in plain text. While this probably isn't a big deal in most cases (since it's the default user/pw on a local network) we might want to look into other ways of authentication?

Just putting this thought here so it isn't lost.

Something like this, probably make a notebook

import numpy as np

import nengo
from nengo.solvers import NoSolver

import nengo_fpga
from nengo_fpga.networks import FpgaPesEnsembleNetwork

# # Model params
# N = 4
# Din = 2
# Dout = Din

# # Generate neuron params
# with nengo.Network(seed=1) as param_net:
#     a = nengo.Ensemble(N, Din, neuron_type=nengo.RectifiedLinear(),
#                        #                       encoders=[[0, 1], [1, 0],
#                        #                                [0, -1], [-1, 0]],
#                        encoders=[[0.707, 0.707], [-0.707, 0.707],
#                                  [-0.707, -0.707], [0.707, -0.707]],
#                        intercepts=nengo.dists.Choice([0]),
#                        max_rates=nengo.dists.Choice([100])
#                        )
#     b = nengo.Node(size_in=Dout)
#     conn = nengo.Connection(a, b)

# with nengo.Simulator(param_net) as param_sim:
#     bias = param_sim.data[a].bias
#     decoders = param_sim.data[conn].weights
#     encoders = param_sim.data[a].encoders
#     gain = param_sim.data[a].gain

# print("bias")
# print(bias)
# print("decoders")
# print(decoders)
# print("encoders")
# print(encoders)
# print("gain")
# print(gain)

# 4 neurons, each set to represent one quadrant of a sine wave
bias = np.array([0., 0., 0., 0.])
decoders = np.array([[0.00659448, -0.0065236, -0.00656359, 0.00652033],
                     [0.00660615, 0.00646017, -0.00655018, -0.00658202]])
encoders = np.array([[.7071067812, .7071067812],
                     [-.7071067812, .7071067812],
                     [-.7071067812, -.7071067812],
                     [.7071067812, -.7071067812]])
gain = np.array([100., 100., 100., 100.])


with nengo.Network() as model:
    stim = nengo.Node(
        output=lambda t: [np.sin(t*5*np.pi), np.cos(t*5*np.pi)])

    fpga = FpgaPesEnsembleNetwork('pynq',
                                  n_neurons=4,
                                  dimensions=2,
                                  learning_rate=0,
                                  function=lambda x: [0, 0])  # just to show this is ignored

    # Explicitly set ensemble parameters
    # Take a look at nengo.Ensemble and nengo.Connection docs for all options
    fpga.ensemble.bias = bias
    fpga.ensemble.encoders = encoders
    fpga.ensemble.gain = gain

    # To set the decoders we need to use nengo.NoSolver to tell the builder
    # we will provide out own decoders (you can also set "weights" here
    # instead, see nengo.solvers.NoSolver for more info)
    fpga.connection.solver = NoSolver(decoders.T)

    nengo.Connection(stim, fpga.input)

Just as the nengo-gui has a installable nengo script, we should do the same for the nengo-fpga repo. Envisaged uses:

• nengo-fpga : Automatically starts up nengo (gui) with the nengo-fpga backend option. I.e., nengo -b nengo-fpga
• nengo-fpga --configure : Instantiates a text-gui to prompt user for information to configure the fpga_config file.
• nengo-fpga --get_dna <device name> : Queries <device name> for it's DNA value. I.e., calls out to the DNA_extractor.py script.

Move to Python logging, Nengo removed this functionality (nengo/nengo#1566)

A synapse (that is not directly configurable) is created on this connection to the output node. This results in slightly different results between FpgaPesEnsembleNetwork and a corresponding Ensemble both with the same nengo.Simulator. A workaround is to configure fpga_pes_ensemble_network.connection.synapse = None after creating the network.

Here's a quick unit test that demonstrates identical behaviour with this work-around:

import nengo
from nengo_fpga.networks import FpgaPesEnsembleNetwork

n = 1
neuron_type = nengo.SpikingRectifiedLinear()
rate = 100
probe_tau = 0.1
sim_t = 20

# 1) Nengo Ensemble implementation
with nengo.Network() as model1:
    a1 = nengo.Ensemble(
        n_neurons=n,
        dimensions=1,
        neuron_type=neuron_type,
        gain=np.zeros(n),
        bias=rate*np.ones(n),
    )
    
    p1 = nengo.Probe(a1.neurons, synapse=probe_tau)
    
# 2) Equivalent FpgaPesEnsembleNetwork implementation
with nengo.Network() as model2:
    a2 = FpgaPesEnsembleNetwork(
        fpga_name='foobar',
        n_neurons=n,
        dimensions=n,
        learning_rate=0,
    )
    a2.ensemble.gain = np.zeros(n)
    a2.ensemble.bias = rate*np.ones(n)
    a2.ensemble.neuron_type = neuron_type
    a2.connection.solver = nengo.solvers.NoSolver(np.eye(n))
    a2.connection.synapse = None  # <- this is the important line
    
    p2 = nengo.Probe(a2.output, synapse=probe_tau)

# Compare for equality using the same simulator
with nengo.Simulator(model1) as sim1:
    sim1.run_steps(sim_t)

with nengo.Simulator(model2) as sim2:
    sim2.run_steps(sim_t)
    
assert np.allclose(sim1.data[p1], sim2.data[p2])