uci-carl / carlsim4 Goto Github PK

Valgrind reports memory leaks in pthread_attr_setaffinity_np. I'm not exactly sure what the problem is. But, could this explain why the test suite "gets stuck", as repeated allocations/runs would make the simulation slower and slower? Your local machine probably has more memory than the average Travis node, so Travis should run into problems earlier.

Tested with the "Hello World" project with NO_CUDA:

$ valgrind --leak-check=full --log-file="log.out" ./hello_world

Here's an example and the summary of the report:

==21728== 1,280,000 bytes in 10,000 blocks are definitely lost in loss record 16 of 16
==21728==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21728==    by 0x4C2FDEF: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21728==    by 0x5053855: pthread_attr_setaffinity_np@@GLIBC_2.3.4 (pthread_attr_setaffinity.c:47)
==21728==    by 0x427E13: SNN::clearExtFiringTable() (in /home/mbeyeler/source/CARLsim4/projects/hello_world/hello_world)
==21728==    by 0x43EBE0: SNN::runNetwork(int, int, bool) (in /home/mbeyeler/source/CARLsim4/projects/hello_world/hello_world)
==21728==    by 0x40899B: CARLsim::runNetwork(int, int, bool) (in /home/mbeyeler/source/CARLsim4/projects/hello_world/hello_world)
==21728==    by 0x40589E: main (in /home/mbeyeler/source/CARLsim4/projects/hello_world/hello_world)
==21728==
==21728== LEAK SUMMARY:
==21728==    definitely lost: 10,251,008 bytes in 80,024 blocks
==21728==    indirectly lost: 0 bytes in 0 blocks
==21728==      possibly lost: 0 bytes in 0 blocks
==21728==    still reachable: 72,736 bytes in 2 blocks
==21728==         suppressed: 0 bytes in 0 blocks

CARLsim still segfaults when ./results doesn't exist, no explanation given. Just learned this the hard way. It's a problem when running executables from subdirectories, such as ./carlsim/tests/carslim_tests.

At the very least, check if directory exists before attempting to create a file within it. Notify the user accordingly.

Even better, make it possible to set a different save directory before CARLsimInit gets called.

The full-no-direct connection type gives the same connections as the full connection type. The control does not enter this condition in the code.(might be something that I am overlooking, please let me know)

After the ECJ parameter-tuning interface update is complete, add an example to the projects directory of with a .cpp model and a .params ECJ parameter file.

I'm not sure the C++ code for the PTI interface builds anymore, and the build process should be simplified regardless. Can we integrate the PTI helper classes and tests into the main build, so it's not separate anymore?

Hello, can someone look into the issue with spikeMonitor methods, which return 0 in all the failure cases? Ting and I tested on Ubuntu 14.04 (same as Travis) and on Windows on our local machine and all the tests pass. However, when we push, some of the tests fail on Travis and spikeMonitor getter methods return 0 in all those cases.

https://travis-ci.org/UCI-CARL/CARLsim4/builds/288848259?utm_source=github_status&utm_medium=notification

It is required for OAT NetworkMonitor and PTI, atleast. It is still deactivated in CARLsim4.

Running CARLsim4 with Matlab 2017b, I get the following error when trying to run the OAT with the smooth output:

Expected one output from a curly brace or dot indexing expression, but there were 0 results.

Error in NetworkMonitor/setGroupPlotType (line 609)
        obj.groupMonObj{gId}.setDefaultPlotType(plotType);

Error in demoSmooth (line 27)
NM.setGroupPlotType(-1, 'heatmap')

I think that it could be fixed by updating demoSmooth.m with the following patch:

*** doc/source/tutorial/4_image_processing/scripts/demoSmooth.m.orig	2018-10-18 11:23:24.753124357 -0700
--- doc/source/tutorial/4_image_processing/scripts/demoSmooth.m	2018-10-18 11:23:28.205065731 -0700
*************** createStimFromImage('../input/carl.jpg',
*** 24,29 ****
  % Plot network activity
  clear NM
  NM = NetworkMonitor('../results/sim_smooth.dat')
! NM.setGroupPlotType(-1, 'heatmap')
  NM.plot
  
--- 24,29 ----
  % Plot network activity
  clear NM
  NM = NetworkMonitor('../results/sim_smooth.dat')
! NM.setGroupPlotType(NM.groupNames{end}, 'heatmap')
  NM.plot

However, I'm not sure if the root cause is the version of Matlab that I am using.

When we first created the ECJ PTI, I was only loosely connected with the ECJ project and didn't have commit rights—so we added some Java code to CARLsim's source tree to make it easier to use ECJ to tune an arbitrary external binary.

These features can now be moved into the ECJ project itself as a general feature.

Reconsider which of our custom ECJ extensions are really necessary (Sean wondered why created a new problem form and evaluator, instead of just using GroupedProblemForm), and rewrite or move it into ECJ.

The result should be that CARLsim no longer has any extra Java code with a separate build procedure.

I am trying to use sim.setWeight to apply a normal distribution to the synaptic weights between 2 neuron groups. However, when I call sim.setWeight I get the following assertion error:

carlsim/kernel/src/snn_manager.cpp:1337: void SNN::setWeight(short int, int, int, float, bool): Assertion `managerRuntimeData.connIdsPreIdx[pos_ij] == connId' failed.

It seems like the wrong connection ID is being retrieved, however I cannot seem to figure out what the variable pos_ij is and therefore can't figure out why the error is occurring (I tried printing pos_ij from snn_manager.cpp but nothing would print).

I believe I am giving the setWeight method the correct inputs (connection ID and 0-indexed neuron numbers), so I am thinking this may be a bug. Also, when calling setWeight on a connection from one neuron group to itself there is no problem, only when it is called on 2 separate neuron groups.

Jeffrey Kopsick from GMU reported inconsistent results on the 80-20 example, when different GPU partitioning setups were adopted using 3 GPUs vs. 4 GPUs. I could reproduce the same using CPUs as well.

Of the four groups, I get consistent output for 1, 2, and 4 CPUs. In case of three CPUs:

1. exc1=>0, exc2=>0, exc3=>1, inh =>2 is consistent with 1,2, 4 CPUs
2. When I let the inhibitory group share a CPU with an excitatory group, it becomes inconsistent. Which is not an expected behavior.

On a system with two GPUs, and a run with 2 CPUs and 2 GPUs was fine.

I get the following error attempting to open (never mind compile/build) the CARLsim solution file in Visual Studio:

C:\Users\USERNAME\Documents\working\CARLsim4\carlsim\kernel\carlsim.vcxproj(74,5): The imported project "C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\BuildCustomizations\CUDA 8.0.props" was not found. Confirm that the path in the declaration is correct, and that the file exists on disk.

I tried on VS 2015 and 2017

I wrote a class inheriting ConnectionGenerator to connect groups using normally distributed weights. in the connect method I explicitly set delay = 1. Despite using this custom connection to connect all groups, when trying to add STP I get the error:
[ERROR carlsim/kernel/src/snn_manager.cpp:4864] STP with delays > 1 ms is currently not supported.

This must mean that somewhere glbNetworkConfig.maxDelay must be being set to > 1 incorrectly.

Izhi-9, Compartment, and LIF are missing.

Tutorial 5 motion energy does not work. It might be the issue of changing Grid3D structure in CARLsim4.
CUDA error at cppME/src/motion_energy.cu:543 code=11(cudaErrorInvalidValue) "cudaMemcpy(d_stim,stim,nrC_*nrX_*nrY_,cudaMemcpyHostToDevice)"

The Makefile of tutorial 7 is out of date. Update it to address makefile changes in CARLsim4

I'm having a bit of a strange problem. I've run a test simulation with a single LIF neuron and different levels of input current (sim.setExternalCurrent(I)) where 1 < I < 1000 and I found that the neuron responds fairly normally (although the f-I curve has steps and is not linear as it should be for LIF). However, when I try to run a simulation with multiple groups of LIF neurons (3000 neurons in total), the simulation is not producing any spikes, even for excessively large input current values or input Poisson spike train rates. I will attach the project file here.
main_real_net.cpp.zip

Please let me know what other information I could provide to help resolve this. Thanks!

Trying to build with VS 2019 Community on Windows 10.
Trying the hello_world but get a missing file:

'C:\CARLsim\x64\Release\carlsim.lib'

It isn't part of the GetHub download.

On Tegra, OS_SIZE is not correctly extracted. It should be a number (32 or 64), but instead on your platform is aarch64.

Fix by replacing line 55:
https://github.com/UCI-CARL/CARLsim4/blob/master/carlsim/configure.mk#L55

with:

OS_SIZE = $(shell uname -m | sed -e "s/i.86/32/" -e "s/x86_64/64/" -e "s/armv7l/32/" -e "s/aarch64/64/")

I exactly followed the customized synaptic connection codes (copy-paste) and saw the following error:
error: ‘MyConnection::MyConnection()’ is private within this context
MyConnection myConn;
^~~~~~
src/main_Project_1.cpp:28:6: note: declared private here
MyConnection() {}
^~~~~~~~~~~~
src/main_Project_1.cpp:81:15: error: ‘virtual MyConnection::~MyConnection()’ is private within this context
MyConnection myConn;
^~~~~~
src/main_Project_1.cpp:29:6: note: declared private here
~MyConnection() {}
^
src/main_Project_1.cpp:82:44: error: no matching function for call to ‘CARLsim::connect(int&, int&, MyConnection&, bool)’
sim.connect(gExc, gExc, myConn,SYN_PLASTIC);

Any help please?

Hi there, I was trying to install CARLsim in my Windows OS. While building the carlsim solution file in Visual STUDIO 2015,i got total 36 errors and most of them were 'carlsim_datastructures.h': No such directory found.
Can anyone provide the step by step guidelines to install carlsim in windows ? Thankyou.

Best regards

the spike timings and counts between single runtime and multi-runtime are different

Tutorial 3 works on Windows but not on Linux. Allocating PoissonRate() on GPU caused CUDA error

The issue was revelead by Compartments spikeTimesCPUvsGPU test.
The issue occurs at some, but not all timesteps.

CPU + release, CPU + debug, and GPU + debug modes all produce the same spike times when running compartment models. The GPU + release mode, however, deviaties from the other three models.

Overview of the issue:
The GPU release mode seems to suffer from occassional calculation 'errors' that slowly snowball out of control. The cause of these calculation 'errors' has not been determined.

More in depth look:
The deviation is first evident in regard to voltage values then recovery & current variables.
The deviation slowly increases in size, eventually resulting in deviation between spike times.
The deviation in voltage values begins early on (within first 100ms).

The GPU release voltage values deviate from GPU debug voltage.
The CPU release voltage and CPU debug voltage do not deviate.

Print statements in GPU release mode seem to affect the reported values.
This is not the case in GPU debug mode.

Attempted:
Equating different optimization flags between debug & release modes for both C/C++ and CUDA C/C++.

https://devtalk.nvidia.com/default/topic/551571/different-results-when-using-gpu-debug-option-g-/?offset=6

https://www.researchgate.net/post/Debug_mode_VS_release_mode_in_visual_studio

https://devtalk.nvidia.com/default/topic/670121/release-and-debug-modes-on-cuda-5-0/

Disabling FMAD:
https://stackoverflow.com/questions/14552576/disabling-fused-multiply-add-in-cuda-under-visual-studio-2010

Potential Causes:

1. Race condition?
2. ???

In a customized network, I saw that the SNN cannot run when we have more that 256 synapses totally in M groups. BTW, I am using "nocuda" version. I changed the assertion error in my computer to: `++numConnections_ <= MAX_CONN_PER_SNN +1000 ' to fix the error for now and continue running the SNN. Is there any way to change the value of this variable (MAX_CONN_PER_SNN) and also its type? Its type is short-int which is not enough to cover my large networks (syn>32765).

spikeCountD1/D2 are updated only when simulation runs more than 1000ms.

Hi there,

I have an issue while compiling 'make test' with carlsim4.
See the picture below, the error seems related to a new LIF neuron model that is not present in carlsim4...

any idea how to fix this?

thank you for your good work.
Federico

The README is completely outdated. Need to update installation instructions.

When setRates is called on a PSG obejct to reset input spike rates, setSpikeRate must be called again. This is documented in the user guide, however the Tutorial 1 does not have this and as a result the spike rates are not updated.

float inputRateHz = i*10.0f;
sim.setSpikeRate(gIn, &in);
in.setRates(inputRateHz);

The docs and make help both speak of a make debug and make debug_nocuda target.

Not sure these still exist. I see them as listed as .PHONY targets in carlsim/carlsim.mk, but they don't seem to actually be implemented anywhere.

Took me a minute to figure out (as a newcomer) that make release_nocuda works just fine, and that it's specifically debug and debug_nocuda that don't work.

We need to make sure that CPU and GPU results are consistent and correct when run using 4 GPUs and 4 CPU cores, at least. Jinwei has found some issues with memory copy when running on 4 GPUs on SDSC cluster.

Use Travis to add continuous integration, such that the regression suite is run whenever a new PR is submitted, followed by a coverage report.

In carlsim/interface/src/carlsim.cpp

UserErrors::assertTrue(carlsimState_ == CONFIG_STATE,
UserErrors::CAN_ONLY_BE_CALLED_IN_STATE, funcName, funcName, "CONFIG or SETUP.");
should be changed to:
UserErrors::assertTrue(carlsimState_ == CONFIG_STATE,
UserErrors::CAN_ONLY_BE_CALLED_IN_STATE, funcName, funcName, "CONIG.");

When cloning the program the username is not necessary.

This problem should be related to the difference of compilers since the code runs well on linux and windows. I write functions called isinf_xplat and isnan_xplat to replace them and it could be compiled on OS X now. The code will be tested on linux, windows and projects on OS X later.

I'm about to install Ubuntu on my new PC for running CARLsim4. Has CARLsim4 been tested on newer Ubuntu LTS releases like 18.04 or 20.04?

We need to add an example CMakeLists.txt for the projects, say for hello_world, and explain in the docs how to use it for building the project and the arguments to cmake.

Imported from CARLsim3, applies to README/User Guide Chapter 1 of CARLsim4, too:

From @SigmaX :

Since we include gtest as a submodule, building tests is easy and convenient.

But the README suggest that building the tests is as simple as running make test after checkout. This won't work until gtest has been built—that that takes a few steps.

We should update the README to direct users to the gtest README, and to build the gtest library in the designated path.

I get:
C:\CARLsim\doc\source\tutorial\8_compartments\tutorial_8_compartments.vcxproj : error : The imported project "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\BuildCustomizations\CUDA 8.0.props" was not found.

Using the latest CUDA install - cuda_10.1.168_425.25_win10.exe
Windows 10
Visual Studio Microsoft Visual Studio Community 2019
Version 16.0.0
VisualStudio.16.Release/16.0.0+28729.10
Microsoft .NET Framework
Version 4.7.03190