julestalloen / pytorch-hebbian Goto Github PK

View Code? Open in Web Editor NEW

76.0 76.0 26.0 236.1 MB

A lightweight and flexible framework for Hebbian learning in PyTorch.

License: MIT License

Python 100.00%

hebbian-learning image-classification local-learning pytorch unsupervised-learning

pytorch-hebbian's People

Contributors

Stargazers

Watchers

pytorch-hebbian's Issues

doesn't work with multi-process DataLoader

Hi,

I'm trying to speed up training by using a multi-processing in the DataLoader, however this throws an error. Any insight would be appreciated.

Here is a snippet of code

trainloader = torch.utils.data.DataLoader(train_dataset, batch_size=128, shuffle=True)
valloader = torch.utils.data.DataLoader(val_dataset, batch_size=128, shuffle=True, num_workers = 4, pin_memory=True)

learning_rule = KrotovsRule()
optimizer = Local(named_params=model.named_parameters(), lr=0.01)

evaluator = HebbianEvaluator(model=model, score_name='accuracy',
                                score_function=lambda engine: engine.state.metrics['accuracy'], epochs=1, supervised_from=-1)

trainer = HebbianTrainer(model=model, learning_rule=learning_rule, optimizer=optimizer, supervised_from=-1, device='cuda')

evaluator.attach(trainer.engine, Events.EPOCH_COMPLETED(every=1), trainloader, valloader)

trainer.run(train_loader=trainloader, epochs=1)

Here is the error:

Traceback (most recent call last):
  File "cornet_hebbian_training.py", line 84, in <module>
    trainer.run(train_loader=valloader, epochs=1)
  File "/home/vayzenbe/GitHub_Repos/GiNN/Models/pytorch_hebbian/trainers.py", line 31, in run
    self.engine.run(train_loader, max_epochs=epochs)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/ignite/engine/engine.py", line 702, in run
    return self._internal_run()
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/ignite/engine/engine.py", line 775, in _internal_run
    self._handle_exception(e)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/ignite/engine/engine.py", line 469, in _handle_exception
    raise e
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/ignite/engine/engine.py", line 745, in _internal_run
    time_taken = self._run_once_on_dataset()
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/ignite/engine/engine.py", line 850, in _run_once_on_dataset
    self._handle_exception(e)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/ignite/engine/engine.py", line 469, in _handle_exception
    raise e
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/ignite/engine/engine.py", line 801, in _run_once_on_dataset
    self.state.batch = next(self._dataloader_iter)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
    return self._process_data(data)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
    data.reraise()
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torch/_utils.py", line 429, in reraise
    raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/vayzenbe/GitHub_Repos/GiNN/Models/load_without_faces.py", line 62, in __getitem__
    tensor_image = self.transform(image)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 60, in __call__
    img = t(img)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 869, in forward
    i, j, h, w = self.get_params(img, self.scale, self.ratio)
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 831, in get_params
    log_ratio = torch.log(torch.tensor(ratio))
  File "/home/vayzenbe/anaconda3/lib/python3.7/site-packages/torch/cuda/__init__.py", line 161, in _lazy_init
    "Cannot re-initialize CUDA in forked subprocess. To use CUDA with "
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

Fix TensorBoard hparams metrics

Hyperparameter metrics do not show in TensorBoard.

Hebbian rule throws an error

Hi!

I find this repository very curious, thank you!

However, I've tried to run the code with the Hebbian update rule, and I run across several exceptions.

It seems that in pytorch_hebbian/handlers/tensorboard_logger.py in line 8, BaseWeightsHandler should be imported instead of BaseWeightsHistHandler.
Running HebbsRule from hebb.py results with RuntimeError: 1D tensors expected, but got 2D and 1D tensors. I believe it is due to the torch.dot(w, x) operation: in PyTorch, dot only supports computing the dot product of two 1D tensors with the same number of elements, in contrast to NumPy.

Is there a way to fix it? Do you have any plans on continuing working on this project?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.