g1910 / hypernetworks Goto Github PK
View Code? Open in Web Editor NEWPyTorch implementation of HyperNetworks (Ha et al., ICLR 2017) for ResNet (Residual Networks)
License: GNU General Public License v3.0
PyTorch implementation of HyperNetworks (Ha et al., ICLR 2017) for ResNet (Residual Networks)
License: GNU General Public License v3.0
Great implementation Sir ! It matches at most of the places with the original paper. I had an doubt regarding the embedding in inference.Hope to clear it from you.
Hi, Thank you for sharing your code.
Could I ask your final result of training this HyperNetwork code for CIFAR-10?
In the code given below , I find that we are not specifying weights in the resnet layers. If so do we except that the weights will get updated through the optimizer and not through the HyperNetwork ? -
class ResNetBlock(nn.Module):
def __init__(self, in_size=16, out_size=16, downsample = False):
super(ResNetBlock,self).__init__()
self.out_size = out_size
self.in_size = in_size
if downsample:
self.stride1 = 2
self.reslayer = nn.Conv2d(in_channels=self.in_size, out_channels=self.out_size, stride=2, kernel_size=1)
else:
self.stride1 = 1
self.reslayer = IdentityLayer()
self.bn1 = nn.BatchNorm2d(out_size)
self.bn2 = nn.BatchNorm2d(out_size)
def forward(self, x, conv1_w, conv2_w):
residual = self.reslayer(x)
print(x.shape)
out = F.relu(self.bn1(F.conv2d(x, conv1_w, stride=self.stride1, padding=1)), inplace=True)
out = self.bn2(F.conv2d(out, conv2_w, padding=1))
out += residual
out = F.relu(out)
return out
Hello!
I'm currently getting this error when I try to run train.py:
Traceback (most recent call last): File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main prepare(preparation_data) File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path main_content = runpy.run_path(main_path, File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 288, in run_path return _run_module_code(code, init_globals, run_name, File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 97, in _run_module_code _run_code(code, mod_globals, init_globals, File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "C:\dev\HyperNetworks\train.py", line 80, in <module> for d in enumerate(trainset_loader): File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\data\dataloader.py", line 435, in __iter__ return self._get_iterator() File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\data\dataloader.py", line 381, in _get_iterator return _MultiProcessingDataLoaderIter(self) File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\data\dataloader.py", line 1034, in __init__ w.start() File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\multiprocessing\process.py", line 121, in start self._popen = self._Popen(self) File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 327, in _Popen return Popen(process_obj) File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__ prep_data = spawn.get_preparation_data(process_obj._name) File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 154, in get_preparation_data _check_not_importing_main() File "C:\Users\raho272696\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main raise RuntimeError(''' RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. python-BaseException
And it breaks when it gets to iterate through the train dataloader.
Thank you!
Hi,
I was wondering what is the reason the hypernetwork parameters are initialized like this:
torch.fmod(torch.randn(dim),2)
By the way, thanks for the repo, it is awesome.
Thank you very much for the released code.
I wonder if there are any experimental results for this implementation.
May I ask what version do you use for torch, torch_vision and Cuda? It seems to have some problems running your code on colab because of the version.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.