zhanghang1989 / regnet-search-pytorch Goto Github PK

Search for RegNet using PyTorch

License: MIT License

Python 100.00%

regnet-search-pytorch's Introduction

Search-RegNet-PyTorch

Implemention for neural architecture search of RegNet using PyTorch and AutoTorch.

This example and Fast AutoAugment will be used in the tutorial on From HPO to NAS: Automated Deep Learning at CVPR 2020.

model	ref	Acc	config
RegNet-0.4GF	official	72.38	link
RegNet-0.4GF	ours	72.18	link
RegNet-4.0GF	official	79.03	link

official: using official configuration. ours: using our searched configuration.

Training HP setting: learning rate: 0.2, batch size: 512, weight decay: 1e-4,

Quick Start

Install Dependencies

Install PyTorch, following the instruction.
Install other dependencies:

pip install autotorch thop torch-encoding

Install Apex (optional):

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

Test #Params and FLOPs from config file

python test_flops.py --config-file configs/RegNetX-4.0GF.ini

Single Model Training

Prepare ImageNet Dataset

cd scripts/
# assuming you have downloaded the dataset in the current folder
python prepare_imagenet.py --download-dir ./

Train a single model from a config file

python train.py --dataset imagenet --config-file configs/RegNetX-4.0GF.ini --lr-scheduler cos --epochs 120 --checkname default --lr 0.025 --batch-size 64 --amp

Architecture Search

Generate config files with expected GFLOPs

python generate_configs.py --gflops 4 --num-configs 32 --config-file configs/RegNetX-4.0GF

The generated configuration files will be saved as configs/RegNetX-4.0GF-1.ini, configs/RegNetX-4.0GF-2.ini ...

Search best model for the config files in a folder

In this example, each model will be trained using a single gpu for 25 epochs.

python search.py --config-file-folder gen_configs/RegNet-0.4GF/ --output-folder out_configs/ --epochs 25

The accuracy will be written into the output config file after training.

regnet-search-pytorch's People

Contributors

Stargazers

Watchers

regnet-search-pytorch's Issues

Quick Start error

I followed the quick start step,util No module named 'arch'. I solved it with pip install arch.
However,when i runpython test_regnet.py --config-file RegNet-Search-PyTorch/configs/RegNetX-4.0GF.ini ,it get a similar problem which is ModuleNotFoundError: No module named 'arch.regnet'

To be specific，
(pytorch-OpCounter) root@w-16:~/wxb/pytorch-OpCounter/benchmark# python test_regnet.py --config-file RegNet-Search-PyTorch/configs/RegNetX-4.0GF.ini Traceback (most recent call last): File "evaluate_regnet_zyx.py", line 41, in <module> main() File "evaluate_regnet_zyx.py", line 28, in main arch = importlib.import_module('arch.' + args.arch) File "/root/anaconda2/envs/pytorch-OpCounter/lib/python3.6/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 994, in _gcd_import File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked ModuleNotFoundError: No module named 'arch.regnet'

MXNet reimplement

Good job, but I have some questions:

It is known to all that FAIR's RegNet source code had released(pycls codebase), so what's the difference ?(RegNet-0.4GF is 0.2% less than official) and why u reimplement ?
Have u and your team use MXNet or gluon to implement RegNet? If not, why? Does phenomenon implies Pytorch more usability than MXNet ?

Thanks.

design ShuffleNetV2 search space

Thanks for your excellent work!
Are the search criteria in regnet suitable for ShuffleNetV2? If I want to search a shuffleNet model with FLOPs 60M, can I design a search space based on this code?

C++ error

hello,When I run search.py,I find a error like this.

My pytorch version is 1.4.0. Thank you.

本项目是否有预训练的模型权重？怎么导入原论文作者的预训练模型

项目训练好的模型，能不能分享以下，由于机器性能较差，没有训练的能力，希望获取一份已经训练好的模型。我使用您的模型，导入原论文作者的权重失败，不知道原因？

About some warnings

I got the fllowing logs after running: python test_flops.py --config-file configs/RegNetX-4.0GF.ini，i wonder whether these warnings are normal?

[INFO] Register count_convNd() for <class 'torch.nn.modules.conv.Conv2d'>.
[INFO] Register count_bn() for <class 'torch.nn.modules.batchnorm.BatchNorm2d'>.
[INFO] Register zero_ops() for <class 'torch.nn.modules.activation.ReLU'>.
[WARN] Cannot find rule for <class 'arch.regnet.ConvBnAct'>. Treat it as zero Macs and zero Params.
[WARN] Cannot find rule for <class 'arch.regnet.Bottleneck'>. Treat it as zero Macs and zero Params.
[WARN] Cannot find rule for <class 'torch.nn.modules.container.Sequential'>. Treat it as zero Macs and zero Params.
[WARN] Cannot find rule for <class 'arch.regnet.Stage'>. Treat it as zero Macs and zero Params.
[WARN] Cannot find rule for <class 'arch.regnet.GlobalAvgPool2d'>. Treat it as zero Macs and zero Params.
[INFO] Register count_linear() for <class 'torch.nn.modules.linear.Linear'>.
[WARN] Cannot find rule for <class 'arch.regnet.RegNet'>. Treat it as zero Macs and zero Params.
macs: 3.989G, params: 22.118M

无法使用autotorch,如何修改autotorch部分的代码？

由于个人原因我无法使用pip下载autotorch,请问如何改写autotorch的代码？我可以人工输入参数，不进行网格搜索。

不是很懂作者怎么将RegNet嵌套在ResNet结构中的

我运行了arch里面的regnet.config_network函数，读取的是configs文件里面的RegNet-0.4GF.iNI文件，我打印了出来，发现和原来ResNet网络没什么太大区别，当然可能是我没看懂