samsungsailmontreal / ghn3 Goto Github PK
View Code? Open in Web Editor NEWCode for "Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?" [ICML 2023]
Home Page: https://arxiv.org/abs/2303.04143
License: MIT License
Code for "Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?" [ICML 2023]
Home Page: https://arxiv.org/abs/2303.04143
License: MIT License
Hello thank you for your work.
Why random INIT performance is so high in Table 7 CIFAR-10 if the model has not been trained before.
does random init mean model initialize with Xavier initialization?
when using swint without any pretrained weights on CIFAR-10 as random initialize the performance is around 5 to 10. so I do not understand how you get that huge performance with no pretraining with the baseline random init.
thank you
Thank you for your interesting research. I have some questions regarding the paper:
Thank you.
Can you explain the few-shot setting whose result are reported in
Table 7. Transfer learning from ImageNet to few-shot CIFAR-10
and CIFAR-100 with 1000 training labels with 3 networks: ResNet50 (R-50), ConvNext-B (C-B) and Swin-T (S-T)
How many shots in the training set as well as in the test set or query set.
This is the code i use to evaluate ghn3clm16 on cifar-10 but the top1 results was 0.41 and 1.68 for top5
did I do something wrong.
here is my test code
`import torch
import torchvision
from ppuda.config import init_config
from ghn3 import from_pretrained, norm_check, Graph, Logger
from ppuda.config import init_config
from ppuda.utils import infer, AvgrageMeter, adjust_net
from ppuda.vision.loader import image_loader
args = init_config(mode='eval', debug=0, arch='resnet50', split='torch') # load arguments from the command line
assert args.arch is not None, ('architecture must be specified using, e.g. --arch resnet50', args.arch)
def bn_set_train(module):
if isinstance(module, torch.nn.BatchNorm2d):
module.track_running_stats = False
module.training = True
ghn = from_pretrained(args.ckpt, debug_level=args.debug).to(args.device) # get a pretrained GHN
is_imagenet = args.dataset.startswith('imagenet')
is_torch = args.split == 'torch'
print('loading the %s dataset...' % args.dataset)
val_loader, num_classes = image_loader(args.dataset,
args.data_dir,
test=True,
test_batch_size=args.test_batch_size,
num_workers=args.num_workers,
noise=args.noise,
im_size=224,#args.imsize,
seed=args.seed)[1:]
model = eval(f'torchvision.models.{args.arch}()').to(args.device) # create a PyTorch model
if is_torch and not is_imagenet:
model = adjust_net(model, large_input=False) # adjust the model for small images such as 32x32 in CIFAR-10
with torch.no_grad(): # to improve efficiency
model = ghn(model) # predict parameters of the model
model.eval()
model.eval() # set to the eval mode to disable dropout, etc.
model.apply(bn_set_train)
top1, top5 = infer(model.to(args.device), val_loader, verbose=False)
print(f'top1: {top1} and top5:{top5} ')`
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.