mseitzer / pytorch-fid Goto Github PK
View Code? Open in Web Editor NEWCompute FID scores with PyTorch.
License: Apache License 2.0
Compute FID scores with PyTorch.
License: Apache License 2.0
I'm calculating FID score for a small dataset - folder1 - 1341 images and folder 2 - 1000 images with parameters as --dims 768. Please see below for the error-
100%|██████████| 10/10 [00:11<00:00, 1.11s/it]Warning: batch size is bigger than the data size. Setting batch size to data size
ValueError Traceback (most recent call last)
/usr/lib/python3.6/runpy.py in run_module(mod_name, init_globals, run_name, alter_sys)
203 run_name = mod_name
204 if alter_sys:
--> 205 return _run_module_code(code, init_globals, run_name, mod_spec)
206 else:
207 # Leave the sys module alone
7 frames
/content/drive/My Drive/pytorch-fid-master/pytorch_fid/fid_score.py in get_activations(files, model, batch_size, dims, cuda)
100 pred_arr = np.empty((len(files), dims))
101
--> 102 for i in tqdm(range(0, len(files), batch_size)):
103 start = i
104 end = i + batch_size
ValueError: range() arg 3 must not be zero
/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py:2590: UserWarning: Unknown failure executing module: <pytorch_fid>
warn('Unknown failure executing module: <%s>' % mod_name)
Hi,
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Can you please help?
Hi, I was trying to make a pytorch implementation of inception_score when i came across your weights. They also contain a FC layer in the end, I wanted to know if those weights are also the same as tensorflow or are they different.
12%|█████████▏ | 12/100 [00:03<00:25, 3.47it/s]
Traceback (most recent call last):
File "fid_score.py", line 261, in
args.dims)
File "fid_score.py", line 246, in calculate_fid_given_paths
dims, cuda)
File "fid_score.py", line 228, in _compute_statistics_of_path
dims, cuda)
File "fid_score.py", line 213, in calculate_activation_statistics
act = get_activations(files, model, batch_size, dims, cuda, verbose)
File "fid_score.py", line 111, in get_activations
for f in files[start:end]])
ValueError: could not broadcast input array from shape (256,256,3) into shape (256,256)
Because imread is deprecated from scipy 1.3, i use imread from matplotlib instead. But problem occurs on the line:
images = np.array([imread(str(f)).astype(np.float32) for f in files[start:end]])
It returns a nested numpy array, likes array([array([...]), ...]), rather than 2d array. To make the code precise, do you know how to convert the nested array to 2d numpy array?
Using the updated weights still get wrong result. See this repo https://github.com/AtlantixJJ/PytorchInceptionV3 for detail.
Run this code (need to store some image in data/cifar10_test, or go to the repo above):
"""
A script to test Pytorch and Tensorflow InceptionV3 have consistent behavior.
"""
import sys, argparse, os, pathlib
sys.path.insert(0, ".")
import numpy as np
import tensorflow as tf
import torch
from inception_modified import inception_v3
from PIL import Image
parser = argparse.ArgumentParser()
parser.add_argument("--load_path", default="", help="The path to changed pytorch inceptionv3 weight. Run change_statedict.py to obtain.")
args = parser.parse_args()
def check_or_download_inception(inception_path):
''' Checks if the path to the inception file is valid, or downloads
the file if it is not present. '''
INCEPTION_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz'
if inception_path is None:
inception_path = '/tmp'
inception_path = pathlib.Path(inception_path)
model_file = inception_path / 'classify_image_graph_def.pb'
if not model_file.exists():
print("Downloading Inception model")
from urllib import request
import tarfile
fn, _ = request.urlretrieve(INCEPTION_URL)
with tarfile.open(fn, mode='r') as f:
f.extract('classify_image_graph_def.pb', str(model_file.parent))
return str(model_file)
def torch2numpy(x):
return x.detach().cpu().numpy().transpose(0, 2, 3, 1)
torch.backends.cudnn.benchmark = True
torch.manual_seed(1)
torch.cuda.manual_seed(1)
data_dir = "data/cifar10_test/"
imgs_pil = [Image.open(open(data_dir + s, "rb")).resize((299,299), Image.BILINEAR) for s in os.listdir(data_dir)]
imgs = [np.asarray(img).astype("float32") for img in imgs_pil]
x_arr = np.array(imgs)
x_arr_tf = x_arr
# TF InceptionV3 graph use [0, 255] scale image
feed = {'FID_Inception_Net/ExpandDims:0': x_arr_tf}
# This is identical to TF image transformation
x_arr_torch = x_arr / 255. #(x_arr - 128) * 0.0078125
x_torch = torch.from_numpy(x_arr_torch.transpose(0, 3, 1, 2)).float().cuda()
model = inception_v3(pretrained=True, aux_logits=False, transform_input=False)
if len(args.load_path) > 1:
# default: pretrained/inception_v3_google.pth
print("=> Get changed weight from %s" % args.load_path)
try:
model.load_state_dict(torch.load(args.load_path))
except RuntimeError:
pass
model.cuda()
model.eval()
if x_torch.size(2) != 299:
import torch.nn.functional as F
x_torch = F.interpolate(x_torch,
size=(299, 299),
mode='bilinear',
align_corners=False)
features = model.get_feature(x_torch)
feature_pytorch = features[-1].detach().cpu().numpy()
if len(feature_pytorch.shape) == 4:
feature_pytorch = feature_pytorch[:, :, 0, 0]
inception_path = check_or_download_inception("pretrained")
with tf.gfile.FastGFile("pretrained/classify_image_graph_def.pb", 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def( graph_def, name='FID_Inception_Net')
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
layername = "FID_Inception_Net/pool_3:0"
layer = sess.graph.get_tensor_by_name(layername)
ops = layer.graph.get_operations()
for op_idx, op in enumerate(ops):
for o in op.outputs:
shape = o.get_shape()
if shape._dims != []:
shape = [s.value for s in shape]
new_shape = []
for j, s in enumerate(shape):
if s == 1 and j == 0:
new_shape.append(None)
else:
new_shape.append(s)
# print(o.name, shape, new_shape)
o.__dict__['_shape_val'] = tf.TensorShape(new_shape)
tensor_list = [n.name for n in tf.get_default_graph().as_graph_def().node]
target_layer_names = ["FID_Inception_Net/Mul:0", "FID_Inception_Net/conv:0", "FID_Inception_Net/pool_3:0"]
target_layers = [sess.graph.get_tensor_by_name(l) for l in target_layer_names]
sess.run(tf.global_variables_initializer())
res = sess.run(target_layers, feed)
x_tf = res[0]
feature_tensorflow = res[-1][:, 0, 0, :]
print("=> Pytorch pool3:")
print(feature_pytorch[0][:6])
print("=> Tensorflow pool3:")
print(feature_tensorflow[0][:6])
print("=> Mean abs difference")
print(np.abs(feature_pytorch - feature_tensorflow).mean())
def get_tf_layer(name):
return sess.run(sess.graph.get_tensor_by_name(name + ':0'), feed)
result:
=> Pytorch pool3:
[0.42730308 0.00819586 0.27243498 0.2880235 0.10205843 0.05626289]
=> Tensorflow pool3:
[0.13085267 0.5260418 0.22931635 0.02919772 0.2439549 0.50965333]
=> Mean abs difference
0.34774715
is there a way to calculate the distance between a dataset and just one sample (instead of a set of them), producing consistent results?
Hello!
Thanks for your wonderful project for the FID score.
I have an issue with this line of code.
Line 121
pred = model(batch)[0]
# If model output is not scalar, apply global spatial average pooling.
# This happens if you choose a dimensionality not equal 2048.
if pred.shape[2] != 1 or pred.shape[3] != 1:
pred = adaptive_avg_pool2d(pred, output_size=(1, 1))
pred_arr[start:end] = pred.cpu().data.numpy().reshape(batch_size, -1)
It seems that only a single image per batch is used for calculating FID. I am confused by this. Please help me solve this issue. Thank you!
Best,
Simon
time-consuming of the FID computation
Hi, I want to know why the FID computation is very slow. When I calculate the FID of 28000 images, it sometimes got stuck and spent almost one day or more to calculate once! Is there any idea to help me fix this problem? Thanks!
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-jnses9_7/pytorch-fid/setup.py'"'"'; __file__='"'"'/tmp/pip-install-jnses9_7/pytorch-fid/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-v7ho1gid
cwd: /tmp/pip-install-jnses9_7/pytorch-fid/
Complete output (7 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-jnses9_7/pytorch-fid/setup.py", line 5, in <module>
long_description = fh.read()
File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4: ordinal not in range(128)
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Hi! Thanks for your work.May i have you instruction on how to calculate FID score for Mnist dataset which has only 1 channels ,no idea to solve it now.Thanks.
Thanks for the great work. I have a question of if the number of images in both two image folders have to be equal when calculating the FID score. Because the FID distance is represented as the distance between two normal distributions. Sorry to bother you.
Thanks for your great work!
And I have a question now. You said ''Note that this changes the magnitude of the FID score '',and what exactly does this mean?
Waiting for your reply~
Thanks for your great effort! Do you have any plan in implementing Inception score in pytorch?
Hello,
Is there a special reason as to why you choose num_classes=1008
instead of using the default value 1000
when creating the model. Refer to: https://github.com/mseitzer/pytorch-fid/blob/master/pytorch_fid/inception.py#L193
Best, Jan
I noticed that the evaluations are measured on a large datasets to get statistics performance, is there any way to evaluate on a single image?
I use you module as follows
from my_utils.pytorch_fid import fid_score
def load_inception_model(self, dims):
block_idx = fid_score.InceptionV3.BLOCK_INDEX_BY_DIM[dims]
self.model = fid_score.InceptionV3([block_idx])
self.model.cuda()
self.model.eval()
But this load function takes a very long time. I think this is coupled with pytorch upgrade. However I am not sure. Can you make some comments?
Hi,
First of all, thanks for the nice implementation. Could you explain why you normalized inputs like here?
The normalized results seems different from below.
# mean and var for ImageNet
mean = (0.485, 0.456, 0.406)
std = (0.229, 0.224, 0.225)
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(mean=mean, std=std)])
Best,
Yunjey
I found that the FID changes drastically when images are saved to the disk. Basically, it is caused by the change of pixel values when rounded from the continuous real values to 0-255 integers. Is this common in FID testing? How does everyone minimize its effect?
Hello, what is the data format required to calculate the fid score? Thank you
Hello all,
I used following command to calculate FID distance from two different folders.
: ./fid_score.py /data/vision/pytorch-CycleGAN-and-pix2pix/datasets/synthia2kitti_ex1/testAB /data/vision/pytorch-CycleGAN-and-pix2pix/datasets/synthia2kitti_ex1/testB
I've come across the error as follows.
:Traceback (most recent call last):
File "./fid_score.py", line 262, in
args.dims)
File "./fid_score.py", line 249, in calculate_fid_given_paths
dims, cuda)
File "./fid_score.py", line 223, in _compute_statistics_of_path
imgs = imgs.transpose((0, 3, 1, 2))
ValueError: axes don't match array
Please give me any comment.
Thank you.
I noticed you merged setup.py but I don't see it in pypi, anything I can help with?
RuntimeError: The expanded size of the tensor (3) must match the existing size (864) at non-singleton dimension 3. Target sizes: [32, 3, 3, 3]. Tensor sizes: [864]
Traceback (most recent call last):
File "/home/multiai3/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/multiai3/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/multiai3/anaconda3/lib/python3.7/site-packages/pytorch_fid/main.py", line 4, in
pytorch_fid.fid_score.main()
File "/home/multiai3/anaconda3/lib/python3.7/site-packages/pytorch_fid/fid_score.py", line 252, in main
args.dims)
File "/home/multiai3/anaconda3/lib/python3.7/site-packages/pytorch_fid/fid_score.py", line 237, in calculate_fid_given_paths
dims, cuda)
File "/home/multiai3/anaconda3/lib/python3.7/site-packages/pytorch_fid/fid_score.py", line 219, in _compute_statistics_of_path
dims, cuda)
File "/home/multiai3/anaconda3/lib/python3.7/site-packages/pytorch_fid/fid_score.py", line 204, in calculate_activation_statistics
act = get_activations(files, model, batch_size, dims, cuda)
File "/home/multiai3/anaconda3/lib/python3.7/site-packages/pytorch_fid/fid_score.py", line 107, in get_activations
for f in files[start:end]])
ValueError: could not broadcast input array from shape (256,256,3) into shape (256)
I check that evey image is in 3 channels and 256 * 256 resolution.
What is the problem?
Thanks for the valuable work.
From your code, I cannot find where you load the weight of the inception v3, so I doubt if you forgot it.
When the code load the inception model, it appear this kind of error。
./fid_score.py /home/liuyx/dataset/cyclegan/fastblind-test/testA/bike-packing/ /home/liuyx/dataset/cyclegan/fastblind-test/fakeA/bike-packing/fakea/
Traceback (most recent call last):
File "./fid_score.py", line 262, in
args.dims)
File "./fid_score.py", line 242, in calculate_fid_given_paths
model = InceptionV3([block_idx])
File "/home/liuyx/codes/pytorch-fid/inception.py", line 80, in init
inception = fid_inception_v3()
File "/home/liuyx/codes/pytorch-fid/inception.py", line 207, in fid_inception_v3
inception.load_state_dict(state_dict)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 721, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Inception3:
Unexpected key(s) in state_dict: "Conv2d_1a_3x3.bn.num_batches_tracked", "Conv2d_2a_3x3.bn.num_batches_tracked", "Conv2d_2b_3x3.bn.num_batches_tracked", "Conv2d_3b_1x1.bn.num_batches_tracked", "Conv2d_4a_3x3.bn.num_batches_tracked", "Mixed_5b.branch1x1.bn.num_batches_tracked", "Mixed_5b.branch5x5_1.bn.num_batches_tracked", "Mixed_5b.branch5x5_2.bn.num_batches_tracked", "Mixed_5b.branch3x3dbl_1.bn.num_batches_tracked", "Mixed_5b.branch3x3dbl_2.bn.num_batches_tracked", "Mixed_5b.branch3x3dbl_3.bn.num_batches_tracked", "Mixed_5b.branch_pool.bn.num_batches_tracked", "Mixed_5c.branch1x1.bn.num_batches_tracked", "Mixed_5c.branch5x5_1.bn.num_batches_tracked", "Mixed_5c.branch5x5_2.bn.num_batches_tracked", "Mixed_5c.branch3x3dbl_1.bn.num_batches_tracked", "Mixed_5c.branch3x3dbl_2.bn.num_batches_tracked", "Mixed_5c.branch3x3dbl_3.bn.num_batches_tracked", "Mixed_5c.branch_pool.bn.num_batches_tracked", "Mixed_5d.branch1x1.bn.num_batches_tracked", "Mixed_5d.branch5x5_1.bn.num_batches_tracked", "Mixed_5d.branch5x5_2.bn.num_batches_tracked", "Mixed_5d.branch3x3dbl_1.bn.num_batches_tracked", "Mixed_5d.branch3x3dbl_2.bn.num_batches_tracked", "Mixed_5d.branch3x3dbl_3.bn.num_batches_tracked", "Mixed_5d.branch_pool.bn.num_batches_tracked", "Mixed_6a.branch3x3.bn.num_batches_tracked", "Mixed_6a.branch3x3dbl_1.bn.num_batches_tracked", "Mixed_6a.branch3x3dbl_2.bn.num_batches_tracked", "Mixed_6a.branch3x3dbl_3.bn.num_batches_tracked", "Mixed_6b.branch1x1.bn.num_batches_tracked", "Mixed_6b.branch7x7_1.bn.num_batches_tracked", "Mixed_6b.branch7x7_2.bn.num_batches_tracked", "Mixed_6b.branch7x7_3.bn.num_batches_tracked", "Mixed_6b.branch7x7dbl_1.bn.num_batches_tracked", "Mixed_6b.branch7x7dbl_2.bn.num_batches_tracked", "Mixed_6b.branch7x7dbl_3.bn.num_batches_tracked", "Mixed_6b.branch7x7dbl_4.bn.num_batches_tracked", "Mixed_6b.branch7x7dbl_5.bn.num_batches_tracked", "Mixed_6b.branch_pool.bn.num_batches_tracked", "Mixed_6c.branch1x1.bn.num_batches_tracked", "Mixed_6c.branch7x7_1.bn.num_batches_tracked", "Mixed_6c.branch7x7_2.bn.num_batches_tracked", "Mixed_6c.branch7x7_3.bn.num_batches_tracked", "Mixed_6c.branch7x7dbl_1.bn.num_batches_tracked", "Mixed_6c.branch7x7dbl_2.bn.num_batches_tracked", "Mixed_6c.branch7x7dbl_3.bn.num_batches_tracked", "Mixed_6c.branch7x7dbl_4.bn.num_batches_tracked", "Mixed_6c.branch7x7dbl_5.bn.num_batches_tracked", "Mixed_6c.branch_pool.bn.num_batches_tracked", "Mixed_6d.branch1x1.bn.num_batches_tracked", "Mixed_6d.branch7x7_1.bn.num_batches_tracked", "Mixed_6d.branch7x7_2.bn.num_batches_tracked", "Mixed_6d.branch7x7_3.bn.num_batches_tracked", "Mixed_6d.branch7x7dbl_1.bn.num_batches_tracked", "Mixed_6d.branch7x7dbl_2.bn.num_batches_tracked", "Mixed_6d.branch7x7dbl_3.bn.num_batches_tracked", "Mixed_6d.branch7x7dbl_4.bn.num_batches_tracked", "Mixed_6d.branch7x7dbl_5.bn.num_batches_tracked", "Mixed_6d.branch_pool.bn.num_batches_tracked", "Mixed_6e.branch1x1.bn.num_batches_tracked", "Mixed_6e.branch7x7_1.bn.num_batches_tracked", "Mixed_6e.branch7x7_2.bn.num_batches_tracked", "Mixed_6e.branch7x7_3.bn.num_batches_tracked", "Mixed_6e.branch7x7dbl_1.bn.num_batches_tracked", "Mixed_6e.branch7x7dbl_2.bn.num_batches_tracked", "Mixed_6e.branch7x7dbl_3.bn.num_batches_tracked", "Mixed_6e.branch7x7dbl_4.bn.num_batches_tracked", "Mixed_6e.branch7x7dbl_5.bn.num_batches_tracked", "Mixed_6e.branch_pool.bn.num_batches_tracked", "Mixed_7a.branch3x3_1.bn.num_batches_tracked", "Mixed_7a.branch3x3_2.bn.num_batches_tracked", "Mixed_7a.branch7x7x3_1.bn.num_batches_tracked", "Mixed_7a.branch7x7x3_2.bn.num_batches_tracked", "Mixed_7a.branch7x7x3_3.bn.num_batches_tracked", "Mixed_7a.branch7x7x3_4.bn.num_batches_tracked", "Mixed_7b.branch1x1.bn.num_batches_tracked", "Mixed_7b.branch3x3_1.bn.num_batches_tracked", "Mixed_7b.branch3x3_2a.bn.num_batches_tracked", "Mixed_7b.branch3x3_2b.bn.num_batches_tracked", "Mixed_7b.branch3x3dbl_1.bn.num_batches_tracked", "Mixed_7b.branch3x3dbl_2.bn.num_batches_tracked", "Mixed_7b.branch3x3dbl_3a.bn.num_batches_tracked", "Mixed_7b.branch3x3dbl_3b.bn.num_batches_tracked", "Mixed_7b.branch_pool.bn.num_batches_tracked", "Mixed_7c.branch1x1.bn.num_batches_tracked", "Mixed_7c.branch3x3_1.bn.num_batches_tracked", "Mixed_7c.branch3x3_2a.bn.num_batches_tracked", "Mixed_7c.branch3x3_2b.bn.num_batches_tracked", "Mixed_7c.branch3x3dbl_1.bn.num_batches_tracked", "Mixed_7c.branch3x3dbl_2.bn.num_batches_tracked", "Mixed_7c.branch3x3dbl_3a.bn.num_batches_tracked", "Mixed_7c.branch3x3dbl_3b.bn.num_batches_tracked", "Mixed_7c.branch_pool.bn.num_batches_tracked".
I use the fid.py to measure fid score of my images datasets, I generated 10000 images in tImages directory and used the command 'python fid.py ./tImages fid_stats_celeba.npz' or 'python fid.py ./tImages ./sImages ' (sImages directory is another images datasets), but waiting a while, I always get a ValueError. e.g. 'ValueError: Imaginary component 89928690.58258057' or
'ValueError: Imaginary component 1.376687186290827e+24'. I don't know which step I did wrong. Could anyone tell me what the problem is, thanks!
error information:
Traceback (most recent call last):
File "fid.py", line 334, in
fid_value = calculate_fid_given_paths(args.path, args.inception, low_profile=args.lowprofile)
File "fid.py", line 317, in calculate_fid_given_paths
fid_value = calculate_frechet_distance(m1, s1, m2, s2)
File "fid.py", line 155, in calculate_frechet_distance
raise ValueError("Imaginary component {}".format(m))
ValueError: Imaginary component 1.376687186290827e+24
Hey there,
I ran into problems as my images are of variable size (couldn't convert the list read-in to np.array).
Does it make sense to consider resizing images in imread?
Image.open(filename).resize((299, 299))
Best,
Mike
Hi!
Thanks Indeed for the code! It was really easy to follow.
I was working on Similar areas, where I need to check the quality of the Image Generated by the GANs for a CIFAR10/100 dataset.
I had a small query though. When I load the Inception model, I find the pretrained model is based on an Imagenet Dataset. Now supposing that I am doing a task based on CIFAR10 Dataset on GAN. How can I use the model for Imagenet since CIFAR10 has a size of 32x32 and number of classes is 10, where as in Imagenet it is about 224x224 and classes are 1000.
Do I need to change anything while calculating IS and FID for dataset such as CIFAR10/100 or CelebA?
Any comments or suggestion would be really appreciated!
Regards,
Nitin Bansal
Hi,
Have you compared it with the official TF implementation? Are the score significantly different? Are there pitfalls one should be about when using it that might get significantly different results than TF?
Thanks, and thank you for writing that code.
I am getting this error when testing a model.
RuntimeWarning: invalid value encountered in double_scalars arg2 = norm(X.dot(X) - A, 'fro')**2 / norm(A, 'fro')
I did not find a solution.
By definition, the FID score has a minimum bound of 0 but I got -4.8e-5 when I calculated the distance of a dataset to itself. Obviously, this is very close to zero. Nevertheless, can't think of a reason why would this be non-zero from your implementation. Any ideas?
please add the 'jpeg' etc. image extensions as well to reduce the time for saving the entire datasets into 'jpg' or 'png' extensions.
I get the following error:
~/w/sg2 ❯❯❯ python -m pytorch_fid --batch-size 1 ~/Documents/dataset/r-sample ~/Documents/dataset/g-sample/6/088000
100%|█████████████████████████████████████████████████████████████████| 2702/2702 [02:58<00:00, 15.17it/s]
Warning: batch size is bigger than the data size. Setting batch size to data size
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/udemegane/workspace/sg2/venv38sg2/lib/python3.8/site-packages/pytorch_fid/__main__.py", line 4, in <module>
pytorch_fid.fid_score.main()
File "/home/udemegane/workspace/sg2/venv38sg2/lib/python3.8/site-packages/pytorch_fid/fid_score.py", line 249, in main
fid_value = calculate_fid_given_paths(args.path,
File "/home/udemegane/workspace/sg2/venv38sg2/lib/python3.8/site-packages/pytorch_fid/fid_score.py", line 238, in calculate_fid_given_paths
m2, s2 = _compute_statistics_of_path(paths[1], model, batch_size,
File "/home/udemegane/workspace/sg2/venv38sg2/lib/python3.8/site-packages/pytorch_fid/fid_score.py", line 218, in _compute_statistics_of_path
m, s = calculate_activation_statistics(files, model, batch_size,
File "/home/udemegane/workspace/sg2/venv38sg2/lib/python3.8/site-packages/pytorch_fid/fid_score.py", line 204, in calculate_activation_statistics
act = get_activations(files, model, batch_size, dims, cuda)
File "/home/udemegane/workspace/sg2/venv38sg2/lib/python3.8/site-packages/pytorch_fid/fid_score.py", line 102, in get_activations
for i in tqdm(range(0, len(files), batch_size)):
ValueError: range() arg 3 must not be zero
This error occurs despite a batch size that is clearly smaller than the dataset (eg 1).
I don't know the cause.
Thank you.
I am getting this error when computing FID between 2 datasets.
Traceback (most recent call last):
File "fid_score.py", line 260, in <module>
args.dims)
File "fid_score.py", line 248, in calculate_fid_given_paths
fid_value = calculate_frechet_distance(m1, s1, m2, s2)
File "fid_score.py", line 184, in calculate_frechet_distance
raise ValueError('Imaginary component {}'.format(m))
ValueError: Imaginary component 0.6888144870124462
it is seems that the gpu is changed to device, but how to use the device?
I tried to use this: cuda:0.
But gpu is not used.
How can we do for this? Thanks.
I used it via powershell with two different datasets with same size and same format(png).
However, I got an error likes
Traceback (most recent call last):
File "C:\Users\usrname\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\usrname\anaconda3\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\usrname\anaconda3\lib\site-packages\pytorch_fid_main.py", line 4, in
pytorch_fid.fid_score.main()
File "C:\Users\usrname\anaconda3\lib\site-packages\pytorch_fid\fid_score.py", line 249, in main
fid_value = calculate_fid_given_paths(args.path,
File "C:\Users\usrname\anaconda3\lib\site-packages\pytorch_fid\fid_score.py", line 238, in calculate_fid_given_paths
m2, s2 = _compute_statistics_of_path(paths[1], model, batch_size,
File "C:\Users\usrname\anaconda3\lib\site-packages\pytorch_fid\fid_score.py", line 218, in _compute_statistics_of_path
m, s = calculate_activation_statistics(files, model, batch_size,
File "C:\Users\usrname\anaconda3\lib\site-packages\pytorch_fid\fid_score.py", line 204, in calculate_activation_statistics
act = get_activations(files, model, batch_size, dims, cuda)
File "C:\Users\usrname\anaconda3\lib\site-packages\pytorch_fid\fid_score.py", line 110, in get_activations
images = images.transpose((0, 3, 1, 2))
ValueError: axes don't match array
I don't know what is the problem
Hi, I know you are looking for a bug of why pytorch fid behaves different with tf fid, and I am also searching for why.
I found that the bilinear interpolation output is different of pytorch (1.1) and tensorflow (1.13.1). See this issue (pytorch/pytorch#10604).
Hope it helps. b.t.w. could you tell me how different is FID in this repo with that of original? It is quite necessary for me. Thank you!
Hi,
I am trying to apply this implementation for grayscale image datasets, but that doesn't work. I think the function imread has to have a condition to convert grayscale images to RGB ones.
Line 68 in 011829d
I've tried to use this:
def imread(filename): """ Loads an image file into a (height, width, 3) uint8 ndarray. """ img_ = np.asarray(Image.open(filename), dtype=np.uint8) if len(img_.shape) < 3: img_ = np.asarray(cv2.cvtColor(img_, cv2.COLOR_GRAY2RGB), dtype=np.uint8) return img_[..., :3]
But it does not work for me, the resulting Images array has the dimension (batch_size,), and an error occurs when trying to transpose it to the shape (batch_size, 3, hight, width) at line 116
images = images.transpose((0, 3, 1, 2))
can you help to overcome this issue, please
Using calculate_frechet_distance.py, The FID between cifar-10 and the standard npz provided by TTUR, the FID is 15. Any idea why?
PIL has some issues with reading TIF files as it automatically converts to uint8. What do you think about adding something like:
if path.suffix == ".tif":
img = tifffile.imread(path).astype(np.float32)
else:
img = Image.open(path).convert('RGB')
here. FWIW I've tested this modification locally and it's not a breaking change.
First, thank you for the high-quality implementation!
scipy.misc.imread
was deprecated in SciPy 1.0.0, and was removed in 1.2.0. Nevertheless fid_score.py
imports this function. This will fail unless the user downgrades their SciPy.
SciPy recommends using imageio.imread
instead. Another alternative is np.asarray(PIL.Image.open(...))
. If you have a preference for which dependency you'd like to add, I will submit a PR.
All best,
Jon
Hello,
Why is the PyTorch FID different from the TF implementation? Is it because of different weights for the Inception Network used in both frameworks or is it because of numerical differences in the implementation of the operations in both frameworks?
Which magnitude do the differences have?
Thank you very much!
Jan
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.