sicara / easy-few-shot-learning Goto Github PK
View Code? Open in Web Editor NEWReady-to-use code and tutorial notebooks to boost your way into few-shot learning for image classification.
License: MIT License
Ready-to-use code and tutorial notebooks to boost your way into few-shot learning for image classification.
License: MIT License
Hi Diego, thank you so much for contributing to EasyFSL! This topic seems to have raised a lot of concern among users and it's very nice to have some help 😄
I agree it would be good to have a tool to easily feed a support set to
FewShotClassifier.process_support_set()
from a fileroot_dir, structure. It's a good idea to useImageFolder
for this.However, I'm not sure it is relevant to integrate it into a novel
FewShotPredictor
class. I'm concerned that this would raise some confusion withFewShotClassifier
, and I think we can find a lighter solution.I'm thinking this could be achieved with a class
SupportSetFolder
that would extendImageFolder
and implement two methodsget_images() -> Tensor
andget_labels() -> Tensor
that would respectively return images and labels in the specified device.Then the user would ony have to do this:
device = "cuda" support_set = SupportSetFolder(root=path/to/root, transform=my_inference_transform, device=device) with torch.no_grad(): my_classifier.eval() my_classifier.process_support_set(support_set.get_images(), support_set.get_labels()) predictions = my_classifier(query_images.to(device))What do you think?
Hi,
I was looking for same question , and tried your above example in my use case but I get an error
AttributeError: 'SupportSetFolder' object has no attribute 'to'
following is the code I am using
DEVICE = "cuda"
support_set = SupportSetFolder(root='C:/Users/wn00204104/PycharmProjects/Data-Preparation/easy-few-shot-learning/data/test_set', transform=transform, device=DEVICE)
with torch.no_grad():
few_shot_classifier.eval()
few_shot_classifier.process_support_set(support_set.get_images(), support_set.get_labels())
predictions = few_shot_classifier(query_images.to(DEVICE)).argmax(dim=1)
Thanks in advance
Originally posted by @shraddha291996 in #49 (comment)
Hello, When I run Train a model with Classical, there is one problem during validation, how can I solve this
validation_accuracy = evaluate(few_shot_classifier, test_loader, device=DEVICE, tqdm_prefix="Validation")
Hey, @gabrielsicara I trained a model using GPU on google colab but was unable to take its inference on the CPU. Also how to make inference for any input single image on CPU? Like how to define a support set that the model will use for processing? I m trying to explore these on my own and will definitely do a pull request. But if you can answer those it will be helpful.
Tried different ways of loading as well. It works fine on GPU.
Hello, after training, I even checked the accuracy.
But now I want to visualize part of the result.
What should I do?
Problem
Thanks for your sharing about FSL, there is one problem:
When I finished the tutorial 'Discovering Prototypical Networks' , I want to use my own photo data to build test_set, how can I do that and How should I construct my data's structure
Hi Thank you for your great work and sharing it with everyone.
I want to implement few shot learning for a task that I have. where I have collected few samples(10) each for both positive and negative class. How do I train the model on these novel classes using my custom dataset.
Thank you for your help
Hello,
Is there a way to make predtictions on test set in evaluation phase , in your notebook in episodic training and classical training you calculate the accuracy but I also want to predict the class on test set. Can you please suggest regarding this ?
Thanks in advance
thanks
I want to train this model on custom data but I did not understand the split for CUB and I could not even get documentation on EasySet ? do you know where it is ?
i just have 2 classes in my data btw
how about the 1-shot and 5-shot acc by training on mini-imagenet?
Hi,
I've been using your library for my project and I am confused on two of your notebooks.
In [1] you mention that "To train the model, we are just going to iterate over a large number of randomly generated few-shot classification tasks, and let the fit method update our model after each task. This is called episodic training."
However, in [2], you mention that you are also using episodic training but here you add the concept of epochs to it, encapsulating several episodes in each one. Is it because you now use a validation set that you did not used in [1] or that epochs in [1] are implicit because of the 40 000 episodes default value ?
I must say that I am quite confused by this. Do I have a misunderstanding somewhere?
Thank you for clarifying that for me.
[1] https://github.com/sicara/easy-few-shot-learning/blob/master/notebooks/my_first_few_shot_classifier.ipynb
[2] https://github.com/sicara/easy-few-shot-learning/blob/master/notebooks/episodic_training.ipynb
Hi, I am quite new to the topic few-shot learning and would like to thank you for providing this insightful tutorial. I managed to run your code successfully with some slight tweaks to the dataset I use which is Mini-ImageNet.
I wanted to to plot a confusion matrix using PyTorch to make it something similar to this:
Youtube link
Blog link
Do you think it is possible to carry it out in your tutorials on Jupyter Notebook? I have trouble following their methods and implementing in classical_training.ipynb and my_first_few_shot_classifier.ipynb.
The reason I wanted to plot a confusion matrix is because I think it can help me in explaining the results a bit better.
I want to thank you for your contributions again as I felt it really helped me a lot.
It will be great if there was a tutorial for Few shot object detection as well.
hello i have a question.
Is it correct that the backbone should adjust the batch norm statistics when processing the prototypes (s. https://arxiv.org/abs/1603.04779)? I get significant differences when I set the backbone to eval() in the method process_support_set() before computing the prototytpes like:
def process_support_set(
self,
support_images: torch.Tensor,
support_labels: torch.Tensor,
):
self.backbone.eval() # <- Ive added this line the rest is unchanged
support_features = self.backbone.forward(support_images)
self.prototypes = compute_prototypes(support_features, support_labels)
This way the backbone wont update the BN statistics. However I dont know if this is intended and some clarification would be really helpful.
Thanks in advance
I was trying to use the RelationNet class with a backbone that uses single channel inputs. The default compute_backbone_output_shape
method in the AbstractMetaLearner
class expects a 3 channel image input.
I believe the solution would be to convert this method to an instance method and allow subclasses to override it if needed.
Or, Am I missing something and there is an alternate method to achieve this?
Hi @ebennequin, Thanks for this elegant code base, some questions(can be a feature request)
Hello! First of all, thank you very much for your code! I have a few questions for you.
The first question is: I want my model to accurately classify my own pictures. Must the test set and training set be mutually exclusive? Because I have fewer categories, only 6.
The second question is: how can I change your code so that I can run it myself?
Thank you sincerely!
I created an example that might help you.
import torchvision.transforms as tt
import torch
from torchvision.datasets import ImageFolder
from easyfsl.methods import FewShotClassifier
from torch.utils.data import DataLoader
class FewShotPredictor :
"""
This class aims to implement a predictor for a Few-shot classifier.
The few shot classifiers need a support set that will be used for calculating the distance between the support set and the query image.
To load the support we have used an ImageFolder Dataset, which needs to have the following structure:
folder:
|_ class_name_folder_1:
|_ image_1
|_ …
|_ image_n
|_ class_name_folder_2:
|_ image_1
|_ …
|_ image_n
The folder must contain the same number of images per class, being the total images (n_way * n_shot).
There must be n_way folders with n_shot images per folder.
"""
def __init__(self ,
classifier: FewShotClassifier,
device,
path_to_support_images,
n_way,
n_shot,
input_size=224):
"""
:param classifier: created and loaded model
:param device: device to be executed
:param path_to_support_images: path to creating a support set
:param n_way: number of classes
:param n_shot: number of images on each class
:param input_size: size of image
"""
self.classifier = classifier
self.device = device
self.predict_transformation = tt.Compose([
tt.Resize((input_size, input_size)),
tt.ToTensor()
])
self.test_ds = ImageFolder(path_to_support_images, self.predict_transformation)
self.val_loader = DataLoader(
self.test_ds,
batch_size= (n_way*n_shot),
num_workers=1,
pin_memory=True
)
self.support_images, self.support_labels = next(iter(self.val_loader))
def predict (self, tensor_normalized_image):
"""
:param tensor_normalized_image:
Example of normalized image:
pil_img = PIL.Image.open(img_dir)
torch_img = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor()
])(pil_img)
tensor_normalized_image = tt.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])(torch_img)[None]
:return:
Return
predict = tensor with prediction (mean distance of query image and support set)
torch_max [1] = predicted class index
"""
with torch.no_grad():
self.classifier.eval()
self.classifier.to(self.device)
self.classifier.process_support_set(self.support_images.to(self.device), self.support_labels.to(self.device))
pre_predict = self.classifier(tensor_normalized_image.to(self.device))
predict = pre_predict.detach().data
torch_max = torch.max(predict,1)
class_name = self.test_ds.classes[torch_max[1].item()]
return predict, torch_max[1], class_name
Originally posted by @diego91964 in #17 (comment)
Good morning guys and many thanks for the awesome and very helpful code and the effort put to achieve that
I have a question regarding novel image class prediction:
Is there a way to calculate as in 'classical' classification the percentage/probability of a novel image to belonging to each class?
Do you believe a softmax maybe at the 'return predict, torch_max[1], class_name' at the return tensor would have a meaning?
Thanks in advance
Problem
Hello, In Classical_Training.py, I have five classes in teset set, by using evaluate function, can get five classes' average accuary, but I want to get every class's accuary, how can i achieve this?, thanks
Problem
What do you want to do? What is blocking you?
I am reading an article and would like to discuss the possibility of implementing it in the prototypical network (https://arxiv.org/abs/1805.10123).
It would require the creation of a block to represent TEN layer, however the implementation of resnet12 uses a library and would have to be replaced by its own implementation.
What do you think about the proposal in this article for improving the calculation of the loss function?
So i have trained your episodic training notebook on custom data. but I had a question about how would we view the output I got the accuracy also but how would we view the classification
I have a suggestion of improvement for your model API. I would take the support images as parameter only in the constructor and not in forward() and precompute their feature vectors once for all at model creation time.
And then in your forward method you take only query images as input.
It allows to make multiple forward calls without recomputing support feature vectors each time.
It is also more intuitive because forward usually takes as input only elements for which we want a prediction (query images here).
You can optionally add methods to update the support images (to add or remove some for example).
Problem
I am trying to train backbone using classical training, and use Finetune in methods to fine-tune the model by episodic_training.ipynb. How should I implement it? I see that the episodic_training.ipynb you wrote has fixed the parameters for backbone, but when I import the pre-trained model for fintune, it does not work properly.
Another question is, how much is n_validation_tasks generally set to? Is there a standard? Because the setting of this hyperparameter will affect the result. I look forward to your answer.
convolutional_network = resnet50(num_classes=2).to(DEVICE)
convolutional_network.load_state_dict(torch.load('save_model/resnet50.pt'))
few_shot_classifier = Finetune(convolutional_network).to(DEVICE)
Hi,
I have a question about size of batch and the number of images in each batch in episodic and my-first-few-shot-classifier notebooks . As you mentioned in your https://github.com/sicara/easy-few-shot-learning/blob/master/notebooks/my_first_few_shot_classifier.ipynb number of images in each batch calculated with (nway*(nshot+nquery)) then what is the size of batch or its same like number of images in each batch?
Thanks in Advance
Describe the bug
This is very small typo but in README.md, if you click on example of classical training, it goes to episodic notebook.
To Reproduce
Steps to reproduce the behavior:
Not sure if I should add pull request or raise an issue when I find small typos. Let me know :)
Problem
What do you want to do? What is blocking you?
Considered solutions
What have you tried but didn't work?
How can we help
Be as clear and concise as possible so we can help you in the most efficient way.
I am a complete beginner in python coding and I am tasked with an image classification problem.
I have a dataset of 25000 images with 2 classes (50-50). 1 class being photos of corrosion and the other with photos of non-corrosion.
How do I change the code in my_first_few_shot_classifier.ipynb so that I can upload my own dataset from my local drive as well as split the dataset into training and test set.
Thank you!
Problem
Hello. This is nice repo and make me easily to understand and implement FSL model to my project. But i would like to ask you How to implement and train Transductive Fine-tuning model
Since this model is classical training (if i understand correctly) so i use the same classical training tutorial and just replace PrototypicalNetworks
to TransductiveFinetuning
in few_shot_classifier()
But in training stage, This error is show up
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
/tmp/ipykernel_16618/1151720753.py in <module>
6 for epoch in range(n_epochs):
7 print(f"Epoch {epoch}")
----> 8 average_loss = training_epoch(model, train_loader, train_optimizer)
9
10 if epoch % validation_frequency == validation_frequency - 1:
/tmp/ipykernel_16618/3987028259.py in training_epoch(model_, data_loader, optimizer)
7
8 loss = LOSS_FUNCTION(model_(images.to(DEVICE)), labels.to(DEVICE))
----> 9 loss.backward()
10 optimizer.step()
11 # model_(images.to(DEVICE))
~/.local/lib/python3.7/site-packages/torch/_tensor.py in backward(self, gradient, retain_graph, create_graph, inputs)
361 create_graph=create_graph,
362 inputs=inputs)
--> 363 torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
364
365 def register_hook(self, hook):
~/.local/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)
173 Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
174 tensors, grad_tensors_, retain_graph, create_graph, inputs,
--> 175 allow_unreachable=True, accumulate_grad=True) # Calls into the C++ engine to run the backward pass
176
177 def grad(
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
So i comment loss.backward()
and optimizer.step()
line as it already have in forward
function of Transductive Fine-tuning model but next problem is the loss of model in training process does not reduce and i still don't know why?
Can you provide me how to fix this error and how to implement Transductive Fine-tuning or other classical model in EasyFSL?
Thanks a lot
Thank you for the sharing. I trained the few shot algorithm you provided on a dataset that I dispose. However, how do I do the inference on one image? I mention that my train and test datasets belong to the same classes, but with different images.
HI,
I really loved your repo and implementation. Just want to check if there any timeline on when the MAML based code will be out?
First of all, great repo, very useful!
I was wondering when will you release the next version that includes the SupportSetFolder class to the latest PyPi distribution.
Apologies for the beginner question, but could you please provide me some information on how to create a custom dataset for EasyFSL?
I know there are various labeling tools, different output file formats, and many examples of PyTorch custom datasets. Is there a particular labeling tool, file format, and dataset creation code that would be required here?
I've used Label Studio before and exported in a JSON format, and I've seen other folks use Labelbox and the Darknet format (for YOLOv5). Your example code starts with an existing dataset, so I want to make sure I create mine in a compatible way.
Thank you for any help you can provide.
Hi, Thanks for making such a simple and beautiful library for Few-Shot Learning. I have a query when we run a particular cell from your notebook for training meta-learning model, does it also train the ResNet18 Model on the given Dataset for generating a better representation of Feature Images like we do it while we do transfer learning when we train classifier model on our custom dataset using Imagenet pre-trained parameters or Does it only trains Prototype network?
Please, clarify this doubt.
Thanks again.
Can you provide an example of how the code can be run with relation networks?
Hello, Thank you very much for your amazing work, its very helpful.
I have one question on getting prediction on custom dataset, so basically I am using Easyset for my custom dataset and classical training notebook and I want to see prediction/classification also , for example from which class my test image belongs to.
I hope my quetsion is clear to you.
Thanks in advance
Problem
When combining the "My First Few Shot Classifier" with a custom dataset from the "Easy Set", I got this error:
Exception has occurred: PicklingError
Can't pickle <function <lambda> at 0x13d8f6e50>: attribute lookup <lambda> on __main__ failed
File "/Users/yannusinovich/Documents/ONX_AI/machine-learning/recommender/issue_logs/few_shot_learning_easyfsl.py", line 279, in <module>
) = next(iter(test_loader))
Considered solutions
test_set
in "My First Few Shot Classifier" to dataset
since test_set
wasn't defined.'EasySet' object has no attribute '_flat_character_images'
.self._flat_images_and_labels: List[Tuple[str, int]] = list(zip(self.images, self.labels))
in the __init__
of the EasySet
class, and I changed the sampler code to:N_WAY = 4 # Number of classes in a task
N_SHOT = 4 # Number of images per class in the support set
N_QUERY = 2 # Number of images per class in the query set
N_EVALUATION_TASKS = 10
dataset.get_labels = lambda: [
instance[1] for instance in dataset._flat_images_and_labels
]
The value of self._flat_images_and_labels
for EasySet looks exaclty the same as Omniglot's _flat_character_images
, but it runs into the error above at this step:
(
example_support_images,
example_support_labels,
example_query_images,
example_query_labels,
example_class_ids,
) = next(iter(test_loader))
How can we help
Could you please provide me some guidance on how to use the "My First Few Shot Classifier" notebook with a custom dataset from the "Easy Set"?
Hello, author.
Can you tell me the corresponding paper of this source code?
thank you
Hello,
I am Running Episodic training with Matching Network and backbone ViT. I get the error CUDA out of memory.
My FSL setting is :
Nway- 9
Nshot- 10
Nquery- 10
batch size- 2
I am also resizing the my dataset rgb images with 224 by 224 .
Could anyone please help me with this issue?
One thing when I am running Classical training with Prototypical network and Efficient backbone with same setting its running fine.
Thanks in advance
Hi, thanks for your code, it helps me a lot.
But it also got some problems for a newbie like me.
Although I make the code run successfully now, I also make a lot of compromises to some errors.
I combined the code from classical_training.ipynb and my_first_few_shot_classifier.ipynb.
I post all my code step by step and point out the problems I met.
I am running Windows10.
The environment is created by Anaconda.
Cuda10.2, Cudnn 7.0, PyTorch 1.10.1
At last, great thanks for your code again.
Let's discuss this together.
Hi ,
I want to try out few things with freezing and relaxing some network layers, but I am not sure exactly how I can do that with few shot dual architecture.
Thanks in Advance
Hi,
I am using your classical and episodic Training notebooks, they are very helpful for my project , although I want to try different backbones like EfficientNet. I am new in this so do you have any idea if I can use different backbone than ResNet if yes what changes I will have to consider in the code?
Thanks in adavance
After training a model based on Prototype networking and evaluating on test data. I want to do prediction on new image set. How can I do that?
Problem
I'm trying to modify and run the my_first_few_shot_classifier notebook with the miniImageNet data set.
But not sure about the data (where to get it/how to store it/how to load it).
What is the recommended/correct/required form ? I have downloaded various (mini) imagenet data sets and each are quite different.
(1) All images (60k) are in a single directory (/images) with the naming format:
n0153282900000005.jpg
(2) Test images are in one test directory & train/val are in a directory p/class.
test-2015 / ILSVRC2015 / Data / DET / test
ILSVRC2012_test_00000003.JPEG
train-val-2015 / ILSVRC2015 / Data / DET / train / ILSVRC2013_train / n00007846
n00007846_71814.JPEG
(3) No images but a python dict with the (I'm guessing) RGB values of each image.
{'image_data': array([[[[200, 242, 240],
[204, 235, 249],
[202, 235, 250],
...,
[210, 222, 238],
[206, 219, 232],
[218, 229, 237]],
[[167, 171, 219],
[172, 176, 224],
[175, 179, 227],
...,
[ 68, 74, 74],
[ 47, 53, 53],
[ 80, 85, 91]]]], dtype=uint8), 'class_dict': {'n01930112': [0, 1, 2, 3, 4
I got 1 from a Google drive. I got 2 from official imagenet ILSVRC2015 download page. And the last one from Kaggle.
Considered solutions
I will show what I have tried re data set number 1 (where all images are in a single directory).
base = "\images"
train_set = MiniImageNet(root=base, split="train", training=True)
test_set = MiniImageNet(root=base, split="test", training=False)
And also tried:
train_set = MiniImageNet(
split="train",
root=base,
transform=transforms.Compose(
[
transforms.Grayscale(num_output_channels=3),
transforms.RandomResizedCrop(image_size),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
]
),
)
But in both cases, getting error.
Traceback (most recent call last):
File "C:/git_repos/dcu/fsl/easy-few-shot-learning/gary-code/first-fsl-orig.py", line 172, in <module>
) = next(iter(test_loader))
File "C:\git_repos\dcu\fsl\easy-few-shot-learning\venv\lib\site-packages\torch\utils\data\dataloader.py", line 521, in __next__
data = self._next_data()
File "C:\git_repos\dcu\fsl\easy-few-shot-learning\venv\lib\site-packages\torch\utils\data\dataloader.py", line 561, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "C:\git_repos\dcu\fsl\easy-few-shot-learning\venv\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\git_repos\dcu\fsl\easy-few-shot-learning\venv\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\git_repos\dcu\fsl\easy-few-shot-learning\easyfsl\datasets\mini_imagenet.py", line 110, in __getitem__
Image.open(self.data_df.image_path[item]).convert("RGB")
File "C:\git_repos\dcu\fsl\easy-few-shot-learning\venv\lib\site-packages\pandas\core\series.py", line 882, in __getitem__
return self._get_value(key)
File "C:\git_repos\dcu\fsl\easy-few-shot-learning\venv\lib\site-packages\pandas\core\series.py", line 990, in _get_value
**loc = self.index.get_loc(label)**
File "C:\git_repos\dcu\fsl\easy-few-shot-learning\venv\lib\site-packages\pandas\core\indexes\range.py", line 358, in get_loc
raise KeyError(key)
KeyError: tensor(9552)
Process finished with exit code 1
It looks like there are no labels assigned to the images.
How can we help
So the 2 questions are:
(1) What is the correct way to store the data ?
(2) How to load the data ?
Any help will be much appreciated. An example of this notebook running on mini imagenet would be most helpful.
Thanks,
Gary
Hi !
Thanks a lot for your great work. Is it possible to add the improvement of TIM https://proceedings.neurips.cc/paper/2021/file/4d7a968bb636e25818ff2a3941db08c1-Paper.pdf ?
Kindest regards,
Pierre
Problem
I am new to FSL and have a simple problem in my scientific domain that I thought I would try as a learning example. I am trying to perform classical training for a 5 shot 2 way problem. When I am running the code from the tutorial notebook as it is after using EasySet to create a custom data object, I am getting the following error when I encounter the validation epoch during my training:
ValueError : Sample Larger than population or is negative
Considered solutions
I've tried changing the batch size and n_workers so far, and neither have worked
How can we help
I can't figure out what is going wrong here. I am very new to machine learning and would love to have your help in any way possible!
Hi. Thank you for your great work and sharing it with everyone.
I have a question.
For custom datasets, how to divide the class? (train, val, test) Randomly select some classes as the training set, or?
Do you have any tricks?
Hello,
I have a very basic question on support images used in testing phase, I know that they are used for evaluation of the model but what is their exact role ? As support images in training phase are used for model training and prototypes are created in the training phase. What happens in testing with support images?
Thanks in advance
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.