Git Product home page Git Product logo

Comments (11)

arthurdouillard avatar arthurdouillard commented on September 23, 2024 1

Oh ok, so that just RotMNIST with a kind of task sampler...

Well, I think your solution @TLESORT is nice. No need to create a new classes for that I guess. Although, we can add in the doc an explanation how to do it.

from continuum.

TLESORT avatar TLESORT commented on September 23, 2024 1

This implementation will be added in the documentation.

from continuum.

arthurdouillard avatar arthurdouillard commented on September 23, 2024

I'm not sure to understand this dataset.

  • an angle is associated to a pair? Or every pair can be seen in any angle?
  • for the pair (0,1), what is the class to predict? Is it multilabel?

from continuum.

TLESORT avatar TLESORT commented on September 23, 2024

Hi Kishaan, thanks for your issue and for trying to add a new scenario to continuum! :)

This setting is a bit tricky I need to think a bit how to implement it in the best way in continuum.
But I think you will need to implement a new class in continuum for that.
If I understand correctly, in MNIST-360 the rotation depends not only on the task but also on the class, which is not possible yet in continuum.

One trick to make it possible, could be to create two incremental scenarios and concatenate after batches.
Something like this (but it is very dirty :`) )

continual_dataset = MNIST(.....)
list_rotation_first = [ trsf.rotation(angle * i) for i in range(10) ]
first_digit_scenario = ClassIncremental(continual_dataset, increment=1, transformations=list_rotation_first)

list_rotation_second = [ trsf.rotation(45 +angle * i) for i in range(10) ]
second_digit_scenario = ClassIncremental(continual_dataset, increment=1, transformations=list_rotation_second)


for task_id in range(3*9):
     taskset_1 = first_digit_scenario[task_id%9]
     taskset_2 = second_digit_scenario[1+task_id%9]
     # batch_1 ~ taskset_1
     # batch_2 ~ taskset_2
     # batch = concatenate(batch_1, batch_2 )

Tell me if it helps you :)
I will think of a way to make it cleaner.
Any better idea @arthurdouillard ?

from continuum.

TLESORT avatar TLESORT commented on September 23, 2024

@arthurdouillard I think the goal is to predict the label of a given class whatever the rotation. The 1's data stay labelized as one whatever the rotation.
if I understand correctly, this is a normal classification scenario but with some variation to enable better transfer.

from continuum.

arthurdouillard avatar arthurdouillard commented on September 23, 2024

@TLESORT hum, but there are two digits on the image, which one is supposed to be predicted?

from continuum.

TLESORT avatar TLESORT commented on September 23, 2024

@arthurdouillard you are speaking about this representation?
image

I think that each "subplot" is just 9 images together sampled form the same task. So each image has only one label. Or do I miss something?

from continuum.

TLESORT avatar TLESORT commented on September 23, 2024

@Kishaan did you manage to make the proposed implementation run?
Is it doing what is expected?

from continuum.

Kishaan avatar Kishaan commented on September 23, 2024

HI,
Sorry for the delay, I was not expecting such a prompt reply. I was going through the implementation from the author's who proposed this dataset. Like you said, it seems a bit tricky to implement it directly. They make batches with pairs of classes at a time, {0,1}, {1,2} ... etc which ends up in 9 pairs. Every class's samples are rotated with increasing rotation spanning an entire 360* angle. They fix a round parameter R (which is set to 3) where these batches of 9 pairs repeat (from what I understood, with increasing rotation angles). In summary, the model sees batches of digit pairs, in which the digits are rotated with gradually increasing angles (the distribution changes gradually).

In the shared image, as @TLESORT pointed out, every subplot contains 9 images, one label for each image, and the rotations in each of this subplot increase gradually. I haven't had time to try out your solution yet. From what I see in your example, the rotation angle you specified was one per class, am I right? In MNIST-360, they do it in per class manner, as in, every sample in a class would be rotated with incrementally increasing angle.

I will post here if I find a way to do this.

from continuum.

TLESORT avatar TLESORT commented on September 23, 2024

hmm, then maybe with a supplementary loop in could work.
(something like this)

continual_dataset = MNIST(.....)
for i in range(3):
  start_angle = 120 * i
  angle = 120 / 9
  list_rotation_first = [ trsf.rotation(start_angle + angle * i) for i in range(10) ]
  first_digit_scenario = ClassIncremental(continual_dataset, increment=1, transformations=list_rotation_first)
  
  list_rotation_second = [ trsf.rotation(start_angle  + 45 +angle * i) for i in range(10) ]
  second_digit_scenario = ClassIncremental(continual_dataset, increment=1, transformations=list_rotation_second)
  
  
  for task_id in range(9):
       taskset_1 = first_digit_scenario[task_id]
       taskset_2 = second_digit_scenario[1+task_id%9]
       # batch_1 ~ taskset_1
       # batch_2 ~ taskset_2
       # batch = concatenate(batch_1, batch_2 )

from continuum.

TLESORT avatar TLESORT commented on September 23, 2024

Hi, I made an implementation for MNIST-360

    import os
    import torch
    from continuum.scenarios import ClassIncremental
    from torchvision import transforms

    folder = "tests/samples/mnist360/"
    if not os.path.exists(folder):
        os.makedirs(folder)
    continual_dataset = MNIST(data_path=DATA_PATH, download=False, train=True)

    for i in range(3):
        start_angle = 120 * i
        angle = int(120 / 9)

        list_rotation_first = [[transforms.RandomAffine(degrees=[start_angle + angle * j, start_angle + angle * j + 5])]
                               for j in range(10)]
        list_rotation_second = [
            [transforms.RandomAffine(degrees=[start_angle + 45 + angle * j, start_angle + 45 + angle * j + 5])] for j in
            range(10)]
        first_digit_scenario = ClassIncremental(continual_dataset, increment=1, transformations=list_rotation_first)
        second_digit_scenario = ClassIncremental(continual_dataset, increment=1, transformations=list_rotation_second)

        for task_id in range(9):
            taskset_1 = first_digit_scenario[task_id]
            taskset_2 = second_digit_scenario[1 + task_id % 9]

            # / ! \ we can not concatenate taskset here, since transformations would not work correctly

            loader_1 = DataLoader(taskset_1, batch_size=64)
            loader_2 = DataLoader(taskset_2, batch_size=64)

            nb_minibatches = min(len(loader_1), len(loader_2))
            for minibatch in range(nb_minibatches):
                x_1, y_1, t_1 = next(iter(loader_1))
                x_2, y_2, t_2 = next(iter(loader_2))

                x, y, t = torch.cat([x_1, x_2]), torch.cat([y_1, y_2]), torch.cat([t_1, t_2])

                # train here on x, y, t

                #### to visualize result ####
                # from continuum.viz import visualize_batch
                # visualize_batch(batch=x[:100], number=100, shape=[28, 28, 1], path=folder + f"MNIST360_{task_id + 9 * i}.jpg")

I close the issue since it has been inactive since few weeks.
Feel free to reopen it if needed :)

from continuum.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.