leslietrue / cpp Goto Github PK

hello is it possible to use my arbitrary data with this?

Hello @LeslieTrue, and very nice work! Congrats on the ICLR acceptance!

I am really interested in reproducing your results on ImageNet. To this end, I would like to ask 2 things:

1. Hyperparameters on the paper and script args

After trying to match the code to the paper and supp. (table 7), I end up with the following hyperparameters for ImageNet

 "hidden_dim": 2048, 
 "z_dim": 1024, 
 "n_clusters": 1000, 
 "epo": 20, 
 "bs": 1024, 
 "lr": 0.0001, 
 "lr_c": 0.0001, 
 "momo": 0.9, 
 "pigam": 0.05, 
 "wd1": 0.0001, 
 "wd2": 0.005, 
 "eps": 0.1,   #  used in MLCLoss
 "pieta": 0.12,  # sinkhon knop for imagenet
 "piiter": 5, 
 "seed": 42, 
 "warmup": 2000,  # is this correct? Is this what you mean by 1-2 epochs on imagenet I guess?

Could you please confirm and let me know if there is any other hyperparameter I need to specify that may not be in the paper? Saving the args you specified to get state-of-the-art results to a .json file would help.

2. Evaluation after training

During training the MLPs on top of CLIP, you have an intermediate evaluation, which, to my understanding, is based on the mini-batch. Thus, the provided script main_efficient.py does not have any evaluation to reproduce the NMI and ACC from the paper.

How do I do that? So far, my best guess is that I need to compute z, logits = model(x) for the whole dataset and store the results and afterward:

self_coeff = (logits @ logits.T).abs().unsqueeze(0)
Pi = sink_layer(self_coeff)[0]
Pi = Pi * Pi.shape[-1]
Pi = Pi[0]
Pi_np = Pi.detach().cpu().numpy()
acc_lst, nmi_lst, _, _, pred_lst = spectral_clustering_metrics(Pi_np, n_clusters, y_np)

Is this how you actually evaluated? I guess if the test set has $n$ samples, that means that we need to compute the eigenvalues of an $n \times n$ matrix, which, as far as I can recall, is of $O(n^3)$.

Your help would be highly appreciated and will help us report your method in other datasets!

Thanks in advance, and have a great day!

Nikolas

Hello,

Firstly, I'd like to express my gratitude for sharing this code and your dedication to the project. After going through your paper, my understanding was that the CPP algorithm estimates the number of clusters. However, in the optimalcluster.py file, it seems that the number of clusters is required as an input.

Have I misunderstood something, or is there a part of the implementation that is yet to be completed?

Thank you for your time and clarification.

I have been following your work since last August! Well done and congratulations!

Thanks for your GREAT WORK!!

But it seems that when training ImageNet, the cluster head's output is never used. In main_efficient.py,

with autocast(enabled=True):
                z, logits = model(x)
                logits = z
                self_coeff = (logits @ logits.T).abs().unsqueeze(0)

Could you please offer some explanations? Thanks a lot