yyguo0536 / svin Goto Github PK

Thanks for sharing your research, which inspire me a lot!
There are still some question on the framework after reading your paper and checking the code. It will be very pleasure if you can help me better understanding it.

1. In my opinion, the interpolation part of the network is a fine-tuning of deformation field output by the first part. Why don't we derectly output the target interpolated results in the interpolation part?
2. Can Eq5 and 6 help in the experiments?

File "/media/sc1/BED0CCE0D0CCA04F/lizhipeng/BIS/SVIN/interpolation/motion_model.py", line 4, in
from grid import Dense3DSpatialTransformer
ImportError: No module named grid

Hello !

I find your paper and I'm very interested about it.

So I want to try your code and I was wondering if you can give some details about the dataset.

1. What type of file for input ? Is it 3D file, or 2D slices ?
2. What type of medical image can be used ? DICOM, nifti ?
3. What are the dimensions that are allowed ? Is 96 the maximum ?
4. What kind of value should be in the column 'training file' and 'ground_truth' ? and why 24, 48, 96 ?

data = pd.read_csv('your training data with scale/1')
training_l = data['training_file']
label_l = data['ground_truth']
for i in range(len(training_l)):
    training_rows96.append(training_l[i])
    trainning_label96.append(label_l[i])

Thank you in advance !

Hi there,

1. Load netG-epoch500-all
   To be able to use the Interpolation, do we have to run the Motion Field Estimation first and then use the saved netG-epochXXX ?
   Or is it possible to run the interpolation from scratch without any pretrained generator from the motion field ?

Because actually I have tried to use the netG-epoch from the motion field but I have a mismatch error.

# STM_Net_Dir = 'netG-epoch500-all'
STM_Net_Dir = './../motion-net/save_motion/netG-epoch400'

Error :

Traceback (most recent call last):
  File "train_4d.py", line 72, in <module>
    STM_Net.load_state_dict(tmp)
  File "/home/yiheng/anaconda3/envs/svin/lib/python2.7/site-packages/torch/nn/modules/module.py", line 830, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for DataParallel:
	size mismatch for module.up_block6.convb.convs.0.weight: copying a param with shape torch.Size([32, 32, 3, 3, 3]) from checkpoint, the shape in current model is torch.Size([32, 56, 3, 3, 3]).

Maybe I misunderstand this part and it doesn't mean to load this weight.

2. Skip loading

I have tried to bypass this loading option by commenting these line:

    STM_Net = Net_3d_ALL()
    STM_Net.to(device)
    STM_Net = torch.nn.DataParallel(STM_Net, [0])
    # tmp = torch.load(STM_Net_Dir)
    # STM_Net.load_state_dict(tmp)
    # STM_Net.eval()

But therefore I meet this error

Traceback (most recent call last):
  File "train_4d.py", line 174, in <module>
    model.optimize_parameters()
  File "/home/yiheng/bacasable/SVIN/interpolation/models/model.py", line 261, in optimize_parameters
    self.forward()
  File "/home/yiheng/bacasable/SVIN/interpolation/models/model.py", line 174, in forward
    self.netG(self.combined_image, self.D96_combine, self.D48_combine, self.D24_combine)
AttributeError: InterpolationModel instance has no attribute 'D48_combine'

And in fact, self.D48_combine as well for self.D24_combine don't appear in the model.py file.

    def set_input(self, combined_image, img0_96, img0_48, img0_24, \
        img1_96, img1_48, img1_24, imgt_96, imgt_48, imgt_24, D0t_96, D1t_96, index_l):

        self.real96_A = imgt_96.to(self.device)
        self.real48_A = imgt_48.to(self.device)
        self.real24_A = imgt_24.to(self.device)
        self.combined_image = combined_image.to(self.device)
        self.img0_96 = img0_96.to(self.device)
        self.img1_96 = img1_96.to(self.device)
        self.img0_48 = img0_48.to(self.device)
        self.img1_48 = img1_48.to(self.device)
        self.img0_24 = img0_24.to(self.device)
        self.img1_24 = img1_24.to(self.device)
        self.D0t_96 = D0t_96.to(self.device)
        self.D1t_96 = D1t_96.to(self.device)
        self.D96_combine = torch.cat([self.D0t_96, self.D1t_96], 1)
        self.index_l = index_l.to(self.device)

Thank you in advance

Could you share the dataset link ?
I can't find "4D-C-CT" on the Internet.
Thank you!

Hi, thanks for sharing your outstanding work. Recently I read your another article ' Automatic left ventricular cavity segmentation via deep spatial sequential network in 4D computed tomography'. You have also done an outstanding job in this paper. And it mentioned that you plan to release our source codes at Github. But I did not find the source code of this paper in your repository. Is it convenient for you to provide the source code？

Hi !

I have an issue about the memory and I remember that you said in your paper that you trained on two 11GB 1080Ti GPU.

RuntimeError: CUDA out of memory. Tried to allocate 108.00 MiB (GPU 0; 10.92 GiB total capacity; 10.18 GiB already allocated; 73.38 MiB free; 10.27 GiB reserved in total by PyTorch)

I was hoping if you can tell me what did you change in your code (or maybe just an option that I missed) to work on both GPU to avoid this memory issue.

Thanks in advance !

from .warp_layer import Dense3DSpatialTransformer

ImportError: cannot import name Dense3DSpatialTransformer

Hi,

Motion :
I wanted to try the testing of the motion field but I have an issue with the variable index_list, it's not defined in your test_motion.py

Interpolation :

1. Your ````eval_data.py has an emptymain```, is it the same as the one in the motion-net ?
2. The data to use is the original testing set ? Or the deformed image produced by the test of motion-net ?

data = pd.read_csv(\
        'test data')
testdata_l = data['']
testlabel_l = data['']

3. What .pth I should put in dir_deform = '.pth' of the test.py ?

Thank you !