activevisionlab / ray-onet Goto Github PK

Hello, I've found this work really interesting and I would like to ask a question about the scaling factor in your framework.

From the paper I can find the following paragraphs:

about training:

and inference:

So is the inferred 3D model mesh generated up-to-scale? In practice the scaling calibration during training is just used to improve shape reconstruction results, yet leaving the scale factor unknown during inference, isn't it?

Thanks in advance.

I'm trying to use the model with my own images but the resulting 3d mesh is always have problems with the z axis.
Is there any preprocessing required to be done for the input images before using the model in inference?

When installing dependencies, the minimum version of pyembree is 0.1.8, the minimum requirement for python is 3.8, and the installation of python3.6 in element.yaml, and the installation of pyembree is 0.1.4

After creating the conda env and compiling the extension modules, in order to test your code on the provided input images in the demo folder, I used python generate.py configs/demo.yaml .
The mesh created for all the demo folder's images is like this one:

Can anyone help me with this issue?
Thank you for your attention

Hi, test on custom data gives very poor outcomes. Is that because the model provided is only trained on 13 shapenet categories. Would it help to train on entire shapenet dataset. Any suggestions? Thanks

I want to train Ray-ONet from scratch.

But I have a problem in the datasets preprocessing, can you provide the preprocessed data set or help me solve it ?

Thanks!

Hi,
I am trying to train rayonet on custom data. I have prepared my data two times but I am getting the same error while training:

File "trainVTPxy90CLAHE.py", line 139, in
loss = trainer.train_step(batch)
File "/home/ubuntu/ray-onet/im2mesh/rayonet/training.py", line 52, in train_step
loss = self.compute_loss(data)
File "/home/ubuntu/ray-onet/im2mesh/rayonet/training.py", line 202, in compute_loss
occ_pred = self.model.decode(scale_factor, points_xy, c, c_local) # (B, n_points, num_samples)
File "/home/ubuntu/ray-onet/im2mesh/rayonet/models/init.py", line 75, in decode
logits = self.decoder(scale_factor, points_xy, c_global, c_local)
File "/home/ubuntu/anaconda3/envs/rayonet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/ray-onet/im2mesh/rayonet/models/decoder.py", line 166, in forward
net = self.fc_geo1(net)
File "/home/ubuntu/anaconda3/envs/rayonet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/ray-onet/im2mesh/layers.py", line 40, in forward
net = self.fc_0(self.actvn(x))
File "/home/ubuntu/anaconda3/envs/rayonet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/anaconda3/envs/rayonet/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 87, in forward
return F.linear(input, self.weight, self.bias)
File "/home/ubuntu/anaconda3/envs/rayonet/lib/python3.6/site-packages/torch/nn/functional.py", line 1372, in linear
output = input.matmul(weight.t())
RuntimeError: size mismatch, m1: [65536 x 258], m2: [259 x 256] at /tmp/pip-req-build-808afw3c/aten/src/THC/generic/THCTensorMathBlas.cu:290

Maybe I am missing sth during data preparation .
Any help would be greatly appreciated.