tancik / fourier-feature-networks Goto Github PK

Hi there,

how much RAM is needed to run it with the Aramdillo.ply? Running it with my local .ply files works fine. But the 500GB RAM on our server are not enough for the Armadillo (had to stop it manually). Do you have some advice here?

Best,
Timon

The demo notebook's imports do not work out of the box in Colab in their current state.
I get
ImportError: cannot import name 'stax' from 'jax.experimental' (/usr/local/lib/python3.7/dist-packages/jax/experimental/__init__.py)

and

ImportError: cannot import name 'optimizers' from 'jax.experimental' (/usr/local/lib/python3.7/dist-packages/jax/experimental/__init__.py)

Switching the stax and optimizers import to from jax.example_libraries import stax, optimizers fixed the import issues for me

Hi,

I am trying to run the occupancy experiment, but I don't have the 3D models you used. Can you tell me where I can find them?

Thank you

Regarding the 2D image regression task, is an MLP associated with a single image? Do you need to train a new MLP if you want to regress another image?

I have ipynb that you had made based on pytorch cnn
but it is now deleted from your github
Is there any problem with pytorch cnn?
I have had diffculty understanding JAX library...so I am gonna try to use Fourier Feature mapping net with pytorch cnn

I installed jax with the below command,

pip install --upgrade "jax[cpu]" install -q neural_tangents==0.2.2 livelossplot

but I come across the error jax.api does not exist while importing neural tangent package and
searched to find that I need to upgrade neural tangent package, or downgrade jax.

Both worked, but in either case I run into another kind of error while executing the other codes.

I also tried running the codes on Colab, by clicking on the badge in the notebook file, but also had the same error: jax.api does not exist.

I find it hard to find any proper combination of those two packages; neural tangent and jax.

Could anyone help me please?

Your work is really impressive. I visited your git to reproduce your result hoping that it could be the baseline for my research.
But there seem to be some issues here.
An error "'ShapedArray' object has no attribute 'val'" shows up if I try running '1d_ntk_opt.ipynb' and '1d_regression.ipynb'.
Can you help me with this problem?

Thank you

I am trying to run 3d_MRI.ipynb locally but I haven't found any way to adjust the memory usage. There isn't an obvious batch_size variable -- any hints?

Thanks!

Hello,

Congrats on this amazing work and thanks for open-sourcing the code. I am attempting to run the notebook for 3D shape reconstruction locally and I face the error attached below,

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-8f4605d1a63b> in <module>
     32         print(expname)
     33 
---> 34         out = run_training(embed_params, mesh, corners, test_pts, render_args_lr, expname)
     35         tests_all[expname] = out[1]
     36         out_all[expname] = out

<ipython-input-5-b42092951119> in run_training(embed_params, mesh, corners, test_pts, render_args_lr, name)
     81             xs.append(i)
     82             slices = [outputs, pred, np.abs(pred - outputs)]
---> 83             renderings = list(render_rays(get_params(opt_state), ab, *render_args_lr))
     84             renderings.append(make_normals(render_args_lr[0], renderings[0]) * .5 + .5)
     85 

ValueError: Non-hashable static arguments are not supported. An error occured while trying to hash an object of type <class 'list'>, [TrackedArray([-0.001     ,  0.24340938,  0.20585178]), TrackedArray([0.90775315, 0.74892081, 0.81103368])]. The error was:
TypeError: unhashable type: 'list'

Could this be due to version mismatch in JAX? I'm using jax on GPU with Cuda 11.3. Also, along with the package requirement, can you also please upload an environment.yml or requirement.txt if possible? Thanks

It seems that the page has restricted access:

When I use neural-tangents 0.3.2 to run the 1d_regression.ipynb, I got ValuError:

Could you help me with this?

1. I find that the gaussian basis must be fixed instead of random sampling in the training process. This makes me confused about the connection of the bochner's theorem.
2. Have you considered that combining VAE to parameterize the fourier feature will make some surprising results? Maybe the disentengled representation learning can have another direction in the research.

Hi, could you please tell me how to download the Text and Natural datasets?

From what I understand, this method can reconstruct crisp image just using coordinates.
Is it possible to encode a whole video (x,y,t) using this model?

I have tried to introduce x,y,t on this, but the PSNR is getting smaller deeper in t. (starts at 25ish for the first few images, and getting lower to 16 by the 25th image)
I tried to encode t using normal positional encoding.

If it is possible to encode t, Would it be beneficial for NERF with t-axis?

Thanks for helping me out with the previous question.
It was really helpful!!

I have an additional issue after running 3d_shape_occupancy with my .ply file.

TypeError: No constant handler for type: <class 'trimesh.caching.TrackedArray'>
arises while running run_training function.

I am very new to jit and JAX such that I am having trouble debugging with the code.
Have you happened to bump into this problem before?
Thank you!

As discussed in A.3, training a and b does not seem to influence the performance. An intuition is as what you mentioned: "as the b_j values do not deviate significantly from their initial values" (How about a_j?). Do you have any theoretic evidence of why this is true? To my knowledge, in language processing tasks, they let the network learn also the embedding of each token, and it makes the performance better.

Is there a notebook to experiment with this (Figure 8)?

This is a very excellent work. I've tested it. It performs very well for image regression and novel-view thesis on 'single-object'. But when I try to train it with multiple objects from same category, the proposed methods seems not work very well. I am trying to figure out a solution to this problem: given images of multi-objects from same category, predict image under certain view for a specific object. This is my code for multi-objects training https://github.com/jingma-git/NeRF_Pytorch. Any insights to solve this problem? Here is some useful links: https://github.com/thunguyenphuoc/HoloGAN

Thanks for sharing this excellent code.
When running the 3d_MRI demo, I also found some functions for 3D CT data processing(such as get_shepp_dataset_3D) in it, I am wondering is there also a CT Projection functions (similar to that in 2d_CT.ipynb) to run 3D CT reconstruction correctly?
In other words, do you have any demo for 3D_CT reconstruction?
Thanks.

Hi,

I have downloaded the ATLAS Dataset from the website http://fcon_1000.projects.nitrc.org/indi/retro/atlas.html.
But the ATLAS Dataset is in the 'nii.gz' format.
So I am wondering is there a data preprocessing function to convert the origin 'nii.gz' file to the 'npz' file?

Thank you