I can't find a way of exporting the point cloud.

If someone can explain, I will add the exporting to the notebook.

im guessing its this line num_points=[1024, 4096 - 1024] or maybe this? batch_size=4096, grid_size=128 but dunno how it works. Do i just double all values?

if I understood the model is trained this way: text -> generated rendering img by GLIDE -> 3D. So in theory it would be possible to generate 3D from an image. My question is if it possible even with images without. shadows like cartoon characters or drawings. Great work btw and thank you for making it open source

I had my poor Nvidia 3060 Laptop GPU.
But when I tried to run the code, it simply tells me out of memory. (OutOfMemoryError: CUDA out of memory.)
So I was forced to change the code to use CPU, device = torch.device('cpu')
And it is so slow, which is annoying.
Do anyone know the minimum requirement for the GPU?
Thanks!

Hi there, thanks for this great project. I have been playing around with it and can now generate pointclouds but I am having problems converting them to a mesh using the example pointcloud2mesh.ipynb . I have skimage installed correctly (as seen below) but I consistently get this error, I don't know how to work around it. Any help would be appreciated!

Skimage installed

ubuntu@132-145-133-39:~/point-e$ pip show scikit-image
Name: scikit-image
Version: 0.19.3
Summary: Image processing in Python
Home-page: https://scikit-image.org
Author: 
Author-email: 
License: Modified BSD
Location: /home/ubuntu/.local/lib/python3.8/site-packages
Requires: imageio, networkx, numpy, packaging, pillow, PyWavelets, scipy, tifffile
Required-by: point-e

The error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-28-9f81ccec745d> in <module>
      1 # Produce a mesh (with vertex colors)
----> 2 mesh = marching_cubes_mesh(
      3     pc=pc,
      4     model=model,
      5     batch_size=4096,

~/point-e/point_e/util/pc_to_mesh.py in marching_cubes_mesh(pc, model, batch_size, grid_size, side_length, fill_vertex_channels, progress)
     68         volume_np -= np.mean(volume_np)
     69 
---> 70     verts, faces, normals, _ = measure.marching_cubes_classic(
     71         volume=volume_np,
     72         level=0,

AttributeError: module 'skimage.measure' has no attribute 'marching_cubes'

Maybe im doing something wrong here but i tried using a couple custom images and the point cloud produced seems good but when im passing it to the mesh converter the result is crap. Any ideas?

UPDATE: Yea image2cloud is broken. I tried everything but its not working. Created images as close to the examples as possible and the results are completely random and irrelevant to the input.

Thank you for releasing point-e! I was wondering whether there would a way to sample more points than 4096?

I tried to do two-steps upsampling but it does not work.

sampler = PointCloudSampler(
    device=device,
    models=[base_model, upsampler_model, upsampler_diffusion],
    diffusions=[base_diffusion, upsampler_diffusion, upsampler_diffusion],
    num_points=[1024, 4096 - 1024, 4096 * 4 - 4096],
    aux_channels=['R', 'G', 'B'],
    guidance_scale=[3.0, 3.0, 3.0],
    use_karras = (True, True, True),
    karras_steps = (64, 64, 64),
    sigma_min = (1e-3, 1e-3, 1e-3),
    sigma_max = (120, 160, 160),
    s_churn = (3, 0, 0),
)

# Load an image to condition on.
# img = Image.open('example_data/cube_stack.jpg')
img = Image.open('example_data/render_img.png')

# Produce a sample from the model.
samples = None
for x in tqdm(sampler.sample_batch_progressive(batch_size=1, model_kwargs=dict(images=[img]))):
    samples = x

It gives the following errors.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In [48], line 7
      5 # Produce a sample from the model.
      6 samples = None
----> 7 for x in tqdm(sampler.sample_batch_progressive(batch_size=1, model_kwargs=dict(images=[img]))):
      8     samples = x

File ~/miniconda/envs/3d_cu116_py38/lib/python3.8/site-packages/tqdm/notebook.py:259, in tqdm_notebook.__iter__(self)
    257 try:
    258     it = super(tqdm_notebook, self).__iter__()
--> 259     for obj in it:
    260         # return super(tqdm...) will not catch exception
    261         yield obj
    262 # NB: except ... [ as ...] breaks IPython async KeyboardInterrupt

File ~/miniconda/envs/3d_cu116_py38/lib/python3.8/site-packages/tqdm/std.py:1195, in tqdm.__iter__(self)
   1192 time = self._time
   1194 try:
-> 1195     for obj in iterable:
   1196         yield obj
   1197         # Update and possibly print the progressbar.
   1198         # Note: does not call self.update(1) for speed optimisation.

File ~/project/text2shape/repos/3DGen/de_package/point-e/point_e/diffusion/sampler.py:135, in PointCloudSampler.sample_batch_progressive(self, batch_size, model_kwargs)
    133 if stage_guidance_scale != 1 and stage_guidance_scale != 0:
    134     for k, v in stage_model_kwargs.copy().items():
--> 135         stage_model_kwargs[k] = torch.cat([v, torch.zeros_like(v)], dim=0)
    137 if stage_use_karras:
    138     samples_it = karras_sample_progressive(
    139         diffusion=diffusion,
    140         model=model,
   (...)
    149         guidance_scale=stage_guidance_scale,
    150     )

TypeError: zeros_like(): argument 'input' (position 1) must be Tensor, not list

when running the text2pointcloud.ipynb example I got the error:

TypeError                                 Traceback (most recent call last)
Cell In [8], line 6
      4 # Produce a sample from the model.
      5 samples = None
----> 6 for x in tqdm(sampler.sample_batch_progressive(batch_size=1, model_kwargs=dict(texts=[prompt]))):
      7     samples = x

File c:\anaconda3\envs\sd\lib\site-packages\tqdm\std.py:1195, in tqdm.__iter__(self)
   1192 time = self._time
   1194 try:
-> 1195     for obj in iterable:
   1196         yield obj
   1197         # Update and possibly print the progressbar.
   1198         # Note: does not call self.update(1) for speed optimisation.

File c:\point-e\point_e\diffusion\sampler.py:164, in PointCloudSampler.sample_batch_progressive(self, batch_size, model_kwargs)
    156     samples_it = diffusion.p_sample_loop_progressive(
    157         model,
    158         shape=(internal_batch_size, *sample_shape[1:]),
   (...)
    161         clip_denoised=self.clip_denoised,
    162     )
--> 163 for x in samples_it:
...
    105     out = self.diffusion.p_mean_variance(
    106         self.model, x_t * c_in, t, clip_denoised=clip_denoised, model_kwargs=model_kwargs
    107     )

torch version 1.12.1+cu116
Python version 3.10.4
Windows 10

I would like to know the CUDA version for this code to run.

When trying to get this working using the GPU on M1 with

Both of these return True

print(torch.backends.mps.is_available())
print(torch.backends.mps.is_built())

This also works

device = torch.device('mps')

All steps pass until this part

for x in tqdm(sampler.sample_batch_progressive(batch_size=1, model_kwargs=dict(images=[img]))):
    samples = x

Here is the error

File ~/Downloads/point-e-main/point_e/diffusion/gaussian_diffusion.py:1016, in _extract_into_tensor(arr, timesteps, broadcast_shape)
   1006 def _extract_into_tensor(arr, timesteps, broadcast_shape):
   1007     """
   1008     Extract values from a 1-D numpy array for a batch of indices.
   1009 
   (...)
   1014     :return: a tensor of shape [batch_size, 1, ...] where the shape has K dims.
   1015     """
-> 1016     res = th.from_numpy(arr).to(device=timesteps.device)[timesteps].float()
   1017     while len(res.shape) < len(broadcast_shape):
   1018         res = res[..., None]

TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

I am working on getting the text2pointcloud.ipynb example code running in Blender 3.4's built-in Python environment (Windows 11, Python 3.10.8 (main, Oct 18 2022, 21:01:35) [MSC v.1928 64 bit (AMD64)] on win32, numpy 1.24.1, torch 1.13.1+cpu)

In doing so, I have encountered the error below when producing a sample from the model:

creating base model...
MODEL_CONFIGS[base_name] dict{'cond_drop_prob': 0.1, 'heads': 8, 'init_scale': 0.25, 'input_channels': 6, 'layers': 12, 'n_ctx': 1024, 'name': 'CLIPImagePointDiffusionTransformer', 'output_channels': 12, 'time_token_cond': True, 'token_cond': True, 'width': 512, 'cache_dir': 'C:\Users\MyUser\Blender-Project'}
creating upsample model...
downloading base checkpoint...
downloading upsampler checkpoint...
creating base model...
creating upsample model...
downloading base checkpoint...
downloading upsampler checkpoint...
Producing a sample from the model...
0it [00:01, ?it/s]
Error: Python: Traceback (most recent call last):
File "C:\Users\MyUser\Blender-Project\pgu.blend\pgu.py", line 72, in
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\tqdm-4.64.1-py3.10.egg\tqdm\std.py", line 1195, in iter
for obj in iterable:
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\point_e-0.0.0-py3.10.egg\point_e\diffusion\sampler.py", line 163, in sample_batch_progressive
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\point_e-0.0.0-py3.10.egg\point_e\diffusion\k_diffusion.py", line 181, in karras_sample_progressive
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\torch-1.13.1-py3.10-win-amd64.egg\torch\autograd\grad_mode.py", line 43, in generator_context
response = gen.send(None)
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\point_e-0.0.0-py3.10.egg\point_e\diffusion\k_diffusion.py", line 265, in sample_heun
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\point_e-0.0.0-py3.10.egg\point_e\diffusion\k_diffusion.py", line 173, in guided_denoiser
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\point_e-0.0.0-py3.10.egg\point_e\diffusion\k_diffusion.py", line 160, in denoiser
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\point_e-0.0.0-py3.10.egg\point_e\diffusion\k_diffusion.py", line 105, in denoise
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\point_e-0.0.0-py3.10.egg\point_e\diffusion\gaussian_diffusion.py", line 285, in p_mean_variance
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\torch-1.13.1-py3.10-win-amd64.egg\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\point_e-0.0.0-py3.10.egg\point_e\models\transformer.py", line 287, in forward
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\point_e-0.0.0-py3.10.egg\point_e\models\transformer.py", line 208, in _forward_with_cond
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\torch-1.13.1-py3.10-win-amd64.egg\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\python\lib\site-packages\torch-1.13.1-py3.10-win-amd64.egg\torch\nn\modules\linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: Expected size for first two dimensions of batch2 tensor to be: [2, 4] but got: [2, 6].

My setup is admittedly non-standard, but I am hoping that someone can help. To get to this point, I also made a couple of changes to the example code, including passing a custom 'cache_dir' to the model, since Blender defaults to using C:\Windows\system32 otherwise:

MODEL_CONFIGS[base_name]['cache_dir'] = cache_dir
MODEL_CONFIGS['upsample']['cache_dir'] = cache_dir

I also encountered a problem in which the PointCloudSampler instance called 'sampler' wasn't instantiated, so I am creating an instance of this class as follows:

sampler = PointCloudSampler(
device=device,
models=[base_model, upsampler_model],
diffusions=[base_diffusion, upsampler_diffusion],
num_points=[1024, 2048],
aux_channels=["normals"]
)

this is called before the "for x in tqdm(sampler.sample_batch_progressive(batch_size=1, model_kwargs=dict(texts=[prompt]))):
samples = x"

invocation.

Finally, I have applied the float to int TypeError fix described in Issue 6 (in point_e\diffusion\k_diffusion.py), but this does not solve the problem.

Is there a way to process multiple images (at different angles) to generate a more accurate point cloud?

im using text to point cloud and i get to the last step. And it shows 4 graphs with the model and 4 views.

im on the below step and i usually stop it after 10 mins

# Set a prompt to condition on. prompt = 'boat'

Should I let this stage complete ?
I stopped it after about ten minutes and then started the next step which showed the model in a graph viewer png.

When the iteration goes to 90/130, the program will break down and rebuilding again.
torch.cuda.OutOfMemoryError: CUDA out of memory.

my devices are:
GPU:RTX3060
CUDA:V12.0.76
Pytorch:1.13.1

I'm getting really poor output from the PLY:

Am I using marching_cubes_mesh incorrectly?

mesh = marching_cubes_mesh(
    pc=pc,
    model=model,
    batch_size=4096,
    grid_size=256, # increase to 128 for resolution used in evals
    progress=True,
)

Thank you so much for sharing the blender_script ! I was wondering what is the version of blender that is used.

To build own training data, it would be really appreciated if you could share more about how to re-built the 3d point clouds from the renders and camera matrices generated by blender.

Would be awesome to see built in PLY support

PointCloud2Mesh produces mesh with no color. I've tried adding fill_vertex_channels=True inside of mesh = marching_cubes_mesh( but it still won't work. Any help?

Hello,

I'm trying to get this to work on my Nvidia GTX graphics card. I created a new virtualenv using the cuda setup instructions from pytorch but I get the following error

torch.load torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB

I see that this can be caused by failing to clear unneeded information during traing but can't see an obvious way to resolve this

https://discuss.pytorch.org/t/runtimeerror-cuda-out-of-memory-tried-to-allocate-20-00-mib-gpu-0-3-94-gib-total-capacity-3-36-gib-already-allocated-13-06-mib-free-78-58-mib-cached/56632/4

Full traceback:

    base_model.load_state_dict(load_checkpoint(base_name, device))
  File "...\point_e\models\download.py", line 78, in load_checkpoint
    return torch.load(path, map_location=device)
  File "...\site-packages\torch\serialization.py", line 789, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "...\site-packages\torch\serialization.py", line 1131, in _load
    result = unpickler.load()
  File "...\site-packages\torch\serialization.py", line 1101, in persistent_load
    load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location))
  File "...\site-packages\torch\serialization.py", line 1083, in load_tensor
    wrap_storage=restore_location(storage, location),
  File "...\site-packages\torch\serialization.py", line 1055, in restore_location
    return default_restore_location(storage, str(map_location))
  File "...\site-packages\torch\serialization.py", line 215, in default_restore_location
    result = fn(storage, location)
  File "...\site-packages\torch\serialization.py", line 187, in _cuda_deserialize
    return obj.cuda(device)
  File "...\site-packages\torch\_utils.py", line 80, in _cuda
    untyped_storage = torch.UntypedStorage(

Thanks a lot for your contribution, I hope to do further research on point-E.
How can I train the model myself, Could you please provide the code for training model.
Sincere thanks !

Does anyone know if it would be possible to speed up the pointcloud generation using something like Cython? Are there any existing forks aimed at achieving this?

I want to combine the cloud points by extracting the cloud points of images taken from various directions.
ex)

I want to create a mesh by extracting two cloud points

Hello, thanks for the awesome project !
I love the results and respect for the authors' efforts.

By the way, I cannot find the information of training data.
I wonder the details of training data such as how the training data is collected.

I think that many researchers are also curious about the details of training data & data gathering process.

Thank you :)

Can we use any reconstruction methods from Open3D to increase the quality of the output model

http://www.open3d.org/docs/latest/tutorial/Advanced/surface_reconstruction.html

I have tried out a few methods but it was just making them even worse

Hello, when I try to run text2pointcloud with 300M or 1B models I'm getting the following error:

Cell In[8], line 6
      4 # Produce a sample from the model.
      5 samples = None
----> 6 for x in tqdm(sampler.sample_batch_progressive(batch_size=1, model_kwargs=dict(texts=[prompt]))):
      7     samples = x

File ~/.local/lib/python3.8/site-packages/tqdm/notebook.py:259, in tqdm_notebook.__iter__(self)
    257 try:
    258     it = super(tqdm_notebook, self).__iter__()
--> 259     for obj in it:
    260         # return super(tqdm...) will not catch exception
    261         yield obj
    262 # NB: except ... [ as ...] breaks IPython async KeyboardInterrupt

File ~/.local/lib/python3.8/site-packages/tqdm/std.py:1195, in tqdm.__iter__(self)
   1192 time = self._time
   1194 try:
-> 1195     for obj in iterable:
   1196         yield obj
   1197         # Update and possibly print the progressbar.
   1198         # Note: does not call self.update(1) for speed optimisation.

File ~/mirage-dev/aman/deep-learning/point-e/point_e/diffusion/sampler.py:130, in PointCloudSampler.sample_batch_progressive(self, batch_size, model_kwargs)
...
    318 _ = batch_size
    319 with torch.no_grad():
--> 320     return dict(embeddings=self.clip.embed_images_grid(model_kwargs["images"]))

KeyError: 'images'

Works fine with image2pointcloud

Appreciate the help!

increase points an quality o that no have a sence, maximum what can bi gen with nom quality its some banana, in 5 steps an 5 hous, in cloud points, maybe, what a u dont can use in any 3d programm coz format of thats cooooooool cloud poinit some like a .......... ply bugaga, points cloud thet a not points/vertex comon wutfk, 1024 point, 1024 that a vely small, if low poly banan have 128 vertex thay a be have in clouds some like 8000 an u ai point e do only 1000 points thets not fo banan small thats small fo pensil all what can be gen in 1000 point its..... axis........ ONE axis, z axis fo sample 1000 it small

It would be great to hear more, at a high level, about limits on the point density. Is it variable to how features are made and attached?

Like if I want a living room -> in a house -> on a basketball -> in the hand -> of a person, sitting in an airplane, would there be a variable mesh all the way into that chain?

Or would some of that just get cut off? Is it averaged overall to some viewport-concept of the whole object?

Hello,

It would be very nice to be able to save the point cloud (not the ply mesh) for visualisation in Cloud Compare / Blender / Excel / another viewer. Maybe a simple xyz file format would be best as that is very generic, human readable and should be accepted by almost all visualisation tools. The pointclouds are not large so a compressed file format like e57 is not needed.

I think some data is lost when exporting to a ply mesh so that does not solve the issue.

Hi, thank you for your hard work.

I'm trying to reproduce the results from your paper but I can't seem to get it. I'm taking code from text2pointcloud.ipynb along with the captions mentioned in the paper but I can't see to generate the same results.

Captions from the paper:

Using the same caption a pair of 3d glasses, left lens is red right is blue, I get something like this instead:

Is it possible to tell me where I went wrong, or if anyone else has ran into the same issue as well?
Thanks

How to use it with Pycharm?

I'm trying to convert the pt files in point_e_model_cache folder to ONNX, but it keeps returning this error: AttributeError: 'collections.OrderedDict' object has no attribute 'modules'. Anyone know how to fix?

My file -
Google Colab

Hi, i am trying to run the text2pointcloud script locally. During the first 67 iteration, the speed of the iteration was acceptable but after that the model iterates once every 70-80 seconds. I am testing the model with "corgi.jpg", which worked fine in the colab.

106it [1:08:34, 71.01s/it]

GPU: 1660 Ti Max-Q (6 GB VRAM)
Python Version: 3.8.15
Model: base40M

base_model = model_from_config(MODEL_CONFIGS[base_name], device) in text2pointcloud.ipynb causes the error OSError: [Errno 30] Read-only file system: '/point_e_model_cache'

Running macOS Ventura 13.2, M1, Python 3.9.13

Thank you for your amazing work. I have a question about fine-tuning "GLIDE". In your paper, you state that your dataset has 3D data and multiple projected images, but not text data. How do you fine-tuning GLIDE to make GLIDE match the distribution of your dataset?

Running into a Jupyter kernel crash in last cell.
I tried to set KMP_DUPLICATE_LIB_OK=true to see if it changes anything, but no difference.

Output copy/pasted:
info 23:47:26.002: Loading webview. View is notset
info 23:47:26.003: Loading web view...
info 23:47:26.004: Webview panel created.
info 23:47:26.414: Web view react rendered
warn 23:47:59.496: StdErr from Kernel Process OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

error 23:48:00.563: Disposing session as kernel process died ExitCode: 3, Reason: c:\Users\andre\anaconda3\envs\point-e\lib\site-packages\traitlets\traitlets.py:2547: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use 'hmac-sha256' instead of '"hmac-sha256"' if you require traitlets >=5.
warn(
c:\Users\andre\anaconda3\envs\point-e\lib\site-packages\traitlets\traitlets.py:2498: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0. Use '06a75dfe-7f86-4306-9e7d-773b07153587' instead of 'b"06a75dfe-7f86-4306-9e7d-773b07153587"'.
warn(
OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

info 23:48:00.563: Dispose Kernel process 43428.
error 23:48:00.564: Raw kernel process exited code: 3
error 23:48:00.566: Error in waiting for cell to complete Error: Canceled future for execute_request message before replies were done
at t.KernelShellFutureHandler.dispose (c:\Users\andre.vscode-insiders\extensions\ms-toolsai.jupyter-2023.1.1003441034\out\extension.node.js:2:32353)
at c:\Users\andre.vscode-insiders\extensions\ms-toolsai.jupyter-2023.1.1003441034\out\extension.node.js:2:51405
at Map.forEach ()
at y._clearKernelState (c:\Users\andre.vscode-insiders\extensions\ms-toolsai.jupyter-2023.1.1003441034\out\extension.node.js:2:51390)
at y.dispose (c:\Users\andre.vscode-insiders\extensions\ms-toolsai.jupyter-2023.1.1003441034\out\extension.node.js:2:44872)
at c:\Users\andre.vscode-insiders\extensions\ms-toolsai.jupyter-2023.1.1003441034\out\extension.node.js:2:2181854
at t.swallowExceptions (c:\Users\andre.vscode-insiders\extensions\ms-toolsai.jupyter-2023.1.1003441034\out\extension.node.js:7:108503)
at p.dispose (c:\Users\andre.vscode-insiders\extensions\ms-toolsai.jupyter-2023.1.1003441034\out\extension.node.js:2:2181832)
at t.RawSession.dispose (c:\Users\andre.vscode-insiders\extensions\ms-toolsai.jupyter-2023.1.1003441034\out\extension.node.js:2:2186853)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
warn 23:48:00.567: Cell completed with errors {
message: 'Canceled future for execute_request message before replies were done'
}
info 23:48:00.568: Cancel all remaining cells true || Error || undefined

demo：text2pointcloud.ipynb

with errors：
File ~/ai3d/point-e/point_e/diffusion/gaussian_diffusion.py:1016, in _extract_into_tensor(arr, timesteps, broadcast_shape)
1006 def _extract_into_tensor(arr, timesteps, broadcast_shape):
1007 """
1008 Extract values from a 1-D numpy array for a batch of indices.
1009
(...)
1014 :return: a tensor of shape [batch_size, 1, ...] where the shape has K dims.
1015 """
-> 1016 res = th.from_numpy(arr).to(device=timesteps.device)[timesteps].float()
1017 while len(res.shape) < len(broadcast_shape):
1018 res = res[..., None]

IndexError: tensors used as indices must be long, byte or bool tensors

i dont know, how to export as pointcloud, please expain in alaymen language, if possible, please provide google notebook with changes, Thanks

Using macOS Ventura, M1 iMac, keep getting this error message:

/Users/blendersushi/miniforge3/lib/python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: dlopen(/Users/blendersushi/miniforge3/lib/python3.9/site-packages/torchvision/image.so, 0x0006): Symbol not found: __ZN3c106detail19maybe_wrap_dim_slowExxb
  Referenced from: <8080486D-E510-3000-AA6A-F3AD49ACC172> /Users/blendersushi/miniforge3/lib/python3.9/site-packages/torchvision/image.so
  Expected in:     <8ADDD67A-2C07-3290-B140-DF1BD644AB8B> /Users/blendersushi/miniforge3/lib/python3.9/site-packages/torch/lib/libc10.dylib
  warn(f"Failed to load image Python extension: {e}")

Faced an issue while importing skimage.measure. Related to this issue: scikit-image#4266

Fix was to update the imports in the file -> point_e/util/pc_to_mesh.py:

• Line no: link
• From:

import skimage

• To:

import skimage.measure as skm

• skimage.__version__ == 0.18.3

Not too sure how to pip install point-e. I saw pip install -e, but believe that is an editable install for the setup.py

Any suggestions? Thanks for your time!

am using conda latest version i have cudatoolkit installed on my environment and when run the text 2point cloud it chooses my CPU instead of my GPU any solutions ?

I am trying to run the text2pointcloud on windows using VSC with the jupyter notebook.

I am able to make it past the first box where torch is imported.

Now I am getting stuck in the second block when it reaches model_from_config(MODEL_CONFIGS[base_name], device). The error is TypeError: __init__() got an unexpected keyword argument 'device'.

Here is the full error log:

TypeError                                 Traceback (most recent call last)
Cell In[2], line 5
      3 print('creating base model...')
      4 base_name = 'upsample'
----> 5 base_model = model_from_config(MODEL_CONFIGS[base_name], device)
      6 base_model.eval()
      7 base_diffusion = diffusion_from_config(DIFFUSION_CONFIGS[base_name])

File c:\point-e\point_e\models\configs.py:129, in model_from_config(config, device)
    127     return UpsamplePointDiffusionTransformer(device=device, dtype=torch.float32, **config)
    128 elif name == "CLIPImageGridUpsamplePointDiffusionTransformer":
--> 129     return CLIPImageGridUpsamplePointDiffusionTransformer(
    130         device=device, dtype=torch.float32, **config
    131     )
    132 elif name == "CrossAttentionPointCloudSDFModel":
    133     return CrossAttentionPointCloudSDFModel(device=device, dtype=torch.float32, **config)

File c:\point-e\point_e\models\transformer.py:428, in CLIPImageGridUpsamplePointDiffusionTransformer.__init__(self, device, dtype, n_ctx, cond_drop_prob, frozen_clip, cache_dir, **kwargs)
    413 def __init__(
    414     self,
    415     *,
   (...)
    422     **kwargs,
    423 ):
...
---> 55 self.c_fc = nn.Linear(width, width * 4, device=device, dtype=dtype)
     56 self.c_proj = nn.Linear(width * 4, width, device=device, dtype=dtype)
     57 self.gelu = nn.GELU()

TypeError: __init__() got an unexpected keyword argument 'device'

I was hoping someone knew how to fix this.

This is all amazing thank you!

I was wondering if it's possible to export and save the point cloud data in the .ply format?