Comments (5)
Thanks! I fixed that issue, but now I'm receiving another problem, saying something about the xFormers
Example usage
%cd ../OpenLRM
EXPORT_VIDEO=True
EXPORT_MESH=True
INFER_CONFIG="./configs/infer-b.yaml"
MODEL_NAME="zxhezexin/openlrm-mix-base-1.1"
IMAGE_INPUT="./assets/sample_input/owl.png"
!python -m openlrm.launch infer.lrm --infer $INFER_CONFIG model_name=$MODEL_NAME image_input=$IMAGE_INPUT export_video=$EXPORT_VIDEO export_mesh=$EXPORT_MESH
........................................................................................................................................................................................................
/OpenLRM
[2024-03-21 13:24:10,947] openlrm.models.modeling_lrm: [INFO] Using DINOv2 as the encoder
/OpenLRM/openlrm/models/encoders/dinov2/layers/swiglu_ffn.py:51: UserWarning: xFormers is not available (SwiGLU)
warnings.warn("xFormers is not available (SwiGLU)")
/OpenLRM/openlrm/models/encoders/dinov2/layers/attention.py:33: UserWarning: xFormers is not available (Attention)
warnings.warn("xFormers is not available (Attention)")
/OpenLRM/openlrm/models/encoders/dinov2/layers/block.py:46: UserWarning: xFormers is not available (Block)
warnings.warn("xFormers is not available (Block)")
0% 0/1 [00:00<?, ?it/s]/OpenLRM/openlrm/datasets/cam_utils.py:153: UserWarning: Using torch.cross without specifying the dim arg is deprecated.
Please either pass the dim explicitly or simply use torch.linalg.cross.
The default value of dim will change to agree with that of linalg.cross in a future release. (Triggered internally at ../aten/src/ATen/native/Cross.cpp:63.)
x_axis = torch.cross(up_world, z_axis)
^C
from openlrm.
Hi,
No module named openlrm
means your working dir is not the root dir of your cloned repo.
You can run !realpath .
to see which is the working dir for your notebook.
from openlrm.
https://colab.research.google.com/drive/1WLkWLOZ-w_PUVhvt3x1zor1VFkopiwjZ?usp=sharing
from openlrm.
I am still working on the colab implementation, for now, these are the script modifications on app.py & the errors on it
..............................................................................................................................................................................................
Copyright (c) 2023-2024, Zexin He
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
import os
from PIL import Image
import numpy as np
import gradio as gr
def assert_input_image(input_image):
if input_image is None:
raise gr.Error("No image selected or uploaded!")
def prepare_working_dir():
import tempfile
working_dir = tempfile.TemporaryDirectory()
return working_dir
def init_preprocessor():
from openlrm.utils.preprocess import Preprocessor
global preprocessor
preprocessor = Preprocessor()
def preprocess_fn(image_in: np.ndarray, remove_bg: bool, recenter: bool, working_dir):
image_raw = os.path.join(working_dir.name, "raw.png")
with Image.fromarray(image_in) as img:
img.save(image_raw)
image_out = os.path.join(working_dir.name, "rembg.png")
success = preprocessor.preprocess(image_path=image_raw, save_path=image_out, rmbg=remove_bg, recenter=recenter)
assert success, f"Failed under preprocess_fn!"
return image_out
def demo_openlrm(infer_impl):
def core_fn(image: str, source_cam_dist: float, working_dir):
dump_video_path = os.path.join(working_dir.name, "output.mp4")
dump_mesh_path = os.path.join(working_dir.name, "output.ply")
infer_impl(
image_path=image,
source_cam_dist=source_cam_dist,
export_video=True,
export_mesh=True,
dump_video_path=dump_video_path,
dump_mesh_path=dump_mesh_path,
)
return dump_video_path
return dump_mesh_path
def example_fn(image: np.ndarray):
from gradio.utils import get_cache_folder
working_dir = get_cache_folder()
image = preprocess_fn(
image_in=image,
remove_bg=True,
recenter=True,
working_dir=working_dir,
)
video = core_fn(
image=image,
source_cam_dist=2.0,
working_dir=working_dir,
)
mesh = core_fn(
mesh=mesh,
working_dir=working_dir,
)
return image, video, mesh
_TITLE = '''OpenLRM: Open-Source Large Reconstruction Models'''
_DESCRIPTION = '''
<div>
<a style="display:inline-block" href='https://github.com/3DTopia/OpenLRM'><img src='https://img.shields.io/github/stars/3DTopia/OpenLRM?style=social'/></a>
<a style="display:inline-block; margin-left: .5em" href="https://huggingface.co/zxhezexin"><img src='https://img.shields.io/badge/Model-Weights-blue'/></a>
</div>
OpenLRM is an open-source implementation of Large Reconstruction Models.
<strong>Image-to-3D in 10 seconds with A100!</strong>
<strong>Disclaimer:</strong> This demo uses `openlrm-mix-base-1.1` model with 288x288 rendering resolution here for a quick demonstration.
'''
with gr.Blocks(analytics_enabled=False) as demo:
# HEADERS
with gr.Row():
with gr.Column(scale=1):
gr.Markdown('# ' + _TITLE)
with gr.Row():
gr.Markdown(_DESCRIPTION)
# DISPLAY
with gr.Row():
with gr.Column(variant='panel', scale=1):
with gr.Tabs(elem_id="openlrm_input_image"):
with gr.TabItem('Input Image'):
with gr.Row():
input_image = gr.Image(label="Input Image", image_mode="RGBA", width="auto", sources="upload", type="numpy", elem_id="content_image")
with gr.Column(variant='panel', scale=1):
with gr.Tabs(elem_id="openlrm_processed_image"):
with gr.TabItem('Processed Image'):
with gr.Row():
processed_image = gr.Image(label="Processed Image", image_mode="RGBA", type="filepath", elem_id="processed_image", width="auto", interactive=False)
with gr.Column(variant='panel', scale=1):
with gr.Tabs(elem_id="openlrm_render_video"):
with gr.TabItem('Rendered Video'):
with gr.Row():
output_video = gr.Video(label="Rendered Video", format="mp4", width="auto", autoplay=True)
with gr.Column(variant='panel', scale=1):
with gr.Tabs(elem_id="openlrm_render_mesh"):
with gr.TabItem('Rendered Mesh'):
with gr.Row():
output_mesh = gr.Mesh(label="Rendered Mesh", format="ply", width="auto", autoplay=True)
# SETTING
with gr.Row():
with gr.Column(variant='panel', scale=1):
with gr.Tabs(elem_id="openlrm_attrs"):
with gr.TabItem('Settings'):
with gr.Column(variant='panel'):
gr.Markdown(
"""
<strong>Best Practice</strong>:
Centered objects in reasonable sizes. Try adjusting source camera distances.
"""
)
checkbox_rembg = gr.Checkbox(True, label='Remove background')
checkbox_recenter = gr.Checkbox(True, label='Recenter the object')
slider_cam_dist = gr.Slider(1.0, 3.5, value=2.0, step=0.1, label="Source Camera Distance")
submit = gr.Button('Generate', elem_id="openlrm_generate", variant='primary')
# EXAMPLES
with gr.Row():
examples = [
['assets/sample_input/owl.png'],
['assets/sample_input/building.png'],
['assets/sample_input/mailbox.png'],
['assets/sample_input/fire.png'],
['assets/sample_input/girl.png'],
['assets/sample_input/lamp.png'],
['assets/sample_input/hydrant.png'],
['assets/sample_input/hotdogs.png'],
['assets/sample_input/traffic.png'],
['assets/sample_input/ceramic.png'],
]
gr.Examples(
examples=examples,
inputs=[input_image],
outputs=[processed_image, output_video],
fn=example_fn,
cache_examples=bool(os.getenv('SPACE_ID')),
examples_per_page=20,
)
working_dir = gr.State()
submit.click(
fn=assert_input_image,
inputs=[input_image],
queue=False,
).success(
fn=prepare_working_dir,
outputs=[working_dir],
queue=False,
).success(
fn=preprocess_fn,
inputs=[input_image, checkbox_rembg, checkbox_recenter, working_dir],
outputs=[processed_image],
).success(
fn=core_fn,
inputs=[processed_image, slider_cam_dist, working_dir],
outputs=[output_video],
).success(
fn=core_fn,
inputs=[processed_mesh, working_dir],
outputs=[output_mesh],
)
demo.queue()
demo.launch(share = True)
def launch_gradio_app():
os.environ.update({
"APP_ENABLED": "1",
"APP_MODEL_NAME": "zxhezexin/openlrm-mix-base-1.1",
"APP_INFER": "./configs/infer-gradio.yaml",
"APP_TYPE": "infer.lrm",
"NUMBA_THREADING_LAYER": 'omp',
})
from openlrm.runners import REGISTRY_RUNNERS
from openlrm.runners.infer.base_inferrer import Inferrer
InferrerClass : Inferrer = REGISTRY_RUNNERS[os.getenv("APP_TYPE")]
with InferrerClass() as inferrer:
init_preprocessor()
if not bool(os.getenv('SPACE_ID')):
from openlrm.utils.proxy import no_proxy
demo = no_proxy(demo_openlrm)
else:
demo = demo_openlrm
demo(infer_impl=inferrer.infer_single)
if name == 'main':
launch_gradio_app()
..............................................................................................................................................................................................
config.json: 100% 322/322 [00:00<00:00, 1.49MB/s]
pytorch_model.bin: 100% 1.04G/1.04G [00:08<00:00, 116MB/s]
[2024-03-22 19:07:46,354] openlrm.models.modeling_lrm: [INFO] Using DINOv2 as the encoder
/content/OpenLRM/openlrm/models/encoders/dinov2/layers/swiglu_ffn.py:43: UserWarning: xFormers is available (SwiGLU)
warnings.warn("xFormers is available (SwiGLU)")
/content/OpenLRM/openlrm/models/encoders/dinov2/layers/attention.py:27: UserWarning: xFormers is available (Attention)
warnings.warn("xFormers is available (Attention)")
/content/OpenLRM/openlrm/models/encoders/dinov2/layers/block.py:39: UserWarning: xFormers is available (Block)
warnings.warn("xFormers is available (Block)")
Downloading: "https://dl.fbaipublicfiles.com/dinov2/dinov2_vitb14/dinov2_vitb14_reg4_pretrain.pth" to /root/.cache/torch/hub/checkpoints/dinov2_vitb14_reg4_pretrain.pth
100% 330M/330M [00:01<00:00, 174MB/s]
Downloading data from 'https://github.com/danielgatis/rembg/releases/download/v0.0.0/u2net.onnx' to file '/root/.u2net/u2net.onnx'.
100%|████████████████████████████████████████| 176M/176M [00:00<00:00, 861GB/s]
Traceback (most recent call last):
File "/content/OpenLRM/openlrm/utils/proxy.py", line 34, in wrapper
return func(*args, **kwargs)
File "/content/OpenLRM/app.py", line 132, in demo_openlrm
output_mesh = gr.Mesh(label="Rendered Mesh", format="ply", width="auto", autoplay=True)
AttributeError: module 'gradio' has no attribute 'Mesh'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/content/OpenLRM/app.py", line 226, in
launch_gradio_app()
File "/content/OpenLRM/app.py", line 221, in launch_gradio_app
demo(infer_impl=inferrer.infer_single)
File "/content/OpenLRM/openlrm/utils/proxy.py", line 36, in wrapper
os.environ['http_proxy'] = http_proxy
File "/usr/lib/python3.10/os.py", line 685, in setitem
value = self.encodevalue(value)
File "/usr/lib/python3.10/os.py", line 757, in encode
raise TypeError("str expected, not %s" % type(value).name)
TypeError: str expected, not NoneType
..............................................................................................................................................................................................
Thanks!
from openlrm.
Hi,
Plz refer to this issue here. #24 (comment)
The newest commit has solved this problem.
from openlrm.
Related Issues (20)
- Issue serving a model trained with the provided training code HOT 14
- How to inference custom-trained model HOT 5
- Why encoder is not freeze in training config? HOT 1
- Can you share large training config?
- MVImgNet code HOT 1
- Not able to run blender_script.py and also how do I train on objaverse? HOT 24
- How to fix "Cannot find valid data directory for uid" error? HOT 3
- Inference Issue HOT 2
- 训练需要什么样的显卡配置 HOT 4
- Training with .obj files HOT 2
- How to evaluate metrics HOT 1
- Matrix multiplication: not supported between 'Matrix' and 'Vector' types HOT 2
- ValueError: math domain error HOT 5
- Inference result quality on trained model is not good HOT 10
- CUDA out of memory for inferring HOT 1
- What resources to train from scratch HOT 1
- Matrix multiplication: not supported between 'Matrix' and 'Vector' types HOT 1
- Pretrained models config files HOT 3
- Whether the camera system is OpenGL?
- Which parts are covered by Nvidia license? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openlrm.