Git Product home page Git Product logo

menpo3d's Introduction

menpo

BSD License Python 3.6 Support Python 3.7 Support

menpo3d - Tools for manipulating meshes

A library inside the Menpo Project that makes manipulating 3D mesh data a simple task. In particular, this project provides the ability to import, visualize and rasterize 3D meshes. Although 3D meshes can be created within the main Menpo project, this package adds the real functionality for working with 3D data.

Installation

Here in the Menpo team, we are firm believers in making installation as simple as possible. Unfortunately, we are a complex project that relies on satisfying a number of complex 3rd party library dependencies. The default Python packing environment does not make this an easy task. Therefore, we evangelise the use of the conda ecosystem, provided by Anaconda. In order to make things as simple as possible, we suggest that you use conda too! To try and persuade you, go to the Menpo website to find installation instructions for all major platforms.

Visualizing 3D objects

menpo3d adds support for viewing 3D objects through Mayavi, which is based on VTK. One of the main reasons menpo3d is a seperate project to the menpo core library is to isolate the more complex dependencies that this brings to the project. 3D visualization is not yet supported in the browser, so we rely on platform-specific viewing mechanisms like QT or WX.

In order to view 3D items you will need to first use the %matplotlib qt IPython magic command to set up QT for rendering (you do this instead of %matplotlib inline which is what is needed for rendering directly in Jupyter/Ipython notebooks). As a complete example, to view a mesh in IPython you would run something like:

import menpo3d
mesh = menpo3d.io.import_builtin_asset('james.obj')
%matplotlib qt
mesh.view()

menpo3d's People

Contributors

jabooth avatar loubnar avatar mmcauliffe avatar nontas avatar patricksnape avatar steliosploumpis avatar trigeorgis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

menpo3d's Issues

Building Statistical Shape Model from 3D meshes?

Is it possible, in Menpo3d, or another of the Menpo packages to import and build a statistical shape model from 3D mesh data? Also, is there any code available to show how this would be done? Finally, is there code for the 2D active appearance or active shape modeling to show how to do it within menpo?

ArrtributeError:'NoneType' object has no attribute 'GetMapper'

When i run your code in my facial models,i get a error 'ArrtributeError:'NoneType' object has no attribute 'GetMapper'' in the path of 'menpo3d/io/input/mesh/base.py' in line 119. My os is Windows10,my vtk version is 7.0,my python version is py35.Can anyone help me?

3D Morphable Models

Hi, has the 3DMM from 3D Face Morphable Models "In-the-Wild" been implemented in Menpo3d yet? I couldn't find any documentation, nor could I find an implemented model in the codebase. Thanks.

I believe @jabooth is still working on releasing the publicly ready version of the code

Originally posted by @patricksnape in #45 (comment)

Make it easier to set QT Environment variable

Following #16, thinking we could probably do a little more to make it easier to set the env variable for Qt-based visualisation.

Few options:

  • try/except the 'QString' has already been set to version 1 error, and raise one that offers some guidance, e.g.:

To use menpo3d visualisation, you need to set this environment variables

export QT_API=pyqt
export ETS_TOOLKIT=qt4
  • Double check we can't just set these env variables ourselves on menpo3d import before importing mlab in the menpo3d codebase.
import os

if 'QT_API' not in os.environ and 'ETS_TOOLKIT' not in os.environ: 
    os.environ['QT_API'] = 'pyqt'
    os.environ['ETS_TOOLKIT'] = 'qt4'

from mayavi import ....

This would be the cleanest solution IMHO.

  • If the above doesn't work, and if we move forward with some form of menpo download-and-run option, we could at least set these env variables in our own custom launchers:
./Launch Jupyer Notebook.cmd 
SET QT_API=pyqt
SET ETS_TOOLKIT=qt4
src\python src\Scripts\jupyter-notebook-script.py --notebook-dir=notebooks

GLRasterizer dll load failure in RasterizationBasics.ipynb

This issue occurs in the 5th cell RasterizationBasics.ipynb in the menpo3d notebooks.
from menpo3d.rasterize import GLRasterizer
# Build a rasterizer configured from the current view
r = GLRasterizer(**viewer_settings)

The resulting error looks like this:
ImportError Traceback (most recent call last)
<ipython-input-7-5266371db03a> in <module>()
3 # Build a rasterizer configured from the current view
4 # r = GLRasterizer(**viewer_settings)
----> 5 r = GLRasterizer()

~\Anaconda3\envs\menpo\lib\site-packages\cyrasterize\base.py in __init__(self, width, height, model_matrix, view_matrix, projection_matrix, verbose)
98 view_matrix=None, projection_matrix=None, verbose=False):
99 # delay import so we only check for GL setup at first initialization
--> 100 from .glrasterizer import GLRasterizer
101 self._opengl = GLRasterizer(width, height, verbose=int(verbose))
102 if not self._opengl.successfully_initialized():

ImportError: DLL load failed: The specified module could not be found.

My laptop runs a Windows 10 OS and I was wondering if that might be a part of the issue. I've checked that the latest versions of menpo3d and cyrasterize are installed. Would you have any idea what might be causing the error?

How to use menpo3d/extractimage.py to generate the mesh occlusion mask for UV map?

from menpo3d.extractimage import *

import menpo3d.io as m3io
import menpo.io as mio
import cv2
import menpo3d
import pdb

obj_path = '/data/DATA/3dFace/UVGAN_DATA/Visualise_demo/demo_input/001_01_01_051_10.obj'

img_path = "/data/DATA/3dFace/UVGAN_DATA/Visualise_demo/demo_input/img/"
mesh = m3io.import_mesh(obj_path)
image = mio.import_images(img_path, verbose=True)

#from menpowidgets import visualize_images
#visualize_images(image)
pdb.set_trace()

#mesh_temp.view()
points = mesh.points
mask = per_vertex_occlusion_accurate(mesh)
colours = extract_per_vertex_colour(mesh, image[0])

I use the menpo3d, and import mesh and image, but the mask results is false, so every points(53125) is occluded, I output the result as below. Could you help me to get the occlusion mask for uvmap? Thanks a lot!
image

ModuleNotFoundError: No module named 'menpo3d'

Hi, do we need to install menpo3d separately? I was getting an ImportError: No module named menpo3d If menpo package supports python 3.6+, and menpo3d supports python2, how is this gonna work out?

Proposal: add support for CGAL

CGAL seems like a pretty solid library for performing lots of geometric tasks.

It's like VTK, but purely focused on accurate computation.

For instance, they have a nice efficient surface intersection object:
http://doc.cgal.org/latest/AABB_tree/

The benchmarks seem promising. I'm wondering if it is worth making CGAL a dependency and wrapping methods to get high performance utilities for our 3D deformable modelling work.

How to output unwrapped UV texture

It is stated in the project how to unwrap texture cylindrically. But is there a way output/extract the unwrapped texture file? It is a question related to rendering and rasterization.

per_vertex_occlusion gives Segmentation fault

Hi, I'm trying to extract vertex colour with occlusion, and the following code gives me Segmentation fault:

import menpo3d.io as m3io
from menpo3d.extractimage import per_vertex_occlusion

mesh = m3io.import_mesh('/data/body_template/16joints/Male_63119_126226.obj')
mask = per_vertex_occlusion(mesh)

I traced back to the source code and this line:

model_to_image_transform, shape_image = render_hi_res_shape_image(mesh, render_width=render_width)

in the definition of per_vertex_occlusion is causing the error, so I think it's a problem with glrasterizer.

Any suggestions on how to fix this?

Add scikit-sparse to conda deps

Now we have it build for all versions of Python (at least for linux) we should include this in the conda deps, such a game changer for NICP perf.

NICP hangs in Jupyter notebook

When NICP (non_rigid_icp etc) is called in a Jupyter notebook, the operation hangs.
Tested in ipykernel 6.3.1 with ipython 7.27.0 and and ipython 8.4. There is no such issue in Ipykernel 5.5.3 and ipython 7.22.0.
The issue is with sys.stdout.close(), which hangs (perhaps it has to do with issue 868 at ipykernel project). Also, in the same matter, fileno has a different behaviour in the newer versions as it does not raise an UnsupportedOperation exception. Perhaps is it better to check __IPYTHON__ instead of fileno ?

Questions of 3D Models in Menpo3D?

Hi, I am using the menpo3D to display the james.obj in 3D, and it works perfectly, and I want to know if there is also a library which permits us to generate 3D models?

And I find the 3D model of James has a high resolution, can you tell me which equipements you apply to get a realistic 3D model with such a high resolution like this one? Thank you very much for your responses.

VTK OBJ importer duplicates vertices for each triangle

Importing the james.obj file from the built-in assets creates a mesh with 205,458 vertices (meshlab lists 34,913). The number of vertices is three times the number of faces (68,486). Exporting the mesh to a .stl file in meshlab and then importing it in Menpo results in the correct number of vertices. It looks like it's a bug in vtk.vtkOBJReader, since the number of verticies reported at https://github.com/menpo/menpo3d/blob/master/menpo3d/io/input/mesh/base.py#L200 is the erroneously high number.

the cholesky_AAt in Non-Rigid ICP can't work

Hello, When there are many vertices whose matching reliability weight is set to 0, the Hessian ATA is not full rank.
So only the slow solve method scipy_spsolve can work. The sksparse.cholmod.cholesky_AAt just throw an ERROR:
sksparse.cholmod.CholmodNotPositiveDefiniteError: c:\cholmod-scikit-sparse-window\suitesparse-metis-for-windows-1.3.1\suitesparse\cholmod\supernodal\t_cholmod_super_numeric.c:911: matrix not positive definite (code 1)

how do you handle with this? or I got the wrong version of scikit-sparse?

Unable to use ColouredTriMeshViewer3d

Traceback (most recent call last):
File "", line 1, in
File "/home/cwfang/.local/lib/python2.7/site-packages/menpo3d/visualize/viewmayavi.py", line 439, in render
specular_light=specular_light, alpha=alpha)
File "/home/cwfang/.local/lib/python2.7/site-packages/menpo3d/visualize/viewmayavi.py", line 420, in _render_mesh
mapper.set_input_data(pd)
AttributeError: 'PolyDataMapper' object has no attribute 'set_input_data'

unable to view TriMesh

Currently this fails to show the mesh in the VTK window:

from menpo.shape import TriMesh
tm = TriMesh.init_from_depth_image(x_im)
tm.view(new_figure=True)

Question about menpo3d.io.import_mesh

Hello,
I have a question about the function menpo3d.io.import_mesh. When I use the import_mesh, the returned mesh has 156783 points(mesh.points.shape == (156783, 3)), but actually my obj file has only 26317 points. The two are not equal. Can someone tell me why this happens?
Thanks in advance!

Meet errors when install the menpo3d on Windows.

Hello developers,
I met a problem when I attempt to install the menpo3d on Windows system. I try to install the menpo3d in conda environment but it shows the following error.

C:\Users\zjcch.conda\envs\cleft_full\lib\site-packages\Cython\Compiler\Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: C:\Users\zjcch\AppData\Local\Temp\pip-install-5_4g4yl2\menpo3d_d2eb2978a5c247a08dd8df3785d89771\menpo3d\rasterize\tripixel.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
error in menpo3d setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; .* suffix can only be used with == or != operators
moderngl>=5.6.*,<6.0

It seems that there are some issues with Cython, about * suffix.
How can I fix it?

Error importing WRL mesh

Error when importing mesh:
m = m3io.import_mesh('/vol/hci2/Databases/video/BU-3DFE/F0001/F0001_FE04WH_F3D.wrl')

Stack trace:

AttributeError Traceback (most recent call last)
in ()
----> 2 m = m3io.import_mesh('/vol/hci2/Databases/video/BU-3DFE/F0001/F0001_FE04WH_F3D.wrl')

/vol/atlas/homes/mengjiao/miniconda/envs/normrecov/lib/python2.7/site-packages/menpo3d/io/input/base.pyc in import_mesh(filepath, landmark_resolver, texture_resolver)
98 landmark_ext_map=mesh_landmark_types,
99 landmark_attach_func=_import_object_attach_landmarks,
--> 100 importer_kwargs=kwargs)
101
102

/vol/atlas/homes/mengjiao/miniconda/envs/normrecov/lib/python2.7/site-packages/menpo/io/input/base.pyc in _import(filepath, extensions_map, landmark_resolver, landmark_ext_map, landmark_attach_func, asset, importer_kwargs)
858 if importer_kwargs is None:
859 importer_kwargs = {}
--> 860 built_objects = importer_callable(path, asset=asset, **importer_kwargs)
861
862 # landmarks are iterable so check for list precisely

/vol/atlas/homes/mengjiao/miniconda/envs/normrecov/lib/python2.7/site-packages/menpo3d/io/input/mesh/base.pyc in wrl_importer(filepath, asset, texture_resolver, **kwargs)
127 # Get the Data
128 polydata = vtk.vtkPolyData.SafeDownCast(mapper_dataset)
--> 129 polydata.Update()
130
131 # We must have point data!

AttributeError: 'vtkCommonDataModelPython.vtkPolyData' object has no attribute 'Update'

error in import_mesh

I wanna import a TexturedTrimesh. there is an obj mesh file and a jpg texture file in the folder with the same name, no extra tcoords file. the tcood information is provided in obj file with "vt " line, but it seems that "import_mesh" function can handle it.

the warning says:
UserWarning: texture was found, but no tcoords were recovered, reverting to an untextured mesh.
warnings.warn('texture was found, but no tcoords were recovered, '

Support for numpy>1.24.0

Hi,

A lot of codebase seem to use np.float which was deprecated in numpy 1.20 and removed in 1.24. Therefore, importing menpo3d raises error. An update would be much appreciated.

Cheers,

Allow updating of existing mayavi mesh visualizations

Have this in my jabphd repo - basic use case is you need to render out a whole load of similar meshes with same camera setup:

fig = mlab.figure()
set_camera(fig, nice_pose_camera)
# render_function shows the mesh and returns a function that will update the view
# for new mesh data
update = render_mesh(mean)
mlab.safefig(...)
for mesh in meshes_to_visualize:
    update(mesh)
    mlab.safefig(...)

Issue is it takes something OOP ATM in menpo (per-mesh visualizer) and changes it to persist for many meshes.

Unable to view using the sample code

I'm trying to visualize a 3D textured mesh using this sample code:
import menpo3d import matplotlib import PyQt4 import time mesh = menpo3d.io.import_builtin_asset('/home/eman/Desktop/testMesh.obj') mesh.view() time.sleep(0.8)
But it is only opening the viewer but not showing the mesh. Why is this happening?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.