khushgrover / smart-match Goto Github PK

Team: TheChibiTeam

Members: Piyush Aggarwal, Kartikey Tiwari, Khushnuma Grover

University: Thapar Institute of Engineering and Technology

We show how a MGN network can be fine-tuned on new images. The network will produce 3d-garments of person and the 3d-body parameters. These 3d-body parameters can be layered on top of SMPL body.

We show how these 3d-garments can be used to mix and match clothing on a model. This can be rendered a website in real-time using three.js. For demo purposes we have used https://p3d.in.

https://drive.google.com/file/d/1QRDrG15-JAS8AP1cJY5Hg9Mgm1-XceyG/view

https://drive.google.com/file/d/1WFXqZGwt4ARoZs8HLbwpgdle7-PyiVSE/view

https://drive.google.com/file/d/1eqPVxptgWA76aZt2NUSrmHBpPWXXzEDi/view

https://drive.google.com/file/d/1juuHeB4G0OUVO6JLMe8celSGWUG561fk/view

The local changes have also been shown on Colab Notebooks: (Colab doesn't have a display so we could not display out the resulta and have used SSH and ngrok int local machine for visualization.)

MGN: https://github.com/khushgrover/smart-match/blob/main/MultiGarmentNetwork.ipynb

PGN Segmentation: https://github.com/khushgrover/smart-match/blob/main/pgn_segmentation.ipynb

Training U-NET: https://github.com/khushgrover/smart-match/blob/main/Train_UNET.ipynb

Extracting dresses using U-NET: https://github.com/khushgrover/smart-match/blob/main/Train_UNET.ipynb

Extracting dresses using GrabCut: https://github.com/khushgrover/smart-match/blob/main/OpenCv_GrabCut.ipynb

Extracting Keypoints of body OpenPose: https://github.com/khushgrover/smart-match/blob/main/Openpose1_6_0.ipynb

https://github.com/bharat-b7/MultiGarmentNetwork.git

This repository is the official implementation for the paper "Multi-Garment Net: Learning to Dress 3D People from Images, ICCV'19" in Python 2.7 and Tensorflow 1.13.

Link to paper: https://arxiv.org/abs/1908.06903

In this, the trained model is provided and we plan to use the fine tuning of the network, which can be done using anywhere between 1-8 images of a person.

• Installed DIRT: https://github.com/pmh47/dirt (provides Fast Rendering for Tensorflow)
• Downloaded and installed Mesh packages for visualization: https://github.com/MPI-IS/mesh

Preprocessing for Inputs

• Run semantic segmentation on images. PGN semantic segmentation used. https://github.com/Engineering-Course/CIHP_PGN
• Run OpenPose body_25 for 2D joints. https://github.com/CMU-Perceptual-Computing-Lab/openpose

SMPL is a function M that maps pose θ and shape β to a mesh of V = 6890 vertices.

Downloaded the neutral SMPL model from http://smplify.is.tue.mpg.de/