Comments (2)
The robust label maps are generated from images via some pre-trained robust semantic segmentation network. The intuition here is to have a segmentation of the input image that is semantically meaningful and roughly consistent across synthetic and real data. The consistency is generally a challenge and methods just trained on one dataset from a narrow domain is unlikely to work well on some other dataset. This is why we used a method that had been trained on multiple datasets and shown to generalize well. The maps are supposed to be one-hot encoded maps at the same resolution as an input image and with each channel representing a different semantic class.
The gbuffer file in our case was an npz-file containing multiple G-buffers that correspond to the input image at a pixel level. The G-buffers contained e.g., surface normals, the view vector reflected over the surface normal, distance to the camera, surface albedo, glossiness, or approximate irradiance. The intuition is to encode geometry, materials, and lighting information for each pixel in the input image such that the network does not need to learn how to extract this data from an image itself.
The ground truth label map is a semantic segmentation obtained from the synthetic data. The goal here is to cluster pixels with approximately the same material/appearance properties. The resulting maps are then used b the network to process G-buffers in separate streams. Each stream is intended to roughly correspond to some class of materials (or objects if derived from a semantic segmentation map). Concretely, the ground truth label maps would be one-hot-encoded maps as the robust label maps above.
Sorry for missing the generate_fake_gbuffers.py. I'll add it shortly.
from photorealismenhancement.
I have some problems about the ground truth label map.
What is the difference between ground truth label map and robust label map of synthetic data? I think they are the same.
In readme, it says "The pipeline expects for each dataset a txt file containing paths to all images. Each line should contain paths to image, robust label map, gbuffer file, and ground truth label map, all separated by commas." So in fake_dataset of config file, is the robust label map and ground truth label map the same here?
from photorealismenhancement.
Related Issues (20)
- Loading model for training HOT 2
- ValueError when saving model HOT 3
- RuntimeError while training HOT 3
- usage
- Tensor size mismatch error for batch_size>1 - File:gb_encoder.py HOT 1
- step: Matching patches across datasets. Error num_samples = 0 HOT 3
- Train without using robust labels map from Mseg HOT 6
- Problem on MSeg, took 10 minutes for 1 image and only produced the gray scaled image? HOT 17
- Error Training -> Num_samples = 0
- License? HOT 2
- Error Training: RecursionError
- Datasets, txt and csv files. HOT 2
- ModuleNotFoundError: No module named 'epe'
- Is this repo still mantained???? can anyone run this repo without any issue? HOT 2
- Anyone knows the difference between robust label map and ground truth label map?
- How do you change the number of classes? HOT 1
- How to setup a dataset description like PfDDataset?
- IENet 2 training goes to Nan very quickly
- Is labeling data needed for both datasets? HOT 3
- Can pre-training models such as checkpoint be provided?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from photorealismenhancement.