Comments (9)
@AdarshMJ Thanks for the comment. I am a little confused on your last comment. As far as I understand, if each frame image is clear then the encoded video output must be clear as well. Are you saying that each stylized frame is clear before the encoding but the output comes out blurry?
If so, there is one thing that is different from your approach with ours, which I suspect as the source of blurriness. For video stylization, we used a single style image (sunset) for stylizing the entire content video frames. This may be the reason for the blurred outputs that come from different stylization of nearby frames. Is it necessary to stylize your content video frame with each corresponding style video frame? If not, please try with a single representative style frame and let me know if this trick works or not.
from wct2.
from wct2.
@AdarshMJ Hi. I hope that works. About the training, you know, making a neat release-version code needs quite a time and efforts π . I am thinking of releasing the training code for enc-dec networks in a near future but I cannot guarantee that the release will come soon.
from wct2.
from wct2.
@AdarshMJ Yes, it does and its amount depends on the content and style. Generally, including a segmentation map would provide way much better stylization results. This can be easily seen when you try a nightscape of a city as a style. If you do not have a segmentation map, the stylized output would be very dark overall (because your model cannot map sky to sky and building to building). Dependency on the semantic map is inevitable for photorealistic stylization until now. Still, I would say ours are better in maintaining the structure.
from wct2.
Im using the segmentation code from this [CSAILVision/semantic-segmentation-pytorch ](git clone https://github.com/mingyuliutw/semantic-segmentation-pytorch segmentation). I was able to generate this segmentation mask from that code.
Should I give this whole image as content-segment input? or Just crop out the segmentation map and give just that?
When I crop the segmentation mask and give it as input, I get this error -
Traceback (most recent call last):
File "transfer.py", line 205, in
run_bulk(config)
File "transfer.py", line 155, in run_bulk
content_segment = load_segment(_content_segment, config.image_size)
File "/Users/adarsh/Desktop/project/WCT2/utils/io.py", line 88, in load_segment
image = change_seg(image)
File "/Users/adarsh/Desktop/project/WCT2/utils/io.py", line 64, in change_seg
dist = np.sum(np.abs(np.asarray(key) - arr_seg[x, y, :]))
ValueError: operands could not be broadcast together with shapes (3,) (4,)
from wct2.
@AdarshMJ First of all, please check the tutorial given by PhotoWCT authors: https://github.com/NVIDIA/FastPhotoStyle/blob/master/TUTORIAL.md#prepare-label-maps
By doing so, you would be able to follow the exact same procedure of which trained dataset is made.
Secondly, please check how the example images I gave looks like. They should share the same filename as well as image size. For semantic maps, a unique label set of your style image should match with that of your content image. I guess this is the reason why you are having such error.
I hope this helps to clarify your issue.
from wct2.
@jaejun-yoo Thank you for the clarifications! The problem was the resolution of the segmentation maps and its corresponding content/style maps do not have same resolution. I guess that's why the error. I will retry it by having similar image sizes. Thank you!
from wct2.
Good. Since the problem asked is clarified and this thread is already too long, I will close this issue. Please open another one if you need to ask about the semantic label map. Thx!
from wct2.
Related Issues (20)
- WCT2 seems cannot transfer styles without segmentation maps? Could you please indicate how to make it works without segmentation maps since you claimed in your paper that it supports transfer directly and without segmentation. HOT 1
- Can I train with my data? HOT 1
- Some questiona about network? HOT 4
- Query regarding "conv0" layer HOT 1
- Segmentation maps precision HOT 1
- In "cat5" skip network, only the last layer matters HOT 2
- Qs about the Dataset HOT 2
- Does it support large-size photos style transfer? HOT 1
- some questions about the figure 9 HOT 2
- How to add temporal consistency HOT 2
- QS about the depth map of transfered images HOT 1
- About training source HOT 4
- why the encoder has the layer conv0 HOT 1
- How does it work for an indoor dataset? HOT 1
- Question about the loss functions
- LL components vs. avg. pooling HOT 2
- Details about training decoder
- About SSIM metircs
- Without Seg HOT 1
- Create
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wct2.