Question re: DeepSDF training about shapegan HOT 3 CLOSED

Just to document a few things, totally unscientific.
I've also been experimenting with different normalizations on the generator w/ the point discriminator.

The following experiments were done with the same discriminator, and only the generator was modified. I train for only ~100 epochs, and visually check if the results look like they are converging.

• Spectral norm - fails, most values from the sdf network are very extreme
• Weak spectral norm (only apply norm if value is greater than 1) - fails, either invalid values or only shapes for cones/blobs
• Instance norm - works, but visually seems worse than layer/weight norm
• Group norm - works, I wonder if this would help in the multi-class case
• Weight norm - works (haven't benchmarked, visually looks comparable to layer norm, seems less diverse)
• Layer norm (repo default)

Seems quite robust to choice of activation:

• Leaky relu w/ slope 0.1 - works
• ELU - works
• ReLU (repo default)

Edit: training with instance norm is less stable than layer norm, at least in terms of generator loss.

from shapegan.

Thanks for taking the time to respond

If you want to be sure, you should test different batch compositions and BN layers.

Yeah, definitely - doing that now. Was just trying to bootstrap off of you :)

Thanks again for sharing your code - best of luck in your research.

from shapegan.

I'm afraid I don't have good answers for both of these questions.

I remember trying batch normalization in my DeepSDF implementation, but IIRC training was unsuccessful with that. (The DeepSDF authors say that batch normalization didn't work for them either, but weight normalization does work)

For the batch composition, I changed the implementation of the data loader throughout development and used the most "convenient" way of composing the batches. The current implementation expects all data to be in a single .to file, so this would of course fail if there is not enough RAM. I don't remember finding that the composition of the batch (few shapes vs many shapes) makes a difference, although testing for this parameter wasn't my goal.

If you want to be sure, you should test different batch compositions and BN layers.

from shapegan.