Comments (2)
Hi, thanks for the questions! Better perceptual losses is certainly an area for further investigation.
First off, as you can see from the code:
https://github.com/magenta/ddsp/blob/master/ddsp/losses.py
the standard autoencoder configuration
https://github.com/magenta/ddsp/blob/master/ddsp/training/gin/models/ae.gin#L41
actually does use a log-amplitude scaled spectrogram (dB scale).
The configuration that jointly learns to predict frequency
https://github.com/magenta/ddsp/blob/master/ddsp/training/gin/models/ae_abs.gin
uses mel-frequency scale as input to the encoder, and addtionally uses a pretrained CREPE network to capture higher level audio statistics, similar to "perceptual losses" in the image literature.
All models use A-Frequency weighting for calculation of "loudness_db" which is used as a conditioning feature.
There are certainly more options for losses inspired by psychophysics (including several even in losses.spectral_loss()
, but we simply haven't used them yet because they haven't yet been necessary.
Feel free to try them out and if you find one that clearly works better, write it up and contribute it to the library.
from ddsp.
@jesseengel Thanks for the response!
You didn't quite answer my second question. Let me ask it in a different way...
With regards to the autoencoder configuration...
ddsp/ddsp/training/gin/models/ae.gin
Line 40 in cd98116
Is there a benefit to training on an amplitude spectrogram? It looks like it's also used in the loss in addition to the log scaled spectrogram.
from ddsp.
Related Issues (20)
- OnlineF0PowerPreprocessor cannot function with compute_power = False.
- No module crepe
- AttributeError: module 'hmmlearn.hmm' has no attribute 'CategoricalHMM'
- AttributeError: module 'hmmlearn.hmm' has no attribute 'CategoricalHMM'
- AttributeError: module 'hmmlearn.hmm' has no attribute 'CategoricalHMM'
- AttributeError: module 'collections' has no attribute 'Iterable'
- python environment Mac M1 HOT 1
- train_autoencoder.ipynb error I got HOT 1
- ImportError: cannot import name 'dtensor_api' from 'keras.dtensor' HOT 5
- vst notebook
- error when training !
- pip is repeatedly installing various versions of same packages HOT 9
- Question About Midi Autoencoder
- Failed building wheel for llvmlite, Could not build wheels for numba, llvmlite, which is required to install pyproject.toml-based projects HOT 1
- timbre_transfer.ipynb is broken on Colab
- train_autoencoder.ipynb is broken on Colab
- Installation Guide HOT 2
- pitch_detection.ipynb is broken in Colab
- VST3 file format no detected not working with fl studio
- Training Script is not starting
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ddsp.