So far the only models trained were focused on single string pitch recognition, let's see what happens if multiple strings are to be recognized by the model.

Create a model class - MonoPitchNet

• just a few CNN layers
• do not use reshape, use stack, chunk or permute
• 360 logits on the output, just like crepe
• take a lot of inspiration from penn

FCNF0++ should be build and delivered audio data correct with the below

• 5 cents of bin width
• 1440 pitch bins
• 128 batch
• 8kHz sample rate
• 1024 samples frame size ~ 128ms
• no normalization in FCNF0++

This requires modifications to the current dataset class, quite many things actually.

Let's consider the easiest case where pitch is only estimated for frames which do not contain silence. This way the network will only be pitch-focused. This should solve many issues.

MonoPitch1D is struggling with recognizing short-term pitch deviations. Let's try to fix this!
It seems that the approach with just CQT on the input cannot catch all the information about how the pitch is changing. My guess is that the bins are just way too wide and small pitch deviation information is gone. CQT is great for long-term pitch values, it needs to be balanced off with something else for the short-term, perhaps a short time STFT, pure audio in time domain or autocorrelation? Enhanced Autocorrelation?

Scenarios to test:

• CQT + TIME
• STFT + TIME
• CQT + TIME + AC
• CQT + TIME + EAC
• STFT + TIME + AC
• STFT + TIME + EAC
• HCQT
• HCQT + TIME
• HCQT + TIME + EAC

Quite many scenarios, ideed.

Currently MonoPitchNet model outputs a one-hot vector of size 360, add one more class to make it aware of silence, in total 361 classes. This should enhance the overall quality of the model.

Make new conversion functions that will remember that 1 in the silence class means SILENCE, 1 in the pitch class means PITCH. When converting to a pitch array put 0 where there is silence to indicate NO pitch state, even if the array is in cents.

Weights and Biases are a much more comprehensive yet more transparent way of providing model statistics, even whilst training. Check it out and implement!