Comments (16)
@groadabike great to hear that convtasnet works. Are you getting to results similar to demucs?
Not really, at the moment I have really poor performances.
As soon as I have a decent trained model, I would send it to you.
from asteroid.
Wow that would be great! We'd be really glad to have your contributions obviously. Music separation was on the list but we didn't get to it yet.
A nice hello from the sigsep gang,
This looks like a very nice and ambitious approach. Would love to contribute here. Would you be interested in adding music separation things such as
-
The musdb dataset
Yes please !
-
Music specific augmentations
Yes please ! Can you link to a paper or an existing implementation? What do you have in mind for it? Implementation in numpy or torch?
-
multichannel support throughout the package
Yes please ! Again, what do you have in mind exactly? We are very interested.
The docs was not updated yet but you might be interested by #42, our STFT now supports arbitrary number of channels, the only convention is that time is last (like nn.Conv1D)
-
Open-unmix model
Sure, it would be great !
Feel free to join the slack, it will make some things easier.
I'm going to moderately refactor some things in the next days, maybe I can ping you when it's ready.
Thanks again for your message
from asteroid.
@faroit Released 0.1.0 yesterday, should be ready to use now ! 🚀
from asteroid.
Thats great. Will check it out soon. In the meantime, I wanted to ask if you are interested to work on a new evaluation tool that should serve the whole community and replace mir_eval.separation and museval. I can add you to the discussion if you are interested
from asteroid.
Hi @faroit, I am testing a model (ConvTasNet) by using the current version of the musdb18 dataset and it works great.
I just did small modifications for my specific problem:
- Transform to mono
- work with 16KH
- Target = vocals.
Can I ask you why the mixture is the addition of all sources instead of using the mixture.wav from each directory?
What can I expect in the second draft of MUSDB18 dataset?
Thank you
from asteroid.
Please correct me if I'm wrong.
Can I ask you why the mixture is the addition of all sources instead of using the mixture.wav from each directory?
If you pass only ['bass', 'drums']
you'll have a mixture that only contains bass and drums, and separation still makes sense in this way.
If you load the mixture, you'll always have all the sources.
What can I expect in the second draft of MUSDB18 dataset?
I think he meant to add the directions to download and convert MUSDB18 to wav, right?
Now that I have you here, @groadabike, would you like to share your pre-trained model?
from asteroid.
I f you pass only ['bass', 'drums'] you'll have a mixture that only contains bass and drums, and separation still makes sense in this way.
If you load the mixture, you'll always have all the sources.
You are right about this. But, currently, all sources become targets. So, I had to modify the dataset to just get the vocals
and background
as targets.
I think he meant to add the directions to download and convert MUSDB18 to wav, right?
Do you mean to have the ability to manage both MUSDB18 and MUSDB18HQ?
I also asked because, in the current MUSDB18dataset, the sources are returned in a dictionary instead of a vstacked np array.
Now that I have you here, @groadabike, would you like to share your pre-trained model?
Yes, sure!!
Let me fix some errors first. There is no novelty here, just adaptations.
How can I share the model with you?
from asteroid.
Oh great thank you !
The idea was to return a dict here because the sources are not of the same nature (as opposed to speech/speech separation), and you can still use the default Dataset and make vocals
and backgrounds
your target when you receive the dict.
I think that was the intended usage.
Regarding sharing the model (thanks), did you use the plain ConvTasNet importable from asteroid or did you change things there?
from asteroid.
@groadabike great to hear that convtasnet works. Are you getting to results similar to demucs?
from asteroid.
Regarding sharing the model (thanks), did you use the plain ConvTasNet importable from asteroid or did you change things there?
Yes, I did use the plain ConvTasNet model. I am re-running again. When I finished, I can tell you how it went.
from asteroid.
Which loss do you use?
Did you check the demucs implementation?
from asteroid.
Which loss do you use?
That was my first error, didn't change it to L1 loss.
Did you check the demucs implementation?
Yes, I did.
Just to confirm that I am not mixing terminology and have another silly mistake.
Are these parameters correct?
filterbank: # Conv-TasNet symbols from table 1
n_filters: 256 # N
kernel_size: 20 # L
stride: 10 # L/2
masknet:
n_blocks: 8 # X
n_repeats: 4 # R
mask_act: relu # mask_nonlinear
conv_kernel_size: 3 # P
bn_chan: 256 # B
skip_chan: 256 # Sc
hid_chan: 512 # H
In terms of the augmentation, will it be a standard way to do it in Asteroid or should I implement it for the specific case? If I am trying to replicate demucs performances with Asteroid, I should implement, as it says in demucs paper, shuffling sources within one batch to generate one new mix, randomly swapping channels
from asteroid.
These parameters are correct yes.
There is no default way to do this is Asteroid yet but we've been thinking about it with @faroit but it not clear yet.
You can check this script on MUSDB, or the dynamic mixing on WHAM.
from asteroid.
Hi, I've recently been doing audio source separation tasks by using the Conv-Tasnet model. However, I have no way to obtain the MUSDB18HQ dataset, I would be very grateful if you could provide it to me.
from asteroid.
@YadChen request here: https://zenodo.org/record/3338373
from asteroid.
�Thans!
from asteroid.
Related Issues (20)
- Legacy Install Failure HOT 1
- X-UMX: lr_scheduler_step() missing 1 required positional argument: 'metric' HOT 4
- About the usage HOT 1
- torchmetrics>=0.11.0 HOT 2
- HOW TO USE OTHER DATASET TO TRAIN Multi-Decoder-DPRNN MODEL HOT 3
- val loss in distribute training HOT 2
- refer to asteroid/dsp/overlap_add.py:31 LambdaOverlapAdd model use file_separate output audio file is All noise HOT 2
- KeyError while evaluating musdb18 (X-UMX) HOT 2
- RunTimeError related with NCCL when training librimix recipe HOT 3
- Input and output mean scales updated during training (X-UMX) HOT 5
- how to train the dataset HOT 3
- Transfer learning of pretrained model on custom dataset HOT 2
- Inference script to hear samples from AVSpeech Model ?
- Support for mobile development (Android)? HOT 3
- PIT Loss for multichannel audio for speech separation HOT 1
- test model
- mossFormer2 & sepTDA models HOT 2
- KeyError: 'source_2_path' when training ConvTasNet using enh_single mode HOT 3
- ConvTasNet pretrained huggingface model inference setup
- How to use GPU rather than CPU
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from asteroid.