Comments (7)
I think I don't 100% understand your use case here. Is the problem that the audio does not live on disk?
If so, it wouldn't be hard to modify muda to support in-memory audio data that's dynamically generated. Everything after that should be pretty easy.
from muda.
No, it's not the audio does not live on disk but rather it does not exist yet. Regardless of this point... the main thing is that I think it could be useful to to decouple the transformation of the jams files from the transformation of the audio... and then be able to load the transformed jams file at a later point in time and process the audio then.
This could be useful if you are "statically" augmenting a dataset (rather than dynamically at training time) in which you want to save the augmented dataset to use for training with another learner... but you don't want to save all of the processed audio files because of disk constraints.
Once these two are decoupled, it also doesn't seem necessary to provide the original audio file or signal in order to simply transform the jams.
from muda.
Okay, thanks for clarifying. I think I understand your point now. (I think loading from in-memory audio is a good idea too though, and should be a separate issue.)
I get a little nervous about decoupling things in this way, but I do like the idea of delayed audio deformation. The current design makes this almost possible, since all transformation state is logged in the jams sandbox. Something like the following:
- Add a
delayed
flag toBaseTransformer.transform
andPipeline.transform
, and_transform
. When this is true, theaudio
step is skipped in_transform
. - Add a
rebuild
method that reconstitutes the audio from a jams with a muda history entry.
Then you should be able to do something like the following:
>>> # Build a muda pipeline
>>> for jam_out in pipeline.transform(jam_in, delayed=True):
jam_out.save(SOMEWHERE)
>>> # Some time later
>>> jam_reload = jams.load(SOMEWHERE)
>>> audio_out = muda.rebuild(jam_reload, audio_in)
muda.rebuild
would need to do the following:
- inspect
jam_reload
for muda history - reconstruct all transformers in the history
- apply transformers to
audio_in
in order, with the correspondingstate
variables
Sound reasonable?
from muda.
Yeah, that sounds perfect.
from muda.
While reviewing #40 , I realized a slight snag in this plan. The transformer in question needs there to be an audio buffer in place to calculate noise clip durations and generate the state
object.
This might require some careful thought to get right.
from muda.
Coming back to this having implemented a prototype of the rebuild op in #62 , and reviewed the current suite of deformer ops. TLDR: i think this isn't gonna work in general, but you might be able to hack around it to get something going.
As mentioned above, many of the deformers require access to the audio signal to determine their deformation states, notably pitch-shifting requires a tuning estimate, and background-noise requires a sample-accurate duration count. These things really do need to be pre-computed so that the deformations stay as a deterministic, self-contained set of functions of the state variables. I don't see a good way around that (eg by delayed computation).
@mcartwright if you're still interested in this, how hacky would it be to pre-compute a dummy signal for generating the deformation jamses, and then use the rebuild functionality afterward on real data?
from muda.
I think this is as fixed as it's going to get, having merged the replay() function.
from muda.
Related Issues (20)
- Add explicit int cast in pitch.py HOT 1
- Switch from nose to py.test HOT 1
- Drop sklearn dependency
- Deformation union HOT 1
- OSError: [Errno 2] No such file or directory HOT 6
- Errors in tests HOT 1
- Support passing jams kwargs to muda's load/save functions HOT 2
- Remove in-place annotation modification HOT 3
- BackgroundNoise fails if len(soundf)==n_target HOT 4
- How to apply deformations from annotated jams file? HOT 7
- Can I load a wav file without JAMS? HOT 4
- Running Error on Spyder in windows 10 HOT 2
- External dependency I/O overhead for out-of-core pipelines HOT 5
- Getting TypeError: expected string or buffer HOT 1
- Cannot save jams with random seeds
- Future-proofing against deprectations
- Old jams BackgroundNoise lacking start and stop HOT 3
- Modernization
- ColoredNoise deformer bug HOT 1
- Tapdancing (use pedalboard as backend)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from muda.