Git Product home page Git Product logo

brownify's Introduction

Brownify

Brownify allows you to make your music brown. It splits audio tracks separate tracks for individual instruments, and it allows for performing basic modifications to those tracks. The tracks can then be recombined to help form a brown version of the original audio track.

Documentation Status Unit Test Status Unit Test Coverage End to End Test Status License: MIT

What is "brown" music?

In some ways, that is up to you! But the terminology at least comes from the band Ween.

Various people have expressed their opinion on what it means for music to be brown over on this Ween subreddit thread, but one of the founding members of Ween summarized it in this way:

I can’t really explain ‘brown’ except to give examples... Ween opened for The Ramones in 1991. The Ramones showed up in a Country Squire station wagon that was smashed on one side, and missing a window. Dee Dee, Joey, and Marky. That’s brown. The beautiful side of brown. That is brown – that’s real. That’s brown. It can be really bad, it can be a horrible thing, when you get browned out by some band. A band shows up and they’re brown – none of their equipment works or whatever. But it’s a strength. You know you’re getting the real thing.

How does Brownify split audio tracks?

Brownify uses the Spleeter library for this. Spleeter does the heavy lifting.

At this point in time, Brownify supports the 5-stem splitter, which splits songs into the following separate tracks:

  • Bass
  • Drums
  • Piano
  • Vocals
  • Other (i.e. everything else)

Installation

At this time, installation requires obtaining the source code. In the future, Brownify will be installable through pypi. It is recommended to install through a venv to avoid cluttering your python installation's packages.

1. Obtain the source code

Clone brownify from github.

2.a. Install using venv (recommended)

cd <path-to-brownify>

# (optionally) with venv
python3 -m venv .env
source .env/bin/activate

# Install an editable version
pip3 install -e .

2.b. Install at system level

cd <path-to-brownify>

# Install
pip3 install .

Usage

In order to use Brownify, you can either use the python API or a command line program.

What are Brownify recipes?

Brownify allows for user-specified modifications to audio tracks and a user-specified definition of how to merge audio tracks. To support his, Brownify relies on simple configurations called "recipes". Some example recipes are provided in the recipes directory.

Recipe syntax

Recipes are defined by a series of pipeline specifications. Each pipeline is comprised of a track source, a series of actions, and a track sink. Below is an example:

vocals -> flat -> early -> save(newVocals);

In this example, the vocals of the original audio track are made flat and a little bit early compared to the original audio. The resulting track is marked to be merged into the final track using the save wrapper around the sink.

Here is a more complicated example:

vocals -> octaveup -> early -> save(vocals1);
vocals -> octavedown -> late -> save(vocals2);
bass -> late -> late -> late -> late -> lateBass;
lateBass -> flat -> save(flatBass);
lateBass -> halfflat -> save(halfFlatBass);
drums -> save(drums);
other -> save(other);
piano -> save(piano);

In the above example, the original vocals are duplicted and modified. The original vocals are not part of the final track. A temporary sink called lateBass is defined and used to create two versions of the bass track. Note that "lateBass" is not part of the final track, as it does not include the save wrapper around the sink. However, it is used to define two tracks flatBass and halfFlatBass which are saved as part of the final track.

When defining recipes, the following are available to you:

Original sources:

  • bass: The bass track split out by the audio splitter
  • drums: The drum track split out by the audio splitter
  • other: The track of remaining instruments split out by the audio splitter
  • piano: The piano track split out by the audio splitter
  • vocals: The vocal track split out by the audio splitter

Names: Sinks must be named. Names are alphanumeric with no punctuation or whitespace.

Defined sources: A defined source is a named sink which was already defined in a previous pipeline.

Actions:

  • early: Make the audio slightly (but noticeably) early.
  • flat: Tune the audio pitch down by a full step.
  • halfflat: Tune the audio pitch down by a half step.
  • halfsharp: Tune the audio pitch up by a half step.
  • late: Make the audio slightly (but noticeably) late.
  • octavedown: Tune the audio pitch down by a full octave.
  • octaveup: Tune the audio pitch up by a full octave.
  • sharp: Tune the audio pitch up by a full step.

Sinks: A sink must be named. It can either be temporary (to be used as a defined source) or part of the final track. Temporary sinks are useful for reusing sets of actions over a common audio source. In order to make a source temporary, it is simply named with no wrapper. To mark a sink as part of the final audio track, the name of the sink should be wrapped with the save wrapper, like this:

save(NAME)

More examples can be found in the recipes directory.

Command line program

After installing Brownify, the brownify command will be placed in the path. If installed in a venv, the environment must be activated for the command to be available. Please run brownify --help for more details.

Note that the spleeter library used by brownify will look for files it needs in the current working directory from wherever brownify is invoked. It is recommended that you navigate to the brownify source directory each time you run brownify so that you only need to download those files once. This is not required, but it will speed up running time by avoiding re-downloading the files.

Examples

The simplest way to get started with Brownify is to find a song you want to modify on youtube, obtain the URL, and pick an existing recipe from the provided recipes directory. Below is an example that adds a dash of brown to a Christmas song:

# This example assumes that the brownify source is located in your home directory
cd ~/brownify
brownify https://www.youtube.com/watch?v=B4RqeAvE7iA boognish-christmas.mp3 --recipe-file recipes/boognish-brown

The result of this command is a file boognish-christmas.mp3 located in the current directory.

Python API

The python documentation for the latest release can be found here.

brownify's People

Contributors

gth828r avatar

Stargazers

Guillaume Dumoulin avatar Max Vogel avatar

Watchers

 avatar James Cloos avatar

brownify's Issues

Add a volume effect

We should add an action that allows for turning the volume of a track up or down.

Consider adding support for parameterized actions

This is a larger task for the future. We should support generic paramaterization of our functions.

  • I'm not exactly sure what this means for the recipe grammar yet. If it complicates the grammar too much, it may not be worth it.
  • For the pipeline creation and processing, we would need to support functions with arbitrary signatures. This might mean creating classes for each action with a unique constructor and a common "exec" function or something like that.

Add an end-to-end test and run it through CI

We should have a canned end-to-end test which we can easily run to confirm everything is working. Ephemeral errors may occasionally occur with this test, but it will give us supplemental information beyond the unit tests about whether or not things are working as expected. Once we have this test, we should run it using CI.

Investigate improved pitch shift quality

I saw a comment on stackoverflow that someone recommended trying pyrubberband over librosa to improve pitch shift sound quality. In particular, librosa's output often has a tinny or metalic sound. From what I read, this is inherent to the algorithm that both of these libraries use, but pyrubberband may have better performance.

We should at least try it out and see if it is better. We can also investigate if there are any other alternatives. The current quality is not so bad that this is a big problem for such a silly project.

Generate and publish sphinx documentation

We should set up documentation in our library, given that we have already started doing a reasonable amount of docstring writing and type hinting.

  • Initialize the repository for sphinx documentation generation
  • Publish the documentation somewhere
  • Update the setup.cfg to point to where the documentation lives
  • Update the top level README to point to where documentation lives

Remove docstring type hint duplication

The docstrings currently provide type hints alongside the type hints in the method signatures. This is not necessary, and it is error prone. We should remove the docstring type hints and leave the inline type hints in place.

Consider refactoring the runner module

The runner module is doing too much right now. We should probably have the caller directly interact with the AudioMerger, or perhaps have a separate class which calls the PipelineProcessor and AudioMerger. But the PipelineProcessor calls the AudioMerger for now, and that seems a bit messy.

End to end test failing in CI with error about protobuf

We are seeing the following error:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.12/x64/bin/brownify", line 5, in <module>
    from brownify.cli import main
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/brownify/cli.py", line 10, in <module>
    from brownify.runners import PipelineProcessor
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/brownify/runners.py", line 14, in <module>
    from brownify.splitters import AudioSplitter
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/brownify/splitters.py", line 6, in <module>
    from spleeter.separator import Separator
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/spleeter/separator.py", line 26, in <module>
    import tensorflow as tf
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 40, in <module>
    from tensorflow.python.eager import context
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tensorflow/python/eager/context.py", line 32, in <module>
    from tensorflow.core.framework import function_pb2
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

The protobuf version being installed in CI is 3.9.x, which is well behind 3.19.0. See if we can fix this error.

Define some initial unit tests

We should start creating some very basic unit tests to at least create the structure and create some templates for more unit tests as time permits. The ActionParser methods seem like a good thing to test at this time.

We shouldn't need to manage protobuf versions in our install

One of our dependencies (definitely at least tensorflow) uses protobuf. However, it seems like there is an issue where the way we are installing it may be too lax about how it manages the protobuf dependency. In #34, we worked around this by forcing a specific version of protobuf through our package, but that is not the best long term solution.

Figure out what the real root cause of the issue is, and see if we can remove the fix we put in place in #34.

Clean up logging mechanisms

We should make use of logging to get better insight into our program. We should also turn off some of the verbose logs that our libraries use unless they are desired by the caller. Finally, we should do some nicer exception handling by catching things and doing proper logging.

Update pytube version

Pytube broke after changes on the youtube side. There are pull requests pending to fix it. Once a new version is released, we need to update our requirements.txt.

Allow for intermediate files to be persisted

Brownify currently needs to create some intermediate files based on how our current third party libraries work. In a normal run, we want to clean up those files after we finish, leaving only the desired processed audio file. To help with debugging, it can be useful to persist those intermediate files if the processed audio does not come out as one expects it to. Specifically, this can help isolate if the input file matches expectations, and it can show how well the splitter was able to split the source audio into separate tracks.

Investigate enabling pip cache on Github actions

It would be nice if we could enable the pip cache in Github actions. Unfortunately, it requires a requirements.txt file, which we would separately need to maintain from our setup.cfg file. Keep an eye on this and decide if we should add a requirements.txt or do something else.

Some RTD documentation not rendered due to libsndfile being missing

Our documentation is being generated, but much of the API is missing. When looking at the raw logs from the RTD builds, it appears that our version of soundfile installed through pip checks for the underlying C library, and if it is not present, it raises an exception. We need to install that library.

Consider making the splitter part of the actions

Right now, we always assume that the audio sources are split, then we use the split sources as the starting points for our pipeline. Perhaps we should instead allow for splitting to be part of the pipeline and assume that the only source is the one provided as the input file.

Consider refactoring the parser

Right now, the parser does both a syntactic ingestion of a program using pyparsing as well as does semantic interpretation of the parsed syntax into a pipeline. Some of the methods are quite long, and they include some code which seems unreachable.

We might want to consider splitting out the syntax and semantic parsing into separate classes with public methods and wrap those with an outer class which the user calls.

Fix docstring line continuation indentation

When we need to continue lines within docstrings for method arguments (or similar), we need to indent. Otherwise, the rendering of the sphinx documentation is not correct.

Convert time shifting parameter to be time-based

Right now, the API for shifting time is an "early" or "late" call, but that passes on a value which represents the number of samples to shift by to another API. That value should really be time based and tied to the sample rate for the audio.

Add a license

Based on which libraries we use, we need to pick a license for our library. Then we need to:

  • Add a license file
  • Update our setup.cfg to reference the license we use

RTD build is broken: cannot import attr module

The readthedocs build is failing to create our documentation. It is complaining that myst-parser is trying to import attrs, but the attrs module cannot be found. Investigate and fix.

Improve error handling of internal libraries

We are not currently catching many exceptions thrown by the libraries we use. We should do that and convert them into Brownify errors. As long as we know which exceptions to expect, this model makes it easy to use exception handling internally within the library and perform logging either at the command line program or at a select number of strategic places within the library.

Set the version for the package

We should set the version for the package in setup.cfg and make that available through the __version__ variable at the module level.

Add badges to repo

We should add any badges that make sense to the repo readme prior to the v0.1.1 release.

Below is text that can be put into the README with known badges:

[![Documentation Status](https://readthedocs.org/projects/brownify/badge/?version=latest)](https://brownify.readthedocs.io/en/latest/?badge=latest)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.