rbeato / tabgen Goto Github PK

This technical documentation is to accompany the tabgen tablature generation system. The full source code is available on GitHub. Additional comments and interface documentations can be found as inline comments in the code.

In order to fully run tabgen, you need to install the following software (recommended versions in brackets):

• Python 3 (3.5), inluding the libraries:
  • tensorflow (1.2), ideally as tensorflow-gpu on top of CUDA
  • keras (2.0.5)
  • scikit-learn (0.18.1)
  • numpy (1.13.0)
  • pandas (0.20.2)
  • tqdm (4.14.0)
• MuseScore (2.0.3)

To run the system, ...

• Make sure your tablature files are split into the folders training_input and evaluation_input.
• Edit the enumeration class tabgen.definitions.FeatureConfig to support the desired instrument and features.
• Make sure that tabgen.definitions.Path.MSCORE points to a working installation of MuseScore or MuseScore portable.
• Run the do_preprocessing script.
• Run the estimate_accuracy script. You may alternatively want to use run_sample or make a copy of it and adjust the settings.

The tabgen package consist of the following six modules:

This is the basic configuration file, containing global settings.

The enumeration class Path contains file paths as strings. Path.MSCORE has to point to an executable binary file of MuseScore in order for the system to work with input files other than *.mscx.

The enumeration class FeatureConfig contains the configuration of features to extract from training data and to use during evaluation time. A short info is given inline. For detailed information on the features, please refer to the project report.

The preprocessing module defines functions for batch-processing of tablatures, turning them into usable training data. Simply run this module using the do_preprocessing executable.

A collection of base classes:

• ChordFrettingEvaluatorBase: The abstract base class for any evaluator. If you wish to add a new evaluator, e.g. a hand-tuned heuristic, inherit from this class and implement the evaluate(fretting) method.
• StringConfigBase: An abstract class for StringConfig, to decouple dependencies
• PruningConfig: A configuration object for pruning in the tree search.

This is the core package, modelling the nature of the guitar fretting problem. We conceptually divide classes into pitch view, capturing the musical intention or output, and fretting view, capturing the mechanics of the fretboard.

Pitch view and fretting view are connected by the StringConfig class, which defines the strings and frets of an instrument. Conceptually, an object from the fretting view can be applied to a StringConfig, yielding an object in Pitch view. Similarly, the StringConfig can be used to generate possible fretting view objects from a pitch view object. The implementation of StringConfig pre-defines some typical instruments, e.g. StringConfig.STANDARD_24_FRETS for a six-string guitar with 24 frets in standard tuning.

The following pitch view classes are implemented:

• Pitch: represents a single pitch. The pitch class is based on the MIDI integer representation, but can generate note names with the method get_note_name(). It can further produce viable _NoteFretting_s with the method get_note_frettings(string_config).
• Chord: A chord is implemented as a collection of _Pitch_es with a duration. It can produce viable _ChordFretting_s with the method get_note_frettings(string_config, ...).

The following fretting view classes are implemented:

• NoteFretting: Captures a single note fretting, defined by a combination of string and fret. The corresponding Pitch object can be generated by calling get_pitch(string_config).
• ChordFretting: Captures a fretting of a Chord. ChordFretting can be understood as the main class in tabgen.modelling.
  • The features are extracted in this class and are available through the property ChordFretting.features.
  • The property ChordFretting.cost captures the cost of the fretting, depending on the evaluator (class ChordFrettingEvaluatorBase) the ChordFretting was initialised with.
  • The property next_pitches implements the pitch lookahead feature for LSTM predictions.
  • The property previous can be used to traverse backwards in a tree of ChordFrettings.
  • The method get_chord(string_config) yields the associated chord.
• ChordFrettingSequence: A sequence of _ChordFretting_s, i.e. a potential tablature. Offers a simple text tablature printout with to_ascii_tab() and implements the model-based aspects of the tree search implemented by tabgen.processing.Solver.

The evaluation module is a collection of evaluation classes, i.e. subclasses of tabgen.base.ChordFrettingEvaluatorBase which return a cost function tabgen will optimise for. The following implementations are available:

• RandomChordFrettingEvaluator: Returns a random number, resulting in a random tablature. Use as a lowest baseline for your experiments.
• BaselineChordFrettingEvaluator: A generic class for heuristics, implemented as a hand-tuned linear model. Use the weights dictionary to assign weights to features. If you set tabgen.definitions.FeatureConfig.heuristics = True, you will have access to a predefined set of heuristics, available as features 'heuristics_*'. For example, use dict(heuristic_distance_move=1.0) to generate frettings based on the distance travelled between consecutive frettings.
• ProbabilityLookupEvaluator: Looks up probabilities from preprocessing. Falls back to a low probability for unseen data.
• RegressionChordFrettingEvaluator: Estimates the cost function as a predicted negative log probabilty. Refer to the project report for details.
• LSTMChordFrettingEvaluator: Predicts a new fretting based on previous frettings. As this does not clearly identify a "second best" match, you may want to set your PruningConfig to (0,1,0,1). This effectively disables the tree search and selects the best fit at every time step.

This module implements two classes tying the whole system together:

• Parser: Parse MuseScore XML files to an internal class construct with parse(file_path) or write a previously parsed class construct back to an XML file with save(file_path). Note that save does not generate a new file, but copy the existing structure and change only the tablature to the generated one. The parser is tested with XML files converted from Guitar Pro with MuseScore 2.0.3, but may not fully work with other input formats.
• Solver: The Solver implements the tree search used to find the best fretting sequence for a chord sequence input. Call Solver.solve(chord_sequence, string_config) to create a new tablature from a list of _Chord_s, or use Solver.solve_multi(...) to process a batch of input files with the tablature generator.

• do_preprocessing: Scans files in Path.TRAINING_INPUT and generates a training data file Path.FEATURE_FILE
• estimate_accuracy: trains models on Path.FEATURE_FILE and evaluates them. Change the evaluators dictionary to select or define the models to train. Note that only one cost neural network and one direct prediction network can be trained at the same time. You may however backup the weight files.
• run_sample: Use this file as a skeleton for running tabgen with your own settings.