Git Product home page Git Product logo

timbre-dissimilarity-metrics's Introduction

Timbre Dissimilarity Metrics

A collection of metrics for evaluating timbre dissimilarity using the TorchMetrics API. Work in progress and subject to sudden change โ€” use in projects at your own risk.

Installation

pip install -e .

Usage

import timbremetrics

datasets = timbremetrics.list_datasets()
dataset = datasets[0] # get the first timbre dataset

# MAE between target dataset and pred embedding distances
metric = timbremetrics.MAE(
    dataset=dataset, distance=timbremetrics.l1
)

# get numpy audio for the timbre dataset
audio = timbremetrics.get_audio(dataset)

# get arbitrary embeddings for the timbre dataset's audio
embeddings = net(audio)

# compute the metric
metric(embeddings)

Metrics

The following metrics are implemented.

Mean Squared Error

Gives the mean squared error between the upper triangles of the predicted distance matrix and target distance matrix:

$MSE(D_X,D_Y)=\dfrac{2}{n(n-1)}\sum\limits_{i=1}^n\sum\limits_{j=i+1}^n(D_X-D_Y)^2$

Mean Absolute Error

Gives the mean squared error between the upper triangles of the predicted distance matrix and target distance matrix:

$MAE(D_X,D_Y)=\dfrac{2}{n(n-1)}\sum\limits_{i=1}^n\sum\limits_{j=i+1}^n|D_X-D_Y|$

Item Rank Agreement

Gives the proportion of distances ranked per-item that match between the predicted distance matrix and target distance matrix.

$IRA(R_X,R_Y)=\dfrac{1}{n^2-n}\left[\sum\limits_{i=1}^n\sum\limits_{j=1}^n(1_{{0}}(R_{X_{i,j}}-R_{Y_{i,j}}))-n\right]$

Where $1_A(x)$ is the indicator function given by:

Indicator function

and $R_X$ & $R_Y$ are distances matrices ranked per item such that each row contains the ordinal distances from the corresponding item. We also provide a top-k version which computes this metric considering only the closest k items in each row.

Triplet Agreement

Samples pseudo-triplets from the target distance matrix according to a positivity radius and margin, and returns the proportion of these triplets for which ordering is retained in the predicted distance matrix, with the margin optionally enforced.

Triplet K-NN Agreement

For each anchor (a) from the target distance matrix $D_X$, all triplets (a, i, j) are sampled where i and j are in a's K-nearest neighborhood and D(a, i) < D(a, j). The metric returns the proportion of these triplets for which ordering is retained in the predicted distance matrix $D_Y$.

all triplets

triplet agreement

Mantel Test

The Mantel test computes Pearson's r or Spearman's rho on the condensed form of the upper triangles of the predicted and target distance matrices. The significance of the given result can be estimated using permutation analysis.

References

@article{thoret2021learning,
  title={Learning metrics on spectrotemporal modulations reveals the perception of musical instrument timbre},
  author={Thoret, Etienne and Caramiaux, Baptiste and Depalle, Philippe and Mcadams, Stephen},
  journal={Nature Human Behaviour},
  volume={5},
  number={3},
  pages={369--377},
  year={2021},
  publisher={Nature Publishing Group}
}

timbre-dissimilarity-metrics's People

Contributors

ben-hayes avatar cyrusvahidi avatar tiianhk avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.