Git Product home page Git Product logo

contextual-biasing-dataset's Introduction

Contexual-Biasing-Dataset

paper: CB-Conformer: Contextual biasing Conformer for biased word recognition

This repository contains the Mandarin biased words dataset with five sub-datasets, namely the person-name dataset, the place-name dataset, the organization-name dataset, and the full set of biased words containing all categories called "all_biased" dataset as well as the dataset without biased words called "no_biased" dataset.

Each sub-dataset is divided into train, test, and dev sets, and contains a collection of biased words. The specific dataset sizes are as follows:

train dev test time(h) number of biased words
person-name 1000 1000 9997 10.24 73
place-name 1000 1000 10191 12.96 42
organization 1000 500 2035 3.94 183
all_biased 2992 2497 21972 26.77 298
no_biased 12774458 / / 793.30 /

This dataset is filtered and divided from the WenetSpeech dataset, if you need to use this dataset, you need to do the following steps

  1. download the

    [WenetSpeech]: https://wenet.org.cn/WenetSpeech/ ""WenetSpeech""

    dataset

  2. Save the wav.scp from WenetSpeech/WenetSpeech/train_m or your own wav.scp to each sub-dataset

  3. Call fix_data_dir.sh of the

    [Kaldi]: https://github.com/kaldi-asr/kaldi ""Kaldi""

    tool to process each sub-dataset and you will get a filtered and constructed dataset

citation:

@inproceedings{CB-Conformer,
  title={CB-Conformer: Contextual biasing Conformer for biased word recognition},
  author={Xu, Yaoxun and Liu, Baiji and Huang, Qiaochu and Song, Xingchen and Wu, Zhiyong and Kang, Shiyin and Meng, Helen},
  booktitle={ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={1--5},
  year={2023},
  organization={IEEE}
}

contextual-biasing-dataset's People

Contributors

xuyaoxun avatar

Stargazers

brightLLer avatar  avatar  avatar  avatar Yasutaka Odo avatar  avatar chenchen avatar kk avatar  avatar

Watchers

 avatar

contextual-biasing-dataset's Issues

paper

hi,can you put the paper pdf on arxiv?

Utterance Number

The total number of utterance of wenetspeech 'train_m' subset in wenet repo is 1514500.
However the total number of utterances of all_biased(dev+test+train) + no_biased is 27461 + 1274458 = 1301919.
What's the difference between these two numbers?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.