Git Product home page Git Product logo

dacidian's Introduction

DaCiDian (大词典)

DaCiDian is an open-sourced lexicon for Chinese Automatic Speech Recognition(ASR)


Design

In mainstream ASR system, lexicon is a core component, that maps word into acoustic modeling units(such as phone). In DaCiDian, we break the mapping into 2 independent layers:

word --> PinYin syllable --> phoneme

The purpose of this design is as follows:

  • Anyone who is familiar with PinYin (basically every mandarin speaker), can enrich DaCiDian's vocabulary, by adding new entry(word) into the layer-1 mapping.

  • ASR system developers can easily adapt DaCiDian to their own phone set by defining their own layer-2 mapping.


Layer-1: Word -> Syllable Mapper (word_to_pinyin.txt)

  • word and its pronunciation(s) are seperated by tab
  • multiple pronunciations are seperated by ;
  • each pronunciation is a sequence of Chinese PinYin syllables, seperated by space
  • tone infomation are encoded into 0,1,2,3,4

examples:

...
裤子    KU_4 ZI_0
好事    HAO_4 SHI_4;HAO_3 SHI_4
教授    JIAO_1 SHOU_4;JIAO_4 SHOU_4
...
语音识别  YU_3 YIN_1 SHI_2 BIE_2
傅里叶变换 FU_4 LI_3 YE_4 BIAN_4 HUAN_4

Layer-2: Syllabel->Phone Mapper (pinyin_to_phone.txt)

pinyin_to_phone is a user-defined mapping from PinYin syllables to target phone set

Take traditional PinYin's Initial-Final structure for example, a mapping should be defined as follows:

A	$0 a
AI	$0 ai
AN	$0 an
ANG	$0 ang
AO	$0 ao
BA	b a
BAI	b ai
BAN	b an
BANG	b ang
BAO	b ao
...
...
...
ZONG	z ong
ZOU	z ou
ZU	z u
ZUAN	z uan
ZUI	z ui
ZUN	z un
ZUO	z uo
  • syllable and its phoneme representation are seperated by tab
  • $0 is the NULL-Initial phone.

Notes

  • English word pronunciation in Chinese context is a problem, at the moment, we would rather leave it alone.

dacidian's People

Contributors

dophist avatar

Watchers

James Cloos avatar GRU avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.