uygned / nhpylm Goto Github PK
View Code? Open in Web Editor NEWThis project forked from fgnt/nhpylm
Python bindings for a c++ based implementation of the Nested Hierarchical Pitman-Yor Language model
This project forked from fgnt/nhpylm
Python bindings for a c++ based implementation of the Nested Hierarchical Pitman-Yor Language model
########## # nhpylm # ########## python bindings for a c++ based implementation of the Nested Hierarchical Pitman-Yor Language model ########### # Contact # ########### In case of questions, suggestions, problems etc. please send an email or check the disussion group. Oliver Walter: [email protected] Jahn Heymann: [email protected] Discussion group: email: [email protected] google groups: https://groups.google.com/d/forum/latticewordsegmentation ############## # References # ############## Teh, Yee Whye. "A hierarchical Bayesian language model based on Pitman-Yor processes." Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2006. Mochihashi, Daichi, Takeshi Yamada, and Naonori Ueda. "Bayesian unsupervised word segmentation with nested Pitman-Yor language modeling." Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1. Association for Computational Linguistics, 2009. ############### # Instalation # ############### git clone [...] nhpylm cd nhpylm pip install --user -e . ################ # Requirements # ################ kaldi: to run the examples, the openfst command line tools installed by kaldi will be needed. Prior to running the examples, set the environment varaible KALDI_ROOT to point to you kaldi installation. (see nhpylm/kaldi.py) ############ # Examples # ############ Example output can be found in examples/lattice_playground All possible segmentations of an example sentence: examples/lattice_playground/wcl-l-e-p/I_loop_L.fst.pdf Example using PHI compositions: examples/Word_Char_Lexicon-LM-Example-Phi.ipynb Example using PHI compositions and acoustig model: examples/Word_Char_Lexicon-LM-Example-Phi-Monophone.ipynb Example using EPS fallback (standard decoding chain): examples/Word_Char_Lexicon-LM-Example.ipynb PHI composition with example from WSJ database: examples/Word_Char_Lexicon-LM-WSJCAM0-Phi.ipynb EPS fallback with example from WSJ database: examples/Word_Char_Lexicon-LM-WSJCAM0.ipynb Decoding of test sentences with both, PHI composition and EPS fallback: examples/Word_Char_Lexicon-LM-WSJCAM0-Decode.ipynb Create (L o G) for WSJCAM0 training data: examples/WSJCAM0_HPYLM.ipynb Create (L o G) for WSJ LM training data: examples/WSJ_HPYLM.ipynb Example using PHI compositions and character only language model: examples/Char-LM-Example-Phi.ipynb
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.