Git Product home page Git Product logo

bilingualcorpus's Introduction

Background

NLP is a hot topic currently! Team AI really want's to leverage the NLP research and this an attempt for all the NLP researchers to explore exciting insights from bilingual data

The Japanese-English Bilingual Corpus of Wikipedia's Kyoto Articles” aims mainly at supporting research and development relevant to high-performance multilingual machine translation, information extraction, and other language processing technologies.

Unique Features

A precise and large-scale corpus containing about 500,000 pairs of manually-translated sentences. Can be exploited for research and development of high-performance multilingual machine translation, information extraction, and so on.

The three-step translation process (primary translation -> secondary translation to improve fluency -> final check for technical terms) has been clearly recorded. Enables observation of how translations have been elaborated so it can be applied for uses such as research and development relevant to translation aids and error analysis of human translation.

Translated articles concern Kyoto and other topics such as traditional Japanese culture, religion, and history. Can also be utilized for tourist information translation or to create glossaries for travel guides.

The Japanese-English Bilingual Kyoto Lexicon is also available. This lexicon was created by extracting the Japanese-English word pairs from this corpus.

Sample

One Wikipedia article is stored as one XML file in this corpus, and the corpus contains 14,111 files in total.

The following is a short quotation from a corpus file titled “Ryoan-ji Temple”. Each tag has different implications. For example:

<j>Original Japanese sentence<j> <e type="trans" ver="1">Primary translation</e> <e type="trans" ver="2">Secondary translation</e> <e type="check" ver="1">Final translation</e> <cmt>Comment added by translators</cmt>

Categories

The files have been divided into 15 categories: school, railway, family, building, Shinto, person name, geographical name, culture, road, Buddhism, literature, title, history, shrines and temples, and emperor (Click the link to view a sample file for each category).

Acknowledgements

The National Institute of Information and Communications Technology (NICT) has created this corpus by manually translating Japanese Wikipedia articles (related to Kyoto) into English.

Licence

Use and/or redistribution of the Japanese-English Bilingual Corpus of Wikipedia's Kyoto Articles and the Japanese-English Bilingual Kyoto Lexicon is permitted under the conditions of Creative Commons Attribution-Share-Alike License 3.0. Details can be found at http://creativecommons.org/licenses/by-sa/3.0/.

bilingualcorpus's People

Contributors

venali avatar

Watchers

James Cloos avatar Bipin Manandhar avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.