Topic: corpus Goto Github

👇 Here are 823 public repositories matching this topic...

adbar / trafilatura
corpus,Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
User: adbar
Home Page: https://trafilatura.readthedocs.io
web-scraping text-extraction nlp html2text news text-mining crawler text-cleaning text-preprocessing article-extractor

blkserene / wordless
corpus,An Integrated Corpus Tool With Multilingual Support for the Study of Language, Literature, and Translation
User: blkserene
corpus corpus-analysis corpus-statistics corpus-linguistics corpus-tools corpus-processing literature translation tokenizer tagger

brightmart / nlp_chinese_corpus
corpus,大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP
User: brightmart
chinese-dataset chinese-corpus pretrain word2vec nlp bert language-model wiki news question-answering

candlewill / dialog_corpus
corpus,用于训练中英文对话系统的语料库 Datasets for Training Chatbot System
User: candlewill
dataset dialog system corpus chatbot

cbluebenchmark / cblue
corpus,中文医疗信息处理基准CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark
Organization: cbluebenchmark
Home Page: https://tianchi.aliyun.com/dataset/dataDetail?dataId=95414&lang=en-us
benchmark chineseblue corpus dataset chinese biomedical-tasks evaluation acl2022

chatopera / efaqa-corpus-zh
corpus,❤️Emotional First Aid Dataset, 心理咨询问答、聊天机器人语料库
Organization: chatopera
Home Page: https://www.chatopera.com/
natural-language-processing natural-language-understanding corpus psychology

chatopera / insuranceqa-corpus-zh
corpus,:helicopter: 保险行业语料库，聊天机器人
Organization: chatopera
Home Page: https://www.chatopera.com/
corpus chatbot qasystem natural-language-processing natural-language-understanding machine-learning dataset question-answering insurance insuranceqa-corpus-zh

christos-c / bible-corpus
corpus,A multilingual parallel corpus created from translations of the Bible.
User: christos-c
bible bible-corpus corpus multilingual translation

cluebenchmark / clue
corpus,中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard
Organization: cluebenchmark
Home Page: http://www.CLUEbenchmarks.com
nlu benchmark chinese corpus dataset bert albert chineseglue glue roberta

cluebenchmark / cluecorpus2020
corpus,Large-scale Pre-training Corpus for Chinese 100G 中文预训练语料
Organization: cluebenchmark
Home Page: https://arxiv.org/abs/2003.01355
chinese chinese-corpus datasets pretrain corpus nlp bert roberta albert

cluebenchmark / cluedatasetsearch
corpus,搜索所有中文NLP数据集，附常用英文NLP数据集
Organization: cluebenchmark
Home Page: https://www.cluebenchmarks.com/dataSet_search.html
nlp datasets chinese ner qa match text-classification machine-translation knowledge-graph corpus

cluebenchmark / cluepretrainedmodels
corpus,高质量中文预训练模型集合：最先进大模型、最快小模型、相似度专门模型
Organization: cluebenchmark
Home Page: https://arxiv.org/abs/2003.01355
pretrained-models chinese bert roberta albert sentence-classification sentence-pairs semantic-similarity sentence-analysis text-classification

coolengineer / sejong-corpus
corpus,Korean sejong corpus download and simple analysis
User: coolengineer
korean corpus morphological-analysis python linux mac sejong

crownpku / small-chinese-corpus
corpus,Some useful Chinese corpus datasets 中文语料小数据
User: crownpku
chinese-nlp corpus

dariusk / corpora
corpus,A collection of small corpuses of interesting data for the creation of bots and similar stuff.
User: dariusk
language words bots corpus

edinburghnlp / code-docstring-corpus
corpus,Preprocessed Python functions and docstrings for automated code documentation (code2doc) and automated code generation (doc2code) tasks.
Organization: edinburghnlp
Home Page: https://arxiv.org/abs/1707.02275
code-generation corpus docstrings documentation-generator neural-machine-translation

endymecy / awesome-deeplearning-resources
corpus,Deep Learning and deep reinforcement learning research papers and some codes
User: endymecy
code corpus deep-learning modelzoo neural-network nlp paper reinforcement-learning video

fendouai / awesome-chatbot
corpus,Awesome Chatbot Projects,Corpus,Papers,Tutorials.Chinese Chatbot =>:
User: fendouai
Home Page: http://www.panchuangai.com/
chatbot tensorflow awesome seq2seq seq2seq-chatbot seq2seq-model corpus tutorial

franck-dernoncourt / pubmed-rct
corpus,PubMed 200k RCT dataset: a large dataset for sequential sentence classification.
User: franck-dernoncourt
nlp machine-learning corpus medical randomized-controlled-trials sentence-classification

gair-nlp / mathpile
corpus,Generative AI for Math: MathPile
Organization: gair-nlp
Home Page: https://gair-nlp.github.io/MathPile/
corpus language-model large-language-models math pre-training

grammarly / ua-gec
corpus,UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language
Organization: grammarly
Home Page: https://ua-gec-dataset.grammarly.ai/
dataset corpus gec grammatical-error-correction ukrainian-language corpus-data corpus-tools natural-language-processing nlp-datasets

guhhhhaa / 4675-scifi
corpus,chinese NLP corpus of chinese science fiction,chinese science fiction corpus : About 4675 Chinese science fiction novels 大约有4675本科幻小说，中文科幻小说自然语言处理语料库，中文科幻小说文本语料库，中文科幻小说文本数据库，科幻小说语料
User: guhhhhaa
scifi corpus corpus-data nlp nlp-datasets nlp-machine-learning nlp-resources science-fiction chinese-nlp datasets

gunthercox / chatterbot-corpus
corpus,A multilingual dialog corpus
User: gunthercox
Home Page: http://chatterbot-corpus.readthedocs.io
chatterbot corpus dialog language yaml

helsinki-nlp / prosody
corpus,Helsinki Prosody Corpus and A System for Predicting Prosodic Prominence from Text
Organization: helsinki-nlp
natural-language-processing sequence-labeling prosody bert pytorch machine-learning corpus dataset speech-synthesis

jiangyanting / pre-modern_chinese_corpus_dataset
corpus,近代汉语语料库数据集 自然语言处理 语料库 古代汉语 古汉语 文言文 数字人文 计算语言
User: jiangyanting
corpus natural-language-processing data-mining machine-learning dataset

kirralabs / indonesian-nlp-resources
corpus,data resource untuk NLP bahasa indonesia
Organization: kirralabs
dataset nlp corpus-linguistics corpus indonesian-language indonesian named-entity-recognition pos-tagging dependency-parser crawler

ko-ichi-h / khcoder
corpus,KH Coder: for Quantitative Content Analysis or Text Mining
User: ko-ichi-h
Home Page: http://khcoder.net/en
content-analysis text-mining visualization kwic corpus

ko-nlp / korpora
corpus,Korean corpus repository
Organization: ko-nlp
corpus korean-natural-language

lil-lab / nlvr
corpus,Cornell NLVR and NLVR2 are natural language grounding datasets. Each example shows a visual input and a sentence describing it, and is annotated with the truth-value of the sentence.
Organization: lil-lab
Home Page: http://lic.nlp.cornell.edu/nlvr/
computer-vision corpus machine-learning natural-language-processing

louisowen6 / nlp_bahasa_resources
corpus,A Curated List of Dataset and Usable Library Resources for NLP in Bahasa Indonesia
User: louisowen6
natural-language-processing bahasa-indonesia dataset library packages nlp sentiment-analysis nlp-bahasa-resources indonesian indonesian-language

lucasjinreal / weibo_terminater
corpus,Final Weibo Crawler Scrap Anything From Weibo, comments, weibo contents, followers, anything. The Terminator
User: lucasjinreal
scraper weibo sina corpus chinese chatbot

m1-llie / tumcc
corpus,[IP&M 2022] Telegram地下市场中文黑话识别语料集。Telegram Underground Market Chinese Corpus. Paper: Identification of Chinese Dark Jargons in Telegram Underground Markets Using Context-Oriented and Linguistic Features (IP&M, 2022).
User: m1-llie
chinese corpus dataset telegram

mesolitica / malaysian-dataset
corpus,We gather Malaysian dataset! https://malaysian-dataset.readthedocs.io/
Organization: mesolitica
Home Page: https://malaysian-dataset.readthedocs.io/
text-mining corpus malaysia malay-dataset bahasa-melayu manglish

mhbashari / awesome-persian-nlp-ir
corpus,Curated List of Persian Natural Language Processing and Information Retrieval Tools and Resources
User: mhbashari
persian-language natural-language-processing information-retrieval language-detection persian-nlp corpus part-of-speech-tagger normalizer named-entity-recognition embeddings

niutrans / classical-modern
corpus,非常全的文言文（古文）-现代文平行语料
Organization: niutrans
corpus parallel-corpus traditional-chinese traditional-and-simplified-chinese

nonamestreet / weixin_public_corpus
corpus,微信公众号语料库
User: nonamestreet
corpus nlp chinese-nlp natural-language-processing linguistics weixin weixin-data wei-xin corpora yu-liao

oroszgy / awesome-hungarian-nlp
corpus,A curated list of NLP resources for Hungarian
User: oroszgy
Home Page: https://oroszgy.gitbook.io/awesome-hungarian-nlp-resources/
nlp natural-language-processing text-mining information-retrieval information-extraction hungarian hungarian-language awesome awesome-list nlu

oye93 / chinese-nlp-corpus
corpus,Collections of Chinese NLP corpus
User: oye93
chinese-nlp datasets corpus

plexpt / chatgpt-corpus
corpus,ChatGPT 中文语料库 对话语料 小说语料 客服语料 用于训练大模型
User: plexpt
Home Page: https://chat.aimakex.com/
corpus awesome corpus-data question-answering

quanteda / quanteda
corpus,An R package for the Quantitative Analysis of Textual Data
Organization: quanteda
Home Page: https://quanteda.io
r text-analytics natural-language-processing corpus quanteda

several27 / fakenewscorpus
corpus,A dataset of millions of news articles scraped from a curated list of data sources.
User: several27
fakenews dataset database corpus natural-language-processing nlp machine-learning artificial-intelligence

soskek / bookcorpus
corpus,Crawl BookCorpus
User: soskek
bookcorpus corpus crawler nlp scraper

srvk / how2-dataset
corpus,This repository contains code and metadata of How2 dataset
Organization: srvk
Home Page: https://srvk.github.io/how2-dataset/
corpus dataset how2-dataset language machine-translation multimodality speech-recognition video

strongcourage / fuzzing-corpus
corpus,My fuzzing corpus
User: strongcourage
corpus file-format fuzzing testsuite vulnerability

tensorlayer / seq2seq-chatbot
corpus,Chatbot in 200 lines of code using TensorLayer
Organization: tensorlayer
Home Page: https://github.com/tensorlayer/tensorlayer
tensorlayer tensorflow chatbot rnn lstm bot nlp chat corpus python

thunlp-aipoet / datasets
corpus,Poetry-related datasets developed by THUAIPoet (Jiuge) group.
Organization: thunlp-aipoet
poetry-generation chinese corpus

wainshine / chinese-names-corpus
corpus,中文人名语料库。人名生成器。中文姓名,姓氏,名字,称呼,日本人名,翻译人名,英文人名。可用于中文分词、人名实体识别。
User: wainshine
Home Page: https://open.namemoe.com/
names corpus dict ner dataset

wainshine / company-names-corpus
corpus,公司名语料库。机构名语料库。公司简称,缩写,品牌词,企业名。可用于中文分词、机构名实体识别。
User: wainshine
Home Page: https://open.namemoe.com/
company corpus dict ner dataset

yohasebe / wp2txt
corpus,A command-line toolkit to extract text content and category data from Wikipedia dump files
User: yohasebe
wikipedia wikipedia-dump ruby nlp corpus machine-learning

zake7749 / gossiping-chinese-corpus
corpus,PTT 八卦版問答中文語料
User: zake7749
Home Page: https://www.kaggle.com/zake7749/pttgossipingcorpus
chatbot chatbot-corpus chinese-chatbot chinese-corpus chinese-dataset chinese-nlp corpus dataset dialog ptt question-answering