Comments (5)
pkuseg:北京大学/生前/来/应聘
jieba:北京/大学生/前来/应聘
看了楼上的,pkuseg好像对'生前'有点执念...
from pkuseg-python.
测试结果:
print(seg.cut('买水果然后来世博园最后去世博会'))
['买', '水', '果然', '后来', '世博园', '最后', '去世', '博', '会']
print(seg.cut('欢迎新老师生前来就餐'))
['欢迎', '新', '老师', '生前', '来', '就', '餐']
from pkuseg-python.
weibo模型会表现更好一些
from pkuseg-python.
pkuseg:北京大学/生前/来/应聘
jieba:北京/大学生/前来/应聘看了楼上的,pkuseg好像对'生前'有点执念...
您好,这个是由于使用的词典问题造成的,如果您不导入词典的话,我们的分词软件是可以分对北京/大学生/前来/应聘的,不过还是很感谢您提供的样例。
from pkuseg-python.
实测ctb8模型,与jieba默认模型对比稍差一些,建议出一些最佳实践的文档
print(seg.cut('工信处女干事每月经过下属科室都要亲口交代24口交换机等技术性器件的安装工作'))
['工信', '处女', '干事', '每', '月', '经过', '下属', '科室', '都', '要', '亲口', '交代', '24', '口', '交换机', '等', '技术性', '器件', '的', '安装', '工作']
print(", ".join(jieba.cut("工信处女干事每月经过下属科室都要亲口交代24口交换机等技术性器件的安装工作")))
工信处, 女干事, 每月, 经过, 下属, 科室, 都, 要, 亲口, 交代, 24, 口, 交换机, 等, 技术性, 器件, 的, 安装, 工作
print(seg.cut('贱狗奴,鸡巴套子把爸爸加上'))
['贱', '狗', '奴', ',', '鸡', '巴', '套子', '把', '爸爸', '加上']
print(", ".join(jieba.cut("贱狗奴,鸡巴套子把爸爸加上",)))
贱狗奴, ,, 鸡巴, 套子, 把, 爸爸, 加上
谢谢您提供的这些数据,这些样例其实另一个方面诠释了给不同领域数据选择不同预训练模型的重要性。您提供的这些样例都是正常的新闻领域预训练模型中遇到的很棘手的问题,所以我们也愿意尝试使用更多网络用语的语料提供一种专有的处理这种情况的预训练模型~
from pkuseg-python.
Related Issues (20)
- 能不能用自训练模型,对文件进行分词啊? HOT 1
- 打包缺少文件情况 FileNotFoundError: \\cache\\_MEI183882\\pkuseg\\dicts\\default.pkl' HOT 2
- 是否可以支持在 M1下使用 目前直接安装会报错。 HOT 2
- AttributeError: type object 'pkuseg' has no attribute 'pkuseg'
- TypeError: train() got an unexpected keyword argument 'nthread' HOT 1
- 词性标注效果过好 HOT 1
- Python3.9/3.9均无法pip安装成功 HOT 7
- 如何进行领域自适应训练? HOT 1
- 虚拟环境下使用出现FileNotFoundError,已单独下载postag还是无果
- cannot install in the environment of python 3.9 HOT 6
- pip install failed on Python 3.10.8 HOT 4
- 越训练,f-score越低 HOT 4
- pip install failed on Python 3.10.9 HOT 7
- 词性标注不准
- 预训练模型“default_v2”导入报错 HOT 2
- 祈使句/动宾短语,分词好像有些不对劲呢,望指点,谢谢
- 分词结果严重错误 HOT 5
- 词性输出错误
- Win10 3.10.14 安装失败 HOT 5
- Using Pre-Downloaded Domain Model is messy HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pkuseg-python.