Comments (15)
duterscmy
commented on July 22, 2024
这个文件本质就是语料集里每个问题对应的bert识别出的实体，我给改成了输入问题直接调用bert模型得到mention的方式，更新了train_ner.py mention_extractor.py kb.py这几个文件，你可以重新下载一下。
train_ner.py得需要去下载谷歌发布的中文预训练模型再执行微调。
…------------------ 原始邮件 ------------------
发件人: "zhengxiaoxuer"<[email protected]>;
发送时间: 2019年11月1日(星期五) 中午1:56
收件人: "duterscmy/ccks2019-ckbqa-4th-codes"<[email protected]>;
抄送: "Subscribed"<[email protected]>;
主题: [duterscmy/ccks2019-ckbqa-4th-codes] question_2_mention.pkl怎么生成 (#3)
您好，感谢作者提供的代码，我运行helper文件程序后，再运行mention_extractor.py,发现缺少question_2_mention.pkl
helper文件没找到生成该字典的程序。
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
from ccks2019-ckbqa-4th-codes.
zhengxiaoxuer
commented on July 22, 2024
恩恩感谢！请问您用的requirement及其版本都有哪些
from ccks2019-ckbqa-4th-codes.
zhengxiaoxuer
commented on July 22, 2024
请问keras版本用的哪个，版本问题会报错
from ccks2019-ckbqa-4th-codes.
duterscmy
commented on July 22, 2024
font{
line-height: 1.6;
}
ul,ol{
padding-left: 20px;
list-style-position: inside;
}
keras2.3.1
tensorflow 1.14.0
在2019年11月1日 14:52，zhengxiaoxuer<[email protected]> 写道：
请问keras版本用的哪个，版本问题会报错
—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.
from ccks2019-ckbqa-4th-codes.
zhengxiaoxuer
commented on July 22, 2024
我的版本和你的是一样的，最后发现是keras_bert包的问题，更新包之后就可以了，感谢！
from ccks2019-ckbqa-4th-codes.
MrRace
commented on July 22, 2024
question_2_mention.pkl这个文件在prop_extractor.py还是有用到。那我是在train_ner.py的时候保存train和valid数据集中问句对应的实体吗？
from ccks2019-ckbqa-4th-codes.
duterscmy
commented on July 22, 2024
这个没啥用了，可以删掉。。我个人倾向于在识别实体的时候直接调模型而不是提前保存结果。。评测时候因为是合作的才弄了这个文件
…---原始邮件---
发件人: "JaonLiu"<[email protected]>
发送时间: 2019年11月19日(星期二) 晚上7:44
收件人: "duterscmy/ccks2019-ckbqa-4th-codes"<[email protected]>;
抄送: "Mention"<[email protected]>;"Caomingyu"<[email protected]>;
主题: Re: [duterscmy/ccks2019-ckbqa-4th-codes] question_2_mention.pkl怎么生成 (#3)
question_2_mention.pkl这个文件在prop_extractor.py还是有用到。那我是在train_ner.py的时候保存train和valid数据集中问句对应的实体吗？
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
from ccks2019-ckbqa-4th-codes.
ZainZhou
commented on July 22, 2024
请问作者prop_extractor.py运行后发现recall只有0.31是否过于低了？
单主语且主语为属性值问题中，能找到所有主语属性值的比例为:0.31
平均每个问题属性为:1.35
[15, 24, 90, 124, 125, 177, 208, 218, 227, 242, 258, 265, 280, 289, 294, 324, 325, 335, 339, 360, 381, 386, 388, 391]
[292, 343, 356, 370]
得到实体mention
耗费时间61.33秒
from ccks2019-ckbqa-4th-codes.
duterscmy
commented on July 22, 2024
font{
line-height: 1.6;
}
ul,ol{
padding-left: 20px;
list-style-position: inside;
}
不应该啊，我记得正常60%左右的，可以把注释掉的那块模糊匹配加上试试。。我记不清最后用的哪个版本了不好意思
在2019年12月6日 18:47，Zain<[email protected]> 写道：
请问作者prop_extractor.py运行后发现recall只有0.31是否过于低了？
单主语且主语为属性值问题中，能找到所有主语属性值的比例为:0.31
平均每个问题属性为:1.35
[15, 24, 90, 124, 125, 177, 208, 218, 227, 242, 258, 265, 280, 289, 294, 324, 325, 335, 339, 360, 381, 386, 388, 391]
[292, 343, 356, 370]
得到实体mention
耗费时间61.33秒
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.
from ccks2019-ckbqa-4th-codes.
ZainZhou
commented on July 22, 2024
@duterscmy 模糊匹配加上也效果很差，而且加上之后速度特别慢。想请问一下 entity_mention = corpus[i]['entity_mention']从之前mention_extractor.py生成的entity_mention字段为什么在prop_extractor.py里导入了但是没有使用？是不是因为没有用这个的原因呢？
from ccks2019-ckbqa-4th-codes.
duterscmy
commented on July 22, 2024
你这个环节的数值是在测试集评价的？验证集上应该是60%+的，因为这个知识库的属性值比较不规范，所以感觉很难提上去。跟entity mention应该没关系。
…---原始邮件---
发件人: "Zain"<[email protected]>
发送时间: 2019年12月7日(周六) 下午3:26
收件人: "duterscmy/ccks2019-ckbqa-4th-codes"<[email protected]>;
抄送: "Mention"<[email protected]>;"Caomingyu"<[email protected]>;
主题: Re: [duterscmy/ccks2019-ckbqa-4th-codes] question_2_mention.pkl怎么生成 (#3)
@duterscmy 模糊匹配加上也效果很差，而且加上之后速度特别慢。想请问一下 entity_mention = corpus[i]['entity_mention']从之前mention_extractor.py生成的entity_mention字段为什么在prop_extractor.py里导入了但是没有使用？是不是因为没有用这个的原因呢？
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
from ccks2019-ckbqa-4th-codes.
ZainZhou
commented on July 22, 2024
@duterscmy 我是在valid上跑的，不是测试集呀
from ccks2019-ckbqa-4th-codes.
ZainZhou
commented on July 22, 2024
@duterscmy 还有一个问题就是在tuple_filter.py中在获取feature时的维度问题。
if len(gold_tuple) == len(set(gold_tuple).intersection(set(t))):
X.append([features[9][0][1]])
Y.append([1])
else:
X.append([features[9][0][1]])
Y.append([0])
在tuple_extractor.py生成的feature里只有['entity','mention',sim2]三个呀
from ccks2019-ckbqa-4th-codes.
ZainZhou
commented on July 22, 2024
@duterscmy answer_bot.py运行会报AttributeError: 'AnswerByPkubase' object has no attribute 'correct_index'
from ccks2019-ckbqa-4th-codes.
duterscmy
commented on July 22, 2024
font{
line-height: 1.6;
}
ul,ol{
padding-left: 20px;
list-style-position: inside;
}
是的..本来有一些相似度特征..后来发现单独用微调后的bert效果就很好了..但是为了方便后续加其他特征代码结构就还是这样写的
在2019年12月8日 00:36，Zain<[email protected]> 写道：
@duterscmy 还有一个问题就是在tuple_filter.py中在获取feature时的维度问题。
if len(gold_tuple) == len(set(gold_tuple).intersection(set(t))):
X.append([features[9][0][1]])
Y.append([1])
else:
X.append([features[9][0][1]])
Y.append([0])
在tuple_extractor.py生成的feature里只有['entity','mention',sim2]三个呀
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.
from ccks2019-ckbqa-4th-codes.
Related Issues (20)
- 文本匹配模型的准确度？
- 请问 keras和keras-bert用的是什么版本？ HOT 1
- neo4j的关系种类数限制是6.5w 请问你是怎么解决的
- 文本匹配模型中dev.csv的构造方式
- 文本匹配模型中dev.csv的构造方式
- 请问train_ner.py里面的corpus_train.pkl和corpus_valid.pkl是怎么得到的呢 HOT 1
- 关于tuple_filter.py中验证集的召回率 HOT 3
- 第一次运行train_ner.py的时候，提示找不到../data/model/ner_model.h5/{saved_model.pbtxt|saved_model.pb} HOT 3
- 有关tuple_filter.py产生的结果指标，一项略低其余正常 HOT 1
- 您好，想问下similarity.py的tf版本和protobuf的版本应该是多少？很多属性值都失效了。 HOT 5
- neo4j查询慢的问题 HOT 3
- 请问pkubase-full.txt可以在哪下载 HOT 8
- tuple_filter.py文件SaveFilterCandiT函数运行error HOT 3
- 该方法是否后续发表论文了？ HOT 2
- segment_dic.txt
- 请问双实体的部分没有代码实现吗？ HOT 1
- 你好，tuple_filter.py用到的LR模型用到的特征，为什么只有len(mention)?bert_score等其他特征不用吗？ HOT 1
- 大佬您好，请问CCKS2019电子病历命名实体识别的数据集有吗 HOT 5
- 请问作者和大家复现时各模块指标都是多少呀？ HOT 8
- 内存炸了，大家都可以跑最后的answer_bot吗？
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ccks2019-ckbqa-4th-codes.