nocoolsandwich / iamqa Goto Github PK
View Code? Open in Web Editor NEW中文wiki百科QA阅读理解问答系统,使用了CCKS2016数据的NER模型和CMRC2018的阅读理解模型,还有W2V词向量搜索,使用torchserve部署
中文wiki百科QA阅读理解问答系统,使用了CCKS2016数据的NER模型和CMRC2018的阅读理解模型,还有W2V词向量搜索,使用torchserve部署
torch-model-archiver --model-name W2V --version 1.0 --serialized-file ./W2V --handler ./Transformer_handler_generalized.py
其中serialized-file ./W2V 是不是不太对啊,W2V目录下又来个W2V?
请问reader的这个pth文件是如何训练的,有没有代码github链接呀,我想自己换个模型试着训练一下
ava.lang.NoSuchMethodError: java.nio.file.Files.readString(Ljava/nio/file/Path;)Ljava/lang/String;
请问是我的java环境有问题吗
你这个ner的模型 Transformer_handler_generalized.py 里面 为啥预处理就直接返回识别的实体结果了 跑了后也出不来结果呢
Traceback (most recent call last):
File "to_pickle.py", line 4, in
W2V = gensim.models.KeyedVectors.load_word2vec_format(model_pt_path,binary = False,unicode_errors='ignore')
File "/usr/local/lib/python3.6/dist-packages/gensim/models/keyedvectors.py", line 1438, in load_word2vec_format
limit=limit, datatype=datatype)
File "/usr/local/lib/python3.6/dist-packages/gensim/models/utils_any2vec.py", line 172, in _load_word2vec_format
header = utils.to_unicode(fin.readline(), encoding=encoding)
File "/usr/local/lib/python3.6/dist-packages/gensim/utils.py", line 355, in any2unicode
return unicode(text, encoding, errors=errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf7 in position 12: invalid start byte
尝试忽略错误,依然报错
import gensim,pickle
model_pt_path = r'sgns.merge.word'
W2V = gensim.models.KeyedVectors.load_word2vec_format(model_pt_path,binary = False,unicode_errors='ignore')
#W2V = gensim.models.Word2Vec.load(model_pt_path)
with open(r'W2V.pickle','wb') as f:
pickle.dump(W2V,f)
请问如何解决呢?
库的版本和requirement.txt基本上完全一样
方便加vx嘛
感谢分享,请问下torchserve用的是什么版本,也麻烦重新更新下requirements
你好,win下怎么启动web服务呀
发送了一条query,得到返回结果是
[{"code": 404, "type": "ModelNotFoundException", "message": "Model not found: reader", "index": 1}]
这是什么原因?
您好,想问一下出现下面的问题如何解决呢?
环境:
Linux gpu
Python 3.6.9
Successfully installed torch-1.7.1+cu101 torchaudio-0.7.2 torchtext-0.8.1 torchvision-0.8.2+cu101
其他基本上都是按照requirement里面安装的
报错日志:
2021-02-06 11:24:43,604 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model W2V
2021-02-06 11:24:43,605 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model W2V
2021-02-06 11:24:43,605 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model W2V loaded.
2021-02-06 11:24:43,605 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: W2V, count: 1
2021-02-06 11:24:43,611 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2021-02-06 11:24:43,639 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:9080
2021-02-06 11:24:43,640 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: EpollServerSocketChannel.
2021-02-06 11:24:43,642 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://0.0.0.0:9081
2021-02-06 11:24:43,642 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2021-02-06 11:24:43,646 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://0.0.0.0:9082
2021-02-06 11:24:43,832 [INFO ] W-9002-W2V_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /tmp/.ts.sock.9002
2021-02-06 11:24:43,833 [INFO ] W-9002-W2V_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]1548
2021-02-06 11:24:43,834 [INFO ] W-9002-W2V_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2021-02-06 11:24:43,834 [DEBUG] W-9002-W2V_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-W2V_1.0 State change null -> WORKER_STARTED
2021-02-06 11:24:43,834 [INFO ] W-9002-W2V_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9002
2021-02-06 11:24:43,834 [INFO ] W-9002-W2V_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.9
2021-02-06 11:24:43,839 [INFO ] W-9002-W2V_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /tmp/.ts.sock.9002.
Model server started.
2021-02-06 11:24:44,414 [ERROR] Thread-1 org.pytorch.serve.metrics.MetricCollector - Traceback (most recent call last):
File "ts/metrics/metric_collector.py", line 10, in
from ts.metrics import system_metrics
File "/usr/local/lib/python3.6/dist-packages/ts/init.py", line 10, in
from . import version
File "/usr/local/lib/python3.6/dist-packages/ts/version.py", line 7, in
version_path = os.path.join(Path(file).resolve().parent, 'version.txt')
File "/usr/lib/python3.6/posixpath.py", line 80, in join
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not PosixPath
2021-02-06 11:26:49,524 [INFO ] Thread-0 org.pytorch.serve.wlm.ModelManager - Model NER unregistered.
2021-02-06 11:26:49,524 [INFO ] Thread-0 org.pytorch.serve.ModelServer - Unregistering model W2V version 1.0
2021-02-06 11:26:49,524 [DEBUG] Thread-0 org.pytorch.serve.wlm.ModelVersionedRefs - Removed model: W2V version: 1.0
2021-02-06 11:26:49,525 [DEBUG] Thread-0 org.pytorch.serve.wlm.WorkerThread - W-9002-W2V_1.0 State change WORKER_STARTED -> WORKER_SCALED_DOWN
2021-02-06 11:26:49,525 [WARN ] Thread-0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9002-W2V_1.0-stderr
2021-02-06 11:26:49,525 [WARN ] Thread-0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9002-W2V_1.0-stdout
2021-02-06 11:26:49,525 [DEBUG] Thread-0 org.pytorch.serve.job.Job - Waiting time ns: 0, Inference time ns: 94509988
2021-02-06 11:26:49,525 [DEBUG] W-9002-W2V_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_SCALED_DOWN
2021-02-06 11:26:49,525 [INFO ] epollEventLoopGroup-5-2 org.pytorch.serve.wlm.WorkerThread - 9002 Worker disconnected. WORKER_SCALED_DOWN
2021-02-06 11:26:49,526 [DEBUG] W-9002-W2V_1.0 org.pytorch.serve.wlm.WorkerThread - Shutting down the thread .. Scaling down.
2021-02-06 11:26:49,527 [WARN ] W-9002-W2V_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: W2V, error: Worker died.
2021-02-06 11:26:49,527 [DEBUG] W-9002-W2V_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-W2V_1.0 State change WORKER_SCALED_DOWN -> WORKER_STOPPED
2021-02-06 11:26:49,527 [WARN ] W-9002-W2V_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9002-W2V_1.0-stderr
2021-02-06 11:26:49,528 [WARN ] W-9002-W2V_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9002-W2V_1.0-stdout
2021-02-06 11:26:49,528 [DEBUG] W-9002-W2V_1.0 org.pytorch.serve.wlm.WorkerThread - Worker terminated due to scale-down call.
2021-02-06 11:26:49,531 [INFO ] W-9002-W2V_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9002-W2V_1.0-stderr
2021-02-06 11:26:49,531 [INFO ] W-9002-W2V_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9002-W2V_1.0-stdout
2021-02-06 11:26:49,597 [INFO ] Thread-0 org.pytorch.serve.wlm.ModelManager - Model W2V unregistered.
root@7a74d96d6c7c:/tmp/pycharm_project_707# torchserve --start --ts-config config.properties --model-store model_store \
--models reader=reader.mar,NER=NER.mar,W2V=W2V.mar
Removing orphan pid file.
root@7a74d96d6c7c:/tmp/pycharm_project_707# java.lang.NoSuchMethodError: java.nio.file.Files.readString(Ljava/nio/file/Path;)Ljava/lang/String;
at org.pytorch.serve.util.ConfigManager.readFile(ConfigManager.java:240)
at org.pytorch.serve.util.ConfigManager.(ConfigManager.java:144)
at org.pytorch.serve.util.ConfigManager.init(ConfigManager.java:290)
at org.pytorch.serve.ModelServer.main(ModelServer.java:83)
你好,请问reader文件夹下inference_test.py文件中读取了bow.pkl文件,在相关项目中没有找到这个文件,可否分享一下?感谢
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.