dnanhkhoa / python-vncorenlp Goto Github PK

When I run the code:
vncorenlp = VnCoreNLP('path/to/VnCoreNLP-1.0.1.jar')
in python shell, I got this output and this command never end.

I have tried your example code but there was an issue

File "<stdin>", line 2, in <module>
  File "/home/cpu10508-local/.virtualenvs/dl4cv/lib/python3.5/site-packages/vncorenlp/vncorenlp.py", line 53, in __init__
    raise FileNotFoundError('Java was not found, please install JRE or JDK 1.8 first.')

I have installed java though

>>> import os
>>> os.system("java -version")
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-0ubuntu0.16.04.1-b11)
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)
0

Hey man, nice work! Just for you to double check the file VnCoreNLPServer.jar is probably corrupted.

I got this problem when tokenizing data using vncorenlp python, and it work fine until it reach 1 of the line in the data file.

org.eclipse.jetty.http.BadMessageException: 400: Unable to parse form content at org.eclipse.jetty.server.Request.getParameters(Request.java:380) at org.eclipse.jetty.server.Request.getParameter(Request.java:1021) at javax.servlet.ServletRequestWrapper.getParameter(ServletRequestWrapper.java:194) at spark.Request.queryParams(Request.java:283) at spark.http.matching.RequestWrapper.queryParams(RequestWrapper.java:141) at vncorenlp.VnCoreNLPServer.handle(VnCoreNLPServer.java:247) at vncorenlp.VnCoreNLPServer.lambda$3(VnCoreNLPServer.java:184) at spark.ResponseTransformerRouteImpl$1.handle(ResponseTransformerRouteImpl.java:47) at spark.http.matching.Routes.execute(Routes.java:61) at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:130) at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1568) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:530) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: java.lang.IllegalStateException: Form too large: 250544 > 200000 at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:523) at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:461) at org.eclipse.jetty.server.Request.getParameters(Request.java:376)

Best regards

Hi,
I am facing an issue with the VnCoreNLP server (the Java server) when I create multiple requests using goroutines. It would be great if you could please take a look:

ERROR VnCoreNLPServer - null
java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
at vncorenlp.VnCoreNLPServer.annotate(VnCoreNLPServer.java:237)
at vncorenlp.VnCoreNLPServer.handle(VnCoreNLPServer.java:270)
at vncorenlp.VnCoreNLPServer.lambda$3(VnCoreNLPServer.java:184)
at spark.ResponseTransformerRouteImpl$1.handle(ResponseTransformerRouteImpl.java:47)
at spark.http.matching.Routes.execute(Routes.java:61)
at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:130)
at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1568)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:530)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 10
at marmot.util.Encoder.append(Encoder.java:68)
at marmot.morph.MorphWeightVector.extractTransitionFeatures(MorphWeightVector.java:584)
at marmot.core.SimpleTagger.addTransitions(SimpleTagger.java:73)
at marmot.core.SimpleTagger.getSumLattice(SimpleTagger.java:407)
at marmot.core.SimpleTagger.tag_states(SimpleTagger.java:565)
at marmot.morph.MorphTagger.tagWithLemma(MorphTagger.java:49)
at vn.corenlp.postagger.PosTagger.tagSentence(PosTagger.java:52)
at vn.pipeline.Sentence.createWords(Sentence.java:60)
at vn.pipeline.Sentence.init(Sentence.java:53)
at vn.pipeline.Sentence.(Sentence.java:30)

The server doesn't serve big text and constantly returns this message: WARN HttpParser - URI is too large >8192

I'm starting server using this line

vncorenlp -Xmx2g ../out/VnCoreNLP-1.1.jar -p 9000 -a "wseg"

However, I got this error
2019-05-04 01.57.19 INFO VnCoreNLPServer - Using annotators: wseg 2019-05-04 01:57:19 INFO WordSegmenter:24 - Loading Word Segmentation model 2019-05-04 01.57.19 ERROR VnCoreNLPServer - null java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at vncorenlp.VnCoreNLPServer.loadVnCoreNLP(VnCoreNLPServer.java:79) at vncorenlp.VnCoreNLPServer.main(VnCoreNLPServer.java:175) Caused by: java.io.IOException: WordSegmenter: ../out/models/wordsegmenter/wordsegmenter.rdr is not found! at vn.corenlp.wordsegmenter.WordSegmenter.<init>(WordSegmenter.java:27) at vn.corenlp.wordsegmenter.WordSegmenter.initialize(WordSegmenter.java:34) ... 6 more

When I submit multiple requests async to VnCoreNLPServer I got ArrayIndexOutOfBoundsException: -1

2019-11-05 15:39:37,233 [qtp950350040-18] ERROR vncorenlp.VnCoreNLPServer.handle(VnCoreNLPServer.java:278) - -1
java.lang.ArrayIndexOutOfBoundsException: -1
	at marmot.util.Encoder.append(Encoder.java:68)
	at marmot.morph.MorphWeightVector.extractStateFeatures(MorphWeightVector.java:366)
	at marmot.core.SimpleTagger.getStates(SimpleTagger.java:181)
	at marmot.core.SimpleTagger.getSumLattice(SimpleTagger.java:322)
	at marmot.core.SimpleTagger.tag_states(SimpleTagger.java:565)
	at marmot.morph.MorphTagger.tagWithLemma(MorphTagger.java:49)
	at vn.corenlp.postagger.PosTagger.tagSentence(PosTagger.java:52)
	at vn.pipeline.Sentence.createWords(Sentence.java:60)
	at vn.pipeline.Sentence.init(Sentence.java:53)
	at vn.pipeline.Sentence.<init>(Sentence.java:30)
	at vncorenlp.VnCoreNLPServer.annotate(VnCoreNLPServer.java:236)
	at vncorenlp.VnCoreNLPServer.handle(VnCoreNLPServer.java:269)
	at vncorenlp.VnCoreNLPServer.lambda$main$2(VnCoreNLPServer.java:178)
	at spark.ResponseTransformerRouteImpl$1.handle(ResponseTransformerRouteImpl.java:47)
	at spark.http.matching.Routes.execute(Routes.java:61)
	at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:134)
	at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1671)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:505)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)
	at java.lang.Thread.run(Thread.java:748)

Hi,

Thank for this awesome toolkit @dnanhkhoa ,

Could you please show me how to run your toolkit as a service so I can send request to it when using python?

I follow the instruction in this link but when I enter the command vncorenlp -Xmx2g E:\Data\Projects\MobiFone5\NLP\VnCoreNLP-master\VnCoreNLP-1.1.jar -p 9000 -a "wseg,pos,ner,parse" into command prompt, it returns 'vncorenlp' is not recognized as an internal or external command, operable program or batch file.

How can I run this command properly?