Comments (23)
Most likely unrelated, this also fails:
sbt 'corenlp/testOnly org.clulab.processors.TestParallel'
...
[info] - should match processing documents serially *** FAILED ***
[info] java.util.NoSuchElementException: None.get
[info] at scala.None$.get(Option.scala:529)
[info] at scala.None$.get(Option.scala:527)
[info] at org.clulab.utils.ToEnhancedDependencies$.$anonfun$collapsePrepositionsUniversalDueTo$1(ToEnhancedDependencies.scala:195)
[info] at org.clulab.utils.ToEnhancedDependencies$.$anonfun$collapsePrepositionsUniversalDueTo$1$adapted(ToEnhancedDependencies.scala:194)
[info] at scala.collection.immutable.List.foreach(List.scala:431)
[info] at scala.collection.generic.TraversableForwarder.foreach(TraversableForwarder.scala:38)
[info] at scala.collection.generic.TraversableForwarder.foreach$(TraversableForwarder.scala:38)
[info] at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:47)
[info] at org.clulab.utils.ToEnhancedDependencies$.collapsePrepositionsUniversalDueTo(ToEnhancedDependencies.scala:194)
[info] at org.clulab.utils.ToEnhancedDependencies$.collapsePrepositionsUniversal(ToEnhancedDependencies.scala:145)
[info] ...
[info] Run completed in 5 seconds, 968 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 0, failed 1, canceled 0, ignored 0, pending 0
[info] *** 1 TEST FAILED ***
[error] Failed tests:
[error] org.clulab.processors.TestParallel
from processors.
@MihaiSurdeanu, the processors branch balaur is not compiling for me. Is there a commit that hasn't made it to github? The latest there is 48a754 and it looks incomplete.
from processors.
Sorry! Yes, the headAndLabels
branch in scala-transformers must be published locally first.
from processors.
I had done that, but the problem seems to be in the processors code :-(
from processors.
Hmm. Then I'm not sure. Can you please paste in the error?
from processors.
I've seen this on two separate computers now, so I don't think it's a fluke. The problem might be that there are two Eisner files and they are getting mixed up.
[error] C:\Users\kwa\MyData\Projects\clulab\processors-project\processors\main\src\main\scala\org\clulab\processors\clu\CluProcessor.scala:452:34: value ensembleParser is not a member of org.clulab.processors.clu.Eisner
[error] val headsWithLabels = eisner.ensembleParser(
[error]
[warn] C:\Users\kwa\MyData\Projects\clulab\processors-project\processors\main\src\main\scala\org\clulab\processors\clu\CluProcessor.scala:19:93: imported `Eisner` is permanently hidden by definition of object Eisner in package clu
[warn] import org.clulab.dynet.{AnnotatedSentence, ConstEmbeddingParameters, ConstEmbeddingsGlove, Eisner, Metal, ModifierHeadPair}
[warn] ^
[warn] C:\Users\kwa\MyData\Projects\clulab\processors-project\processors\main\src\main\scala\org\clulab\processors\clu\CluProcessor.scala:4:93: imported `Eisner` is permanently hidden by definition of object Eisner in package clu
[warn] import org.clulab.dynet.{AnnotatedSentence, ConstEmbeddingParameters, ConstEmbeddingsGlove, Eisner, Metal, ModifierHeadPair, Utils}
[warn] ^
[warn] two warnings found
from processors.
Interesting. I don't see this error on my Mac or a linux box.
In any case, I renamed one of the classes to avoid the conflict.
Can you please pull the branch and try again?
Thanks!
from processors.
On the first one, there is an edge from -2 to 83. In EisnerEnsembleParser.generateOutput there is a case where Eisner fails, so there is a reversion to the greedy inference. A bestDep of -47 is found when i = 45. head ends up being -2. The programming here probably needs to be more defensive. Let me know if I should mess with the code.
from processors.
On the second problem, it looks like ToEnhancedDependencies.collapsePrepositionsUniversalDueTo is failing because it needs lemmas and lemmatize has apparently not yet been run on the document.
from processors.
Please go ahead and "mess with the code" :)
The greedy inference is supposed to choose only valid heads that are either -1 (root of the tree) or between 0 and sentence length:
from processors.
OK. That line 25 is just before depHead - 1, which results in the -2.
from processors.
I'm trying to understand the nearby code. In PostProcessor.parserPostProcessing it looks like you are trying to account for "due to" and wanting to change the headsWithLabels(i + 1) to (i, "mwe"), but only if that's not already the case. If so, the condition at
might need to have an || instead:
(headsWithLabels(i + 1)._1 != i || headsWithLabels(i + 1)._2 != "mwe")
from processors.
I think you're right. Nice catch!
Of course, we need parens around the disjunction.
from processors.
The code is complicated enough to be hiding problems. For example, the value
is already an int not needing conversion and the exception handler
should probably be guarding the line all the way over here:
from processors.
@MihaiSurdeanu, when BalaurProcessor.assignDependencyLabels gets called, are the original headLabels coming out of the neural network using absolute or relative values? It looks like relative because of
but then absolute because of
from processors.
Yes, the ML predicts relative head positions (the classifier generalizes much better with relative positions). The code in the processor then converts them to absolute values, and then construct a DependencyGraph.
from processors.
The code is complicated enough to be hiding problems. For example, the value
is already an int not needing conversion and the exception handler
should probably be guarding the line all the way over here:
I think you're right. Thanks for catching it!
from processors.
@MihaiSurdeanu, the model 0.0.4 should be available now. Next time I will be daring and number it 0.1.0!
from processors.
Thank you!
from processors.
I hope to ask tomorrow about some of this code, because there are too many words (for me) for different stages of the same numbers. Here's a code question, though. In
relHead being 0 has special meaning, because something can't be its own head. However, mod + relHead can also be zero, I think, and that doesn't mean quite the same thing: mod + relHead == 0 => i + 1 + relHead == 0 => i + relHead == -1 => absHead == -1 which is invalid. It might be OK to say that in that case it goes to root just like an actual relHead of 0 would. However, then if mod + relHead == -1 => absHead == -2, the same going to root option is not taken because the condition in the second line is not satisfied.
If that is the case, I have an urge to rearrange some things so as not to get into that situation.
from processors.
Let's discuss in our meeting today!
from processors.
Yet another question... AFAICT, the data coming into assignDependencyLabels for the heads and labels have heads all originating from the tasks.3.labels file. These are 160 values ranging from -127 to +77 with some gaps. There are no duplicates. That seems like a constant. I don't think there should be any issue converting any of the heads to integers so that the exception handling is not necessary. If the values are distinct, then code like
would not seem necessary, either. For this particular task it looks like the scores are already delivered sorted so that
is similarly unnecessary. I'll see if it works without these things.
from processors.
The older classifier based on dynet could generate some strings labels. But this is no longer true with transformers. So that exception is no longer needed.
You're probably correct on the sortBy
as well. At some point I wasn't sorting in scala_transformers. But now we do.
from processors.
Related Issues (20)
- Normalize "grand" HOT 7
- Unreliable tests HOT 2
- The webapp messes up the project in IntelliJ HOT 3
- Preferred size of graphs is not set upon Document deserialization HOT 3
- Hashing is inconsistent HOT 2
- equivalenceHash makes use of hashCode
- pounds as measurements HOT 1
- Paths are written for Mentions that are non-arguments but not read back in
- Update dependency on scala-collection-compat HOT 1
- The Core NLP processors don't understand Easter or Mother's Day and throws an exception HOT 8
- Make unit tests compile without dynet HOT 4
- "Due to", the exceptional case handled in PostProcessor HOT 2
- Java interface for processors HOT 5
- Balaur tests need to pass before it can be merged into master. HOT 7
- Date rule doesn't match well with action HOT 3
- WEEK.tsv better handled in code
- Adverse effect of processors v9 on eidos HOT 5
- Quote is shoved to next sentence
- TokenPattern.assignIds never visits the start of MatchLookAhead or MatchLookBehind
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from processors.