cogroo / cogroo4 Goto Github PK

Some classes at cogroo-nlp org.cogroo.util are not necessary with Java 1.8. We can refactor and remove it.

Consider a text with two wrong sentences

"As casas é bonitas. O médico assistiu ao paciente."

When I execute the follow test I have only one mistake and a java.lang.StringIndexOutOfBoundsException in the method toString of the class Mistake.

package org.cogroo.tools.checker.checkers;

import java.util.ArrayList;
import java.util.List;
import java.util.Locale;

import static junit.framework.Assert.*;

import org.cogroo.analyzer.AnalyzerI;
import org.cogroo.analyzer.ComponentFactory;
import org.cogroo.entities.Mistake;
import org.cogroo.text.Document;
import org.cogroo.text.Sentence;
import org.cogroo.text.impl.DocumentImpl;
import org.junit.BeforeClass;
import org.junit.Test;

public class GovernmentChecker2Test {

private static AnalyzerI analyzer;

@BeforeClass
public static void init() {
    analyzer = ComponentFactory.create(new Locale("pt", "BR")).createPipe();
}

private Sentence[] createTwoWrongSentences() {
    String text = "As casas é bonitas. O médico assistiu ao paciente.";

    Document document = new DocumentImpl();
    document.setText(text);

    analyzer.analyze(document);

    return document.getSentences().toArray(
            new Sentence[document.getSentences().size()]);
}

@Test
public void must_have_two_mistakes() {
    GovernmentChecker checker = new GovernmentChecker();
    Sentence[] sentences = createTwoWrongSentences();
    assertEquals(2, sentences.length);

    List<Mistake> mistakes = new ArrayList<Mistake>();

    for (Sentence sentence : sentences) {
        mistakes.addAll(checker.check(sentence));
    }

    assertEquals(2, mistakes.size());
}


@Test
public void must_print_two_mistakes() {
    GovernmentChecker checker = new GovernmentChecker();
    Sentence[] sentences = createTwoWrongSentences();
    assertEquals(2, sentences.length);


    for (Sentence sentence : sentences) {
        for (Mistake mistake: checker.check(sentence)) {
            assertNotNull(mistake.toString());
        }
    }
  }
}

In the ChunkImpl.java file on line 45 should be used only the tokens from the chunk... right?

22:22:38,175 ERROR Pipe:58 - An analyzer raised an exeception. Analyzer: org.cogroo.analyzer.POSTagger Document text: [@DCrizostomo PARABENX DIVA DO VIVER, vc ahaza felicidades tudo d bom te desejo mt grana, cerveja e bota magya love u]
com.google.common.util.concurrent.UncheckedExecutionException: java.nio.BufferOverflowException
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2258)
at com.google.common.cache.LocalCache.get(LocalCache.java:3990)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3994)
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4878)
at org.cogroo.dictionary.impl.FSADictionary.getTags(FSADictionary.java:120)
at org.cogroo.tools.postag.PortuguesePOSSequenceValidator.queryDictionary(PortuguesePOSSequenceValidator.java:117)
at org.cogroo.tools.postag.PortuguesePOSSequenceValidator.validSequence(PortuguesePOSSequenceValidator.java:83)
at org.cogroo.tools.postag.PortuguesePOSSequenceValidator.validSequence(PortuguesePOSSequenceValidator.java:26)
at opennlp.tools.util.BeamSearch.validSequence(BeamSearch.java:87)
at opennlp.tools.util.BeamSearch.bestSequences(BeamSearch.java:151)
at opennlp.tools.util.BeamSearch.bestSequence(BeamSearch.java:198)
at opennlp.tools.postag.POSTaggerME.tag(POSTaggerME.java:227)
at org.cogroo.analyzer.POSTagger.analyze(POSTagger.java:60)
at org.cogroo.analyzer.Pipe.analyze(Pipe.java:56)
at br.com.post.ProccessPostWorker.run(ProccessPostWorker.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.nio.BufferOverflowException
at java.nio.HeapByteBuffer.put(Unknown Source)
at morfologik.stemming.DictionaryLookup.decodeBaseForm(DictionaryLookup.java:338)
at morfologik.stemming.DictionaryLookup.lookup(DictionaryLookup.java:227)
at org.cogroo.dictionary.impl.FSADictionary.tagLookup(FSADictionary.java:74)
at org.cogroo.dictionary.impl.FSADictionary.access$000(FSADictionary.java:45)
at org.cogroo.dictionary.impl.FSADictionary$1.load(FSADictionary.java:54)
at org.cogroo.dictionary.impl.FSADictionary$1.load(FSADictionary.java:52)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3589)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2374)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2337)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2252)
... 17 more

Com o OpenNLP 1.8.0 podemos fazer diversas mudanças no Chunker, HeadFinder e Shallow Parser.

I'm currently using GrammarChecker class to check grammar mistakes.

I know CoGrOO can also detect mistyped words and make some suggestions. Is it availabe in cogroo4 API?

Preciso utilizar essa versão 4 do cogroo e não tenho acesso a documentação da API pois o link assim descrito no readme.md não está funcionando, gostaria que ou o link ficasse disponviel ou me enviassem por email (se possivel) a documentação
fico muito agradecido, um abraço.

Ao usar a biblioteca versão 4.3.1, com o maven e Java 11, o seguinte erro ocorre:

Caused by: javax.xml.bind.JAXBException: Provider com.sun.xml.internal.bind.v2.ContextFactory not found
 - with linked exception:
[java.lang.ClassNotFoundException: com.sun.xml.internal.bind.v2.ContextFactory]
	at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:149)
	at javax.xml.bind.ContextFinder.find(ContextFinder.java:362)
	at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:446)
	at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:409)
	at org.cogroo.config.LanguageConfigurationUtil.unmarshal(LanguageConfigurationUtil.java:101)
	at org.cogroo.config.LanguageConfigurationUtil.get(LanguageConfigurationUtil.java:71)
	... 8 more
Caused by: java.lang.ClassNotFoundException: com.sun.xml.internal.bind.v2.ContextFactory
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	at javax.xml.bind.ContextFinder.safeLoadClass(ContextFinder.java:578)
	at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:146)
	... 13 more

Acredito que seja por que o jaxb não está mais disponível de forma embutida a partir do Java 9.