wenx999 / lamapacos Goto Github PK

sample....
Branch name:

Purpose of code changes on this branch:


When reviewing my code changes, please focus on:


After the review, I'll merge this branch into:
/trunk

Branch name:0.0.1

Purpose of code changes on this branch:
    delete useless jar dependencies.....
           & xyy.java

When reviewing my code changes, please focus on:
   pom.xml

After the review, I'll merge this branch into:
/trunk

Branch name:0.0.1

Purpose of code changes on this branch:
   1:Integrate nutch, we use nutch to crawl data from internet.
     1>add dependencies to pom.xml
     2>configuration in conf/
     3>plugins of nutch in plugins/
     4>lib
  2:NutchRunner is a try to call nutch's crawl API;

When reviewing my code changes, please focus on:
  NutchRunner.java & pom.xml & conf/ & lib/ & plugins/

After the review, I'll merge this branch into:
/trunk

What steps will reproduce the problem?
1. Run NLPIRTokenizerTest
2. You may encounter the following error:
java.lang.UnsatisfiedLinkError: no NLPIR_JNI in java.library.path
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)
    at java.lang.Runtime.loadLibrary0(Runtime.java:845)
    at java.lang.System.loadLibrary(System.java:1084)
    at kevin.zhang.NLPIR.<clinit>(NLPIR.java:126)
    at org.lamapacos.preprocessor.extraction.NLPIRTokenizer.segment(NLPIRTokenizer.java:18)
    at org.lamapacos.prepocessor.extraction.NLPIRTokenizerTest.testSegment(NLPIRTokenizerTest.java:31)
        …………………………
3. Or you may go through this test(as ShaoLin Xiao does)


It is strange if "java.library.path" is computer-different, since if so we have 
to set this Path properly computer by computer.


You may temporarily solve this problem by setting the VM arguments:
  -Djava.library.path=/home/hadoop/workspace/lamapacos/lib  #the correct path

FYI:http://blog.csdn.net/larrylgq/article/details/7515362

Though it's unnecessary to distinguish all tokens from each other, we must 
detect punctuations from a record so as to analyse its intensity properly.

FYI:
   [1]:http://hi.baidu.com/sing520/item/150f385de1ce0511db1635d4
   [2]:http://www.docin.com/p-381420600.html