geekusa / nlp-text-analytics Goto Github PK

Not currently compatible/vetted with Splunk Cloud, though I did manage to get an older version installed on a different instance one, and would really love to be able to use it.

Hi,
I am doing some experiments with this add-on and have tried to perform some sentiment analysis with the "vader" command.
When I perform this test on English sentences it works flawlessly, but when I replace the sentence with Bulgarian, then it doesn't. No errors are found in the splunkd.log or the mlspl.log files.

Here are the two searches:
| makeresults count=1 | eval text="This one works very well." | vader textfield=text
^ this one work fine and returns expected results.

| makeresults count=1 | eval text="Нещо не работи като хората и това не ми харесва." | vader textfield=text
^ this one does not work and returns no results (actually the search never finishes).

Same issue is experienced with other commands from the NLP add-on, e.g. cleantext.
Performing normal SPL searches (both in English and Bulgarian) are working fine on the environment.
The environment actually is all-in-one Splunk Enterprise 8.1.3 server on 64-bit Kali Linux. Python is v.2.7.18.

Any ideas?

Hi @geekusa,

I would like to know what is the reasoning behind removing similar data (with a score of 1 in similarity and 0 in distance) from the output of the similarity command :

if t != c:
   result = self.algo_select(t, c, transposition, set_algo, algo)
   if 'edit' in algo:
         compare_dict[t+'>'+c] = (
         result,
         self.distance_to_ratio(result, len(t), len(c))
          )
   else:
         compare_dict[t+'>'+c] = result

It can be found at
https://github.com/geekusa/nlp-text-analytics/blob/master/bin/similarity.py#L215

If it is not intended I have a pull request ready to remove it :)

Running Splunk 7.2.0
When using Counts and clicking on "Top Parts-of-Speech" or "Top Terms", new sub-panel with title "$row.sentence$ Occurences"