narimann2 / parsianalyzer Goto Github PK

Persian Analyzer For Elasticsearch.

Java 100.00%

analysis analyzer elasticsearch information-retrieval java lucene lucene-analyzer persian persian-language stem-persian

parsianalyzer's Issues

Issue with sorting some characters in which start with گ چ پ ژ

When I try to sort my documents, the documents in which their name starts with one of the characters like گ چ پ ژ, the sort does not work correctly.
This is the index which I have created:

PUT index_persian_names_test_with_nariman_analyzer
{
  "mappings": {
    "properties": {
      "name": {
        "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          },
          "analyzer": "persian_custom_analyzer"
      }
    }
  },
  "settings": {
    "index": {
      "number_of_shards": 5,
      "max_result_window": 5000,
      "analysis": {
        "analyzer": {
          "english_custom_analyzer": {
            "filter": [
              "lowercase",
              "decimal_digit"
            ],
            "tokenizer": "classic"
          },
          "persian_custom_analyzer": {
            "filter": [
              "lowercase",
              "decimal_digit",
              "parsi_normalizer"
            ],
            "char_filter": [
              "zero_width_spaces"
            ],
            "type": "custom",
            "tokenizer": "standard"
          }
        },
        "char_filter": {
          "zero_width_spaces": {
            "type": "mapping",
            "mappings": [
              """\u200C => \u0020""",
              """\u200B => \u0020""",
              """\u200D => \u0020""",
              """\u200E => \u0020""",
              """\u200F => \u0020""",
              """\u001F => \u0020""",
              """\u00AC => \u0020"""
            ]
          }
        }
      },
      "number_of_replicas": 0
    }
  }
}

I've added these documents:

POST index_persian_names_test_with_nariman_analyzer/_doc
{
  "name": "کرگدن"
}


POST index_persian_names_test_with_nariman_analyzer/_doc
{
  "name": "فیل"
}


POST index_persian_names_test_with_nariman_analyzer/_doc
{
  "name": "پاندا"
}


POST index_persian_names_test_with_nariman_analyzer/_doc
{
  "name": "قناری"
}


POST index_persian_names_test_with_nariman_analyzer/_doc
{
  "name": "گراز وحشی"
}


POST index_persian_names_test_with_nariman_analyzer/_doc
{
  "name": "ژیان"
}

POST index_persian_names_test_with_nariman_analyzer/_doc
{
  "name": "یوزپلنگ"
}

And finally when I try to see the sorted results, the document in which starts with پ must come at first, but it does not.

GET index_persian_names_test_with_nariman_analyzer/_search
{
  "query": {
    "match_all": {
      
    }
  },
  "sort": [
    {
      "name.keyword": {
        "order": "asc"
      }
    }
  ]
}

Here is the result:

{
  "took" : 624,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 7,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "index_persian_names_test_with_nariman_analyzer",
        "_type" : "_doc",
        "_id" : "F0qtn4oBMhBe8matcKHy",
        "_score" : null,
        "_source" : {
          "name" : "فیل"
        },
        "sort" : [
          "فیل"
        ]
      },
      {
        "_index" : "index_persian_names_test_with_nariman_analyzer",
        "_type" : "_doc",
        "_id" : "30qtn4oBMhBe8matgqHw",
        "_score" : null,
        "_source" : {
          "name" : "قناری"
        },
        "sort" : [
          "قناری"
        ]
      },
      {
        "_index" : "index_persian_names_test_with_nariman_analyzer",
        "_type" : "_doc",
        "_id" : "3Eqtn4oBMhBe8mateKHt",
        "_score" : null,
        "_source" : {
          "name" : "پاندا"
        },
        "sort" : [
          "پاندا"
        ]
      },
      {
        "_index" : "index_persian_names_test_with_nariman_analyzer",
        "_type" : "_doc",
        "_id" : "pkqtn4oBMhBe8matn6Jp",
        "_score" : null,
        "_source" : {
          "name" : "ژیان"
        },
        "sort" : [
          "ژیان"
        ]
      },
      {
        "_index" : "index_persian_names_test_with_nariman_analyzer",
        "_type" : "_doc",
        "_id" : "FUqtn4oBMhBe8matXqGp",
        "_score" : null,
        "_source" : {
          "name" : "کرگدن"
        },
        "sort" : [
          "کرگدن"
        ]
      },
      {
        "_index" : "index_persian_names_test_with_nariman_analyzer",
        "_type" : "_doc",
        "_id" : "4Eqtn4oBMhBe8mati6HX",
        "_score" : null,
        "_source" : {
          "name" : "گراز وحشی"
        },
        "sort" : [
          "گراز وحشی"
        ]
      },
      {
        "_index" : "index_persian_names_test_with_nariman_analyzer",
        "_type" : "_doc",
        "_id" : "qUqtn4oBMhBe8matpqKq",
        "_score" : null,
        "_source" : {
          "name" : "یوزپلنگ"
        },
        "sort" : [
          "یوزپلنگ"
        ]
      }
    ]
  }
}

I will be grateful for your help...

Error Installing Plugin for Elasticsearch 7.7

Hi, I tried to install this Analyzer for Elasticsearch 7.7 but it throws an error as below:

It needs to update to 7.7 version, I think.
Update it please, so we can use it.
Thanks

Is it possible to use Jalali calendar instead of Gregorian calendar?

Hi. What is the best way to use Jalali calendar instead of Gregorian calendar in opensearch? I found out Locale option for date processor in pipelines... Does it change calendar too? Or do you think it's better to write a plugin for that?
@NarimanN2

Use filters independently

I need to use each filters inside parsi analyzer independently in a custom analyzer. In fact I need to extend parsi analyzer in custom way. Is there a way to achieve that?

So what about English standard analyzer and parsi use together?!

how can use this analyzer and English analyzer at the same ?

and what about elasticsearch 5.6.12 ?

Not compatible

hi,
Not compatible with elastics search 8.9.0 .
please update

Compatibility issue

Not compatible with elastics search 8.8.1

Compatibility for elastic 8.3.3

Hi,
I was wondering could you update this useful repo and make it work for elastic 8.3.3?

Support elastic 7.2

Do you plan to support elastic 7.2? It's a good plugin but 6.4.2 is old and we want to upgrade our elastic to 7.2. Could you please add support for this version?

Error Installing with Elasticsearch 6.4.3

Hi community, I have changed the Elasticsearch dependency to 6.4.3 but on install, i get this error below:

Exception in thread "main" java.nio.file.NoSuchFileException: /usr/share/elasticsearch/plugins/.installing-2138313496221851015/plugin-descriptor.properties
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:215)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:369)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:415)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
        at java.base/java.nio.file.Files.newInputStream(Files.java:154)
        at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:162)
        at org.elasticsearch.plugins.InstallPluginCommand.loadPluginInfo(InstallPluginCommand.java:713)
        at org.elasticsearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:792)
        at org.elasticsearch.plugins.InstallPluginCommand.install(InstallPluginCommand.java:775)
        at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:231)
        at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:216)
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
        at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:77)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
        at org.elasticsearch.cli.Command.main(Command.java:90)
        at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:47)

And here is my plugin-descriptor.properties:

description=Persian analyzer for elasticsearch.
version=1.0
name=ParsiAnalyzer
classname=org.elasticsearch.analyzer.ParsiAnalyzerPlugin
java.version=1.8
elasticsearch.version=6.4.3

Installation file link(zip): https://drive.google.com/open?id=1ac-cPCRqCPXcTrN8xBKUJ_MLLr5ZfOSK
And i have followed this:

1- Download the latest source code
2- Open pom.xml file
3- Under dependecies tag, change elasticserach version from 5.6.3 to 6.1.2 and save it
4- Open plugin-descriptor.properties file
5- Change elasticsearch.version from 6.4.2 to 5.6.13
6- Run this maven command: mvn clean package -DskipTests
7- In the folder target/releases you’ll now find a zip file called ParsiAnalyzer. install the plugin using this command:
bin/elasticsearch-plugin install file:///path/to/ParsiAnalyzer.zip

Notice: I'm using Dockerized Elasticsearch.

What's wrong with my configs?

error when install

C:\elastic\elasticsearch-6.3.2>bin\elasticsearch-plugin install https://www.dropbox.com/s/2sczcj1372xsq7k/ParsiAnalyzer-%206.3.2.zip?dl=1
-> Downloading https://www.dropbox.com/s/2sczcj1372xsq7k/ParsiAnalyzer- 6.3.2.zip?dl=1
[=================================================] 100%بب
ERROR: This plugin was built with an older plugin structure. Contact the plugin author to remove the intermediate "elasticsearch" directory within the plugin zip.

ERROR: Unknown plugin ParsiAnalyzer-1.0-SNAPSHOT.zip

I've followed the build steps for version 7.17.4 and built the ParsiAnalyzer-1.0-SNAPSHOT.zip under the target/releases folder. When I try to install it by elasticsearch-plugin, it fails:

-> Installing /path/to/ParsiAnalyzer/target/releases/ParsiAnalyzer-1.0-SNAPSHOT.zip
-> Failed installing /path/to/ParsiAnalyzer/target/releases/ParsiAnalyzer-1.0-SNAPSHOT.zip
-> Rolling back /path/to/ParsiAnalyzer/target/releases/ParsiAnalyzer-1.0-SNAPSHOT.zip
-> Rolled back /path/to/ParsiAnalyzer/target/releases/ParsiAnalyzer-1.0-SNAPSHOT.zip
A tool for managing installed elasticsearch plugins

Non-option arguments:
command              

Option             Description        
------             -----------        
-E <KeyValuePair>  Configure a setting
-h, --help         Show help          
-s, --silent       Show minimal output
-v, --verbose      Show verbose output

ERROR: Unknown plugin /path/to/ParsiAnalyzer/target/releases/ParsiAnalyzer-1.0-SNAPSHOT.zip

Any idea about this problem?

narimann2 / parsianalyzer Goto Github PK

parsianalyzer's Issues

Issue with sorting some characters in which start with گ چ پ ژ

Error Installing Plugin for Elasticsearch 7.7

Is it possible to use Jalali calendar instead of Gregorian calendar?

Use filters independently

So what about English standard analyzer and parsi use together?!

Not compatible

Compatibility issue

Compatibility for elastic 8.3.3

Support elastic 7.2

Error Installing with Elasticsearch 6.4.3

error when install

ERROR: Unknown plugin ParsiAnalyzer-1.0-SNAPSHOT.zip

Error Installing Plugin for Elasticsearch 7.9.2

Support elastic search 7.11

change & compile

incompatible with elasticsearch 6.1

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent