The vnlp from vngrs-ai

vnlp's Introduction

VNLP: Turkish NLP Tools

State-of-the-art, lightweight NLP tools for Turkish language.

Developed by VNGRS.

https://vngrs.com/

Functionality:

Sentence Splitter
Normalizer
- Spelling/Typo correction
- Convert numbers to word form
- Deasciification
Stopword Remover:
- Static
- Dynamic
Stemmer: Morphological Analyzer & Disambiguator
Named Entity Recognizer (NER)
Dependency Parser
Part of Speech (PoS) Tagger
Sentiment Analyzer
Turkish Word Embeddings
- FastText
- Word2Vec
- SentencePiece Unigram Tokenizer
News Summarization
News Paraphrasing
Summarization and Paraphrasing models are available in the demo. Contact us at [email protected] for API.

Demo:

Try the Demo.

Installation

pip install vngrs-nlp

Documentation:

See the Documentation for the details about usage, classes, functions, datasets and evaluation metrics.

Metrics:

Usage Example:

Dependency Parser

from vnlp import DependencyParser
dep_parser = DependencyParser()

dep_parser.predict("Oğuz'un kırmızı bir Astra'sı vardı.")
[("Oğuz'un", 'PROPN'),
 ('kırmızı', 'ADJ'),
 ('bir', 'DET'),
 ("Astra'sı", 'PROPN'),
 ('vardı', 'VERB'),
 ('.', 'PUNCT')]

# Spacy's submodule Displacy can be used to visualize DependencyParser result.
import spacy
from vnlp import DependencyParser
dependency_parser = DependencyParser()
result = dependency_parser.predict("Oğuz'un kırmızı bir Astra'sı vardı.", displacy_format = True)
spacy.displacy.render(result, style="dep", manual = True)

Citation

@article{turker2024vnlp,
  title={VNLP: Turkish NLP Package},
  author={Turker, Meliksah and Ari, Erdi and Han, Aydin},
  journal={arXiv preprint arXiv:2403.01309},
  year={2024}
}

vnlp's People

Contributors

Stargazers

Watchers

vnlp's Issues

Fix Python 3.10 compatibility

install_requires in setup.py suggests that the package works with python_version < "3.8" and python_version >= "3.8" but the package is not compatible with Python 3.10 due to dependency cython_hunspell that uses cacheman package.

The problem is that in Python 3.10, MutableMapping moved from collections to collections.abc.

You can see that MutableMapping is still being imported from collections in this file.

As far as I can see said packages are not maintained properly. So maybe migrating away from them would be a solution. For the time being, the only solution other than fixing the error by hand is downgrading the Python version, which is not feasible in the long run IMHO.

To Reproduce:

Install and import the package

from vnlp import DependencyParser

Expected Result:

Package working properly.

Actual Result:

Error message saying ImportError: cannot import name 'MutableMapping' from 'collections'

Environment:

Python 3.10

Recommend Projects

vngrs-ai / vnlp Goto Github PK