When analyzing the text of the Great Gatsby, Methodius captured some things it interpreted as ngrams which were just numbers.

We need a test to eliminate numbers.

Project should be in TS b/c that would be easier for documentation.

right now n-grams work by stripping diacritics; all frequencies/ results take away accent marks.

We should at least offer the ability to opt-in to diacritics.
Seeing confusing results on the demo:

in portuguese, ão is listed, but we can't highlight (a problem with the demo, but still)

With French, we have éèê to account for and it may be useful to see this separate.

When trying to build with it, the error
@parcel/core: Failed to resolve 'core-js/modules/es.array.includes.js' from './node_modules/methodius/dist/methodius.class.js

occurs.

This occurs in the Methodius Demo project: https://github.com/paceaux/methodius-demo

Need ability to discover n-grams that commonly occur with other n-grams.

i.e. How would I discover "tion"

e.g.
"ion"
"tio"
nation => nat ati tio ion
vacation => vac aca cat tio ion
station => sta ati tio ion

needs to ignore «»

Add ability to do distance comparisons on strings.

distance comparisons could include:

• levenshtein
• hamming
• levenshtein-damerau
• jaro-winkler

Consider using Wuzzy. But confirm Wuzzy behaves properly. Wuzzy's Levenshtein function returns a float. algorithm says it should be an int.

According to a few diff folks, Rollup may be overkill. Remove it.

Possibly consider ESBuild? Leave Babel in place, though. Maybe.

After analyzing Alice in Wonderland and Huckleberry Finn, some unusual bigrams popped up:

{
    "‘e": 1,
    "‘i": 10,
    "“‘": 3,
    "‘—": 1,
    "“_": 18,
    "_e": 3,
    "—c": 3,
    "‘l": 1,
    "‘s": 1,
    "_”": 11,
    "w-": 1,
    "n—": 12,
    "—e": 10,
    "n—": 57,
    "“_": 54,
    "_m": 53,
    "e-": 115,
    "-y": 11,
    "_”": 32,
    "‘_": 1,
    "_-": 2,
    "‘t": 4,
    "m”": 2,
    "w”": 1,
    "‘h": 1,
    "l”": 1,
    "tb": 1,
    "lh": 1,
    "‘e": 1,
}

1. Need to analyze the texts and see what the context is for these cases
2. determine if we can just do some sort of normalization (which is prob the case for the quotes)
3. otherwise, exclude things like - and _

Create properties/ methods for getting word bigrams. Maybe?

Right now the class doesn't have many methods, but one thing it lacks is the ability to choose an arbitrary ngram (e.g. quadrigram). Create a member method for this.

Create method(s) that can compare two frequency maps and determine

1. shared items (Intersection)
2. Items unique to each (Symmetric Difference)

Add ability to discover where bigrams, trigrams, ngrams are placed within the word.