Strange words - bug? about english-words HOT 14 OPEN

Also is "giggish" actually a word?

Yes it is: https://www.wordnik.com/words/giggish

from english-words.

@NadavB I suspect the apostrophes may have been removed by mistake. Feel free to re-add them in. PR gladly accepted.

from english-words.

How?

from english-words.

@NadavB As far as I can tell, it looks like the process would be ad hoc for this project. Open words.txt in a text editor, use a regex to find pairs of lines like (.*nt$).*n't$ and delete the lines that look bad.
Then, remove copies of the deleted lines from words_alpha, update the corresponding zip files (why?), and submit a pull request.

from english-words.

But where is the current code that generated words_alpha.txt from words.txt so we can modify it?

from english-words.

I don't think it was ever committed. What I see in the history is that someone just added a words_alpha file, and other people modified it directly.

from english-words.

Also is "giggish" actually a word?

from english-words.

@LameLemon I couldn't find a definition for "giggish," and it looks like it came from the original infochimps dataset. You can probably remove it.

To address to the original issue of "are strange words a bug," I think we should say no and close the thread. The underlying reason for the presence of nonwords is the choice of data sources. More carefully curated corpora either cost more or have fewer words.

from english-words.

@dbrakman so can you commit it please? Otherwise people can't contribute to it...

from english-words.

@NadavB I understand why it should be committed, but I don't have that script. I didn't make these lists.

from english-words.

Ahh, I understand. So if someone from the authors see this thread, please commit, thanks...

from english-words.

@dbrakman It won't help. The word "aren" is found in words.txt as well. So unless someone show how the file words.txt was extracted from the corpus, I don't think this whole repository is usable at all.

from english-words.

'aaa' isn't a word either

from english-words.

H

from english-words.