tsproisl / textcomplexity Goto Github PK

Hi,

Thank you for developing such a comprehensive resource for complexity computation. I am trying the package but ran into some errors,

Traceback (most recent call last):
  File "/usr/local/bin/txtcomplexity", line 12, in <module>
    textcomplexity.cli.main()
  File "/usr/local/lib/python3.7/dist-packages/textcomplexity/cli.py", line 147, in main
    results.extend(surface_based(tokens, args.window_size, args.all_measures))
  File "/usr/local/lib/python3.7/dist-packages/textcomplexity/cli.py", line 60, in surface_based
    mean, stdev, _ = surface.bootstrap(measure, tokens, window_size, strategy="spread")
  File "/usr/local/lib/python3.7/dist-packages/textcomplexity/surface.py", line 478, in bootstrap
    for window in windows.disjoint_windows(tokens, window_size, strategy):
  File "/usr/local/lib/python3.7/dist-packages/textcomplexity/utils/windows.py", line 30, in disjoint_windows
    assert window_size <= text_length
AssertionError

I used run_stanza.py to prepare the conllu file. Could you please suggest what might I do wrong?

Best

stanza deprecated convert_dict() and (at least in 1.7.0) it's not supported anymore. Hence, run_stanza.py fails with:

Traceback (most recent call last):
  File "/tmp/stanza/./run_stanza.py", line 41, in <module>
    main()
  File "/tmp/stanza/./run_stanza.py", line 33, in main
    conll = CoNLL.convert_dict(dicts)
AttributeError: type object 'CoNLL' has no attribute 'convert_dict'

which seemingly can be fixed with:

diff --git a/utils/run_stanza.py b/utils/run_stanza.py
index 02318ff..1508ac5 100755
--- a/utils/run_stanza.py
+++ b/utils/run_stanza.py
@@ -29,12 +29,7 @@ def main():
         filename = os.path.basename(fh.name)
         text = fh.read()
         doc = nlp(text)
-        dicts = doc.to_dict()
-        conll = CoNLL.convert_dict(dicts)
-        with open(os.path.join(args.output_dir, filename + ".conllu"), mode="w", encoding="utf-8") as out:
-            for sentence in conll:
-                out.write("\n".join(("\t".join(token) for token in sentence)))
-                out.write("\n\n")
+        CoNLL.write_doc2conll(doc, os.path.join(args.output_dir, filename + ".conllu"))

I met such problem and don't know how to solve it. Could you please help me with that?

The command I use: bin/txtcomplexity --input-format conllu --lang en --window-size 5 output/TEXT.conllu

My input file looks like this:

I've been attempting to run analyses from the textcomplexity library but keep getting the following error:

TypeError: UdToken.__new__() missing 9 required positional arguments: 'form', 'lemma', 'upos', 'xpos', 'feats', 'head', 'deprel', 'deps', and 'misc'

Here's a deeper look at what's happening:
!txtcomplexity -i conllu 'output.conllu'

Traceback (most recent call last):
File "/usr/local/bin/txtcomplexity", line 12, in textcomplexity.cli.main()
File "/usr/local/lib/python3.10/dist-packages/textcomplexity/cli.py", line 194, in main sentences, graphs = zip(*conllu.read_conllu_sentences(f, ignore_case=args.ignore_case))
File "/usr/local/lib/python3.10/dist-packages/textcomplexity/utils/conllu.py", line 16, in read_conllu_sentences for sentence, sent_id in _read_conllu(f, ignore_case):
File "/usr/local/lib/python3.10/dist-packages/textcomplexity/utils/conllu.py", line 66, in _read_conllu sentence.append(UdToken(*fields))

TypeError: UdToken.__new__() missing 9 required positional arguments: 'form', 'lemma', 'upos', 'xpos', 'feats', 'head', 'deprel', 'deps', and 'misc'

Is this related to an error in my conllu file or how I'm using the textcomplexity library? Any help would be much appreciated! :)

hi, nice repo.

i wonder know how to use this package by calling function instead of command way?

thanks.

@thvitt @christofs @fotisj @pielstroem @schtepf @tsproisl

Hello tsproisl,

Thank you for developing this comprehensive package for text complexity! This is not an issue about the package, but more of a question I have. I have very limited experience with Linux and command lines, and was wondering if you can give me an example about using the package via Jupiter's i-notebook? I have the text file in CoNLL-U format, and saw the cli file, but I'm very unfamiliar with the argument thing, and just couldn't make it work myself. Thank you!