Hi! I wanted to reproduce the result on SentEval, but failed because of the following

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

The error was because there is no "model" key in config dict: <div class="snippet-

Great, thanks for the extra info. Will you call <code class="notrans

Please release a requirements.txt for dependencies about declutr HOT 5 CLOSED

johngiorgi commented on May 27, 2024

Please release a requirements.txt for dependencies

from declutr.

Comments (5)

JohnGiorgi commented on May 27, 2024

Hi @leerumor,

We have a setup.py that serves the same purpose as a requirements.txt.

I can't really tell what is happening from the error. I may need more info to help you. For example:

which version of AllenNLP are you using?
can you show me exactly what your call to run_senteval.py looks like? And especially, which pre-trained model are you evaluating?
did you install following the install instructions here? This would have installed the correct dependencies from the setup.py so you shouldn't need a requirements.txt (that would be redundant).

from declutr.

leerumor commented on May 27, 2024

Thanks for your reply.

I did following the instructions, here are the versions:

allennlp                  1.1.0                    pypi_0    pypi
allennlp-models           1.1.0                    pypi_0    pypi
pytorch-metric-learning   0.9.96                   pypi_0    pypi
typer                     0.3.2                    pypi_0    pypi
validators                0.18.2                   pypi_0    pypi
torch                     1.6.0                    pypi_0    pypi

I downloaded SentEval toolkit and moved declutr-base to output folder. Then run command:

python scripts/run_senteval.py allennlp "SentEval" "output" \
 --output-filepath "output/senteval_results.json" \
 --cuda-device 0 \
 --include-package "declutr"

from declutr.

leerumor commented on May 27, 2024

The error was because there is no "model" key in config dict:

Params({'trainer': {'use_amp': True}, 'hidden_dropout_prob': 0.1, 'vocab_size': 50265, 'type_vocab_size': 1, 'hidden_act': 'gelu', 'gradient_checkpointing': False, 'attention_probs_dropout_prob': 0.1, 'layer_norm_eps': 1e-05, 'initializer_range': 0.02, 'bos_token_id': 0, 'eos_token_id': 2, 'output_hidden_states': True, 'max_position_embeddings': 514, 'model_type': 'roberta', 'intermediate_size': 3072, 'hidden_size': 768, 'architectures': ['RobertaForMaskedLM'], 'num_attention_heads': 12, 'num_hidden_layers': 12, 'pad_token_id': 1})

but when I added config["model"]="declutr", another error occured:

Traceback (most recent call last):
  File "scripts/run_senteval.py", line 732, in <module>
    app()
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "scripts/run_senteval.py", line 714, in allennlp
    overrides="{'trainer.use_amp': true}",
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/allennlp/models/archival.py", line 191, in load_archive
    cuda_device=cuda_device,
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/allennlp/models/model.py", line 369, in load
    return model_class._load(config, serialization_dir, weights_file, cuda_device)
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/allennlp/models/model.py", line 285, in _load
    vocab_dir, vocab_params.get("padding_token"), vocab_params.get("oov_token")
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/allennlp/data/vocabulary.py", line 328, in from_files
    base_directory = cached_path(directory, extract_archive=True)
  File "/home/hadoop-aipnlp/anaconda3/lib/python3.7/site-packages/allennlp/common/file_utils.py", line 175, in cached_path
    raise FileNotFoundError(f"file {url_or_filename} not found")
FileNotFoundError: file output/vocabulary not found

from declutr.

JohnGiorgi commented on May 27, 2024

Great, thanks for the extra info.

Will you call ls on wherever the "output" folder is on your system and let me know the output? If you downloaded declutr-base from here, it should contain everything you need.

Alternatively, I have added a notebook in #182 that will walk you through the evaluation process.

from declutr.

JohnGiorgi commented on May 27, 2024

Closing this because the notebook should provide enough information to evaluate any of the pre-trained models (or your own models) against SentEval. Please re-open if you are still having difficulties!

from declutr.

Please release a requirements.txt for dependencies about declutr HOT 5 CLOSED

Comments (5)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent