A toolkit for Spoken Language Understanding Evaluation (SLUE) benchmark. Refer paper https://arxiv.org/abs/2111.10367 for more details. Official website: https://asappresearch.github.io/slue-toolkit/
Home Page: https://asappresearch.github.io/slue-toolkit/
License: MIT License
I sent my submission of the test set evaluation to "[email protected]" , but there was no reply.
I do not know whether I sent the wrong email address or other reasons.
Hi, I have seen the readme of NER in https://github.com/he159ok/slue-toolkit
But I do not see a command to run a NER model directly/only by the speech model.
May I know a command to run it?
Hello
I'm trying to run the ASR baselines you have provided. I have downloaded the datasets with scripts/download_datasets.sh in manifest/slue-voxceleb and manifest/slue-voxpopuli.
When running
bash baselines/asr/ft-w2v2-base.sh manifest/slue-voxceleb save/asr/w2v2-base-vc
I have the following error:
File "/home/brun/fairseq/fairseq_cli/hydra_train.py", line 27, in hydra_main
_hydra_main(cfg)
File "/home/brun/fairseq/fairseq_cli/hydra_train.py", line 56, in _hydra_main
distributed_utils.call_main(cfg, pre_main, **kwargs)
File "/home/brun/fairseq/fairseq/distributed/utils.py", line 369, in call_main
main(cfg, **kwargs)
File "/home/brun/fairseq/fairseq_cli/train.py", line 94, in main
model = task.build_model(cfg.model)
File "/home/brun/fairseq/fairseq/tasks/audio_finetuning.py", line 193, in build_model
model = super().build_model(model_cfg, from_checkpoint)
File "/home/brun/fairseq/fairseq/tasks/audio_pretraining.py", line 197, in build_model
model = super().build_model(model_cfg, from_checkpoint)
File "/home/brun/fairseq/fairseq/tasks/fairseq_task.py", line 335, in build_model
model = models.build_model(cfg, self, from_checkpoint)
File "/home/brun/fairseq/fairseq/models/init.py", line 106, in build_model
return model.build_model(cfg, task)
File "/home/brun/fairseq/fairseq/models/wav2vec/wav2vec2_asr.py", line 209, in build_model
w2v_encoder = Wav2VecEncoder(cfg, len(task.target_dictionary))
File "/home/brun/fairseq/fairseq/tasks/audio_finetuning.py", line 170, in target_dictionary
return self.state.target_dictionary
File "/home/brun/fairseq/fairseq/tasks/fairseq_task.py", line 41, in getattr
self._state[name] = self._factoriesname
File "/home/brun/fairseq/fairseq/tasks/audio_finetuning.py", line 123, in load_target_dictionary
return Dictionary.load(dict_path)
File "/home/brun/fairseq/fairseq/data/dictionary.py", line 226, in load
d.add_from_file(f)
File "/home/brun/fairseq/fairseq/data/dictionary.py", line 239, in add_from_file
raise fnfe
File "/home/brun/fairseq/fairseq/data/dictionary.py", line 236, in add_from_file
with open(PathManager.get_local_path(f), "r", encoding="utf-8") as fd:
FileNotFoundError: [Errno 2] No such file or directory: '/home/brun/slue-toolkit/manifest/slue-voxceleb/dict.ltr.txt'
Could you help me to solve this problem.
(Sorry if the issue is naive, I am completely new to speech processing)
Many thanks!
Hello,
I have two questions about the challenge rule.
What is the difference between pipeline and end2end? In my understanding, the "pipeline" model has two neural network models independently: audio-to-text (ASR) and text-to-entity (NLP). The "E2E" model has one neural network model: audio-to-(entity, word).
If my model can estimate the whole transcript and entity of each word in only one neural network, is it "E2E"?
Or, if my model has multiple training steps(pretraining-finetuning-end2end) but has one inference step(end2end), it can be "E2E"?
I think if there is no "token id to string" stage in the inference step, it can be "E2E", but I'm not sure.
Where can I submit my submissions for participating challenge? Moreover, can we see the leaderboard or test results before the deadline?
Thanks
Hi,
I wanted to reproduce the sentiment analysis baseline through
bash baselines/sentiment/e2e_scripts/ft-w2v2-base-senti.sh manifest/slue-voxceleb save/sentiment/w2v2-base
Fairseq Config log:
[2022-02-14 01:39:15,687][fairseq_cli.train][INFO] - {'_name': None, 'common': {'_name': None, 'no_progress_bar': False, 'log_interval': 100, 'log_format': 'json', 'log_fil[37/1798]
'tensorboard_logdir': 'save/sentiment/w2v2-base/tb_logs', 'wandb_project': None, 'azureml_logging': False, 'seed': 1, 'cpu': False, 'tpu': False, 'bf16': False, 'memory_efficient_b
f16': False, 'fp16': True, 'memory_efficient_fp16': False, 'fp16_no_flatten_grads': False, 'fp16_init_scale': 128, 'fp16_scale_window': None, 'fp16_scale_tolerance': 0.0, 'on_cpu_c$
nvert_precision': False, 'min_loss_scale': 0.0001, 'threshold_loss_scale': None, 'amp': False, 'amp_batch_retries': 2, 'amp_init_scale': 128, 'amp_scale_window': None, 'user_dir': $
/root/pushkal/slue-toolkit/slue_toolkit/fairseq_addon', 'empty_cache_freq': 0, 'all_gather_list_size': 16384, 'model_parallel_size': 1, 'quantization_config_path': None, 'profile':
False, 'reset_logging': False, 'suppress_crashes': False, 'use_plasma_view': False, 'plasma_path': '/tmp/plasma'}, 'common_eval': {'_name': None, 'path': None, 'post_process': None$
'quiet': False, 'model_overrides': '{}', 'results_path': None}, 'distributed_training': {'_name': None, 'distributed_world_size': 1, 'distributed_num_procs': 1, 'distributed_rank'$
0, 'distributed_backend': 'nccl', 'distributed_init_method': None, 'distributed_port': -1, 'device_id': 0, 'distributed_no_spawn': False, 'ddp_backend': 'c10d', 'ddp_comm_hook': '$
one', 'bucket_cap_mb': 25, 'fix_batches_to_gpus': False, 'find_unused_parameters': True, 'gradient_as_bucket_view': False, 'fast_stat_sync': False, 'heartbeat_timeout': -1, 'broadc$
st_buffers': False, 'slowmo_momentum': None, 'slowmo_base_algorithm': 'localsgd', 'localsgd_frequency': 3, 'nprocs_per_node': 1, 'pipeline_model_parallel': False, 'pipeline_balance$
: None, 'pipeline_devices': None, 'pipeline_chunks': 0, 'pipeline_encoder_balance': None, 'pipeline_encoder_devices': None, 'pipeline_decoder_balance': None, 'pipeline_decoder_devi$
es': None, 'pipeline_checkpoint': 'never', 'zero_sharding': 'none', 'fp16': True, 'memory_efficient_fp16': False, 'tpu': False, 'no_reshard_after_forward': False, 'fp32_reduce_scat$
er': False, 'cpu_offload': False, 'use_sharded_state': False, 'not_fsdp_flatten_parameters': False}, 'dataset': {'_name': None, 'num_workers': 0, 'skip_invalid_size_inputs_valid_te$
t': False, 'max_tokens': 1400000, 'batch_size': None, 'required_batch_size_multiple': 8, 'required_seq_len_multiple': 1, 'dataset_impl': None, 'data_buffer_size': 10, 'train_subset$
: 'fine-tune', 'valid_subset': 'dev', 'combine_valid_subsets': None, 'ignore_unused_valid_subsets': False, 'validate_interval': 1000000, 'validate_interval_updates': 0, 'validate_a$
ter_updates': 2000, 'fixed_validation_seed': None, 'disable_validation': False, 'max_tokens_valid': 1400000, 'batch_size_valid': None, 'max_valid_steps': None, 'curriculum': 0, 'ge$
_subset': 'test', 'num_shards': 1, 'shard_id': 0, 'grouped_shuffling': False, 'update_epoch_batch_itr': False, 'update_ordered_indices_seed': False}, 'optimization': {'_name': None$
'max_epoch': 0, 'max_update': 50000, 'stop_time_hours': 0.0, 'clip_norm': 0.0, 'sentence_avg': True, 'update_freq': [1], 'lr': [2e-05], 'stop_min_lr': -1.0, 'use_bmuf': False, 'sk$
p_remainder_batch': False}, 'checkpoint': {'_name': None, 'save_dir': 'checkpoints', 'restore_file': 'checkpoint_last.pt', 'continue_once': None, 'finetune_from_model': None, 'rese$
_dataloader': False, 'reset_lr_scheduler': False, 'reset_meters': False, 'reset_optimizer': False, 'optimizer_overrides': '{}', 'save_interval': 50, 'save_interval_updates': 1000, $
keep_interval_updates': 1, 'keep_interval_updates_pattern': -1, 'keep_last_epochs': -1, 'keep_best_checkpoints': -1, 'no_save': False, 'no_epoch_checkpoints': True, 'no_last_checkp$
ints': False, 'no_save_optimizer_state': False, 'best_checkpoint_metric': 'macro_f1', 'maximize_best_checkpoint_metric': True, 'patience': -1, 'checkpoint_suffix': '', 'checkpoint_$
hard_count': 1, 'load_checkpoint_on_all_dp_ranks': False, 'write_checkpoints_asynchronously': False, 'model_parallel_size': 1}, 'bmuf': {'_name': None, 'block_lr': 1.0, 'block_mome$
tum': 0.875, 'global_sync_iter': 50, 'warmup_iterations': 500, 'use_nbm': False, 'average_sync': False, 'distributed_world_size': 1}, 'generation': {'_name': None, 'beam': 5, 'nbes$
': 1, 'max_len_a': 0.0, 'max_len_b': 200, 'min_len': 1, 'match_source_len': False, 'unnormalized': False, 'no_early_stop': False, 'no_beamable_mm': False, 'lenpen': 1.0, 'unkpen': $
.0, 'replace_unk': None, 'sacrebleu': False, 'score_reference': False, 'prefix_size': 0, 'no_repeat_ngram_size': 0, 'sampling': False, 'sampling_topk': -1, 'sampling_topp': -1.0, '$
onstraints': None, 'temperature': 1.0, 'diverse_beam_groups': -1, 'diverse_beam_strength': 0.5, 'diversity_rate': -1.0, 'print_alignment': None, 'print_step': False, 'lm_path': Non$
, 'lm_weight': 0.0, 'iter_decode_eos_penalty': 0.0, 'iter_decode_max_iter': 10, 'iter_decode_force_max_iter': False, 'iter_decode_with_beam': 1, 'iter_decode_with_external_reranker$
: False, 'retain_iter_history': False, 'retain_dropout': False, 'retain_dropout_modules': None, 'decoding_format': None, 'no_seed_provided': False}, 'eval_lm': {'_name': None, 'out$
ut_word_probs': False, 'output_word_stats': False, 'context_window': 0, 'softmax_batch': 9223372036854775807}, 'interactive': {'_name': None, 'buffer_size': 0, 'input': '-'}, 'mode$
': {'_name': 'wav2vec2_seq_cls', 'w2v_path': '/root/pushkal/slue-toolkit/save/pretrained/wav2vec_small.pt', 'no_pretrained_weights': False, 'dropout_input': 0.0, 'final_dropout': 0$
0, 'dropout': 0.0, 'attention_dropout': 0.0, 'activation_dropout': 0.1, 'conv_feature_layers': '[(512, 10, 5)] + [(512, 3, 2)] * 4 + [(512,2,2)] + [(512,2,2)]', 'encoder_embed_dim'$
768, 'apply_mask': True, 'mask_length': 10, 'mask_prob': 0.65, 'mask_selection': static, 'mask_other': 0.0, 'no_mask_overlap': False, 'mask_min_space': 1, 'mask_channel_length': 6$
, 'mask_channel_prob': 0.5, 'mask_channel_selection': static, 'mask_channel_other': 0.0, 'no_mask_channel_overlap': False, 'freeze_finetune_updates': 2000, 'feature_grad_mult': 0.0$
'layerdrop': 0.1, 'mask_channel_min_space': 1, 'mask_channel_before': False, 'normalize': '${task.normalize}', 'data': '${task.data}', 'w2v_args': None, 'pool_method': 'self_attn'$
'classifier_dropout': 0.2}, 'task': {'_name': 'slue_audio_classification', 'data': '/root/pushkal/slue-toolkit/manifest/slue-voxceleb', 'labels': 'sent', 'binarized_dataset': Fals$
, 'sample_rate': 16000, 'normalize': False, 'enable_padding': False, 'max_sample_size': None, 'min_sample_size': None, 'num_batch_buckets': 0, 'precompute_mask_indices': False, 'in$
erred_w2v_config': None, 'tpu': '${common.tpu}', 'text_compression_level': none, 'label_dir': '???'}, 'criterion': {'_name': 'slue_sequence_classification'}, 'optimizer': {'_name':
'adam', 'adam_betas': '(0.9,0.98)', 'adam_eps': 1e-08, 'weight_decay': 0.0, 'use_old_adam': False, 'fp16_adam_stats': False, 'tpu': False, 'lr': [2e-05]}, 'lr_scheduler': {'_name':
'tri_stage', 'warmup_steps': 0, 'hold_steps': 0, 'decay_steps': 0, 'phase_ratio': [0.1, 0.0, 0.9], 'init_lr_scale': 0.01, 'final_lr_scale': 0.05, 'max_update': 50000.0, 'lr': [2e-05
]}, 'scoring': None, 'bpe': None, 'tokenizer': None, 'ema': {'_name': None, 'store_ema': False, 'ema_decay': 0.9999, 'ema_start_update': 0, 'ema_seed_model': None, 'ema_update_freq'
: 1, 'ema_fp32': False}, 'job_logging_cfg': {'version': 1, 'formatters': {'simple': {'format': '[%(asctime)s][%(name)s][%(levelname)s] - %(message)s'}}, 'handlers': {'console': {'cl
ass': 'logging.StreamHandler', 'formatter': 'simple', 'stream': 'ext://sys.stdout'}, 'file': {'class': 'logging.FileHandler', 'formatter': 'simple', 'filename': 'hydra_train.log'}},
'root': {'level': 'INFO', 'handlers': ['console', 'file']}, 'disable_existing_loggers': False}}
But facing this error:
Traceback (most recent call last):
File "/root/miniconda3/envs/slue/bin/fairseq-hydra-train", line 33, in <module>
sys.exit(load_entry_point('fairseq', 'console_scripts', 'fairseq-hydra-train')())
File "/root/pushkal/slue-toolkit/deps/fairseq/fairseq_cli/hydra_train.py", line 87, in cli_main
hydra_main()
File "/root/miniconda3/envs/slue/lib/python3.8/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "/root/miniconda3/envs/slue/lib/python3.8/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra
run_and_report(
File "/root/miniconda3/envs/slue/lib/python3.8/site-packages/hydra/_internal/utils.py", line 201, in run_and_report
raise ex
File "/root/miniconda3/envs/slue/lib/python3.8/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/root/miniconda3/envs/slue/lib/python3.8/site-packages/hydra/_internal/utils.py", line 347, in <lambda>
lambda: hydra.run(
File "/root/miniconda3/envs/slue/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 107, in run
return run_job(
File "/root/miniconda3/envs/slue/lib/python3.8/site-packages/hydra/core/utils.py", line 129, in run_job
ret.return_value = task_function(task_cfg)
File "/root/pushkal/slue-toolkit/deps/fairseq/fairseq_cli/hydra_train.py", line 27, in hydra_main
_hydra_main(cfg)
File "/root/pushkal/slue-toolkit/deps/fairseq/fairseq_cli/hydra_train.py", line 56, in _hydra_main
distributed_utils.call_main(cfg, pre_main, **kwargs)
File "/root/pushkal/slue-toolkit/deps/fairseq/fairseq/distributed/utils.py", line 369, in call_main
main(cfg, **kwargs)
File "/root/pushkal/slue-toolkit/deps/fairseq/fairseq_cli/train.py", line 97, in main
criterion = task.build_criterion(cfg.criterion)
File "/root/pushkal/slue-toolkit/deps/fairseq/fairseq/tasks/fairseq_task.py", line 352, in build_criterion
return criterions.build_criterion(cfg, self)
File "/root/pushkal/slue-toolkit/deps/fairseq/fairseq/criterions/__init__.py", line 29, in build_criterion
return build_criterion_(cfg, task)
File "/root/pushkal/slue-toolkit/deps/fairseq/fairseq/registry.py", line 55, in build_x
cls = REGISTRY[choice]
KeyError: 'slue_sequence_classification'
Hi,
I am running into an error when trying to run the baseline script for voxpopuli:
Traceback (most recent call last):
File "~/bin/anaconda3/envs/fairseq/bin/fairseq-hydra-train", line 33, in <module>
sys.exit(load_entry_point('fairseq', 'console_scripts', 'fairseq-hydra-train')())
File "~/repos/fairseq/fairseq_cli/hydra_train.py", line 87, in cli_main
hydra_main()
File "~/bin/anaconda3/envs/fairseq/lib/python3.8/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "~/bin/anaconda3/envs/fairseq/lib/python3.8/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra
run_and_report(
File "~/bin/anaconda3/envs/fairseq/lib/python3.8/site-packages/hydra/_internal/utils.py", line 201, in run_and_report
raise ex
File "~/bin/anaconda3/envs/fairseq/lib/python3.8/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "~/bin/anaconda3/envs/fairseq/lib/python3.8/site-packages/hydra/_internal/utils.py", line 347, in <lambda>
lambda: hydra.run(
File "~/bin/anaconda3/envs/fairseq/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 107, in run
return run_job(
File "~/bin/anaconda3/envs/fairseq/lib/python3.8/site-packages/hydra/core/utils.py", line 129, in run_job
ret.return_value = task_function(task_cfg)
File "~/repos/fairseq/fairseq_cli/hydra_train.py", line 27, in hydra_main
_hydra_main(cfg)
File "~/repos/fairseq/fairseq_cli/hydra_train.py", line 56, in _hydra_main
distributed_utils.call_main(cfg, pre_main, **kwargs)
File "~/repos/fairseq/fairseq/distributed/utils.py", line 369, in call_main
main(cfg, **kwargs)
File "~/repos/fairseq/fairseq_cli/train.py", line 164, in main
extra_state, epoch_itr = checkpoint_utils.load_checkpoint(
File "~/repos/fairseq/fairseq/checkpoint_utils.py", line 272, in load_checkpoint
epoch_itr = trainer.get_train_iterator(
File "~/repos/fairseq/fairseq/trainer.py", line 718, in get_train_iterator
self.reset_dummy_batch(batch_iterator.first_batch)
File "~/repos/fairseq/fairseq/data/iterators.py", line 334, in first_batch
return self.collate_fn([self.dataset[i] for i in self.frozen_batches[0]])
File "~/repos/fairseq/fairseq/data/add_target_dataset.py", line 68, in collater
collated["net_input"]["prev_output_tokens"],
KeyError: 'prev_output_tokens'
I have installed fairseq as recommended on the github page. Here is the installed version:
fairseq 1.0.0a0+0f078de dev_0 <develop>
Any pointers to solve this error?
Thank you
Hi,
I think there is inconsistent use of tabs and spaces among the files in this toolkit like -
text_ner/ner_deberta.py uses spaces and text_ner/ner_deberta_modules.py uses tabs.
Would it be possible to apply black - https://pypi.org/project/black/ , to bring consistent formatting of files.
Thanks
Sid
I am trying to run baselines/ner/nlp_scripts/eval-deberta.sh but it seems to be broken quite a bit.
I have fixed some of the bugs, in the PR (#7) but there seems to be some more, which I am unable to fix comfortably and would require someone with expertise of this code to have a look -
eval_obj.get_scores in def eval( of slue_toolkit/text_ner/ner_deberta.py seems to be passing asr_val_dataset which is set to None when eval_asr is set to False.def get_scores function in slue_toolkit/text_ner/ner_deberta_modules.py. As the run_inference invoked by get_scores uses asr_val_dataset in their Dataloader.self.get_entity_tags( call in the run_inference function when eval_asr is set to False is also broken as this calls self.get_tag_map(indices=True) which seems to be calling an undefined variable tag in tag2id_raw[pfx + tag]Could you please review that PR and also suggest the changes for the above errors. I have kept the PR as [WIP] you can make edits to them as you feel fit.
Thanks
Sid
Hello,
Thanks for all your work. This toolkit is a great contribution to the community.
I am trying to reproduce some of your results, and when running bash scripts/download_datasets.sh, the script crashes in slue_toolkit/prepare/prepare_voxpopuli.py because at some point label_str in get_label_lst() get the value of nan. I leave the full error below.
Thanks again!
entity_pair_str = data_utils.prep_text_ner_tsv( File "/disk/scratch1/ramons/myapps/t2s-xling/myapps/slue-toolkit/slue_toolkit/prepare/data_utils.py", line 146, in prep_text_ner_tsv ner_labels_lst = get_label_lst(ner_labels_str, label_type) File "/disk/scratch1/ramons/myapps/t2s-xling/myapps/slue-toolkit/slue_toolkit/prepare/data_utils.py", line 33, in get_label_lst ner_labels_lst = label_str.strip("[[").strip("]]").split("], [") AttributeError: 'float' object has no attribute 'strip
Hi @ankitapasad @fwu-asapp @sshon-asapp,
I really apologize for my many issue creation. I thought of running the baselines again but I think there are still some issues in the slue_toolkit/text_ner/ner_deberta_modules.py file.
For example -
f"{split_name}.{label_type}.tsv" instead.Would it be possible to run a new bare slue-toolkit and see what all bugs pops up?
The command that I am trying to run is -
bash baselines/ner/nlp_scripts/ft-deberta.sh deberta-base combined
Please ignore if you have already caught these issues.
Thanks
Sid
Hello,
When trying to run
bash baselines/sentiment/e2e_scripts/ft-w2v2-base-senti.sh manifest/slue-voxceleb save/sentiment/w2v2-base
I am getting the error below. Did you find this before?
Thanks!
omegaconf.errors.ConfigKeyError: Key 'label_dir' not in 'AudioClassificationConfig' full_key: label_dir reference_type=Optional[AudioClassificationConfig] object_type=AudioClassificationConfig
I attempted to replicate a Named Entity Recognition (NER) experiment but encountered several issues during the process.
Firstly, when executing the command python slue_toolkit/prepare/prepare_voxpopuli_nel.py create_manifest to generate manifest files, I noticed that the dev.tsv, fine-tune.tsv, and test.tsv files were merely symbolic links. They were unusable for running the end-to-end NER model. To resolve this, I had to manually copy dev.tsv and fine-tune.tsv from slue-toolkit/manifest/slue-voxpopuli into the e2e_ner directory.
Additionally, I faced a problem while performing evaluations with the command bash baselines/ner/e2e_scripts/eval-ner.sh w2v2-base test combined nolm. It appears that the processed test files are missing. Could you provide guidance on how to properly prepare these files for evaluation?
README points to fine-tuning command lines and add comments stating that "Assuming that the preprocessed manifest files are in manifest/slue-voxceleb for SLUE-VoxPopuli" the command should work.
But, following the installation process described in the same README, no manifest folder or pre-processed files are created.
It feels like I am missing a really obvious step.
What is it that I'm missing?
thanks in advance!
in slue-voxceleb_fine-tune.tsv, we have the following sentiment classes:
48 <mixed>
227 Negative
4223 Neutral
1279 Positive
and in slue-voxceleb_dev.tsv, we have
14 Disagreement
3 <mixed>
51 Negative
1124 Neutral
262 Positive
I have two questions:
Hello,
I want to use N-Best hypotheses of each audio file in voxceleb. Is this supported out-of-the-box or code changes are required?
Hi, I have some doubts regarding which data can be used for pretraining the models. I plan to do a mixture of self-supervised and supervised pretraining and I wanted to know:
Does anyone know how to fix the below error when installing wav2letter python bindings
Obtaining file:///gpfsssd/scratch/rech/iqh/upp27cx/wav2letter/bindings/python
Preparing metadata (setup.py) ... done
Installing collected packages: wav2letter
Running setup.py develop for wav2letter
error: subprocess-exited-with-error
× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [48 lines of output]
running develop
running egg_info
writing wav2letter.egg-info/PKG-INFO
writing dependency_links to wav2letter.egg-info/dependency_links.txt
writing top-level names to wav2letter.egg-info/top_level.txt
reading manifest file 'wav2letter.egg-info/SOURCES.txt'
writing manifest file 'wav2letter.egg-info/SOURCES.txt'
running build_ext
gmake: Makefile: No such file or directory
gmake: *** No rule to make target 'Makefile'. Stop.
/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated.
Use build and pip and other standards-based tools.
warnings.warn(
/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build
and pip and other standards-based tools.
warnings.warn(
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/gpfsssd/scratch/rech/iqh/upp27cx/wav2letter/bindings/python/setup.py", line 95, in <module>
setup(
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 177, in setup
return run_commands(dist)
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 193, in run_commands
dist.run_commands()
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
self.run_command(cmd)
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run
self.install_for_development()
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/command/develop.py", line 114, in install_for_development
self.run_command('build_ext')
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
self.distribution.run_command(command)
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/gpfsssd/scratch/rech/iqh/upp27cx/wav2letter/bindings/python/setup.py", line 48, in run
self.build_extensions()
File "/gpfsssd/scratch/rech/iqh/upp27cx/wav2letter/bindings/python/setup.py", line 90, in build_extensions
subprocess.check_call(
File "/gpfswork/rech/iqh/upp27cx/conda-envs/fairseq_a100_clone_jan9/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
In the data collator of AddLabelDataset
slue-toolkit/slue_toolkit/fairseq_addon/data/add_label_dataset.py
Lines 28 to 33 in ea0939b
If self.dataset.collater sort the collated data, target and collated will become mismatched.
Hi,
Thanks for the toolkit! I was wondering if there are plans to release the ASR finetuned models (or it is already there but I missed it). If not, are you accepting PRs on the ASR finetuned models by the community? Thanks in advance!
Jeff Hsu
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.