Comments (3)
vince62s
commented on June 6, 2024
Hi, you need to provide you pytorch version as well as whether you installed flash-attn2 as well as your yaml file, otherwise difficult to help you out.
from opennmt-py.
RakshaPRao
commented on June 6, 2024
Hi, pytorch version = 2.0.1. flash-attn2 is not installed and the yaml is
accum_count: 8
accum_steps: 0
adam_beta1: 0.9
adam_beta2: 0.998
batch_size: 4096
batch_size_multiple: 1
batch_type: tokens
bucket_size: 32768
decay_method: noam
decoder_type: transformer
dropout: 0.2
early_stopping: 10
encoder_type: transformer
feat_merge: sum
heads: 12
hidden_size: 768
keep_checkpoint: 20
label_smoothing: 0.1
layers: 6
learning_rate: 2.0
max_generator_batches: 0
max_grad_norm: 0.0
n_src_feats: 1
normalization: tokens
optim: adam
param_init: 0.0
param_init_glorot: 'true'
pool_factor: 8192
position_encoding: 'true'
queue_size: 1024
report_every: 100
save_checkpoint_steps: 5000
seed: 1234
share_vocab: true
src_feats_defaults: N
src_seq_length: 600
src_vocab_size: 38000
tgt_seq_length: 600
train_steps: 1000000
transformer_ff: 3072
valid_batch_size: 16
valid_steps: 5000
warmup_steps: 8000
word_vec_size: 768
Thanks!
from opennmt-py.
vince62s
commented on June 6, 2024
You need 2.1 or 2.2
Sdpa is buggy with 2.0.1
from opennmt-py.
Related Issues (20)
- Supported SentencePiece parameters HOT 1
- List index out of range in onmt.utils.distributed.all_reduce_and_rescale_tensors:51
- Speech to Text Toy Data Could Not Be Downloaded HOT 3
- Translation API Not Working HOT 1
- How to use Huawei‘s NPU Ascend310 to install OpenNMT-py？ HOT 1
- NaN values when training big transformer model HOT 1
- Support for torch 2.2 HOT 5
- Device side assert triggered on AWQ Mistral converted model HOT 2
- (Again, but different) AssertionError: assert model_dim % head_count == 0 HOT 2
- Custom callbacks for metrics, saving checkpoints HOT 3
- AttributeError: 'Namespace' object has no attribute 'block_ngram_repeat'
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 55: invalid start byte HOT 1
- Index out of range
- Canʻt get past Sentencepiece subword tokenization with pretrained embeddings
- Issues with Custom SentencePiece Models and Pretrained Embeddings in Training
- Error when translating with scores (--with_scores) option enabled HOT 1
- Supported languages HOT 1
- [Bug - Translation server] - Missing `tgt`param in `translator.translate` method (allows some multilingual/seq2seq models to work properly)
- Unable to use translate HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opennmt-py.