Hi, thank you for open-sourcing this implementation. I have trained the model and everything was smooth and OK. But the problem is that there seems to be no improvement in TEST ACC. Could you please instruct me how I can solve this issue? Please note that I have not done any customization on your code and the implementation is exactly yours.

hey guys,when i run your code,it seems that there is no data to train?i am confused.

Epoch 000 / 001 : : 1it [00:02, 2.66s/it, training loss=4.0050, iterations=1]
2021-01-25 17:52:40.330843: W tensorflow/core/grappler/optimizers/implementation_selector.cc:310] Skipping optimization due to error while loading function librarie
s: Invalid argument: Functions '__inference_standard_lstm_6471' and '_inference_standard_lstm_6471_specialized_for_lstm_1_5_StatefulPartitionedCall_at___inference
keras_scratch_graph_71971' both implement 'lstm_287408bc-d5fd-45ea-a08f-389dac98439a' but their signatures do not match.
2021-01-25 17:52:55.098128: W tensorflow/core/grappler/optimizers/implementation_selector.cc:310] Skipping optimization due to error while loading function librarie
s: Invalid argument: Functions '__inference_standard_lstm_43101' and '__inference_standard_lstm_43101_specialized_for_lstm_1_64_StatefulPartitionedCall_at___inferen
ce_keras_scratch_graph_77144' both implement 'lstm_cfe8d53e-9ee3-42c4-9653-ac8c91a62f7f' but their signatures do not match.

This is the error, I am getting. Any help is highly appreciated

Hello, can you share the dataset?

Hi, I ran in to u r code and it gives me a lot help. Thank you.
By the way, I just wonder why u use tf.eye for attention layer.
For many cases, attention layer is implemented by each attention cell and for loop.
tf.eye is okay for implement attention layer at once, but im still curious about why u use it.