liangstein / bytenet-keras Goto Github PK

line 75, In ByteNet-Keras/ByteNet_train.py
encoder2 has to input by encoder2. but you wrote input_tensor.
you implemented byte-net so well, but i think you just made a small mistake.

Hi @liangstein,

Thanx a lot for your implementation, it helped me a lot to understand bytenet.

I have a technical question for your implementation. I see that you use kernel_size=3 in your encoder's Convolutions, which makes absolute sense with padding='same' in order to condition each timestep (t) based on timesteps (t-1, t+1)*. My question is why the kernel_size=3 in decoder's Convolutions, where you use padding='causal' and you only need to condition based on the previous timestep (t-1). Is this the correct size or you should probably use kernel_size=2 in order to avoid including extra timesteps (e.g., timestep t-2)?

Thanx, Ilias

• where timestep is the previous word in the first residual block, while it is the context-aware embedding of your current word in the next blocks.

Originally posted by @liangstein in https://github.com/liangstein/ByteNet-Keras/issue_comments#issuecomment-425307037