Comments (9)
Another 2 questions:
- the paper also mentions using Gradient Clipping for language modeling, and I assume that this was done as part of the optimization of the Adam optimizer, but just to clarify, was this norm scaling?
- the paper mentions gating as well, but is this implemented in the current keras version?
from keras-tcn.
@lingdoc this is in my TODO list. I'll dig into that when I have a bit of time. In ~1 week from now!
from keras-tcn.
btw, great implementation! I'm comparing it with a BiLSTM-CNN variant on a text classification problem and getting similar results with half the runtime.
from keras-tcn.
That's great! Thank you :)
from keras-tcn.
Just for info, I'm working on a PR to make the implementation identical to the paper: #42
from keras-tcn.
Merged. Only weight norm is missing
141ef1c
from keras-tcn.
@lukeb88 :
- In the paper, they don't talk about stacking various dilated convolutions (parameter
nb_stacks
):
I think I understood what you discussed here, but how this changes instead of just increasing the number of dilations?
=> fixed I set the nb_stacks to 1 for all the examples. Results are the same.
how this changes instead of just increasing the number of dilations?
cf. https://arxiv.org/pdf/1609.03499.pdf
- In the single residual block (in
residual_block()
), why is there only one convolutional layer instead of 2 as in the paper?
It's just a simplification?
=> Was a simplification. But I fixed it!
from keras-tcn.
@philipperemy I supposed that the idea of stacking dilated convolution was taken from the Wavenet paper!
Thanks for your clarification...
great work, btw!!!
from keras-tcn.
@lukeb88 yeah the idea was taken from wavenet!
Thanks I highly appreciate :)
from keras-tcn.
Related Issues (20)
- ValueError: Unknown layer: TCN when trying to load saved model HOT 3
- Dilations and nb_stack relationship HOT 4
- The parameter of the TCN HOT 7
- Masking time steps in order to use TCN for variable length sequences HOT 2
- data shape of tcn layer HOT 1
- help one regression per sequence HOT 1
- Ensuring unique weights when a model uses multiple TCNs HOT 2
- Question: TCN Dilated Convolutions? HOT 3
- How can i use TCN to build seq2seq model? HOT 1
- keras-tcn for R HOT 3
- Visualization of internal structure of TCN block HOT 1
- Low accuracy in keras-tuner based TCN model for audio classification HOT 6
- Saving a loaded model gets warning and then fails to open HOT 1
- exchange_rate.txt HOT 5
- Question: Skip connections HOT 2
- Why is the data form (batch_size, timesteps, input_dim) instead less accurate than (batch_size, input_dim, timesteps)?
- Keras 3 support HOT 7
- Confusion about regression or classification head HOT 1
- AttributeError: 'tuple' object has no attribute 'as_list' HOT 3
- Question regarding return_sequences=True HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from keras-tcn.