escvm / escvm_yt Goto Github PK
View Code? Open in Web Editor NEWEscVM YouTube Channel Repository. Start from Notebooks ⬅️
Home Page: https://www.youtube.com/channel/UC9_Hh7gM2TO6I0aPWJBhFog
License: Apache License 2.0
EscVM YouTube Channel Repository. Start from Notebooks ⬅️
Home Page: https://www.youtube.com/channel/UC9_Hh7gM2TO6I0aPWJBhFog
License: Apache License 2.0
can you tell why there are zero trainable parameters in spatial attention layers?
in the notebook 3.3 build cnn with attention ,if we see the summary ,there are zero trainable parameters.
conv2d_6 (Conv2D) (None, 32, 32, 32) 896
batch_normalization_4 (Batch (None, 32, 32, 32) 128
conv2d_7 (Conv2D) (None, 32, 32, 32) 9248
batch_normalization_5 (Batch (None, 32, 32, 32) 128
channel_attention (ChannelAt (None, 32, 32, 32) 0
spatial_attention (SpatialAt (None, 32, 32, 32) 0
max_pooling2d_2 (MaxPooling2 (None, 16, 16, 32) 0
conv2d_8 (Conv2D) (None, 16, 16, 64) 18496
batch_normalization_6 (Batch (None, 16, 16, 64) 256
conv2d_9 (Conv2D) (None, 16, 16, 64) 36928
batch_normalization_7 (Batch (None, 16, 16, 64) 256
channel_attention_1 (Channel (None, 16, 16, 64) 0
spatial_attention_1 (Spatial (None, 16, 16, 64) 0
max_pooling2d_3 (MaxPooling2 (None, 8, 8, 64) 0
conv2d_10 (Conv2D) (None, 8, 8, 128) 73856
batch_normalization_8 (Batch (None, 8, 8, 128) 512
conv2d_11 (Conv2D) (None, 8, 8, 128) 147584
channel_attention_2 (Channel (None, 8, 8, 128) 0
spatial_attention_2 (Spatial (None, 8, 8, 128) 0
global_average_pooling2d_1 ( (None, 128) 0
Total params: 289,578
Trainable params: 288,938
Non-trainable params: 640
_________________________________________________________________`
In the paper + the video https://www.youtube.com/watch?v=F7wd4wQyPd8 you mention using layer normalization
To prevent this, FF normalizes the length of the hidden vector before using
it as input to the next layer (Ba et al., 2016b; Carandini and Heeger, 2013) This removes all of the
information that was used to determine the goodness in the first hidden layer and forces the next
hidden layer to use information in the relative activities of the neurons in the first hidden layer. These
relative activities are unaffected by the layer-normalization
However I did not notice layer norm in your implementation. I see that you used per layer training where each layer does not use previous layers outputs which probably achieves similar outcome?, why did you decide to use that instead of layer normalization? or was the reason for not using layernorm that you only have 1 hidden layer?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.