Add literature background to the models about model-zoo HOT 8 OPEN

This is great!

Looking ahead to the website step, I think it would be better and simpler to just host a "model zoo" or "tutorials" section in the Flux docs themselves:

• almost every manual on writing good documentation mentions these types of sections which Flux is severely lacking right now
• added benefit of being verified with every PR to Flux allowing us to keep it up to date and free of the bit rot the model zoo historically suffers
• having information accessible in a central location is easier for users

from model-zoo.

Thank you @darsnack! Totally agree with your comments.

from model-zoo.

I think it is good to add some comments to some of the basics examples, but I wouldn't want them to be too loaded either. Model-zoo examples are supposedly used for bootstrapping user projects, they aren't meant as tutorials (which we lack and should live in Flux.jl).

from model-zoo.

We can use Literate to create a "clean" version of the examples.

from model-zoo.

Regarding the Simple ConvNet (LeNet5) example, I noticed it is using relu activations and max-pooling. I think the original LeCun paper doesn't do this. It is also different from https://d2l.ai/chapter_convolutional-neural-networks/lenet.html, which is cited in the readme of this example.

Any reason for these discrepancies?

from model-zoo.

@cossio I just wrote the text and took the code as it was on the repo. Perhaps somebody else can comment on these discrepancies?

from model-zoo.

I think that's a modernized version of Lenet5 (relu had not been "invented" at the time) which I've found somewhere (don't remember where though). Maybe we can add a comment on that saying that the original model uses sigmoids?

from model-zoo.

Added a note to the model to specify this.

from model-zoo.