Simplify the CIFAR10 example about flax HOT 10 CLOSED

Could it be that this shows an underlying tension between having two different things: "examples" that should be read to learn, and a "model/sota zoo" that has enough so that someone can jump-start towards current SOTA?

Because regarding the two models: WRN is super widely used, but PyramidNet with Shake-shake is (close to) SOTA. If I want to learn about the framework, I'm happy reading WRN without many tricks, but if I want to see if my newest invention increases SOTA, I'd like to get a small code which gets close to SOTA where I can add my invention and get great numbers (or not).

from flax.

That link is now a 404. Where is the current CIFAR10 example code?

Do you still have a cifar10 example?

from flax.

Thanks for adding this issue Alexey, I agree with your observations.

Can we just keep one or two of the above combinations?
Which ones are more relevant / useful / educational?

Fine by me, I don't know much about the details of these architectures, but I would probably pick the one that is most common. I am not sure which one is most educational.

Should we turn some other combinations into HOW-TOs? Which ones?

HOWTOs are based on diffs, so they are most useful if they are being used for some added functionality. It seems that WideResNet is implemented without shake-shake and with shake-shake. Is it possible to create a HOWTO that shows how to add shake-shake to WideResNet?

from flax.

Good point Lucas! I think we are still figuring out which examples belong in Flax exactly (I agree with the tension), but for this specific example I probably lean towards keeping the simpler one (WRN), with possibly one or more HOWTOs, and having the most complex ones in a separate repository.

from flax.

FYI I chatted about this with Alexey, and he told me that he'd like to wait with finishing the regression testing until he will act on this issue. So I've linked that PR here.

from flax.

We have recently removed out official CIFAR example and instead linked to a much better open source repository (https://github.com/google-research/google-research/tree/master/flax_models/cifar)

from flax.

That link is now a 404. Where is the current CIFAR10 example code?

from flax.

We removed the example because it was using the old API and we had nobody actively working on it and willing to port it.

You can see more examples in our examples README:
https://github.com/google/flax/tree/main/examples

Note that for example the vision_transformer codebase has example code to fine-tune a model on CIFAR10:

• https://github.com/google-research/vision_transformer
• https://colab.research.google.com/github/google-research/vision_transformer/blob/master/vit_jax.ipynb

from flax.

@andsteing Is there any example code showing a simple resnet running on CIFAR10 that you know of?

from flax.

You could switch the imagenet2012 dataset with cifar10 in our examples/imagenet - or you could download the pre-trained imagenet checkpoint and fine-tune it on cifar10

I'm not aware of a repo that simply trains a resnet on cifar10.

from flax.