shashvatshah9 / ffpytorch Goto Github PK

def MNIST_loaders(train_batch_size=50000, test_batch_size=10000): dataloader is directly passing all the 50k images in one go, we need to try and split up this process in batches. Look for good practices in python, ideally we should use generator functions with yields

Make sure the FF module can work with any type of image dataset with minimal changes to the way data is served to the model. This will lead us to use the existing encoding technique of overlaying the classification bit on image, or suggesting some new methodology. Ideally, we can create a class which is implemented by other child classes. And when this class is used in the code, we just call it encoding, and we can rapidly swap different encoding techniques.

Doing batching in input data will definitely help. But more importantly, we can leverage the fact that the network trains only 1 layer at a time. So we can load layers at a time and not the entire model and then swap the trained layer with another layer.

Can we use different network architectures?
How about residual layers, some dropout or funnel-shaped fully connected layers? Need to experiment with this and empirically see the results.

extend and overload https://pytorch.org/docs/stable/_modules/torch/nn/modules/module.html#Module