spino17 / pyglow Goto Github PK

Currently PyGlow uses tqdm progress bar api but it would be more natural to implement one just like keras (Note: to see keras implementation of progress bar see generic_utils in utils)

Currently many of the modules is not handling exceptions and can make program crash in extreme cases.

Write tests for all the modules and increase the unit testing coverage of the library.

Current implementation of fit use either x, y dataset or takes dataloader as arguments. It would be more less expensive to implement a fit method that directly load data in parallel from the file path passed as argument.

I get this error when compiling and training the model on MNIST dataset on Google Colab using their GPU.

`model = IBSequential(input_shape=(1, 28, 28), gpu=True, track_dynamics=True, save_dynamics=True)
model.add(Conv2d(filters=16, kernel_size=3, stride=1, padding=1, activation='relu'))
model.add(Flatten())
model.add(Dense(784, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(10, activation='softmax'))

model.compile(optimizer='SGD', loss='cross_entropy', metrics=['accuracy'])
model.attach_evaluator(HSIC(kernel='gaussian', gpu=True, sigma=5))`

Does anyone know how I can fix it?

I was running the test code in the page https://pyglow.github.io/ on google colab. It gave me this error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-18-0ea5fc0ad3bd> in <module>()
     35 
     36 # train the model along with calculating dynamics
---> 37 model.fit_generator(train_loader, val_loader, num_epochs)

2 frames
/usr/local/lib/python3.7/dist-packages/glow/models/network.py in fit_generator(self, train_loader, val_loader, num_epochs, show_plot)
    291 
    292         """
--> 293         self.training_loop(num_epochs, train_loader, val_loader, show_plot)
    294 
    295     def predict(self, x):

/usr/local/lib/python3.7/dist-packages/glow/models/network.py in training_loop(self, num_epochs, train_loader, val_loader, show_plot)
    181         train_len = len(train_loader)
    182         val_len = len(val_loader)
--> 183         metric_dict = self.handle_metrics(self.metrics)
    184         epoch_collector = []
    185         for epoch in range(num_epochs):

/usr/local/lib/python3.7/dist-packages/glow/models/network.py in handle_metrics(self, metrics)
    134         metric_dict = {}
    135         for metric in metrics:
--> 136             metric_fn = metric_module.get(metric)  # returns the function
    137             metric_dict[metric] = metric_fn
    138 

AttributeError: module 'glow.metrics' has no attribute 'get'

I don't understand what to do about this.

Base implementations for HSIC networks is available. Using that implement HSIC sigma networks as a classifier.

As observed during training, at the start of every epoch there appears a 0 % bar. Fix this bug which is most probably related to how the tqdm updates its iterator.

Implement KNN, KSG, KDE and Parzen Windom estimation methods in information bottleneck module.

As the calculation of IP coordinates for each layer for each batch for each epoch is independent and can be massively calculated in parallel as compared to current implementation which uses nested loops and affects speed of the computation.

Implement module to provide common useful datasets available.

Current implementation uses sequential training of all the models of sigma network but it would massively speed up the computation if those models can be trained parallely.

Implement a complete pipeline of IB principle which includes from distribution estimation to information plane coordinates to plotting IB optimality curves.

Throughout the code there are comments as - ** NOTE - This can be done in parallel !
Which means that the loop can be done in parallel with instruction level parallelism (or some times data level parallelism).

Currently on passing 'adam' as argument in compile method of network it is returing None object. Look up on the issue and remove the bug to make it return the optim.adam() object.

Currently estimation methods are not appropriately implemented to get complete parallel processing of matrix operations and thus more efficient implementations should be employed to speed up the computation.

PyTorch works with tensor objects and to provide the interfacing between the numpy arrays to tensor objects which will be used in internal methods at backend a adapter class must be implemented to provide functionalities that do the inte-rconversions.

Currently setup.py has problems that persist in its 0.0.1 version if you install PyPI then shows no module named glow.

Current implementation of HSIC training process has single core computations but we can very well exploit the layer independence to train each layer on a different processor which can heavily speed up the training process.

Till now only dense layer is implemented so their is a need to extend this to more layers eg. convolutional layers, LSTM and recurrent layers, sampling and pooling layers etc.

Implementations for Deep variational information bottleneck encoders and other related models.

After #11 is closed then we can work on documentation and website of the library for users to get their hands on the code.