Automatic-Structured-Pruning

We have implemented a framework to support developers for an automatic structured pruning of their neural networks. In previous pruning frameworks, only weight pruning is applied here. Although the unnecessary weights are set to zero, memory must still be provided for these weights, which does not result in a reduction of the weights. Thus, no reduction of the required memory space is achieved by this either. For this reason, the Automatic-Structured-Pruning tool was developed. This tool makes it possible to perform pruning for convolutional and fully cross-linked layers. Individual filters or neurons are deleted directly from the respective layers. This results in a reduction of the weights as well as the memory requirements. The tool allows two different pruning approaches:

Factor: For the fully connected and convolutional layers, a factor is specified in each case, which indicates the percentage of neurons or filters to be deleted from the layer.
Accuracy: The minimum accuracy or the maximum loss of accuracy is specified. This defines which accuracy the neural network should still reach after pruning.

The framework is still under development and will be extended from time to time.

Additionally, the Automatic-Structured-Pruning is part of the Tool AutoFlow, which is a tool that helps developers to implement machine learning (ML) faster and easier on embedded devices. The whole workflow of a data scientist should be covered. Starting from building the ML model to the selection of the target platform to the optimization and implementation of the model on the target platform.

Features:

Prune of filters / channels
Prune of neurons
Automatic pruning to minimum accuracy
Prune metric: L1/L2 Norm for neuron Pruning

Restrictions:

The tool is tested with Tensorflow 2.5
It only supports magnitude-based pruning

Upcoming functions:

More prune methods not only l1/l2 norm
not only Sequential models to prune
Resnet support
Train from scratch
Do you have an new idea? Please write to [email protected], [email protected] or create an issue.

How to start?

To make it easier to get started with the framework we have created a notebook which helps you to use the functions directly. You can open the notebook in Google Colab.

When I create a model like this, there is a bug at runtime

state_input = Input(shape=(84, 84, 4), name='state_input', dtype='uint8')
advantage = Input(shape=(1,), name="adv")
state_input_1 = Lambda(layer_function)(state_input)
convlayer = Conv2D(32, (8, 8), strides=(4, 4), activation='relu', padding='valid')(state_input_1)
convlayer = Conv2D(64, (4, 4), strides=(2, 2), activation='relu', padding='valid')(convlayer)
convlayer = Conv2D(64, (3, 3), strides=(1, 1), activation='relu', padding='valid')(convlayer)
flattenlayer = Flatten()(convlayer)
denselayer = Dense(256, activation='relu')(flattenlayer)

out_actions = Dense(4, activation='softmax', name='output_actions')(denselayer)
out_value = Dense(1, name='output_value')(denselayer)
model = Model(inputs=[state_input, advantage], outputs=[out_actions, out_value])

pruned_model = pruning.factor_pruning(model, dense_prune_rate, conv_prune_rate, 'L2', num_classes=10)

Traceback (most recent call last):
File "/home/xys/primary_xingtian/xingtian-pruning/xt/structured_pruning/src/test/pruning_function_test.py", line 86, in
pruned_model = pruning.factor_pruning(model, dense_prune_rate, conv_prune_rate, 'L2', num_classes=10)
File "/home/xys/primary_xingtian/xingtian-pruning/xt/structured_pruning/src/pruning.py", line 77, in factor_pruning
num_new_neurons, num_new_filters, comp)
File "/home/xys/primary_xingtian/xingtian-pruning/xt/structured_pruning/src/pruning_helper_functions.py", line 259, in build_pruned_model
pruned_model.layers[i].set_weights(new_model_param[i])
File "/home/tank/miniconda3/envs/openmmlab/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1826, in set_weights
'shape %s' % (ref_shape, weight.shape))
ValueError: Layer weight shape (1911, 180) not compatible with provided weight shape (1886, 180)

hahn-schickard / automatic-structured-pruning Goto Github PK