- define an old tf.Graph
- train the network with data with [train_load_net] function
- to expand a network, call [extract_weight] function, weights will be returned as numpy.ndarray
- call [Net2Net] functions to expand weights and layers
- define new tf.Graph
- call [train_load_net] to restart from a checkpoint with new graph and new weights
- expand conv2 twice as wide
- add 1 more convolutional layer after conv3
- add skip layer (cautious)
-
Default session will only allocate 512MB of GDDR. Add the following to your session:
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction= any 0~1 number)
with tf.Session(config=tf.ConfigProto(gpu_options=gpu_options)) as sess: -
Total GDDR of K40C is 10GB, call nvidia-smi to see memory allocation status first before run.
-
Let us share memory by calculate how much memory is needed first and then change the factor in code,