Comments (1)
I thought the problem was with BatchToSpace. I was able to prevent BatchToSpace from occurring (occurs when conv1d dilation > 1) and there is still the problem with Regularizer Loss = 0.0. The FLOPs can be calculated, so I assume that reg_outputs/ reg_inputs (network_regularizers/resource_function.py#L477) must be 0.
Further information:
I use the model Jasper (Openseq2seq). For Batch Normalization I set scale and fused = true. channels_last is set.
This is how I included morphnet:
def get_regularization_loss(self):
from open_seq2seq.optimizers import get_regularization_loss as old_regularization_loss
if not self.params['morphnet']:
return old_regularization_loss()
else:
from morph_net.network_regularizers import flop_regularizer
from morph_net.tools import structure_exporter
network_regularizer = flop_regularizer.GammaFlopsRegularizer(output_boundary=[self._logits.op], input_boundary=[self.get_data_layer().input_tensors["source_tensors"][0].op, self.get_data_layer().input_tensors["target_tensors"][0].op], gamma_threshold=1e-3)
regularization_strength = 1e-10
regularizer_loss = (network_regularizer.get_regularization_term() * regularization_strength)
self.reg_loss = regularizer_loss
self.reg_costs = network_regularizer.get_cost()
tf.summary.scalar('RegularizationLoss', regularizer_loss)
tf.summary.scalar(network_regularizer.cost_name, network_regularizer.get_cost())
return tf.add(old_regularization_loss(),regularizer_loss)
from morph-net.
Related Issues (20)
- the morphnet can act on some layers of the model?such as act on backbone. HOT 7
- Questions about batch_normalization HOT 3
- OpRegularizerManager could not handle ops HOT 7
- Getting this Error HOT 1
- Getting error like this HOT 2
- In this case, MorphNet is getting stuck because it doesn't know how to handle the Shape op. You can try a couple options:
- Applying Morph-net while training training Yolov3 using darknet HOT 1
- Query on Morph Net Structure Exporter HOT 2
- What's the tf version this repo used?
- Can morph-net output a pruned model file? HOT 3
- Bad structure HOT 3
- MorphNet not working on google colab HOT 1
- Can I use MorphNet to slim my model trained by Pytorch? HOT 1
- Multiple alive files output HOT 1
- Batch Norm layers Assertion Error
- Latency regularizer for a target hardware. HOT 5
- LayerNorm not support?
- Performance issue in the definition of create_test_input, examples/slim/nets/resnet_v1_test.py HOT 1
- Tensorflow 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from morph-net.