Switched multi-output prediction arrays about bert-for-tf2 HOT 5 CLOSED

@roebius - this is most likely an issue with keras and dict outputs - internally in tf.keras the dict is flattened to a list, and the order might get changed. I thought until now that the order in this case is alphabetical (and would be interested to understand why that's not the case - I believe it is different between TF versions; I also have issued a different but related bug to the TF team - tensorflow/tensorflow#34199).
I don't think it is an issue with bert-for-tf2. I guess you are using BertModelLayer which is just a layer (not aware of the "outer" keras model populating the outputs). You could easily replace it with an EmbeddingLayer and see, if the keras behavior will change.

from bert-for-tf2.

@roebius - (edit: I believe it is name of the output tensor that counts not the dictionary entry ) - it is the dict key that counts (but it didn't worked right with a loss dict, therefore I've switched to using list outputs instead).
I'm using a:

ident = lambda n: tf.keras.layers.Lambda(lambda a: a, name=n)
outputs = [ident("output-a")(out_a), ident("output-b")(out_b)]

to force the names on the output tensors.

from bert-for-tf2.

Thank you for your reply and suggestions.
I confirm I am using BertModelLayer. I also believe it is a Keras issue, but it is a bit complicated for me to change the output layer names: some TF ops I am using force name prefixes for the output layers, so I have to use the actual layer names for my losses but I have to use different names in the output dictionary for the TFRecords I am feeding.
(sorry that I cannot attach a schema as I am not allowed by the rules of a competition I am taking part in :(

from bert-for-tf2.

@roebius - changing the output names can be done with the ident function/layer above. But if I remember correctly it is relevant for the model losses only.

from bert-for-tf2.

I was eventually able to solve the issue.
For the model creation, I am passing the outputs as a list instead of a dictionary: this way the prediction outputs are not switched anymore.

Thanks to your ident suggestion I was able to refactor my code in order to use the output layer names consistently between model output, model losses and TFRecord feature mapping when feeding the model for fit/predict.
Many thanks again.

from bert-for-tf2.