TCAV with TensorFlow Object Detection API models about tcav HOT 4 CLOSED

@BeenKim do you have any examples of people using TCAV for TF/Keras object detection models?

from tcav.

Hi, to update my recent findings and issues.

Findings:

• tracing back to the objectdetection class, I found out how to get the prediction layer of the SSD.

• Also, I successfully recreate the loss function for this layer that works in the same fashion as KerasWrapper
  Issues:

• I copy the KerasWrapper class to build a new KerasWrapperSSD (the SSD layer are Keras layers).

• This class is also on model.py of the TCAV repository

• I load the model as I did to found the model's layers, no error in that, except that the model seems not to load the backbone or any other layer. Hence I cannot extract the bottlenecks.

  def import_keras_model(self, saved_path):
    """Loads keras model, fetching bottlenecks, inputs and outputs."""
    self.ends = {}
    PATH_TO_CFG=saved_path + "/pipeline.config"
    PATH_TO_CKPT = saved_path + "/checkpoint"
    configs = config_util.get_configs_from_pipeline_file(PATH_TO_CFG)
    model_config = configs['model']
    detection_model = model_builder.build(model_config=model_config, is_training=False)
    ckpt = tf.compat.v2.train.Checkpoint(model=detection_model)
    ckpt.restore(os.path.join(PATH_TO_CKPT, 'ckpt-0')).expect_partial()
    self.model = detection_model
    print(self.model)
    print(self.model.feature_extractor.classification_backbone)
    self.get_bottleneck_tensors()
    self.get_inputs_and_outputs_and_ends()
    self.get_image_shape()

• When I load model in this fashion on other script works, but as a class of TCAV, the model is just a simple shell
  I don't know what is missing or what I'm doing wrong. I really, really appreciate any help.
• Being more specific this the output for this command in the wrapper print(self.model._feature_extractor.classification_backbone):
  None
  Bests

from tcav.

Hi, I try different setups of the wrapper but remain the same result. It seems that I cannot load an Object Detection model in this repository. Any help or guidance on why or how to solve this issue is welcome. I really appreciate any help you can provide.

from tcav.

Hi @emmanuelol , I'm a bit unclear on which part of TCAV that doesn't seem to work. You mentioned " Hence I cannot extract the bottlenecks." - what's the error msg you are getting?

I'll close this issue for now since it's been a while (perhaps you've solved it by now), but feel free to reopen if needed.
Been

from tcav.