BART Support about ecco HOT 9 OPEN

Unfortunately BART support wasn't completely there yet. I just made a couple of updates, could you try it now?

Install the latest version of the repo:
!pip install git+https://github.com/jalammar/ecco.git

from ecco.

Hi Anshoo,

Thank you so much for your kind words!

1. We are currently working on BERT support for activation exploration. We don't have immediate plans for supporting other models but we're listening to the community.
2. We are working on more extensive documentation that should make it easier for people to adapt Ecco for their needs and use-cases. Thank you for the initiative.
3. Your best bet is Captum - https://captum.ai/tutorials/Bert_SQUAD_Interpret

from ecco.

Hi Anshoo,

Thank you so much for your kind words!

1. We are currently working on BERT support for activation exploration. We don't have immediate plans for supporting other models but we're listening to the community.
2. We are working on more extensive documentation that should make it easier for people to adapt Ecco for their needs and use-cases. Thank you for the initiative.
3. Your best bet is Captum - https://captum.ai/tutorials/Bert_SQUAD_Interpret

@jalammar Appreciate your time responding to this inquiry & the reference to Captum.

from ecco.

@jalammar Looks like 'facebook/bart-large-mnli' is now supported?

When I instantiate it I get the following error:

lm = ecco.from_pretrained('facebook/bart-large-mnli')

AttributeError                            Traceback (most recent call last)

<ipython-input-5-a9021088d96a> in <module>()
      1 import ecco
----> 2 lm = ecco.from_pretrained('facebook/bart-large-mnli')

2 frames

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in __getattr__(self, name)
   1176                 return modules[name]
   1177         raise AttributeError("'{}' object has no attribute '{}'".format(
-> 1178             type(self).__name__, name))
   1179 
   1180     def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:

AttributeError: 'BartForConditionalGeneration' object has no attribute 'shared'

Any suggestions on how I can fix this?

Thanks in advance!

from ecco.

Thank you Jay. The model now loads.

I'm trying to understand how to use it with the MNLI model. That model accepts a premise and a label. Where the premise is a piece of text ('Computer shut down unexpectedly') and a label ('politics'). So, the model spits out its prediction whether the label is correct for the premise.

I can't figure out how to pass those two elements (premise and label) to any functions in the ecco lm. Any advice?

from ecco.

My guess would be that that they put them in the same string separated by the [sep] token. The paper or model docs should explain that, I think.

from ecco.

Hello, @jalammar

Is there example of using a BERT?
I've seen a lot examples of GPT..

I want to use saliency on BERT's head, not INPUT TOKEN.

from ecco.

@Lim-Sung-Jun There are examples of using BERT for neuron activation, but not for primary feature attribution/saliency. That hasn't been built out yet. I have create issue #64 to track the feature request and invite people to support it.

from ecco.

@Lim-Sung-Jun There are examples of using BERT for neuron activation, but not for primary feature attribution/saliency. That hasn't been built out yet. I have create issue #64 to track the feature request and invite people to support it.

I really appreciate your support!

from ecco.