Large Dataset about contextualized-topic-models HOT 4 CLOSED

Hello!

You could modify this part.

I think the issue is the following one:
CombinedTM takes in input contextualized representations and the vocab.
Contextualized representations are expanded to match the dimension of the vocab.

You could add a Linear layer that reduces the size of the vocab before concatenation.

I believe this should improve performance

from contextualized-topic-models.

Hi,

I tried to add it like this but still doesnt work.

Is this what you meant by adding a linear layer?

35000 is my vocab size. My desired size is 52k.

Thanks in advance.

from contextualized-topic-models.

Hello!

I'd probably do something like this:

self.adapt_bert = nn.Linear(bert_size, 400)
self.adapt_vocab = nn.Linear(input_size, 400)

In this way, embedded dimensions are constrained to 400. You will still need to reconstruct a 52K vocabulary size and the model might struggle a bit, but there's not much we can do for that currently.

from contextualized-topic-models.

Hi,

Thanks for your help! I tried to use a more powerful CPU to compute 52k vocab and it works well without constraining embedded dimensions.

Thanks!

from contextualized-topic-models.