Comments (1)
The most time-consuming part of both the transform() and fit_transform() methods is the extraction of BERT-based embeddings. When you have trained the model, you will still need to extract the embeddings for unseen documents. Unfortunately, this means that it is difficult to speed up transform() as it, computationally, mostly relies on extracting the embedding.
Fortunately, in the case of topic modeling, it is unlikely that you will frequently re-train the model as that would result in creating new topics that need to be interpreted again. Often, you will fit_transform() on a large dataset and use transform() for unseen documents, which is faster as the number of unseen documents are likely to be less frequent.
Having said that, it would be nice to be able to swap out BERT-embeddings for perhaps another feature extraction method. This could result in having a much faster application although it could hinder the quality of the generated clusters. Perhaps flair might be an interesting alternative.
Does this answer your question?
from bertopic.
Related Issues (20)
- Can't update model name when use notebook HOT 4
- Scikit-learn's HDBSCAN Implementation
- Issues with visualizations on loaded models. HOT 1
- (Zero-shot Topic Modeling) TypeError: object of type 'numpy.float64' has no len() HOT 1
- Additional representations did not update with topic reduction HOT 5
- [Guided Topic Modeling] ValueError: setting an array element with a sequence. HOT 6
- Array mismatch when try to fit new data HOT 1
- Does Bertopic support custom keyword extractor? HOT 5
- Why do I lose Names assigned by zero-shot after applying outlier reduction? HOT 1
- get_topic() with KeyBERTInspired? HOT 1
- Mismatch between old OpenAI API in bertopic/backend/_openai and current OpenAI (v1.33.0) HOT 1
- OpenAI Embedding HOT 1
- Potential bug with the PartOfSpeech class due to lower case matching HOT 1
- Chosen represented Topic HOT 5
- merge_topics from a model loaded from safe tensors HOT 1
- Add test coverage around OpenAI(BaseRepresentation) HOT 3
- Can't import bertopic in pycharm but runs fine in Anaconda prompt console HOT 3
- Adding representation model does not change topic_model.get_document_info results() HOT 1
- Installing bertopic cause Spyder 5.5.1 in Anaconda to not launch ith errors. HOT 1
- Updating and Pushing a BERTopic Model with New Documents to Hugging Face Hub still shows old no of training document HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bertopic.