Comments (1)
DistilBERT was fine tuned to achieve 0.8 training accuracy and 0.68 validation accuracy. The peak validation loss accuracy was achieved in one epoch after which point validation performance deteriorated severely as the model overfit the training data and lost its generalized pre-trained weights. Due to checkpointing, the best model at the first epoch was saved, so we don't need to worry about the later epochs.
The validation performance is relatively poor, and much lower than the training performance. A larger model might fit the training data better than 80% solving the underfitting issue. On the other hand, I expect even training accuracy to be limited by the observation below. The overfitting may be addressed by freezing most of the network layers and training just the detection head.
The data labels have been augmented by heavily resampling each review into smaller chunks down to one letter. The smaller chunks inherit the label of the original review, so the target sentiment for "A" and "A series", "occasionally amuses" and "none of which amounts to much of a story" all map to the label of the combination of these. Without more intelligent splitting, this may cap the ability of the network to learn sentiments as datapoints like "A"/"A series" will have variable labels.
Next steps:
- The IMDB dataset is also interesting for sentiment review. Potentially, implement as a separate experiment and then cross validate training on one or both.
- Implement the other common networks and compare performance. For those that come with out-of-the-box sentiment analysis, evaluate performance without fine-tuning.
- Compare fine-tuning with frozen layers and only the sentiment head being allowed to train.
from torch-control.
Related Issues (20)
- Implement sentiment chatbot using several public models
- Fine-tune most popular LLMs for movie sentiment analysis HOT 1
- Evaluate fine tuning only the sentiment head of sentiment analysis models HOT 1
- Add the IMDB dataset to the sentiment analysis task data
- Drop augmented datapoints with variable labels
- Implement time series power consumption regression analysis HOT 1
- Implement Image Classification on dermaMNIST HOT 3
- Add MLOps HOT 3
- Add oversampling/augmentation to mitigate class imbalance
- Implement use of formal parameter search to log neatly with MLFlow HOT 1
- Implement customizable classification head for LLM fine tuning HOT 1
- Improve power consumption prediction model HOT 1
- Build docker container from mlflow and validate it HOT 1
- Upload Docker container to AWS as demo
- Implement production performance monitoring for power consumption model
- Build pipeline to extract structured text from images HOT 2
- Evaluate - Fine tuning the entire LLM network vs default classifier head vs bigger head HOT 5
- Summarize specific values from extracted text
- Detect bounding boxes containing text in images
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from torch-control.