ctarnold / jpllm Goto Github PK
View Code? Open in Web Editor NEWworking on llm research
working on llm research
Instruct model requires chat history of user, assistant, user, assistant alternations. Trying to implement zero-shot prompting while somehow conforming to this requirement is proving difficult.
Given sagor sarker's LID model trained on the LinCE twitter/X data, fine-tune it on the human speech in the bangor miami corpus.
Make slurm scripts consistent in time, memory requested, etc. 30 minutes is shown to be a reasonable time for a few messages to the model. 190GB a reasonable CPU limit. 80GB a reasonable GPU limit.
In web mistral interfaces, the model code-switches to a limited, but consistent, degree. In my implementation, I am seeing a lack of code-switching. I likely have temp to a default of 1, making the model deterministic with a default to English. A preliminary attempt to set temp to some value between 0 and 1 resulted in an error thrown.
Because we need to conform to user, assistant response templates, we can alternate each portion of dialogue. Whenever the speaker switches, switch the user to assistant and vice versa. Steps:
Clean corpus data into chat template
Provide chat template to model
Collect and Save output
Eventually scale to large instruction datasets, needs parallel computations for maximum efficiency. (ex: split 1000 prompts into sets of 50, 100, 250, 500, depending on cluster availability. Test experimentally).
When model with code-switching behavior (at least some code-switching as in the web interface) is implemented, add the component of lang id via the BERT model. Test on the miami corpus, or the LinCE framework.
Compare the success of the LID/POS Tagging Tools to corpus data.
From the linked prompting data in the Southeast Asia code-mixing study, transcribe and modify it to an English/Spanish context.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.