victordibia / llmx Goto Github PK
View Code? Open in Web Editor NEWAn API for Chat Fine-Tuned Large Language Models (llm)
License: MIT License
An API for Chat Fine-Tuned Large Language Models (llm)
License: MIT License
Using Hf provider gives llmx.generators.text.hf_textgen.hftextgenerator() got multiple values for keyword argument 'provider'
error with version 0.0.15a0.
With version 0.0.14a0 works fine.
Hi,
could you please add support for Makersuite PALM API (https://makersuite.google.com/)?
Thank you for previously adding support for Vertex AI PALM API.
Thanks in advance
In many cases it is useful to keep track of the token usage for each query and associated costs. Different model provider apis handle this differently (some provide this information as part of a generate query response) and the task is to return accurate usage states in a unified format for all model providers
Palm api response provides the following fields.
"metadata": {
"tokenMetadata": {
"input_token_count": {
"total_tokens": integer,
"total_billable_characters": integer
},
"output_token_count": {
"total_tokens": integer,
"total_billable_characters": integer
}
}
}
For certain use cases e.g., cli scripts and web apps, it is important to provide some mechanism to externally set the default LLM provider and configuration
llmx.generators.text.textgen.py
, check for an available config and use to instantiate llm()
. Merge provided kwargs with the content of the config file, with kwargs taking precedenceIt would be interesting to try out the recently released lida library with LLMs running locally using Llama.cpp.
Could llmx support such "offline"/embedded or standalone more resource constrained scenarios with LLMs using only CPUs?
If so, can you provide an outline of steps required?
Is there any plan to add in the Anthropic model endpoints (Sonnet, Opus) in to LLMX?
Hi @victordibia, curious - what was missing from litellm to be useful here?
llmx package is used with lida.
Update the openai version please
You tried to access openai.ChatCompletion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.
You can run openai migrate
to automatically upgrade your codebase to use the 1.0.0 interface.
Alternatively, you can pin your installation to the old version, e.g. pip install openai==0.28
A detailed migration guide is available here: openai/openai-python#742
As you can see from below linked website google has decided to depreciate palm apis . So id there any plans to updated it
https://ai.google.dev/palm_docs/deprecation
Hi There,
I found openAI() takes base_url as the mandatory argument to initialize which is mentioned in this vLLM documentation.
https://docs.vllm.ai/en/latest/getting_started/quickstart.html#using-openai-completions-api-with-vllm
from openai import OpenAI
# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"
client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)
However, this base_url
is not mentioned in codebase when initializing openAI()
https://github.com/victordibia/llmx/blob/main/llmx/generators/text/openai_textgen.py#L30
Should this be updated?
Currently the data needs to be stored as a pandas dataframe in memory. Is there any scope to extend this project to large scale databases that use pyspark dataframes or sql
I try to modify the llmx by adding my custom
LLM service and install the update project from source using pip install -e . on windows and I have issues to run Lida with the customized llmx. Could you help me please ?
I've been trying to set up a working example of Azure OpenAI but in vain...
I think the documentation about this LLM provider should be very clear and not just dealt with in a sentence.. it's not about just setting up the env vars.. I've been trying to work around it for almost one hour now, but I don't know how to set the model, etc. I think it's really poorly documented.
Could anyone help please ?
MODEL_NAME=demo-llm
AZURE_OPENAI_BASE=https://******************.openai.azure.com/
AZURE_OPENAI_API_KEY=*********************
AZURE_OPENAI_API_TYPE=azure
AZURE_OPENAI_API_VERSION=2023-07-01-preview
I tried this:
provider = sanitize_provider("azureopenai")
text_gen = OpenAITextGenerator(
# provider="azureopenai",
api_type="azure",
api_base=os.environ["AZURE_OPENAI_BASE"],
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2023-07-01-preview",
models =[os.environ["MODEL_NAME"]],
provider=provider,
)
and this
text_gen = llm(provider="azureopenai",
api_type="azure",
api_base=os.environ["AZURE_OPENAI_BASE"],
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2023-07-01-preview",
model =os.environ["MODEL_NAME"],
) # for azure openai
I get Error processing file: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.
Is there paras like 'num_gpus=2'。How can I load model into mult GPUS
Hello Victor :)
I really like what you are doing and want to use the recently released lida library for projects in my organization.
However, my organization uses Azure OpenAI.
For this we need to be able to specfiy three openai properties:
Here is an example authorization for an Azure OpenAI instance:
import openai
openai.api_type = "azure"
openai.api_base = "https://yourendpoint.openai.azure.com/"
openai.api_version = "2023-07-01-preview"
openai.api_key = os.getenv("OPENAI_API_KEY")
Can you please add these properties to llmx so it is possible to use lida?
Best regards
Marvin
python3 --version
Traceback (most recent call last):
File "", line 1, in
NameError: name 'python3' is not defined
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.