Comments (5)
collindutter
commented on May 20, 2024
1
@maziyarpanahi it looks like you can use TGI through the Inference Client which is already a part of HuggingFaceHubPromptDriver. Do you have an existing TGI endpoint you could help us test with?
from griptape.
maziyarpanahi
commented on May 20, 2024
@collindutter this would be very useful as most productions have their own TGI/vLLM deployments and I guess they are pretty similar in principal (same as OpenAI and other endpoints).
from griptape.
collindutter
commented on May 20, 2024
@maziyarpanahi roger that! I will take a look at implementing this today!
from griptape.
maziyarpanahi
commented on May 20, 2024
HuggingFaceHubPromptDriver
This is great! I have TGI endpoints up and running. I can test any example you provide and share the results.
Thanks again.
from griptape.
collindutter
commented on May 20, 2024
Fantastic, thank you for reporting back! I will add a note to the the docs indicating that HuggingFaceHubPromptDriver can be used with TGI!
from griptape.
Related Issues (20)
- Add Gemini Image Query Driver
- Add Google Gemini support
- BedrockLlamaPromptModelDriver does not apply input truncation, resulting in ValidationError HOT 4
- Add Support For Mistral on AWS Bedrock
- BedrockClaudePromptModelDriver results in ValidationError for Claude V3* models due to incorrect invoke struct HOT 1
- JsonExtractionEngine produces malformed JSON
- ignored_exception_types not working as intended
- vLLM endpoint support HOT 4
- CodeExecutionTask doesn't handle inputs correctly when in a Workflow HOT 2
- Calculating Max Output Tokens Is Inconsistent Across Prompt Drivers
- Driver secrets are serializable
- Azure OpenAI api version `2023-12-01-preview` deprecation HOT 2
- Bing Search Wrapper for WebSearch tool HOT 1
- Add model to emitted events.
- Use rich.print in Chat utility HOT 5
- Bump Pinecone from 2 to 3 HOT 1
- Add StructureRunDrivers to the documentation sidebar
- VectorQueryEngine issue with loading artifacts from query HOT 1
- Add `AzureOpenAiStructureConfig`
- Add DuckDuckGo Search tool
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from griptape.