Comments (1)
不支持。这是原始 FT 里的内容,已移除。
from lmdeploy.
Related Issues (20)
- [Bug] lmdepoly加载的InternXComposer-4kHD推理结果和InternXComposer GitHub项目中直接使用transformers加载的模型推理出的结果不一致。 HOT 2
- 使用turbomind部署CodeQwen1.5模型,推理效果变差 HOT 5
- [Feature] 是否支持enc-dec类型模型中decoder的persistent batch HOT 3
- Lora adapters only work with pytorch enngine. HOT 8
- [Feature] Support for LLaVA-NeXT Qwen1.5-110, Qwen1.5-72B, LLaMA3-8B HOT 12
- [Bug] change h_input_length_buf_ before synchronization HOT 2
- 增加对chatglm3的支持[Feature] HOT 1
- [Feature] progress bar for inference pipeline HOT 1
- [Feature] Support W4A8KV4 Quantization(QServe/QoQ) HOT 1
- Qwen-7B模型的实现与原版本有差异
- [Bug] finish_reason='length' once setting max_tokens with openai API HOT 2
- AsyncEngine 的 stream_infer 函数增加手动传入session_id,实现多次调用 stream_infer 时的并行推理[Feature] HOT 2
- [Bug]: 'ArgumentHelper' has no attribute 'enable_prefix_caching'
- [Bug] llava, cuda out of memory HOT 4
- [Docs] Add docs to NVTX options
- Support for Pali gemma
- [Bug] The response of /v1/chat/completions is not correct, when input params message is a string prompt. HOT 2
- 服务部署完成后,路由转发路径问题 HOT 2
- [Bug] Unrecognized configuration class when quantizing llava HOT 4
- [Bug] 服务在接受到一些截断的图像base64编码后,线程退出,导致服务异常,此时服务没有停止,但是新的post请求无法进来 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lmdeploy.