Comments (7)
head.weight f16 cuda:0 2048 65536
D:\RWKV\backend-python\rwkv_pip\model.py:1824: UserWarning: operator () profile_node %318 : int = prim::profile_ivalue(%316)
does not have profile information (Triggered internally at ..\torch\csrc\jit\codegen\cuda\graph_fuser.cpp:109.)
r, k, v, g, xxx, ss = self.v5_2_before(
INFO: 127.0.0.1:49843 - "POST /switch-model HTTP/1.1" 200 OK
INFO: 127.0.0.1:50019 - "OPTIONS /v1/chat/completions HTTP/1.1" 200 OK
INFO: 127.0.0.1:50019 - "POST /v1/chat/completions HTTP/1.1" 200 OK
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "D:\RWKV\py310\Lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "D:\RWKV\py310\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
return await self.app(scope, receive, send)
File "D:\RWKV\py310\Lib\site-packages\fastapi\applications.py", line 1115, in __call__
await super().__call__(scope, receive, send)
File "D:\RWKV\py310\Lib\site-packages\starlette\applications.py", line 122, in __call__
await self.middleware_stack(scope, receive, send)
File "D:\RWKV\py310\Lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
raise exc
File "D:\RWKV\py310\Lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
await self.app(scope, receive, _send)
File "D:\RWKV\py310\Lib\site-packages\starlette\middleware\cors.py", line 91, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
File "D:\RWKV\py310\Lib\site-packages\starlette\middleware\cors.py", line 146, in simple_response
await self.app(scope, receive, send)
File "D:\RWKV\py310\Lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
raise exc
File "D:\RWKV\py310\Lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
await self.app(scope, receive, sender)
File "D:\RWKV\py310\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in __call__
raise e
File "D:\RWKV\py310\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in __call__
await self.app(scope, receive, send)
File "D:\RWKV\py310\Lib\site-packages\starlette\routing.py", line 718, in __call__
await route.handle(scope, receive, send)
File "D:\RWKV\py310\Lib\site-packages\starlette\routing.py", line 276, in handle
await self.app(scope, receive, send)
File "D:\RWKV\py310\Lib\site-packages\starlette\routing.py", line 69, in app
await response(scope, receive, send)
File "D:\RWKV\py310\Lib\site-packages\sse_starlette\sse.py", line 233, in __call__
async with anyio.create_task_group() as task_group:
File "D:\RWKV\py310\Lib\site-packages\anyio\_backends\_asyncio.py", line 597, in __aexit__
raise exceptions[0]
File "D:\RWKV\py310\Lib\site-packages\sse_starlette\sse.py", line 236, in wrap
await func()
File "D:\RWKV\py310\Lib\site-packages\sse_starlette\sse.py", line 221, in stream_response
async for data in self.body_iterator:
File "D:\RWKV\backend-python\routes\completion.py", line 149, in eval_rwkv
for response, delta, prompt_tokens, completion_tokens in model.generate(
File "D:\RWKV\backend-python\utils\rwkv.py", line 270, in generate
token = self.pipeline.sample_logits(
File "D:\RWKV\backend-python\rwkv_pip\utils.py", line 143, in sample_logits
out = torch.multinomial(probs, num_samples=1)[0]
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
from rwkv-runner.
配置页面截图看一下
from rwkv-runner.
from rwkv-runner.
什么显卡, 试试关掉自定义CUDA算子
from rwkv-runner.
显卡是1650,我关闭cuda后。可以正常输出了。谢谢您的帮助
from rwkv-runner.
可以nvidia-smi看看驱动版本, 尝试更新驱动, 自定义算子能开最好开, 虽然1.5B不开速度也可以
from rwkv-runner.
此外你可以试试WebGPU(Python)模式, 开nf4跑3B, 或许会比1.5B的效果更好
from rwkv-runner.
Related Issues (20)
- 能否增加一个 tokenizer 的 API HOT 1
- 某些模型在 AVOID_PENALTY 的 assert len(dd) == 1 这里会失败 HOT 1
- macOS 14.2.1 没法下载模型,下载进度一直为 0 HOT 1
- expected scalar type Half but found Byte HOT 1
- v1.7.0在Windows Defender中报毒 HOT 1
- v1.7.0 LoRa训练错误 HOT 3
- 启动模型报错:failed to load: MPS backend out of memory HOT 1
- 可以考虑增加 candle-rwkv 作为后端选择
- 400 bad request HOT 11
- 希望能增加本地知识库功能 HOT 4
- 希望能增加min-p
- 请问这个框架支持continous-batching吗 HOT 2
- 启动失败,显示:切换模型失败 - Failed to fetch HOT 2
- 启动失败,提示 错误的PyTorch版本 HOT 1
- Docker version missing: Configs, Models, Downloads and Train options on main screen HOT 1
- Failed to enable custom CUDA kernel HOT 3
- CUDA和Pytorch已经安装,但依然提示Failed CUDA HOT 1
- Fail to connect to the model HOT 1
- Network error when going to sleep or hibernate. HOT 3
- UI Freezes When Focusing on Textbox on Linux HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rwkv-runner.