hizkifw / webchatrwkvstic Goto Github PK

ChatGPT-like Web UI for RWKVstic

Python 71.68% CSS 5.89% HTML 4.78% JavaScript 17.65%

webchatrwkvstic's Introduction

! NO LONGER MAINTAINED !

Warning This repository is no longer maintained. Please see other forks, such as wfox4/WebChatRWKVv2. For more info on the RWKV language model, check the BlinkDL/ChatRWKV repository.

WebChatRWKVstic

RWKV-V4 inference via rwkvstic, with a ChatGPT-like web UI, including real-time response streaming.

How to use

# Clone this repository
git clone https://github.com/hizkifw/WebChatRWKVstic.git
cd WebChatRWKVstic

# Recommended: set up a virtual environment
python -m venv venv
source ./venv/bin/activate

# Install requirements
pip install -r requirements.txt

# Run the webserver
python main.py

The script will automatically download a suitable RWKV model into the models folder. If you already have a model, you can create the models directory and place your .pth file there.

Currently state

Mobile-friendly web UI with autoscroll, response streaming, markdown formatting, and syntax highlighting
Input is formatted into a question/answer format for the model, and earlier chat messages are included in the context

TODO

Tune the model to better match ChatGPT
Clean up the code
Create a Docker image

webchatrwkvstic's People

Contributors

Stargazers

Watchers

Forkers

guhaifudeng stjordanis graphgrailai llucid-97 wfox4

webchatrwkvstic's Issues

done_callback: Callable[[dict[str, Any]], None] = lambda x: None TypeError: 'type' object is not subscriptable

No *.pth models found in the `models` folder, downloading...
 -> RAM: 64264081408
 -> VRAM: 16945709056
Downloading model RWKV-4-Pile-7B-ctx4096
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13.8G/13.8G [04:00<00:00, 61.5MB/s]
Verifying file integrity...
-> Using model models/RWKV-4-Pile-7B-ctx4096.pth
init RWKVOPS, from super
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 582/582 [00:03<00:00, 154.73it/s]
Traceback (most recent call last):
  File "main.py", line 25, in <module>
    import model
  File "/home/ubuntu/RWKV/WebChatRWKVstic/model.py", line 186, in <module>
    class Task:
  File "/home/ubuntu/RWKV/WebChatRWKVstic/model.py", line 190, in Task
    done_callback: Callable[[dict[str, Any]], None] = lambda x: None
TypeError: 'type' object is not subscriptable

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:00:1E.0 Off |                    0 |
| N/A   40C    P0    50W / 300W |      0MiB / 16384MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

when python main.py, show this error

do you have any instructions about enviroments install?
Thank you

What would cause the output to keep repeating

What would cause the output to keep repeating?
Happened on more than one question.

Question:
What are some Python development best practices?

ChatRWKV

The following are some Python development best practices that you can use to improve your code:

    Use an open source or community contributed library.
    Use a consistent coding style throughout the project.
    Use the Github repo and/or website of your project to identify a code convention.
    Use a version control system to keep track of changes.
    
    ...
    
Follow coding standards.
Follow coding conventions.
Follow coding standards.
Follow coding conventions.
Follow coding standards.
Follow coding conventions.
Follow coding standards.
Follow coding conventions.
Follow coding standards.
Follow coding conventions.
Follow coding standards.
Follow coding conventions.
Follow coding standards.
Follow coding conventions.

How to use the webui？

-> Using model models/RWKV-4-Pile-3B-20221110-ctx4096.pth
init RWKVOPS, from super
100%|███████████████████████████████████████████████████| 582/582 [00:44<00:00, 13.13it/s]
Loading context


---

Question:
help me pick an open source license for my project

Full Answer in Markdown:
Choosing an open source license for your project is an important decision that can affect how others use and contribute to your code. Here are a few popular open source licenses to consider:

1. MIT License: This is a permissive license that allows others to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of your code. This license is popular among open source projects because it is very permissive and has very few restrictions.
2. Apache License 2.0: This is another permissive license that allows others to use, modify, distribute, and sublicense your code, but also includes patent protection. The Apache License 2.0 requires that any modified code include a notice of the changes made.
3. GNU General Public License (GPL): This is a copyleft license that requires any derivative works to be distributed under the same license. The GPL is often used for software that is intended to be shared and modified by a community of users.
4. Creative Commons Licenses: These licenses are often used for non-software projects, such as documentation, music, and art. They provide a way to share creative works while retaining some rights.

It's important to consider your goals for your project and how you want others to be able to use and contribute to your code when choosing a license.

Chat context loaded
INFO:     Started server process [19286]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

I try use simple server:

cd WebChatRWKVstic/static
python3 -m http.server 8000
http://127.0.0.1:8000

WebSocket connection to 'ws://127.0.0.1:8000/ws' failed:

中文乱码

下载中文模型后在web端输出是乱码，而命令行输出是正常的

how to use args.strategy = 'cuda fp16i8 *15+' to run 7b model on rtx2060 6G

i can use args.strategy = 'cuda fp16i8 *15+' run 7b model on rtx2060 6G

while use this webchat , i could't find how to set args.strategy

does it not support or i did'nt find it

error when loading RWKV.

came from v100 32G VRAM and 40G RAM.

RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

Importing modules...

**************************************
WARN: CUDA not available, will use CPU
If you want to use CUDA, try running this command:

  pip install torch --extra-index-url https://download.pytorch.org/whl/cu117 --upgrade

For more information, see: https://pytorch.org/get-started/locally/
*************************************

-> Using model models/RWKV-4-Pile-3B-ctx4096.pth
Traceback (most recent call last):
  File "/Users/boyer/hsg/WebChatRWKVstic/main.py", line 25, in <module>
    import model
  File "/Users/boyer/hsg/WebChatRWKVstic/model.py", line 173, in <module>
    model = RWKV(
  File "/Users/boyer/miniconda/lib/python3.10/site-packages/rwkvstic/load.py", line 62, in RWKV
    ops, weights = loadWeights(mode, path, *args, **kwargs)
  File "/Users/boyer/miniconda/lib/python3.10/site-packages/rwkvstic/helpers/loadWeights.py", line 14, in loadWeights
    w: Dict[str, torch.Tensor] = torch.load(
  File "/Users/boyer/miniconda/lib/python3.10/site-packages/torch/serialization.py", line 797, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
  File "/Users/boyer/miniconda/lib/python3.10/site-packages/torch/serialization.py", line 283, in __init__
    super().__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
(base)