llm模型导出onnx模型请使用llm-export
当前支持以模型：
| model | onnx-fp32 | mnn-int4 |
|---|---|---|
| chatglm-6b |  |
 |
| chatglm2-6b |  |
 |
| codegeex2-6b |  |
 |
| Qwen-7B-Chat |  |
 |
| Baichuan2-7B-Chat |  |
 |
| Llama-2-7b-chat |  |
 |
# <model> like `chatglm-6b`
# linux/macos
./script/download_model.sh <model>
# windows
./script/download_model.ps1 <model>
当前构建状态：
| System | Build Statud |
|---|---|
| Linux |  |
| Macos |  |
| Windows |  |
| Android |  |
# linux
./script/linux_build.sh
# macos
./script/macos_build.sh
# windows msvc
./script/windows_build.ps1
# android
./script/android_build.sh
默认使用CPU后端，如果使用其他后端，可以在脚本中添加MNN编译宏
- cuda:
-DMNN_CUDA=ON - opencl:
-DMNN_OPENCL=ON
# linux/macos
./cli_demo # cli demo
./web_demo # web ui demo
# windows
.\Debug\cli_demo.exe
.\Debug\web_demo.exe
# android
adb push libs/*.so build/libllm.so build/cli_demo /data/local/tmp
adb push model_dir /data/local/tmp
adb shell "cd /data/local/tmp && export LD_LIBRARY_PATH=. && ./cli_demo -m model"
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent