Git Product home page Git Product logo

whisper.cpp.android's Introduction

whisper.cpp.android with Translation and TTS

Summary

whisper.cpp.android with CLBlast(OpenCL), Translation (Google ML-Kit) and TTS

WORK IN-PROGRESS

Environment

build all and tested on GNU/Linux

GNU/Linux: Ubuntu 20.04_x64 LTS
Android Studio: android-studio-2023.2.1.24

Test device

Samsung Galaxy A32 (with CLBlast(OpenCL))

Build

// Working Path: $HOME/work
$ mkdir $HOME/work && cd $HOME/work

// Install Android Studio, SDK, ...
// Path: /work/android
//
// ADB: /work/android/sdk/platform-tools/adb


// Clone
$ git clone https://github.com/godmode2k/whisper.cpp.android


// Clone whisper.cpp
$ git clone https://github.com/ggerganov/whisper.cpp.git


// Rename origin dir (whisper.cpp/examples/whisper.android)
$ mv ./whisper.cpp/examples/whisper.android ./whisper.cpp/examples/whisper.android.old

// Copy whisper.cpp.android <whisper.cpp path>/examples/
$ cp -a -r ./whisper.cpp.android/whisper.android ./whisper.cpp/examples/whisper.android


---------------------------------
pull libGLES_mali.so file from device
---------------------------------
Note: lib path is for Samsung Galaxy A32


// Android path: /work/android


// SEE: https://github.com/ggerganov/whisper.cpp/tree/master/examples/whisper.android

$ /work/android/sdk/platform-tools/adb pull /system/vendor/lib64/egl/libGLES_mali.so

// rename
$ cp libGLES_mali.so libOpenCL.so

$ mkdir -p ./whisper.cpp/examples/whisper.android/lib/src/main/jniLibs/arm64-v8a
$ cp libOpenCL.so ./whisper.cpp/examples/whisper.android/lib/src/main/jniLibs/arm64-v8a


---------------------------------
OpenCL-Headers, library
---------------------------------
$ git clone https://github.com/KhronosGroup/OpenCL-Headers.git
$ cd OpenCL-Headers

//$ cmake -S . -B build -DCMAKE_INSTALL_PREFIX=/chosen/install/prefix
//$ cmake --build build --target install

$ cmake -S . -B build
$ cmake --build build

// result: USE CL/*.h

$ cd ..

// renamed above (libGLES_mali.so == libOpenCL.so)
$ cp libOpenCL.so ./OpenCL-Headers/


---------------------------------
CBlast
---------------------------------
$ git clone https://github.com/CNugteren/CLBlast.git
$ cd CLBlast
$ mkdir build && cd build

$ /work/android/sdk/cmake/3.22.1/bin/cmake .. \
    -DCMAKE_SYSTEM_NAME=Android \
    -DCMAKE_SYSTEM_VERSION=33 \
    -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \
    -DCMAKE_ANDROID_NDK=/work/android/sdk/ndk/25.2.9519653 \
    -DCMAKE_ANDROID_STL_TYPE=c++_static \
    -DOPENCL_ROOT=/work/OpenCL-Headers \
    -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \
    -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH \
    -DOPENCL_LIB=/work/libOpenCL.so \
&& make -j4

$ cd ../..


---------------------------------
GGML
---------------------------------
$ git clone https://github.com/ggerganov/ggml.git
$ cd ggml
$ mkdir build && cd build

$ /work/android/sdk/cmake/3.22.1/bin/cmake .. \
    -DGGML_CLBLAST=ON \
    -DCMAKE_SYSTEM_NAME=Android \
    -DCMAKE_SYSTEM_VERSION=33 \
    -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \
    -DCMAKE_ANDROID_NDK=/work/android/sdk/ndk/25.2.9519653 \
    -DCMAKE_ANDROID_STL_TYPE=c++_shared \
    -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH \
    -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \
    -DCLBLAST_HOME=/work/CLBlast \
    -DOPENCL_LIB=/work/libOpenCL.so \
&& make -j4

$ cd ../..


---------------------------------
whisper.cpp
---------------------------------
$ git clone https://github.com/ggerganov/whisper.cpp.git
$ cd whisper.cpp
$ make

$ cd ..


---------------------------------
copy assets
---------------------------------
// whisper.cpp Models
https://huggingface.co/ggerganov/whisper.cpp

// Download Model: Tiny
// https://huggingface.co/ggerganov/whisper.cpp/blob/main/ggml-tiny.bin
$ mkdir models
$ wget https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-tiny.bin -P models/


// whisper.cpp models
$ mkdir -p ./whisper.cpp/examples/whisper.android/app/src/main/assets/models
$ cp ./models/ggml-tiny.bin ./whisper.cpp/examples/whisper.android/app/src/main/assets/models

// whisper.cpp samples
$ mkdir -p ./whisper.cpp/examples/whisper.android/app/src/main/assets/samples
$ cp ./whisper.cpp/samples/jfk.wav ./whisper.cpp/examples/whisper.android/app/src/main/assets/samples


// libs (.so)
// NOTE: libs (.so) below is for Samsung Galaxy A32
//
// You should find dependencies (.so).
//
//
// [Option #1]
//
// e.g., 'libGLES_mali.so' first
//
// $ adb pull /system/vendor/lib64/egl/libGLES_mali.so
// $ readelf -d libGLES_mali.so
// Dynamic section at offset 0x2537748 contains 42 entries:
//   Tag        Type                         Name/Value
//  0x0000000000000001 (NEEDED)             Shared library: [liblog.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libz.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libnativewindow.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libged.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libgpu_aux.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libgpud.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libgralloc_extra.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libcutils.so]
//  0x0000000000000001 (NEEDED)             Shared library: [[email protected]]
//  0x0000000000000001 (NEEDED)             Shared library: [libgralloctypes.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libhidlbase.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libutils.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libm.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libc.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libdl.so]
//  0x0000000000000001 (NEEDED)             Shared library: [libc++.so]
//  0x000000000000000e (SONAME)             Library soname: [libGLES_mali.so]
//  ...
//
// then copy (pull) the (NEEDED) .so files.
// $ mkdir libs
// $ adb pull <path>/.so ./libs
// You have to run 'readelf' for these (NEEDED) .so files again and copy (pull) it.
//
// or
//
//
// [Option #2]
// Easy, but size is too large. I cannot recommend this approach.
//
// copy all libs (*.so)
//
// (Android device)/system/vendor/lib64/*.so
// (Android device)/system/vendor/lib64/egl/libGLES_mali.so
// (Android device)/system/lib64/*.so
//
//
//
// Below instruction is for [Option #1]
//
// NOTE: libs (.so) below is for Samsung Galaxy A32
//
$ mkdir libs
$ adb pull /system/vendor/lib64/egl/libGLES_mali.so ./libs
$ adb pull /system/vendor/lib64/libged.so ./libs
$ adb pull /system/vendor/lib64/libgpu_aux.so ./libs
$ adb pull /system/vendor/lib64/libgpud.so ./libs
$ adb pull /system/vendor/lib64/libgralloc_extra.so ./libs
$ adb pull /system/vendor/lib64/libdpframework.so ./libs
$ adb pull /system/vendor/lib64/[email protected] ./libs
$ adb pull /system/vendor/lib64/libion_mtk.so ./libs
$ adb pull /system/vendor/lib64/libpq_prot.so ./libs
$ adb pull /system/vendor/lib64/[email protected] ./libs
$ adb pull /system/vendor/lib64/[email protected] ./libs
$ adb pull /system/vendor/lib64/[email protected] ./libs
$ adb pull /system/vendor/lib64/[email protected] ./libs
$ adb pull /system/vendor/lib64/[email protected] ./libs
$ adb pull /system/vendor/lib64/libion_ulit.so ./libs
$ adb pull /system/vendor/lib64/libladder.so ./libs
//
$ adb pull /system/lib64/libcutils.so ./libs
$ adb pull /system/lib64/[email protected] ./libs
$ adb pull /system/lib64/libgralloctypes.so ./libs
$ adb pull /system/lib64/libhidlbase.so ./libs
$ adb pull /system/lib64/libutils.so ./libs
$ adb pull /system/lib64/libc++.so ./libs
$ adb pull /system/lib64/libhardware.so ./libs
$ adb pull /system/lib64/[email protected] ./libs
$ adb pull /system/lib64/[email protected] ./libs
$ adb pull /system/lib64/[email protected] ./libs
$ adb pull /system/lib64/[email protected] ./libs
$ adb pull /system/lib64/[email protected] ./libs
$ adb pull /system/lib64/[email protected] ./libs
$ adb pull /system/lib64/libion.so ./libs
$ adb pull /system/lib64/libdmabufheap.so ./libs
$ adb pull /system/lib64/libutilscallstack.so ./libs
$ adb pull /system/lib64/libbase.so ./libs
$ adb pull /system/lib64/android.hardware.graphics.common-V3-ndk.so ./libs
$ adb pull /system/lib64/libvndksupport.so ./libs
$ adb pull /system/lib64/libhidlmemory.so ./libs
$ adb pull /system/lib64/libbacktrace.so ./libs
$ adb pull /system/lib64/android.hardware.common-V2-ndk.so ./libs
$ adb pull /system/lib64/libdl_android.so ./libs
$ adb pull /system/lib64/[email protected] ./libs
$ adb pull /system/lib64/[email protected] ./libs
$ adb pull /system/lib64/libunwindstack.so ./libs
$ adb pull /system/lib64/ld-android.so ./libs
$ adb pull /system/lib64/liblzma.so ./libs
//
$ cp ./libs/*.so ./whisper.cpp/examples/whisper.android/lib/src/main/jniLibs/arm64-v8a/


---------------------------------
build whisper.android
---------------------------------
// Android Studio: gradle.properties
//
GGML_HOME=/work/ggml
GGML_CLBLAST=ON
CLBLAST_HOME=/work/CLBlast
OPENCL_LIB=/work/libOpenCL.so
OPENCL_ROOT=/work/OpenCL-Headers


// Android Studio: build.gradle (:lib)
// Use only here: "arm64-v8a"
//
ndk {
    //abiFilters "arm64-v8a", "armeabi-v7a", "x86", "x86_64
    abiFilters "arm64-v8a"
}


// comment-out
// issues:
//  - ld: error: unknown argument "--copy-dt-needed-entries"
//
// (EDIT) /work/ggml/src/CMakeLists.txt:202
//
$ vim /work/ggml/src/CMakeLists.txt +202
(Before)
        link_libraries("-Wl,--copy-dt-needed-entries")
(After: comment-out)
#        link_libraries("-Wl,--copy-dt-needed-entries")


// Android Studio
clean & rebuild & run...


Run in Release mode is more faster.

Screenshots

UI #1: Debug mode

UI #2: Release mode (more faster)

UI #3: Translation Supported Languages

whisper.cpp.android's People

Contributors

godmode2k avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.