Git Product home page Git Product logo

fardjad / llama-node Goto Github PK

View Code? Open in Web Editor NEW

This project forked from atome-fe/llama-node

0.0 0.0 0.0 30.83 MB

Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on your laptop CPU. support llama/alpaca/gpt4all/vicuna/rwkv model.

Home Page: https://llama-node.vercel.app/

License: Apache License 2.0

JavaScript 20.66% Python 1.43% C 0.01% Rust 45.49% TypeScript 30.97% CSS 0.61% Makefile 0.12% HTML 0.26% MDX 0.45%

llama-node's Introduction

LLaMA Node

llama-node: Node.js Library for Large Language Model

NPM License npm npm Discord twitter

LLaMA generated by Stable diffusion

Picture generated by stable diffusion.



Introduction

This project is in an early stage and is not production ready, we do not follow the semantic versioning. The API for nodejs may change in the future, use it with caution.

This is a nodejs library for inferencing llama, rwkv or llama derived models. It was built on top of llm (originally llama-rs), llama.cpp and rwkv.cpp. It uses napi-rs for channel messages between node.js and llama thread.

Supported models

llama.cpp backend supported models (in GGML format):

llm(llama-rs) backend supported models (in GGML format):

  • GPT-2
  • GPT-J
  • LLaMA: LLaMA, Alpaca, Vicuna, Koala, GPT4All v1, GPT4-X, Wizard
  • GPT-NeoX: GPT-NeoX, StableLM, RedPajama, Dolly v2
  • BLOOM: BLOOMZ

rwkv.cpp backend supported models (in GGML format):

Supported platforms

  • darwin-x64
  • darwin-arm64
  • linux-x64-gnu (glibc >= 2.31)
  • linux-x64-musl
  • win32-x64-msvc

Node.js version: >= 16


Installation

  • Install llama-node npm package
npm install llama-node
  • Install anyone of the inference backends (at least one)

    • llama.cpp
    npm install @llama-node/llama-cpp
    • or llm
    npm install @llama-node/core
    • or rwkv.cpp
    npm install @llama-node/rwkv-cpp

Manual compilation

Please see how to start with manual compilation on our contribution guide


CUDA support

Please read the document on our site to get started with manual compilation related to CUDA support


Acknowledgments

This library was published under MIT/Apache-2.0 license. However, we strongly recommend you to cite our work/our dependencies work if you wish to reuse the code from this library.

Models/Inferencing tools dependencies

Some source code comes from


Community

Join our Discord community now! Click to join llama-node Discord

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.