Git Product home page Git Product logo

aifaq's Introduction

Hyperledger Labs AIFAQ prototype

The scope of this Hyperledger Labs project is to support the users (users, developer, etc.) to their work, avoiding to wade through oceans of documents to find information they are looking for. We are implementing an open source conversational AI tool which replies to the questions related to specific context. This is a prototype which allows to create a chatbot running a RESTful API which requires GPU. Here the official Wiki pages: Hyperledger Labs aifaq and Hyperledger Labs wiki. Please, read also the Antitrust Policy and the Code of Conduct. Every Monday we have a public meeting and the invitation is on the Hyperledger Labs calendar: [Hyperledger Labs] FAQ AI Lab calls.

Background

The system is an open source python project which implements an AI chatbot that replies to HTTP requests. The idea is to implement an open source framework/template, as example, for other communities/organizations/companies. Last results in open LLMs allow to have good performance using common HW resources.
Below the application architecture:

LLM chatbot schema

We use RAG (Retrieval Augmented Generation arxiv.org) for question answering use case. That technique aims to improve LLM answers by incorporating knowledge from external database (e.g. vector database).

The image depicts two workflow:

  1. The data ingestion workflow
  2. The chat workflow

During the ingestion phase, the system loads context documents and creates a vector database. For example, the document sources can be:

  • An online software guide (readthedocs template)
  • The GitHub issues and pull requests

In our case, they are the readthedocs guide and a wiki page.
After the first phase, the system is ready to reply to user questions.

Currently, we use the open source HuggingFace Zephyr-7b-beta, and in the future we want to investigate other open source models.
The user can query the system using HTTP requests, but we want to supply UI samples, as external module.

Open Source Version

The software is under Apache 2.0 License (please check LICENSE and NOTICE files included). We use some 3rd party libraries: here is the ASF 3rd Party License Policy and here is the information for (Assembling LICENSE and NOTICE files).

Installation

This document does not contain commercial advertisement: all the tools/products/books/materials are generic and you have to consider those as examples!
This software needs GPU for the execution: if you do not have a local GPU you could use a Cloud GPU. There are several solutions to use Cloud GPU:

  1. Cloud Provider (AWS, GCP, ...)
  2. On-Demand GPU Cloud (vast.ai, RunPod, ...)
  3. Cloud GPU IDE

Currently, I use a Cloud GPU IDE (Lightning Studio), after signup/login, create new Studio (project):

New Studio Button

select the left solution:

Select Studio Code

click on the Start button, and rename the new Studio:

Rename Studio

Then, and copy-paste the github api repo code:

Copy Paste Code

and create two folders:

  1. chromadb (it will contains vector database files)
  2. rtdocs (it will contains the ReadTheDocs documentation)

That version works with Hyperledger fabric documents (Wiki and ReadTheDocs).

Usage

Download ReadTheDocs documentation

Open a new terminal:

Open Terminal

and download the documentation executing the command below:

wget -r -A.html -P rtdocs https://hyperledger-fabric.readthedocs.io/en/release-2.5/

actually, after a minute we can interrupt (CTRL + C) because it starts to download previous versions:

Wget Command

Now, we can move into rtdocs folder and move the release-2.5 content to rtdocs. We need to compress the content of the folder, moving there and use that command:

Compress files

and move the readthedocs.tar.gz to the parent directory (../):

- mv readthedocs.tar.gz ..
- cd ..

repeating the two commands until we are into rtdocs folder:

Move Command

now remove hyperledger… folder and the content:

Compress files

uncompress the file here and remove compress file:

- tar -xzvf rtdocs.tar.gz
- rm rtdocs.tar.gz

Install requirements

Move to the parent folder and execute the command below:

pip install -r requirements.txt

Activate GPU

After the requirements installation we can switch to GPU before to execute the ingestion script:

Activate GPU

then select the L4 solution:

Select L4

and confirm (it takes some minutes).

Ingest step

Run the ingest.py script:

Run Ingest

it will create content in chromadb folder.

Run API

Now, we can run the API and test it. So, run api.py script:

Run API

and test it:

curl --header "Content-Type: application/json" --request POST --data '{"text": "How to install Hyperledger fabric?"}' http://127.0.0.1:8080/query

below the result:

Show results

Current version notes

That is a proof-of-concept: a list of future improvement below:

  1. This is the first version of the prototype and it will be installed on a GPU Cloud Server
  2. At the same time, we'd like to pass to the next step: the Hyperledger Incubation Stage
  3. We will investigate other open source models
  4. Evaluation of the system using standard metrics
  5. We would like to improve the system, some ideas are: fine-tuning, Advanced RAG, Decomposed LoRA
  6. Add "guardrails" which are a specific ways of controlling the output of a LLM, such as talking avoid specific topics, responding in a particular way to specific user requests, etc.

aifaq's People

Contributors

gcapuzzi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

aifaq's Issues

frontend: Revamp UI components

The following UI components need to be revamped. Note that we are using shadcn/ui and radix for the base components so we can support any first-class dependencies from there.

  • Responsive Navigation Bar
  • Dropdown Menu
  • Responsive Modals

Feature : Add CONTRIBUTING.md and requirements.txt files

It would be better to move correct requirements to a requirements.txt file with following content:

transformers
torch
gradio
bs4
safetensors
accelerate
sentencepiece
bitsandbytes
sentence-transformers
chromadb
langchain

I'd also like to create a CONTRIBUTING.md file describing how to setup the environment.

Any other recommendations @gcapuzzi ?

LLM Infrence speed is slow want to improve it by Groq

  • I noticed that the after asking it usually give answer very approx 2min so I was thinking to play around other LLM Infrence Model like Groq to make the output real-time.

  • Does this project active and how I can contribute to this with other open source models

  • I have setup the project on Google colab
    image

It would be great if you can guide me in the process @gcapuzzi

Feature Request : Implement JWT Authentication for Backend with SQL Database

Description:
We need to add user authentication to our backend API using JWT and store user credentials in a SQL database.

  1. Set up a SQL database to store user information:

    • Create a 'users' table with fields for username, hashed password.
  2. Implement user registration endpoint:

    • Create a /register route
    • Accept username and password in the request body
    • Hash the password before storing it in the database
    • Use SQL INSERT query to add new user to the 'users' table
    • Return appropriate success/error messages
  3. Implement user login endpoint:

    • Create a /login route
    • Use SQL SELECT query to retrieve user credentials from the database
    • Verify provided password against the stored hashed password
    • Generate a JWT token upon successful authentication
    • Return the token in the response
  4. Create JWT authentication middleware:

    • Implement a function to verify JWT tokens
    • Create a dependency that can be used to protect routes
  5. Secure relevant API endpoints:

    • Apply the JWT authentication dependency to the /query route
    • Return appropriate error messages for unauthorized access attempts

Feature Request : Implement rate limiting

Implement rate limiting in the FastAPI backend to control the number of requests a client can make within a specific time period.

Task:

  • Add rate limiting middleware to the FastAPI backend.
  • Configure rate limiting rules.

Feature Request: Collapsible Slide Bar for Frontend Chatbot Interface

A feature to add a collapsible slide bar in the frontend chatbot interface. This will enhance user experience by allowing users to expand or collapse the side bar as needed, providing a cleaner and more focused interface.

Requirements

  • A slide bar on the left side of the chatbot interface.
  • The slide bar should be collapsible with a toggle button.
  • When collapsed, the slide bar should hide completely or minimize to a small icon.
  • Smooth animation for expanding and collapsing the slide bar.
  • Ensure the feature is responsive and works well on both desktop and mobile devices.

Acceptance Criteria

  • A collapsible slide bar is present in the frontend chatbot interface.
  • The slide bar can be toggled open and closed by a button.
  • Smooth animations for the slide bar's collapsing and expanding actions.
  • The slide bar does not interfere with the chatbot's main functionality.
  • The feature is fully responsive on various screen sizes.
  • The UI/UX design follows the project's style guide.

Mockups / Examples

  • Include any relevant mockups or examples here to visualize the feature.

EHN: Integrate TanStack Query and Axios for efficient and reliable frontend-to-backend API communication.

Description

TanStack Query offers auto caching, auto refetching, and dedicated devtools without requiring a backend. It facilitates real-time inquiries, dependent and concurrent queries, window focus refetching, and makes data modifications easier via the Mutations API. In addition to providing infinite scroll, request cancellation, suspense integration, prefetching, offline support, SSR optimization, and data pickers, it effectively manages big datasets.

Task:

  • Set up TanStack Query for managing API calls and state.
  • Use Axios as the HTTP client for making requests to the backend.

ui: Proposing a new theme for the application

This issue will track updates to the application theme. Please open pull requests here with your proposed changes, such as new colors, fonts, etc. Ensure that you use properly licensed assets only.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.