Git Product home page Git Product logo

knowledge_gpt's Introduction

📖KnowledgeGPT

GitHub GitHub Repo stars GitHub forks X (formerly Twitter) Follow

Accurate answers and instant citations for your documents.

Upload your documents and get answers to your questions, with citations from the text.

Demo

Installation

Follow the instructions below to run the Streamlit server locally.

Pre-requisites

Make sure you have Python ≥3.10 installed.

Steps

  1. Clone the repository
git clone https://github.com/mmz-001/knowledge_gpt
cd knowledge_gpt
  1. Install dependencies with Poetry and activate virtual environment
poetry install
poetry shell
  1. (Optional) Avoid adding the OpenAI API every time you run the server by adding it to environment variables.
    • Make a copy of .env.example named .env
    • Add your API key to the .env file

Note: Make sure you have a paid OpenAI API key for faster completions and to avoid hitting rate limits.

  1. Run the Streamlit server
cd knowledge_gpt
streamlit run main.py

Build with Docker

Run the following commands to build and run the Docker image.

cd knowledge_gpt
docker build -t knowledge_gpt .
docker run -p 8501:8501 knowledge_gpt

Open http://localhost:8501 in your browser to access the app.

Customization

You can increase the max upload file size by changing maxUploadSize in .streamlit/config.toml. Currently, the max upload size is 25MB for the hosted version.

Tech Stack

Roadmap

  • Add support for more formats (e.g. webpages, PPTX, etc.)
  • Highlight relevant phrases in citations
  • Support scanned documents with OCR
  • More customization options (e.g. chain type, chunk size, etc.)
  • Visual PDF viewer
  • Support for Local LLMs

Contributing

All contributions are welcome!

Contributors

Big thanks to the following people for their contributions!

License

Distributed under the MIT License. See LICENSE for more information.

Star History

Star History Chart

knowledge_gpt's People

Contributors

mmz-001 avatar rabiaedayilmaz avatar shantanuo avatar umeed-salman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

knowledge_gpt's Issues

Rate limit reached

Rate limit reached for default-global-with-image-limits in organization org-Yu8pTMWbwQ0jvtsUA1H1py29 on requests per min. Limit: 60.000000 / min. Current: 70.000000 / min. Contact [email protected] if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.

Modulenotfounderror

Hello, I keep getting the following error when running main.py. I forked the app and followed the instructions to install it. I apologize if this is a trivial issue that I shouldn't be posting here.

2024-01-14 10:26:33.229 Uncaught app exception
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 534, in _run_script
exec(code, module.dict)
File "/Users/marcionegrao/knowledge_gpt/knowledge_gpt/main.py", line 3, in
from knowledge_gpt.components.sidebar import sidebar
ModuleNotFoundError: No module named 'knowledge_gpt'

This model's maximum context length is 4097 tokens, however you requested 7203 tokens (6947 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.

I uploaded a Chinese txt file(zy.txt).
Input chinese prompt: “什么是精气学说”
input english prompt:"What is the concept of Jingqi theory",
Whether inputting Chinese or English, the following errors will always occur:
This model's maximum context length is 4097 tokens, however you requested 7203 tokens (6947 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.
zy.txt

Use code and functions in script (without streamlit GUI)

Thanks for writing this extremely useful project and making it open source.
I was wondering if you have a script for the functions to be used in another python script (without streamlit functions / UI elements). The use case for this is to automate the process by looping through some locally stored files with nested loops for some questions and save the output into csv file. Unfortunately my current python skills are not strong enough to extract these functions from the code you provided (specially that I have not used streamlit before).
also I second the request by others of being able to save the embeddings locally so the script will not have to re calculate them every time the same file is loaded (saves time and saves bandwidth / api tokens).
thanks again.

Can you automatically generate questions from a document and provide answers? like FAQ

Hi Sasmitha,

I love what you did in this knowledge_gpt. Thank you very much for sharing it!

  1. I think it would be good that GPT can come up questions from the document, then also provide the answers, just like a FAQ. Can you try to add this new feature?
  2. Are you hosting the app in Steamlit cloud? I am new to Steamlit, and I couldn't figure out how you made this app onto Streamlit cloud, especially how to handle those imports. Could you give us a detailed step-by-step guide?

Thanks a lot!

Leo

Windows: ModuleNotFoundError: No module named 'openai'

During the "poetry install step", there is an error msg:
"Warning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run poetry lock [--no-update] to fix it."

Afterwards, when I run "streamlit run main.py", there's error msg:
ModuleNotFoundError: No module named 'openai'

Traceback:
File "C:\Users\souler c\knowledge_gpt.venv\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 565, in _run_script
exec(code, module.dict)
File "C:\Users\souler c\knowledge_gpt\knowledge_gpt\main.py", line 3, in
from openai.error import OpenAIError

I'll also attach cmd & streamlit detail for you:
Streamlit _ERROR_MSG
cmd history.txt

Error message

I get this error message while trying to parse the pdf:

IndexError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you're on Streamlit Cloud, click on 'Manage app' in the lower right of your app).

docker image

Hi, are you planning to create docker image?
I can provide you dockerfile, if needed.

Saved answers for similar queries

What if GPT records the responses in a database and pulls the response from the database instead of sending new requests to OpenAI for similar queries?

Connection timed out

Hey! Ive tried cloning this repo, installed all dependencies and then deployed it to streamlit, but I get a "please wait" that never ends and then a connection time-out. Maybe Im missing something?

ImportError: cannot import name 'faq' from partially initialized module

Not sure why I'm getting this error?
ImportError: cannot import name 'faq' from partially initialized module 'knowledge_gpt.components.sidebar' (most likely due to a circular import) (C:\Users\xxx\OneDrive\Documents\GitHub\knowledge_gpt\knowledge_gpt\components\sidebar.py)
Traceback:
File "C:\Users\xxx\OneDrive\Documents\GitHub\knowledge_gpt.venv\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 565, in _run_script
exec(code, module.dict)
File "C:\Users\xxx\OneDrive\Documents\GitHub\knowledge_gpt\knowledge_gpt\main.py", line 4, in
from knowledge_gpt.components.sidebar import sidebar
File "C:\Users\xxx\OneDrive\Documents\GitHub\knowledge_gpt\knowledge_gpt\components\sidebar.py", line 3, in
from knowledge_gpt.components.sidebar import faq

how can I prevent triggle the limit?

Rate limit reached for default-global-with-image-limits in organization org-ks3x7KSLAWTqCJv1Yq9x3FmZ on requests per min. Limit: 60.000000 / min. Current: 80.000000 / min. Contact [email protected] if you continue to have issues.

how can I prevent triggle the limit?

Thanks

Deprecated functions

Should I change st.cache and st.experimental_memo? When I try with st.cache_data raises an error for an unexpected argument: "allow_output_mutation=True"

Modifying qa.py to allow for the model kwarg

With the release of gpt-4 via the API there can be a parameter passed for model specifying gpt-3.5-turbo or gpt-4; however the current qa.py expects model to be debug or opanai

Error Running App

I recently posted an issue titled "Over resource limits on Streamlit Cloud." but now the website just says, "Oh no. Error running app." Please solve this issue.

Add ability to use other APIs?

For reviewing confidential documents, we'd prefer to use our own APIs, either view oobabooga or koboldcpp. Is there the possibility of including options for using alternate APIs, including local or Claude?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.