This a repo to create a QA Retrieval Bot using both Open LLM as well as OpenAI in Databricks.
To get started please change the configuration notebook in util/notebook-config.py.

Please look at the documentation Below on configuration for specific LLM's

Please change the util/notebook-config.py to point to the location where your files are stored.

config['loc']  = <"Location where the files are stored">

Currently the code supports the following version

• Open AI
• MPT-30b Chat version
• Llama-2-13b HF-chat version
• Llama-2-70-b HF-chat version with 8-bit Quantization

The following code is tested on ML DBR GPU 13.2 Runtime

The Code to run open LLMS has been tested on the below single node cluster configurations:

• AWS : g5-12xlarge [4 A10's]
• Azure : NC24Ads_A100_v4 [1 A100]

The TGI pipeline has support to run on older Generation GPU's like the V100's but has not been tested extensively

There are two ways to convert the PDF to TXT

• Using Azure Form Recognizer To use form recognizer you need to add

config['use_azure_formrecognizer'] = True
and add the URL and KEY from the Azure portal 
config['formendpoint'] 
config['formkey']

• Using Langchain PDF converter set

config['use_azure_formrecognizer'] = False

You can use any cluster to run the OpenAI model and need to set the following configs

set model_id t0 openai
config['model_id'] = 'openai'
os.environ['OPENAI_API_KEY'] = '<your open AI API Key>'

Note : when using OpenAI you do not need to run the 02_load_model Notebook and Azure open AI keys would need extra configurations to be added

To use LLAMA-2 models, you need to agree to the terms and condition of HuggingFace and provide an API key to download the models Refer to these steps to download the key : https://huggingface.co/docs/api-inference/quicktour#get-your-api-token and set the below parameters

config['model_id'] = 'meta-llama/Llama-2-XXb-chat-hf'
config['HUGGING_FACE_HUB_TOKEN'] = '<your HF AI API Key>'

Note : to need to keep 02_load_model Notebook running to have the API running

you need to set the below config

config['model_id'] = 'mosaicml/mpt-30b-chat'

Note : to need to keep 02_load_model Notebook running to have the API running

The current notebook use the following the embedding models

• For openAI : text-embedding-ada-002
• All open LLM's : intfloat/e5-large-v2

The open LLM embedding can be changed by over-riding the Dictionary in utils/notebook-config.py