When using the AWS deployment script, the Terraform does not connect a capacity provider to the LLM Service. Additionally, one of the 2 Fargate tasks continuously fails.

The instructions are also not clear on how to launch the UI and connect it to this piece.

in local (gpt) needs CUDA?

No dependencies to install or update

Installing the current project: rag-stack (0.1.0)
llm/local/ggml-gpt4all-j-v1.3-groovy.bin already exists, skipping download.

💠 Starting Qdrant...
1b4a0b67ed95f9ddce2db5558dafa2c4604e4e714cdf24412febb650643d47cd

🔮 Ragstack is almost ready.
Access the UI at http://localhost:5173 and send queries to http://localhost:8080/ask-question

Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/rag-stack-kU5FsrDq-py3.10/lib/python3.10/site-packages/torch/init.py", line 168, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File "/usr/lib/python3.10/ctypes/init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: libcurand.so.10: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/chatmyfile/server/server/main.py", line 22, in
from vectorstore import QdrantVectorStore
File "/home/chatmyfile/server/vectorstore/init.py", line 1, in
from .qdrant_vectorstore import QdrantVectorStore
File "/home/chatmyfile/server/vectorstore/qdrant_vectorstore.py", line 7, in
from sentence_transformers import SentenceTransformer
File "/root/.cache/pypoetry/virtualenvs/rag-stack-kU5FsrDq-py3.10/lib/python3.10/site-packages/sentence_transformers/init.py", line 3, in
from .datasets import SentencesDataset, ParallelSentencesDataset
File "/root/.cache/pypoetry/virtualenvs/rag-stack-kU5FsrDq-py3.10/lib/python3.10/site-packages/sentence_transformers/datasets/init.py", line 1, in
from .DenoisingAutoEncoderDataset import DenoisingAutoEncoderDataset
File "/root/.cache/pypoetry/virtualenvs/rag-stack-kU5FsrDq-py3.10/lib/python3.10/site-packages/sentence_transformers/datasets/DenoisingAutoEncoderDataset.py", line 1, in
from torch.utils.data import Dataset
File "/root/.cache/pypoetry/virtualenvs/rag-stack-kU5FsrDq-py3.10/lib/python3.10/site-packages/torch/init.py", line 228, in
_load_global_deps()
File "/root/.cache/pypoetry/virtualenvs/rag-stack-kU5FsrDq-py3.10/lib/python3.10/site-packages/torch/init.py", line 189, in _load_global_deps
_preload_cuda_deps(lib_folder, lib_name)
File "/root/.cache/pypoetry/virtualenvs/rag-stack-kU5FsrDq-py3.10/lib/python3.10/site-packages/torch/init.py", line 154, in _preload_cuda_deps
raise ValueError(f"{lib_name} not found in the system path {sys.path}")
ValueError: libcublas.so.*[0-9] not found in the system path ['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/root/.cache/pypoetry/virtualenvs/rag-stack-kU5FsrDq-py3.10/lib/python3.10/site-packages', '/home/chatmyfile/server']
Stopping Docker containers...
qdrant
qdrant

This maybe a simple question for you guys but I am a new guy to all this and still learing.

can anyone please tell me where the PDF are stored while I am using them? I want to start from scratch and i dont know how to remove the old PDFs

Added weave-cleint as dependency but still not getting the ModuleNotFoundError: No module named 'weaviate' ?
how to get rid of this error ?

also getting this when tried to install weave-client explicitly
mac@Hakeye server % poetry install
Installing dependencies from lock file
Warning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run poetry lock [--no-update] to fix it.

Because rag-stack depends on weaviate-client (3.23.2) which doesn't match any versions, version solving failed.

Hey guys! I'm getting the following errors:

INFO: 127.0.0.1:50582 - "POST /upsert-files HTTP/1.1" 401 Unauthorized
true
INFO: 127.0.0.1:50582: "GET /get-previews HTTP/1.1" 401 Unauthorized

It happens every time I just try to upload a file. Did I miss something in the setup?

libcublas.so.*[0-9] not found in the system path
in Local install I get above error.

Did anyone else go through this?

thanks!

Hello everyone!

First, I would like to thank you for the amazing work you've been doing.

I need your help! I'm a newbie on GCP and I'm trying to deploy the rag-stack falcon7b to it.

I'm getting an error on deploy-gcp.sh. Below is the trace I'm getting:

guilhermedomingues@cloudshell:~/rag-stack/scripts/gcp (llama-rag-test)$ sh deploy-gcp.sh
____ _____ __ __
/ __ ____ _____ / // /_ / /
/ // / __ / __ /_ / _/ __ `/ / ///
/ , / // / // // / // // / // ,<
// ||_,/_, ///_/_,/_//||
/____/

Enter your GCP project ID: llama-rag-test
(https://cloud.google.com/iam/docs/keys-create-delete#creating) Enter the path to your GCP service account key file: llama-rag-test-f40c5f7db02f.json
Enter the GCP region (default: us-west1): us-central1-c
Enter your Huggingface API Token: MY_HUGGING_API
Model to deploy (llama2-7b or falcon7b): falcon7b

Initializing the backend...
Initializing modules...

Initializing provider plugins...

• Reusing previous version of hashicorp/kubernetes from the dependency lock file
• Reusing previous version of hashicorp/google from the dependency lock file
• Using previously-installed hashicorp/kubernetes v2.23.0
• Using previously-installed hashicorp/google v4.51.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Success! The configuration is valid.

module.gke-cluster.google_container_cluster.gpu_cluster: Refreshing state... [id=projects/llama-rag-test/locations/us-central1-c/clusters/gpu-cluster]
module.gke-cluster.google_container_node_pool.primary_preemptible_nodes: Refreshing state... [id=projects/llama-rag-test/locations/us-central1-c/clusters/gpu-cluster/nodePools/gpu-node-pool]
data.google_client_config.default: Reading...
data.google_container_cluster.default: Reading...
data.google_client_config.default: Read complete after 0s [id=projects/llama-rag-test/regions/us-central1-c/zones/]
data.google_container_cluster.default: Read complete after 0s [id=projects/llama-rag-test/locations/us-central1-c/clusters/gpu-cluster]
kubernetes_service.falcon7b_service[0]: Refreshing state... [id=default/falcon7b-service]
kubernetes_deployment.falcon7b[0]: Refreshing state... [id=default/falcon7b]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:

• create

Terraform will perform the following actions:

google_cloud_run_service.qdrant will be created

• resource "google_cloud_run_service" "qdrant" {

  • autogenerate_revision_name = false

  • id = (known after apply)

  • location = "us-central1-c"

  • name = "qdrant"

  • project = (known after apply)

  • status = (known after apply)

  • template {

    • spec {

      • container_concurrency = (known after apply)

      • service_account_name = (known after apply)

      • serving_state = (known after apply)

      • timeout_seconds = (known after apply)

      • containers {

        • image = "qdrant/qdrant:v1.3.0"

        • ports {

          • container_port = 6333
          • name = (known after apply)
            }
            }
            }
            }

  • traffic {

    • latest_revision = true
    • percent = 100
    • url = (known after apply)
      }
      }

google_cloud_run_service.ragstack-server will be created

• resource "google_cloud_run_service" "ragstack-server" {

  • autogenerate_revision_name = false

  • id = (known after apply)

  • location = "us-central1-c"

  • name = "ragstack-server"

  • project = (known after apply)

  • status = (known after apply)

  • template {

    • spec {

      • container_concurrency = (known after apply)

      • service_account_name = (known after apply)

      • serving_state = (known after apply)

      • timeout_seconds = (known after apply)

      • containers {

        • image = "jfan001/ragstack-server:latest"

        • env {

          • name = "LLM_URL"
          • value = "http://35.193.123.142"
            }

        • env {

          • name = "QDRANT_PORT"
          • value = "443"
            }

        • env {

          • name = "QDRANT_URL"
          • value = (known after apply)
            }

        • resources {

          • limits = {
            • "memory" = "2Gi"
              }
              }
              }
              }
              }

  • traffic {

    • latest_revision = true
    • percent = 100
    • url = (known after apply)
      }
      }

google_cloud_run_service_iam_member.public will be created

• resource "google_cloud_run_service_iam_member" "public" {
  • etag = (known after apply)
  • id = (known after apply)
  • location = "us-central1-c"
  • member = "allUsers"
  • project = (known after apply)
  • role = "roles/run.invoker"
  • service = "qdrant"
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

google_cloud_run_service.qdrant: Creating...
╷
│ Error: Error creating Service: googleapi: got HTTP response code 404 with body:
│
│
│
│ <title>Error 404 (Not Found)!!1</title>
│ <style>
│ {margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px} > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
│ </style>
│
│

404. That’s an error.
│

The requested URL /apis/serving.knative.dev/v1/namespaces/llama-rag-test/services was not found on this server. That’s all we know.
│
│
│ with google_cloud_run_service.qdrant,
│ on main.tf line 195, in resource "google_cloud_run_service" "qdrant":
│ 195: resource "google_cloud_run_service" "qdrant" {

Can you help me on this?

Thank you again!

Have a nice weekend! :)

I have successfully install the dependencies, but when i try to Run Locally, by running scripts/local/run-dev. There is error below:

Found model file at  llm/local/ggml-gpt4all-j-v1.3-groovy.bin
Invalid model file
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/demo/miniconda3/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/demo/learn/GPT/rag-stack/server/server/main.py", line 38, in <module>
    llm = get_selected_llm()
  File "/home/demo/learn/GPT/rag-stack/server/llm/llm.py", line 24, in get_selected_llm
    return Gpt4AllLLM()
  File "/home/demo/learn/GPT/rag-stack/server/llm/llm.py", line 40, in __init__
    self.llm = GPT4All(
  File "/home/demo/.cache/pypoetry/virtualenvs/rag-stack-5gekgb59-py3.10/lib/python3.10/site-packages/langchain/load/serializable.py", line 74, in __init__
    super().__init__(**kwargs)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
  Unable to instantiate model (type=value_error)
Stopping Docker containers...

Further information:
~ cat /etc/os-release

PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

~ python --version
Python 3.10.9

~ nvidia-smi

Sun Aug  6 11:28:10 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A30          On   | 00000000:00:10.0 Off |                    0 |
| N/A   32C    P0    29W / 165W |   6255MiB / 24576MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA A30          On   | 00000000:00:11.0 Off |                    0 |
| N/A   31C    P0    29W / 165W |      7MiB / 24576MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     12237      G   /usr/lib/xorg/Xorg                  4MiB |
|    0   N/A  N/A    638035      C   /usr/bin/python3                 3196MiB |
|    0   N/A  N/A   2951384      C   ...keai/.venv/bin/python3.10     3052MiB |
|    1   N/A  N/A     12237      G   /usr/lib/xorg/Xorg                  4MiB |
+-----------------------------------------------------------------------------+

~ pip list

Package                   Version
------------------------- ----------
absl-py                   1.4.0
accelerate                0.20.3
aiofiles                  23.1.0
aiohttp                   3.8.5
aiosignal                 1.3.1
altair                    5.0.1
antlr4-python3-runtime    4.9.3
anyio                     3.7.1
appdirs                   1.4.4
APScheduler               3.10.1
arrow                     1.2.3
async-timeout             4.0.2
attrs                     23.1.0
Authlib                   1.2.1
auto-gptq                 0.2.2
backoff                   2.2.1
beautifulsoup4            4.12.2
bioc                      2.0
bitsandbytes              0.39.0
blinker                   1.6.2
boltons                   23.0.0
boto3                     1.26.101
botocore                  1.29.101
bravado                   11.0.3
bravado-core              6.1.0
brotlipy                  0.7.0
cachetools                5.3.0
certifi                   2022.12.7
cffi                      1.15.1
chardet                   5.2.0
charset-normalizer        3.2.0
chromadb                  0.3.25
click                     8.1.6
clickhouse-connect        0.6.8
cmake                     3.27.0
colorama                  0.4.6
coloredlogs               15.0.1
conda                     23.3.1
conda-content-trust       0.1.3
conda-package-handling    2.0.2
conda_package_streaming   0.7.0
contourpy                 1.1.0
cryptography              41.0.3
cycler                    0.11.0
dataclasses-json          0.5.14
datasets                  2.13.0
decorator                 5.1.1
dill                      0.3.6
diskcache                 5.6.1
docutils                  0.20.1
docx2txt                  0.8
duckdb                    0.8.1
effdet                    0.4.1
einops                    0.6.1
et-xmlfile                1.1.0
evaluate                  0.4.0
exceptiongroup            1.1.2
execnet                   2.0.2
faiss-cpu                 1.7.4
fastapi                   0.101.0
ffmpy                     0.3.1
filelock                  3.12.2
filetype                  1.2.0
fire                      0.5.0
Flask                     2.3.2
Flask-SQLAlchemy          3.0.5
flatbuffers               23.5.26
fonttools                 4.42.0
fqdn                      1.5.1
frozenlist                1.4.0
fsspec                    2023.6.0
future                    0.18.3
gitdb                     4.0.10
GitPython                 3.1.32
google-auth               2.17.2
google-auth-oauthlib      1.0.0
gpt4all                   1.0.5
gradio                    3.39.0
gradio_client             0.3.0
greenlet                  2.0.2
grpcio                    1.53.0
h11                       0.14.0
hnswlib                   0.7.0
httpcore                  0.17.3
httptools                 0.6.0
httpx                     0.24.1
huggingface-hub           0.16.4
humanfriendly             10.0
idna                      3.4
importlib-metadata        6.8.0
iniconfig                 2.0.0
InstructorEmbedding       1.0.1
intervaltree              3.1.0
iopath                    0.1.10
isoduration               20.11.0
itsdangerous              2.1.2
Jinja2                    3.1.2
jmespath                  1.0.1
joblib                    1.3.1
jsonlines                 3.1.0
jsonpatch                 1.32
jsonpointer               2.1
jsonref                   1.1.0
jsonschema                4.18.6
jsonschema-specifications 2023.7.1
kiwisolver                1.4.4
langchain                 0.0.250
langsmith                 0.0.19
layoutparser              0.3.4
linkify-it-py             2.0.2
lit                       16.0.6
llama-cpp-python          0.1.73
lm-dataformat             0.0.20
loralib                   0.1.1
lxml                      4.9.3
lz4                       4.3.2
Markdown                  3.4.3
markdown-it-py            2.2.0
MarkupSafe                2.1.2
marshmallow               3.20.1
matplotlib                3.7.1
mdit-py-plugins           0.3.3
mdurl                     0.1.2
monotonic                 1.6
mpmath                    1.3.0
msg-parser                1.2.0
msgpack                   1.0.5
multidict                 6.0.4
multiprocess              0.70.14
mypy-extensions           1.0.0
neptune                   1.2.0
networkx                  3.1
nltk                      3.8.1
numexpr                   2.8.4
numpy                     1.24.3
nvidia-cublas-cu11        11.10.3.66
nvidia-cuda-cupti-cu11    11.7.101
nvidia-cuda-nvrtc-cu11    11.7.99
nvidia-cuda-runtime-cu11  11.7.99
nvidia-cudnn-cu11         8.5.0.96
nvidia-cufft-cu11         10.9.0.58
nvidia-curand-cu11        10.2.10.91
nvidia-cusolver-cu11      11.4.0.1
nvidia-cusparse-cu11      11.7.4.91
nvidia-nccl-cu11          2.14.3
nvidia-nvtx-cu11          11.7.91
oauthlib                  3.2.2
olefile                   0.46
omegaconf                 2.3.0
onnxruntime               1.15.1
openai                    0.27.8
openapi-schema-pydantic   1.2.4
opencv-python             4.8.0.74
openpyxl                  3.1.2
orjson                    3.9.2
overrides                 7.3.1
packaging                 23.1
pandas                    2.0.2
pdf2image                 1.16.3
pdfminer.six              20221105
pdfplumber                0.10.2
peft                      0.4.0
Pillow                    9.5.0
pip                       22.3.1
pip-licenses              4.3.0
pluggy                    1.0.0
portalocker               2.7.0
posthog                   3.0.1
prettytable               3.8.0
protobuf                  3.20.0
psutil                    5.9.5
pyarrow                   12.0.1
pyasn1                    0.4.8
pyasn1-modules            0.2.8
pycocotools               2.0.6
pycosat                   0.6.4
pycparser                 2.21
pydantic                  1.10.12
pydub                     0.25.1
PyJWT                     2.8.0
pynvml                    11.5.0
pyOpenSSL                 23.0.0
pypandoc                  1.11
pypandoc-binary           1.11
pyparsing                 3.1.1
pypdf                     3.14.0
pypdfium2                 4.18.0
pyphen                    0.14.0
PySocks                   1.7.1
pytesseract               0.3.10
pytest                    7.2.2
pytest-xdist              3.2.1
python-dateutil           2.8.2
python-docx               0.8.11
python-dotenv             1.0.0
python-magic              0.4.27
python-multipart          0.0.6
python-pptx               0.6.21
pytz                      2023.3
PyYAML                    6.0.1
referencing               0.30.2
regex                     2023.6.3
replicate                 0.10.0
requests                  2.31.0
requests-file             1.5.1
requests-oauthlib         1.3.1
responses                 0.18.0
rfc3339-validator         0.1.4
rfc3987                   1.3.8
rouge                     1.0.1
rouge-score               0.1.2
rpds-py                   0.9.2
rsa                       4.9
ruamel.yaml               0.17.21
ruamel.yaml.clib          0.2.6
s3transfer                0.6.1
sacrebleu                 2.3.1
safetensors               0.3.1
scikit-learn              1.2.2
scipy                     1.11.1
semantic-version          2.10.0
sentence-transformers     2.2.2
sentencepiece             0.1.99
setuptools                65.6.3
simplejson                3.19.1
six                       1.16.0
smmap                     5.0.0
sniffio                   1.3.0
sortedcontainers          2.4.0
soupsieve                 2.4.1
SQLAlchemy                2.0.19
starlette                 0.27.0
swagger-spec-validator    3.0.3
sympy                     1.12
tabulate                  0.9.0
tenacity                  8.2.2
tensorboard               2.13.0
tensorboard-data-server   0.7.0
tensorboard-plugin-wit    1.8.1
tensorboardX              2.6
termcolor                 2.3.0
text-generation           0.6.0
textstat                  0.7.3
threadpoolctl             3.2.0
tiktoken                  0.4.0
timm                      0.9.5
tokenizers                0.13.3
tomli                     2.0.1
toolz                     0.12.0
torch                     2.0.1
torchvision               0.15.2
tqdm                      4.65.0
transformers              4.30.2
triton                    2.0.0
typing_extensions         4.7.1
typing-inspect            0.9.0
tzdata                    2023.3
tzlocal                   5.0.1
uc-micro-py               1.0.2
ujson                     5.8.0
unstructured              0.9.0
unstructured-inference    0.5.7
uri-template              1.3.0
urllib3                   1.26.16
uvicorn                   0.23.2
uvloop                    0.17.0
validators                0.20.0
watchfiles                0.19.0
wcwidth                   0.2.6
weaviate-client           3.22.1
webcolors                 1.13
websocket-client          1.6.1
websockets                11.0.3
Werkzeug                  2.3.6
wheel                     0.37.1
xlrd                      2.0.1
XlsxWriter                3.1.2
xxhash                    3.3.0
yarl                      1.9.2
zipp                      3.16.2
zstandard                 0.21.0

• I have tried to delete ggml-gpt4all-j-v1.3-groovy.bin and re-download it, but the above error still appeared.
• I have checked that the file llm/local/ggml-gpt4all-j-v1.3-groovy.bin exists
• My Python version is Python 3.10, using Miniconda

Now trying to check model compatibility, environment settings, dependency versions

Dear rag-stack developer,

Greetings! I am vansinhu, a community developer and volunteer at InternLM. Your work has been immensely beneficial to me, and I believe it can be effectively utilized in InternLM as well. Welcome to add Discord https://discord.gg/gF9ezcmtM3 . I hope to get in touch with you.

Best regards,
vansinhu

How hard would it be to rip out supabase for a single-user / pure-local / quick test mode?

Can you provide a proper documentation to use Supabase. I have the URL and the key but getting below error while trying to upload the file:
{'statusCode': 400, 'error': 'Invalid JWT', 'message': 'new row violates row-level security policy for table "buckets"'}

Are there any pre-requisites?

Input to deploy-aks.sh
Model to deploy: llama2-7b or falcon7b (default: falcon7b): llama2-7b

Plan: 9 to add, 0 to change, 0 to destroy.
╷
│ Error: Invalid index
│
│ on main.tf line 297, in resource "kubernetes_deployment" "rag_server":
│ 297: value = "http://${kubernetes_service.falcon7b_service[0].status[0].load_balancer[0].ingress[0].ip}"
│ ├────────────────
│ │ kubernetes_service.falcon7b_service is empty tuple
│
│ The given key does not identify an element in this collection value: the collection has no elements.

The apply searched for falcon7b service where as it wont be applicable for llama-2.
Did any one else try this?

Out of the box on an M2 Mac running Ventura

> git clone
> ./run-dev.sh

ragstack-ui comes up, but the send button posts to the wrong URL http://localhost:5173/undefined/ask-question and dies with an exception.

I edited ragstack-ui/src/utils.tsx and hard coded the post URL to http://localhost:8080/ask-question

Clicking "Send" with an empty text field, or a prompt, or attaching a PDF, generates a gibberish response.
Server log shows e.g.

INFO:     Application startup complete.
[]
DDDH<8<8888H,D<8<,8,D8H,,HH8<8<<H,H,HDH,8D<H8H<HD8HH88,<,88<H,DH8DD,,,<,H8DD88D,8H,8DH<H,H88H8,8<D<,88D,D<<888D,HH,HDHHD8<D,8<<D<,8,DHH8,D,DD<<D,D,HD8,D8H<HHD<<H,<<<H8<,,,,8H8H88H8H<DHDH8<<,8H,H,H8DH8D8H<D,,D,<,H,8,D,<<8DHHDD,H<8HDD888<D,HHHDDH8<8HD,H<D<,<INFO:     127.0.0.1:58636 - "POST /ask-question HTTP/1.1" 200 OK

Is this supported on Mac? Is there something else I need to do for it to function?

I have installed weaviate-client by pip.

🔎 Starting Weaviate...
c6ab34a6f8a9d7bf12c29dde44bc0d66f338723c8e67e9b6c600518e5113c098

🔮 Ragstack is almost ready.
Access the UI at http://localhost:5173 and send queries to http://localhost:8080/ask-question

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/unizhu/Documents/AI/rag-stack/server/server/main.py", line 22, in <module>
    from vectorstore import QdrantVectorStore
  File "/Users/unizhu/Documents/AI/rag-stack/server/vectorstore/__init__.py", line 2, in <module>
    from .weaviate_vectorstore import WeaviateVectorStore
  File "/Users/unizhu/Documents/AI/rag-stack/server/vectorstore/weaviate_vectorstore.py", line 6, in <module>
    import weaviate
ModuleNotFoundError: No module named 'weaviate'

Stopping Docker containers...
weaviate
weaviate

**macOS

**python 3.11.4

**pip list

Package            Version
------------------ ---------
Authlib            1.2.1
certifi            2023.7.22
cffi               1.15.1
charset-normalizer 3.2.0
click              7.1.2
click-params       0.3.0
cryptography       41.0.3
decorator          5.1.1
idna               3.4
pip                23.2.1
pycparser          2.21
requests           2.31.0
setuptools         65.5.0
six                1.16.0
tqdm               4.66.1
urllib3            2.0.4
validators         0.18.2
weaviate-cli       2.2.0
weaviate-client    3.22.1
wheel              0.41.1

Only say:

Hey John, client just called asking about our reports. Could we get this done before Monday?

Hi,

Currently trying to run this locally but it always complaining about some modules not installed, can we get a apt-get install command to install all the pre-requisite software that is needed?

Currently trying it on Ubuntu desktop 22.04

I keep getting several errors like the one shown below:

  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/chris/rag-stack/server/server/main.py", line 22, in <module>
    from vectorstore import QdrantVectorStore
  File "/home/chris/rag-stack/server/vectorstore/__init__.py", line 1, in <module>
    from .qdrant_vectorstore import QdrantVectorStore
  File "/home/chris/rag-stack/server/vectorstore/qdrant_vectorstore.py", line 7, in <module>
    from sentence_transformers import SentenceTransformer
  File "/root/.cache/pypoetry/virtualenvs/rag-stack-dUIYXRLo-py3.10/lib/python3.10/site-packages/sentence_transformers/__init__.py", line 3, in <module>
    from .datasets import SentencesDataset, ParallelSentencesDataset
  File "/root/.cache/pypoetry/virtualenvs/rag-stack-dUIYXRLo-py3.10/lib/python3.10/site-packages/sentence_transformers/datasets/__init__.py", line 1, in <module>
    from .DenoisingAutoEncoderDataset import DenoisingAutoEncoderDataset
  File "/root/.cache/pypoetry/virtualenvs/rag-stack-dUIYXRLo-py3.10/lib/python3.10/site-packages/sentence_transformers/datasets/DenoisingAutoEncoderDataset.py", line 1, in <module>
    from torch.utils.data import Dataset
  File "/root/.cache/pypoetry/virtualenvs/rag-stack-dUIYXRLo-py3.10/lib/python3.10/site-packages/torch/__init__.py", line 229, in <module>
    from torch._C import *  # noqa: F403
ImportError: libcupti.so.11.7: cannot open shared object file: No such file or directory

I keep installing them one by one but seems there is still more to install.

Thanks

I feel there is no detailed info on running UI. I tried building the rag-ui and run with port 5173 exposed.
But getting a blank page in browser (no errors, 200 OK in curl)
curl localhost:5173

<script type="module" src="/@vite/client"></script>

<meta charset="UTF-8" />
<link rel="icon" type="image/png" href="https://res.cloudinary.com/demo/image/upload/v1690994866/Icon_6_mo6skf.png" />
<link rel="preconnect" href="https://fonts.googleapis.com"/>
<link rel="preconnect" href="https://fonts.gstatic.com" crossOrigin="anonymous"/>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100;200;300;400;500;600;700;800;900&display=swap" rel="stylesheet"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Chat My Files</title>

But the browser is just white blank page.

@Ayan-Bandyopadhyay I have few questions:

1. Which key should I use from the below screenshot?
   

2. While trying to reset the password, I am getting below message.
   

Originally posted by @jammy23495 in #25 (comment)

Hey, I tried using rag-stack api :
http://localhost:8080/ask-question

Before this I used upsert-files API to upload a doc:
http://localhost:8080/upsert-files

But on asking a question, I'm getting gibberish response.

kindly guide me, What am I missing.. 😕

Also UI is not loading for me, So I tried using API

Model Found... but is invalid... i try download 5 time and manual with wget and curl too.

Using python3 (3.10.12)
Found model file at llm/local/ggml-gpt4all-j-v1.3-groovy.bin
Invalid model file
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/chatmyfiles/server/server/main.py", line 38, in
llm = get_selected_llm()
File "/home/chatmyfiles/server/llm/llm.py", line 24, in get_selected_llm
return Gpt4AllLLM()
File "/home/chatmyfiles/server/llm/llm.py", line 40, in init
self.llm = GPT4All(
File "/root/.cache/pypoetry/virtualenvs/rag-stack-Radxutw4-py3.10/lib/python3.10/site-packages/langchain/load/serializable.py", line 74, in init
super().init(**kwargs)
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
root
Unable to instantiate model (type=value_error)
Stopping Docker containers...
qdrant
qdrant

I follow the instructions in the ReadMe and the tips provided in other issues to make it work, but when I launch scripts/local/run-dev.sh I get the expected message "INFO: Application startup complete." but when I go to localhost:8080 I get Not Found and in the terminal I see this:

I set USE_WEAVIATE_VECTORSTORE=false in the .env file since there are no instructions in the ReadMe regarding WEAVIATE.

I have never heard of the term RAG before but know the term (embedded) vector database - is there a difference? what is it.

At least on Macs (m2) - I had to install docker (which is a pain - partly docker making a bunch of backward incompatible pivots, but Colima /Orbstack work), Poetry and then upgrade Python version to between 3.10 and 3.12 (and in the process needed pyenv) ..

Net-net you might create an auto-build that is simpler, to be able to benefit a broader community

I'm running into issues with my rag-stack deployment. It initially worked (chatted with the bot) from a local rag-ui connecting to a azure deployed cluster. However, after uploading a document, the bot stopped working.

Now, no matter what I ask I get the infamous: Hey John, client just called asking about our reports. Could we get this done before Monday?

The errors I see in the browser are:
When loading the page after authentication: [/get-previews] 404 - {detail: "Not Found"}
When attempted to chat: [/ask-question] 500 - {detail: "'ScoredPoint' object has no attribute 'title'"}

To be clear, the OPTIONS calls are responding 200OK

I'm using Azure deploy-aks.sh and falcon-7b