ray-project / langchain-ray Goto Github PK

View Code? Open in Web Editor NEW

208.0 9.0 46.0 19.03 MB

Examples on how to use LangChain and Ray

License: Apache License 2.0

Python 100.00%

langchain llm llms ray-distributed fine-tuning-t5 generative-ai langchain-python ray ray-air

langchain-ray's Introduction

🦜️🔗 LangChain and Ray - repository of examples

Welcome to the LangChain and Ray!

Overview

LangChain and Ray are two Python libraries that are emerging as key components of the modern open source stack for LLMs (OSS LLMs). If you're a Python developer or a machine learning practitioner, these tools can be very helpful in rapidly developing LLM-based applications by making it easier to build and deploy these models.

This repo is as a go-to place for all technical examples and use cases on how to use these two libraries together.

Examples

Example	Code	Blog	Video
Open Source LLM Search Engine
Fast and Scalable Embedding Generation
Retrival-Based Question Answering System

Connect with the Ray community

You can learn and get more involved with the Ray community of developers and researchers:

Ray documentation
Official Ray site Browse the ecosystem and use this site as a hub to get the information that you need to get going and building with Ray.
Join the community on Slack Find friends to discuss your new learnings in our Slack space.
Use the discussion board Ask questions, follow topics, and view announcements on this community forum.
Join a meetup group Tune in on meet-ups to listen to compelling talks, get to know other users, and meet the team behind Ray.
Open an issue Ray is constantly evolving to improve developer experience. Submit feature requests, bug-reports, and get help via GitHub issues.
Become a Ray contributor We welcome community contributions to improve our documentation and Ray framework.

langchain-ray's People

Contributors

Stargazers

Watchers

langchain-ray's Issues

Missing dependencies (on Ubuntu 20.04 at least)

On my Ubuntu 20.04 machine, I had to add the following dependencies to get the demo to work:
$ python -m pip install beautifulsoup4
$ python -m pip install accelerate

I also found that the versions of langchain and wandb specified in requirements.txt are not compatible, as indicated by this error message:
ValueError: The Weights & Biases Langchain integration does not support versions 0.0.187 and lower. To ensure proper functionality, please use version 0.0.188 or higher.

The simplest solution was to eliminate the small number of references to wandb:
serve.py: remove the import of wand, and the one call to initialize it
local_pipeline.py: remove import of wand

With these changes, the demo runs.

embed pdf error

(ReadBinary->FlatMap->FlatMap pid=17023, ip=172.31.69.122) OSError: When reading information for key '2205.13708v1.HiJoNLP_at_SemEval_2022_Task_2_Detecting_Idiomaticity_of_Multiword_Expressions_using_Multilingual_Pretrained_Language_Models.pdf' in bucket 'ray-llm-batch-inference': AWS Error [code 15]: No response body. [repeated 157x across cluster]
(ReadBinary->FlatMap->FlatMap pid=27798, ip=172.31.75.31) invalid pdf header: b'<?xml'
(ReadBinary->FlatMap->FlatMap pid=27798, ip=172.31.75.31) EOF marker not found

IndexError: index 0 is out of bounds for axis 0 with size 0

Hi, thanks for the great work in the open-source space. I am facing the below error:
index = faiss.IndexFlatL2(len(embeddings[0])) IndexError: index 0 is out of bounds for axis 0 with size 0

The faiss index is empty. There are no embeddings?

Can you help me debug this? I really appreciate any help you can provide.

Error when querying for open_source_LLM_retrieval_qa example

I followed the example "https://github.com/ray-project/langchain-ray/tree/main/open_source_LLM_retrieval_qa", with the "Building the vector store index" and "Serving" commands as below without any problems:

python build_vector_store.py

serve run serve:deployment

But when trying "Querying"

python query.py 'What is the difference between SERVE and PACK placement groups?'

I get the following error:

Traceback (most recent call last):
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 1332, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 1281, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 1041, in _send_output
    self.send(msg)
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 979, in send
    self.connect()
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
           ^^^^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f06d13f2950>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /?query=What%20is%20the%20difference%20between%20SERVE%20and%20PACK%20placement%20groups? (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f06d13f2950>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/oia2ml/Llama-2/langchain-ray/open_source_LLM_retrieval_qa/query.py", line 6, in <module>
    response = requests.post(f"http://localhost:8000/?query={query}")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /?query=What%20is%20the%20difference%20between%20SERVE%20and%20PACK%20placement%20groups? (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f06d13f2950>: Failed to establish a new connection: [Errno 111] Connection refused'))

Tried to troubleshoot with some different changes (IP and firewall) but could not solve out the problem.
Could you please advise on this matter?
Thank you.

possible langchain prompt template error

Maybe I was wrong, but this line (line 72):
result = self.chain({"input_documents": search_results, "question": query}) in example open_source_LLM_retrieval_qa/serve.py
input_documents should be context, for context is the placeholder to use.

Need more sample code

I have problems to run this demo:

embedding_pdf_documents can run on ray cluster as specified, it's good.
open_source_LLM_retrieval_qa instead only run on local ray
The blog articles is not clear on how to run on ray cluster:

where to store faiss index? s3?
local files e.g. local_embeddings.py need to be specified by runtime_env, how?

Thanks

Unable to use the LocalHuggingFaceEmbeddings("multi-qa-mpnet-base-dot-v1")

Getting below error while loading the localhgembeddings:

error:
huggingface_hub.utils._errors.HfHubHTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/sentence-transformers/multi-qa-mpnet-base-dot-v1

Looks like the embeddings are not more available. Do we have any alternatives ?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.