Gen AI libraries, such as OpenAI, LangChain and LlamaIndex are evolving rapidly and keep introducing breaking changes in the APIs.

The code that is based on the "latest" or unpinned libraries can work today, but will throw unexpected errors tomorrow, just because one of the latest libraries introduced a breaking change.

Identify and pin known working combinations of versions of libraries for all the notebooks.

The target platform for all the tests is Google Colab!!! It might have a different set of base libraries loaded compared to your local/hosted notebook environments. Always make sure Google Colab works. If any other environment works too - that's a nice bonus, but Google Colab is the absolute requirement.

Here are the steps:

• Remove all the pins from the pip install cell
• Test, if needed make changes to the code so it works with the latest libraries, or identify the working version such as !pip -q install "openai==0.28"
• Remove the -q flag from !pip -q install and rerun the cell - at the end of the output you'll see the line like: Successfully installed accelerate-0.25.0 backoff-2.2.1 cohere-4.39 ...
• Pin the library versions to these in the line above.
• Add back -q flag to !pip -q install , and retest the entire notebook.

Explore the possibility of automatically testing the notebooks.

https://github.com/treebeardtech/nbmake

The following command line sequence works locally for some notebooks and fails for othes, but can be a promising avenue to explore:

python3 -m venv venv
source venv/bin/activate
pip install pytest nbmake
export OPENAI_API_KEY="sk-xxxxxxxxx-add-your-openai-api-key-here-xxxxxxxx"
pytest --nbmake **/*ipynb

Plain Redis notebooks (00-03), pass the tests while connecting to the laptop-local redis. OpenAI fails with either getpass if no OPENAI_API_KEY is defined or IndexError: list index out of range.

Dolly-based notebooks timeout (exclude from tests or increase timeout) - executing Dolly on CPU is expected to take an extra long time.

The 5.2 notebook has commend out code (imports and more) than can be removed to clean-up the notebook.

In the 5.1 notebook, Under "Initialize embeddings engine" it says it's using the hugging face engine, but it's actually using OpenAI

Langchain-based notebooks display the following warning:

WARNING:langchain.vectorstores.redis.base:score_threshold is deprecated. Use distance_threshold instead.score_threshold should only be used in similarity_search_with_relevance_scores.score_threshold will be removed in a future release.

The following change would remove the warning:

# retriever=redis.as_retriever()
retriever=redis.as_retriever(search_type="similarity_distance_threshold",search_kwargs={"distance_threshold":0.5}),

Open questions:

• What are the reasonable defaults for distance_threshold?
• Any other defaults such as "k":5 worth adding?

In the 5.2 notebook, in the Initialize LLM text cell, the text incorrectly references Vertex AI as the LLM instead of Dolly.

Git is unable to checkout this repo on Windows, because the folder 06-LlamaIndex_Redis has a trailing space. i.e. the folder is actually named "06-LlamaIndex_Redis "

This was reported by a workshop attendee and I am also able to reproduce the problem.

error: invalid path '06-LlamaIndex_Redis /06.1_OpenAI_LlamaIndex_Redis.ipynb' fatal: unable to checkout working tree warning: Clone succeeded, but checkout failed.

The 5.3 notebook has an unused, commented wget cell that could be removed.