[Bug]: KnowledgeGraphIndex.from_documents ignoring set llm about llama_index HOT 8 OPEN

Hey @mackenziedott, I'm here to help you with any issues you're facing. I can assist with bug resolutions, answer your questions, and guide you on becoming a contributor. Let's tackle this together!

You are experiencing timeouts on the _openAI completion call because the fake API key you are using is not valid. The OpenAI API cannot authenticate the request with a fake key, resulting in the API not responding and leading to a timeout.

The CachedOpenAIApiKeys class sets a fake key ("sk-" + "a" * 48) when set_fake_key is True, which is not a valid API key. Any attempt to make a completion call with this fake key will fail, causing the retries and eventual timeouts you are seeing.

Here is a relevant code snippet that demonstrates this behavior:

class CachedOpenAIApiKeys:
    ...
    def __enter__(self) -> None:
        ...
        if self.set_fake_key:
            os.environ["OPENAI_API_KEY"] = "sk-" + "a" * 48

Additionally, the test_completion_model_with_retry function shows that the OpenAI class will retry the completion call up to the specified max_retries if it encounters an openai.APITimeoutError:

@patch("llama_index.llms.openai.base.SyncOpenAI")
def test_completion_model_with_retry(MockSyncOpenAI: MagicMock) -> None:
    mock_instance = MockSyncOpenAI.return_value
    mock_instance.completions.create.side_effect = openai.APITimeoutError(None)

    llm = OpenAI(model="text-davinci-003", max_retries=3)
    prompt = "test prompt"
    with pytest.raises(openai.APITimeoutError) as exc:
        llm.complete(prompt)

    assert exc.value.message == "Request timed out."
    assert mock_instance.completions.create.call_count == 3

To resolve this issue, you need to use a valid OpenAI API key. The fake key will not work for making actual API calls and will always result in timeouts and retries ^[1][2][3].

To continue talking to Dosu, mention @dosu.

from llama_index.

To resolve the timeouts and retries occurring during the _openAI completion call when loading documents into a KnowledgeGraphIndex using models that don't require an OpenAI key, you can use alternative models such as Konko, which do not rely on OpenAI's API. Here is an example of how you can use the Konko API to avoid using the OpenAI API:

import logging
from unittest.mock import patch
from llama_index.llms.konko import Konko
from llama_index.core.base.llms.types import ChatMessage
from llama_index.legacy.llms.konko_utils import completion_with_retry, acompletion_with_retry

# Configure logging
logger = logging.getLogger(__name__)

# Mock the completion call to avoid using OpenAI's API
@patch("llama_index.llms.konko.base.SyncKonko")
def test_completion_model_basic(MockSyncKonko):
    mock_instance = MockSyncKonko.return_value
    mock_instance.completions.create.return_value = {
        "id": "cmpl-uqkvlQyYK7bGYrRHQ0eXlWi7",
        "object": "text_completion",
        "created": 1589478378,
        "model": "text-davinci-003",
        "choices": [
            {
                "text": "\n\nThis is indeed a test",
                "index": 0,
                "logprobs": None,
                "finish_reason": "length",
            }
        ],
        "usage": {"prompt_tokens": 5, "completion_tokens": 7, "total_tokens": 12},
    }

    llm = Konko(model="text-davinci-003")
    prompt = "test prompt"
    message = ChatMessage(role="user", content="test message")

    response = completion_with_retry(is_chat_model=False, max_retries=3, prompt=prompt)
    assert response["choices"][0]["text"] == "\n\nThis is indeed a test"

    chat_response = completion_with_retry(is_chat_model=True, max_retries=3, messages=[message])
    assert chat_response["choices"][0]["message"]["content"] == "\n\nThis is indeed a test"

# Run the test
test_completion_model_basic()

This code uses the unittest.mock.patch decorator to replace the actual Konko completion call with a mock function that returns a predefined response. Additionally, it uses the completion_with_retry function to handle retries, avoiding timeouts and retries associated with the OpenAI API ^[1][2].

To continue talking to Dosu, mention @dosu.

from llama_index.

To resolve the timeouts on the _openAI completion call when loading documents into a KnowledgeGraphIndex using models that don't require an OpenAI key, you can use the retry mechanism provided by the tenacity library. This approach will help handle various API errors and retry the request automatically.

Here's how you can implement the retry mechanism:

from llama_index.legacy.llms.konko_utils import completion_with_retry

# Define the parameters for the completion call
is_chat_model = False  # Set to True if using a chat model
max_retries = 5  # Number of retries

# Define the kwargs for the completion call
kwargs = {
    # Add your specific parameters here
}

# Use the completion_with_retry function to handle retries
result = completion_with_retry(is_chat_model, max_retries, **kwargs)

# Use the result to load documents into the KnowledgeGraphIndex
# Example:
# knowledge_graph_index.load_documents(result, documents)

By using the completion_with_retry function, you can handle timeouts and other API errors more gracefully, reducing the likelihood of timeouts occurring during the document loading process ^[1][2][3].

To continue talking to Dosu, mention @dosu.

from llama_index.

@dosu
I want to try something. How do i turn off async document loading?

from llama_index.

@dosu

The issue is that I've set my LLM and Embedmodel to models that don't require an OpenAI key

from llama_index.

@dosu
Other functions (such as query_engine) that are set-up similarly do not suffer this issue

from llama_index.

@mackenziedott you are using openai-like, which uses the openai client 🤔 so the logs about using openai are kind of expected?

I'm not sure what the issue is here, but you can check the llm attached to the index with print(index._llm). This is the llm used for extracting triplets

from llama_index.

@mackenziedott you are using openai-like, which uses the openai client 🤔 so the logs about using openai are kind of expected?

I'm not sure what the issue is here, but you can check the llm attached to the index with print(index._llm). This is the llm used for extracting triplets

Thanks! I think I might know the issue, I wonder if i'm overloading my API with aysnc calls. Its weird, because i can call the endpoints with other functions (standard RAG completion) just fine. I experimented with the PropertyGraphIndex, which i run into the same issue after it parses the nodes and starts attempting to extract from text, it can do up to 6 before running into the same issue.

from llama_index.