Broken compatibility with `BaseLangchainStreamingResponse` when using custom `input_variables` about lanarky HOT 5 CLOSED

Thank you, very possible that I am doing something wrong. Thank you also for explaining background.

I know I’m passing a dict correctly.
Did you happen to have tried also with LLMChain instead of ConversationChain?
Is it crucial to pass the Chain as a dependency instead of creating it within the body of the function? (I tried both)

I will try more tomorrow.

from lanarky.

@ajndkr my bad, I was missing the .from_chain (previous version did not need that).

tried now with most recent version, using StreamingResponse, works fine too.

Thank you very much <3

from lanarky.

Hi! thanks for bug report! I will look into this.

from lanarky.

@marcovirgolin I don't think there's a bug in the code but likely a small error in how you are using BaseLangchainStreamingResponse. In case of multiple inputs, your fastapi application should look something like:

@app.post("/chat")
async def chat(
    request: QueryRequest,
    chain: ConversationChain = Depends(conversation_chain),
) -> LLMChainStreamingResponse:
    return LLMChainStreamingResponse.from_chain(
        chain, request.dict(), media_type="text/event-stream"
    )

here request.dict() is a dictionary: {number-one": "<value>", "number-two": "<value>"}. QueryRequest is a pydantic model to define the request body parameters.

Your second question about self.background: it is supposed to be an instance of BackgroundTask. You can learn more about them here: https://fastapi.tiangolo.com/tutorial/background-tasks/

We use it to run some operation after the chain execution is complete. It is useful in situations where you might want to update a database with new input and chain output.

from lanarky.

If possible, can you share your script? It will be helpful for me to debug this error.

from lanarky.