When prompting the LLM through the proxy for something that does not require any of the tools, (Ex: "Write me a poem about the moon" or "How are you today") I get Error: 500 {"tool_calls":[]}.

I see there is logic in place for is_normal_chat but as long as the user has any function defined this variable can never be true.

Also, if forcing is_normal_chat, I get a different error Error calling GroqProvider: Object of type ChatCompletion is not JSON serializable.
And I can see with some debug tests that the LLM is returning a good response, but the proxy is messing up the parsing of it.

I'm testing and trying to find a different check to be implemented so we can have tools defined but still ask a normal question, but I'm not even sure it's possible without major code changes, so it might take a while.

Maybe @unclecode or someone more skilled than me can find a solution first 👍

I tried your shared Google Colab notebook

I'm not sure if it's intended to be like that or if it's meant to be done differently, but I couldn't get that part to work without altering it.

# Here you pass your own GROQ API key

Sending POST request, with full functions implementation

from dotenv import load_dotenv
from duckduckgo_search import DDGS
import requests, os
import json

load_dotenv()

api_key = os.getenv("<your_groq_api_key>")

header = {
    "Authorization": f"Bearer {api_key}",
    "Content-Type": "application/json"
}
proxy_url = "https://groqcall.ai/proxy/groq/v1/chat/completions"

But then received a ConnectTimeout error while making an API request, resulting in an APITimeoutError due to a timed-out request.

Schema-less Function Call 🤩

from dotenv import load_dotenv
from duckduckgo_search import DDGS
import requests, os
import json

load_dotenv()

api_key = os.getenv("<your_groq_api_key>")

Using with PhiData (Or similar libraries)

from phi.llm.openai.like import OpenAILike
from phi.assistant import Assistant
from phi.tools.duckduckgo import DuckDuckGo
import os, json
import dotenv

load_dotenv()


api_key = os.getenv("<your_groq_api_key>")

What i simply mean is that if we can do continuous conversation with it rather than just getting response for once and script ends. I have seen and used some conversational type scripts which use langchain so if you have deeper understanding of its implication with tools and other stuff then please make one or two scripts for that and add it in example :-))

I am actually in China, i find that when i use the requests to post "https://groqcall.ai/proxy/groq/v1/chat/completions" ,it works and there is no error ,but when I deploy the program running locally according to the deployment order described by you, an internal server error occurs when I call the local port

Which means I was intercepted by groq....

But I can access into "http://localhost:8000/proxy/groq/v1/chat/completions" from the browser, and shows the style of the json

Is there any way to fix this without mounting the clash web proxy??

Thank you for answering for me !!!!!!!

hope you can support it, a little complex when coming to function calling with stream