Gamal is a simple, zero-dependency tool designed to quickly provide answers to questions. It finds relevant web pages and uses an LLM to summarize the content, delivering concise answers. Gamal is accessible via the terminal (as a CLI tool), through its minimalist web interface, or as a Telegram bot.

Gamal utilizes SearXNG for web searches and requires an LLM to generate responses based on search results. By default, Gamal integrates with OpenRouter as its LLM service, requiring the configuration of an API key in the LLM_API_KEY environment variable. Please continue reading for detailed instructions on configuring Gamal to use either a local LLM (llama.cpp, Jan, and Ollama) or other managed LLM services (offering over half a dozen options, including OpenAI, Fireworks, and Groq).

To execute Gamal as a CLI tool, run it with Node.js (version >= 18) or Bun:

./gamal.js

For instant answers, pipe the questions directly into Gamal:

echo "List 5 Indonesia's best travel destinations" | ./gamal.js

Gamal also includes a minimalist front-end web interface. To launch it, specify the environment variable GAMAL_HTTP_PORT, for example:

GAMAL_HTTP_PORT=5000 ./gamal.js

Then, open a web browser and go to localhost:5000.

Gamal is capable of functioning as a Telegram bot. Obtain a token (refer to Telegram documentation for details) and set it as the environment variable GAMAL_TELEGRAM_TOKEN before launching Gamal. Note that conversation history in Telegram chats is stored in memory and not persisted to disk.

Gamal can converse in many languages besides English. It always tries to respond in the same language as the question. You can freely switch languages between questions, as shown in the following example:

>> Which planet in our solar system is the biggest?
Jupiter is the largest planet in our solar system [1].
[1] https://science.nasa.gov/jupiter/

>> ¿Y el más caliente?
Venus es el planeta más caliente, con hasta 475°C. [1].
[1] https://www.redastronomy.com/sistema-solar/el-planeta-venus/

Gamal's continuous integration workflows include evaluation tests in English, Spanish, German, French, Italian, and Indonesian.

Gamal is designed to be used with OpenRouter by default, but it can also be configured to work with other LLM services by adjusting some environment variables. The correct API key and a suitable model are required.

Compatible LLM services include Anyscale, Deep Infra, Fireworks, Groq, Lepton, Novita, Octo, OpenAI, and Together.

Refer to the relevant section for configuration details. The example provided is for Llama-3 8B, though any LLM with 7B parameters should also work, such as Mistral 7B, Qwen-2 7B, or Gemma-2 9B.

export LLM_API_BASE_URL=https://api.endpoints.anyscale.com/v1
export LLM_API_KEY="yourownapikey"
export LLM_CHAT_MODEL="meta-llama/Llama-3-8b-chat-hf"

export LLM_API_BASE_URL=https://api.deepinfra.com/v1/openai
export LLM_API_KEY="yourownapikey"
export LLM_CHAT_MODEL="meta-llama/Meta-Llama-3-8B-Instruct"

export LLM_API_BASE_URL=https://api.fireworks.ai/inference/v1
export LLM_API_KEY="yourownapikey"
export LLM_CHAT_MODEL="accounts/fireworks/models/llama-v3-8b-instruct"

export LLM_API_BASE_URL=https://api.groq.com/openai/v1
export LLM_API_KEY="yourownapikey"
export LLM_CHAT_MODEL="llama3-8b-8192"

export LLM_API_BASE_URL=https://llama3-1-8b.lepton.run/api/v1
export LLM_API_KEY="yourownapikey"

export LLM_API_BASE_URL=https://api.novita.ai/v3/openai
export LLM_API_KEY="yourownapikey"
export LLM_CHAT_MODEL="meta-llama/llama-3-8b-instruct"

export LLM_API_BASE_URL=https://text.octoai.run/v1/
export LLM_API_KEY="yourownapikey"
export LLM_CHAT_MODEL="meta-llama-3-8b-instruct"

export LLM_API_BASE_URL=https://api.openai.com/v1
export LLM_API_KEY="yourownapikey"
export LLM_CHAT_MODEL="gpt-4o-mini"

export LLM_API_BASE_URL=https://api.together.xyz/v1
export LLM_API_KEY="yourownapikey"
export LLM_CHAT_MODEL="meta-llama/Llama-3-8b-chat-hf"

Gamal is compatible with local LLM inference tools such as llama.cpp, Jan, and Ollama. Refer to the relevant section for configuration details.

The example provided uses Llama-3 8B. For optimal performance, an instruction-following LLM with 7B parameters or more is recommended. Suitable models include Mistral 7B, Qwen-2 7B, and Gemma-2 9B.

/path/to/llama-server -m Meta-Llama-3-8B-Instruct.Q4_K_M.gguf
export LLM_API_BASE_URL=http://127.0.0.1:8080/v1

export LLM_API_BASE_URL=http://127.0.0.1:1337/v1
export LLM_CHAT_MODEL='llama3-8b-instruct'

ollama pull llama3
export LLM_API_BASE_URL=http://127.0.0.1:11434/v1
export LLM_CHAT_MODEL='llama3'

Gamal includes a built-in evaluation tool. For instance, if a text file named qa.txt contains pairs of User and Assistant messages:

User: Which planet is the largest?
Assistant: The largest planet is /Jupiter/.

User: and the smallest?
Assistant: The smallest planet is /Mercury/.

executing the following command will sequentially search for these questions and verify the answers using regular expressions:

./gamal.js qa.txt

Additional examples can be found in the tests/ subdirectory.

Two environment variables can modify the behavior:

• LLM_DEBUG_FAIL_EXIT: When set, Gamal will exit immediately upon encountering an incorrect answer, and subsequent questions in the file will not be processed.

• LLM_DEBUG_PIPELINE: When set, if the expected regular expression does not match the answer, the internal LLM pipeline will be printed to stdout.

By default, Gamal utilizes the public SearXNG instance at searx.work. To switch to a different SearXNG instance, such as a private one capable of searching additional custom data sources, configure the URL using the SEARXNG_URL environment variable.

For more frequent searches, obtain an API key for JINA reader (its free tier is generous) and set it as JINA_API_KEY. This will increase the rate limit from 20 requests per minute to 200 requests per minute.

To leverage the Brave Search API, a paid search API offering a free trial, acquire the API key and store it as the BRAVE_SEARCH_API_KEY environment variable. With this key configured, Gamal will prioritize Brave Search over SearXNG.