Enable running multiple reverse proxys with different targets. about openai-forward HOT 2 CLOSED

Hi @michaelfeil ,
I understand that your goal is to reverse proxy multiple service addresses (such as api.openai.com and localhost:8080) to the same port. This can be easily achieved by making a few code modifications. For example, you can add another route in openai_forward/app.py:

from sparrow.api import create_app
from .openai import Openai

app = create_app(title="openai_forward", version="1.0")


def add_route(obj: Openai):
    app.add_route(
        obj.ROUTE_PREFIX + "/{api_path:path}",
        obj.reverse_proxy,
        methods=["GET", "POST", "PUT", "DELETE", "OPTIONS", "HEAD", "PATCH", "TRACE"],
    )

openai = Openai()
openai.BASE_URL = "https://api.openai.com"
openai.ROUTE_PREFIX = "/openai"
add_route(openai)

localai = Openai()
localai.BASE_URL = "http://localhost:8080"
localai.ROUTE_PREFIX = "/localai"
add_route(localai)

Then, remove the @classmethod decorator for the _reverse_proxy class method in openai_forward/base.py, otherwise, the modification to BASE_URL will not take effect.

After making these changes, you will be able to access the content of api.openai.com under the /openai route, such as openai/audio/translations, and access all content from the service http://localhost:8080 under the /localai route, such as /localai/v1/completions.

In the future, it's worth considering making ROUTE_PREFIX and BASE_URL configurable for multiple target.

from openai-forward.

Now we can specify multiple targets through configuration, like this:
https://github.com/beidongjiedeguang/openai-forward/blob/e7d09ce3cb647e41af5d3a0c5039fe5659a08d34/.env.example#L6-L10

from openai-forward.