[BUG] Response generation dramatically slower when using aliases in Ninja v1/Pydantic v2 about django-ninja HOT 10 OPEN

So it looks like ninja calls model_validate even if the endpoint returns an instance of the response schema. I'm not sure if this is something that's changed since last version, but I've made a fix for it. Can I get you to test if the change returns you to normal speed?

Change lines 206-210 in ninja/operation.py to the below:

        response_type = response_model.__annotations__["response"]
        if isinstance(response_type, Schema) and isinstance(result, response_type):
            validated_object = response_model.model_construct(response=result)
        else:
            resp_object = ResponseObject(result)
            validated_object = response_model.model_validate(
                resp_object, context={"request": request, "response_status": status}
            )

from django-ninja.

Thanks for the suggestions, @OtherBarry. I suspected that that was the case as well, since returning Schemas directly isn't something I've seen in the docs, and in the code I believe it's just checking for Managers, QuerySets, and dicts. I actually tried returning a Django HttpResponse directly, since that circumvents Ninja's response logic, but unfortunately that didn't improve things either (which is another reason I'm guessing the bottleneck is in the Schema logic somewhere). I'll give your workaround a try though.

EDIT: No appreciable improvement with the workaround.

Another thing that may be worth mentioning: I don't see this slowdown when Pydantic v2 is used with FastAPI. I maintain a lot of FastAPI services, and when I first noticed this in Ninja v1 I was worried it might be affecting all our other services. After testing it though, the speed and throughput appeared the be no different with or without response generation by alias.

from django-ninja.

@vitalik, any suggestions?

from django-ninja.

@jmriebold

did you try returning non validated objects ( I did not investigate this deeply - but most likely it just does double validation)

try this

@api.get('/examples', response=ExampleListSchema, by_alias=True)
def list_examples(request: HttpRequest, **kwargs) -> dict:
    return {"examples": Example.objects.all()}

from django-ninja.

I haven't, but thanks for the suggestion. I'll give that a try and report back.

from django-ninja.

Ok, tried that out basically exactly as you suggested, and while it did improve the response time, it's still approximately 3x slower than with Django-Ninja<1.0 (~150ms vs. ~50ms).

Not sure if this could have anything to do with it, but for what it's worth, the real endpoint and database query I'm testing here JOINs 3 other tables in (which I'm doing with select_related()).

Thanks again for your help on this, I really appreciate it.

from django-ninja.

Have you tested that there aren't any database calls being made as part of the validation?

from django-ninja.

There shouldn't be, since the validation is just standard Pydantic stuff (max length, etc.).

from django-ninja.

I was thinking more in terms of a field that you expect to be prefetched isn't, hence when pydantic goes to accesss the property, django fetches the value from the database in the background. Though with that much of a double validation difference, it's unlikely to be the issue, as the db hit should only happen once regardless of how many times it's validated.

from django-ninja.

I think that's unlikely, since the endpoint code is the same when I'm testing with the two versions. I'm using select_related() for all three tables that are being joined, and a while back I tested it without those calls to verify that the query was faster with them.

from django-ninja.