danielhasan1 / fastapi-listing Goto Github PK

DAO middelware should be optional. allow users to pass db sessions/connections in more than one manner.
This will help:

• users to allow using their existing architecture of managing db connections.
• allow users to manage connection lifecycle more independently
• in managing connections who are running orm other than sqlalchemy
• reduce an external dependency to configure a project

How about we add some metrics that matters?
Often people wanna know which component took most of the time when a request gets stalled or if a request is taking more time:

• in development phase of a listing api
• in maintenance phase of a listing api
  Speaking in terms of this package as everything is broken down into components we can decide to log time component wise.

client request passes through all the components and we can log the difference between entry & exit time.

The major impact will come at pagination level when data actually gets allocated into the memory as the listing query hits sql server at this stage. Other than this in filters we hit sql server for validation purposes which is kinda depend on the stuff or params that we are receiving.

standard unit of time - milliseconds

A standard chart could like something like this

Request enters at: 2023-12-05 13:09:00(1234567890)(unix time)
|
|Prepare Query: 0.1 ms
|___ using xyz strategy listing query received in 0.03ms
|____ query filters applied in 0.06ms (this should expand in case of multiple filters)
|____ query sorting applied in 0.01ms (this should expand in case of multiple sort)
|Paginator: 5ms
|___ query hits sql at: 2023-12-05 13:09:01(1234567890)(unix time)
|____ sql response returned in 5ms
Request exist at: 2023-12-05 13:09:06(1234567890)(unix time)

Added limit for max page size.
remote client can request for variable page size so added an option for user to configure max allowed items per page to put a limiter to avoid performance bottlenecks.
Remote client here could be anyone like frontend or some other backend server.

users will be able to use any ORM.

generate filter documentation from filter doc string.

when allow_count_query_by_paginator=False paginator throws an error

pydantic.error_wrappers.ValidationError: 1 validation error for ListingPageWithoutCount[EmployeeListDetails]
               response -> hasNext
                 none is not an allowed value (type=type_error.none.not_allowed)

• allow logging of listing query
• allow logging of parsed query params

• core service is breaking due to pydantic v2 depricated feature __fields__.
• tests have stopped working since pydantic v2 update due to change in config feature names.