Enhance the python feature server with new `list` endpoints. about feast HOT 10 CLOSED

@tokoko @dmartinol the use-case for this enhancement is to allow for a potential future feast integration with the odh-dashboard. based on some initial discussions i had, it appeared an http server was required.

i've done some more digging this morning though and there appears to have been some more recent developments to the project around gRPC support. i'll continue my research and report back.

from feast.

dev's decision, proto definitions of objects were already in place, so grpc made more sense to me. The main motivation was to expose python-based registries to java/go feature servers, plus I was modelling it after BaseRegistry interface methods, so rpc felt more natural. Nothing that couldn't have been done with rest or http of course.

from feast.

Isn't that the job of a registry server?

from feast.

Just to provide an update on the use of gRPC-web w/ the feast registry's gRPC API. My tests were successful, so updates to the feature server are not necessary.
grpc/grpc-web@master...tchughesiv:grpc-web:dev

from feast.

Isn't that the job of a registry server?

@tokoko yes, and it will still be the job of the registry server. we'd just add new endpoints in feature_server.py that leverage the existing registry methods.

from feast.

I'm not sure I'm a fan, tbh. We'll end up having virtually identical endpoints in registry_server.py and feature_server.py, the only difference being that one is a grpc server and the other's an http one. We'll also have to secure it twice to validate permissions (see this ongoing PR for context). The surface area / complexity that non-python feature servers will need to be implement would also increase...

Do you have some specific use case in mind that can't be covered by the grpc registry server?

from feast.

I'm not sure I'm a fan, tbh. We'll end up having virtually identical endpoints in registry_server.py and feature_server.py ...

Agree, the servers should implement the equivalent Feast interfaces like OnlineStore, OfflineStore and Registry, so that they can be transparently accessed by remote python clients w/o using any 3rd party library (requests, pyarrow.flight, grpc and so on). Just run fs.registry.list_feature_views(). Adding more endpoints would break this pattern.

from feast.

@tokoko just out of curiosity, what was the reason to develop the Registry Server using grpc instead of REST?

from feast.

@tchughesiv is there a list of APIs requested to populate the dashboard and manage the exposed functions?
Maybe a dedicated Backend For Frontend service is more suitable instead of (trying to) use the endpoints of the various Feast servers which are instead targeted to serve the ML workflows.
Of course, this BFF can use the Feast server endpoints, but with interfaces tailored to frontend needs (and yeah, it can also 'talk' gRPC and Arrow Flight-ish), and can be installed by the operator as it needs to refer to the other servers exposing the Feast features.

Oops, I was almost implementing it 🤫 EOM.

from feast.

I'll close this for now, feel free to reopen if circumstances change.

from feast.