Comments (20)
@tokoko @franciscojavierarceo thank you for the write-up/spec. I have concerns around the overall vision for the project and our strategy to achieve that vision, in light of this RFE.
KubeFlow, as the open source MLOps platform, already supports and develops KFP, MR and KServe. There is clearly a gap there for the feature store that Feast can fill. However, in that architecture, Feast would be a service to both KFP and KServe, while MR would be responsible for recording the model metadata necessary to connect serving/inference with the feature store.
Adding model metadata and API into the feature store, as proposed here, seems to invert the dependency graph in that architecture and create an overlap with the responsibilities of those other components. I am afraid that if we move forward with this RFC, it will cause confusion for users and make our integration with kubeflow more complicated, creating friction both from a technical as well as an organizational perspective.
I would advise that we focus on the integration with these other components in the ecosystem instead.
from feast.
yup, an alias sounds fine. Another thing that came to my mind, we will probably have to come up with an appropriate abstraction to put this functionality in (behind FeatureStore). I'm thinking of InferenceEngine
or something similar with LocalInferenceEngine
and RemoteInferenceEngine
as (probably only) two implementations. That would enable us to apply security model to these methods as well. (fyi @dmartinol @redhatHameed)
from feast.
@dmartinol thanks for joining in. (This will probably be lengthy) Let me start from your last point, it's probably crucial that we agree on high-level what we're trying to build here. Yes, this is definitely going out of 'feature store' territory and I'm also a bit uncomfortable about that. Having said that, if we find some utility in extending feast this way that's overlooked by other oss tools out there, we should still proceed. I know we might be committing a terrible case of a feature creep here, happy to be called out if you think we are 😄.
The RFC linked above goes into this, but let me briefly recap here as well... I think there are two general ways to bridge feature retrieval and model execution:
-
If a user already has a somewhat complex serving architecture in place (I'm thinking of a model mesh here, either kserve or seldon) then the best way to go about it would be to treat feature retrieval as a sort of a transformer/preprocessor "model". The control plane of kserve (or seldon) would take turns to call feast first and model endpoint after that passing retrieved feature values. One idea I floated around in the RFC to support this flow is to modify our http/grpc feature servers to make them OIP-compliant effectively treating our
FeatureService
objects as models. If you have a feature service 'example_servicein the registry, calling
inferendpoint of a feature server for a "model"
example_service` would return necessary feature values. Since both kserve and seldon talk OIP, integration should be pretty simple. This flow is probably appropriate when an org has hundreds of models running and relying on kserve/seldon makes sense. -
This ticket comes into play for a simpler deployment flow, when you are able to spin up a model endpoint somewhere and also have feast deployment in place, but have no easy way to bridge the gap between the two. feast itself takes on the role of a very simple control plane and tries to abstract away model endpoints from the caller. There are also some seemingly beneficial side-effects, mainly the ability to rely on a feast online store for the storage of precomputed (meaning batch) or cached (written out after online calls with some ttl) results of particular models. Not sure if the same can be achieved with an external add-on, but that's also a possibility.
Regarding model registry, if registry indeed contains info about current deployment endpoints (not just trained artifacts), that might be a good idea, although I'm not sure where would be keep additional information regarding models (whether we want to enable cache or not, permissions and so on) if we don't have individual Model
objects in our registry and instead rely on a list of models supplied by the registry IIUC.
from feast.
@franciscojavierarceo Your PR prompted me to draw this up. wdyt?
from feast.
Yeah I like this. I think we should use both infer
and predict
that just aliases to infer
. I think that will be intuitive for traditional MLEs working with tabular data.
from feast.
Yeah, that makes sense.
We can use KServe's protobuf definition too.
from feast.
FYI @rimolive
from feast.
from feast.
Yeah, that makes sense.
We can use KServe's protobuf definition too.
I'm pretty sure that's the same thing as OIP, it just had a couple of name changes along the way. V2 Inference Protocol, KServe V2 Protocol, Open Inference Protocol... they are all the same.
from feast.
Yup! I just wanted to provide the link to it.
from feast.
@tokoko (thanks for sharing!) why don't you also look at the Model Registry capabilities?
This component provides a central repository to store and manage models, together with the serving endpoints for each deployment of a model artifact (for this I need to find exact doc references).
In short, instead of abstracting an InferenceEngine
, we could abstract a ModelRegistry
and provide an initial implementation for this Kubeflow component, WDYT?
BTW: This requirement appears more aligned with an external add-on for handling both feature retrieval and model execution together, rather than a core component expected within the 'feature store'. Wouldn't this introduce too many (potentially unnecessary) dependencies?
from feast.
This ticket comes into play for a simpler deployment flow, when you are able to spin up a model endpoint somewhere and also have feast deployment in place, but have no easy way to bridge the gap between the two. feast itself takes on the role of a very simple control plane and tries to abstract away model endpoints from the caller. There are also some seemingly beneficial side-effects, mainly the ability to rely on a feast online store for the storage of precomputed (meaning batch) or cached (written out after online calls with some ttl) results of particular models. Not sure if the same can be achieved with an external add-on, but that's also a possibility.
This was one of my goals when drafting the RFC. (1) Creating some standard abstractions that gives guidance to the community and (2) enabling faster retrieval of predictions.
from feast.
FYI here's the link to the PR @tokoko mentioned: #4288
from feast.
@etirelli thanks for the feedback here! Agree we shouldn't cause confusion and we should have clean and separate responsibilities from the other components.
I don't want to invert the dependency graph but I do want to enable faster model retrieval and I think there's a way to accomplish both of those things. I'll do more research and get back.
from feast.
Maybe the way to go without blurring the lines too much would be to somehow build this as a 3rd party tool (inside feast-dev
organization if possible???). Initially that was my plan a while ago, already have a repository named hefeasto
internally 😄.
We could base that 3rd party tool on existing FeatureStore interface or even expose some new abstractions if needed (for storing cached values in online store, checking whether cache is stale or not, etc...) Some of those new abstractions might even prove useful while integrating with kserve or seldon.
from feast.
To be clear, by better integration with kserve I mean that the flow on kserve side might look instead of like this (get_features_from_feast
-> get_model_prediction_from_model_endpoint
) more like this (check_if_feast_has_cached_result
-(if not)> get_features_from_feast
-> get_model_prediction_from_model_endpoint
-> update_feast_cache
)
from feast.
So I'm going to write a new spec that starts with two goals:
- Batch Models: Serving Pre-computed Predictions for some entity
- Cached Online Models: Caching predictions and updating them only when data sources change
This requires:
a. Retrieving a precomputed score from an online database
b. Recomputing the score somehow (e.g., calling a Kserve endpoint) and retrieving the computed score from an online database
I think that means we can do this in 3 ways:
- KServe orchestrates the call to Feast and the KServe endpoint
- This is how the KServe/Feast demo operates today but it's not meant for Batch Models
- Modifying it to support Batch Models is straight forward but it'd basically just be a call to Feast
- This is how the KServe/Feast demo operates today but it's not meant for Batch Models
- Feast orchestrates
- This is similar to my PR (if instead of an ODFV, we actually called KServe) and satisfies (1) and (2)
- New library to handle orchestration
- This would be a light-weight library that would basically be what (2) does but outside of Feast
There are pros and cons to each and it'd be good to discuss them with the Kubeflow community to work through them to come to a consensus.
I think the solution recommended may end up depending upon the needs of the users. At Affirm, we wanted very low latency since we were serving a customer checkout, so an additional network hop was suboptimal and therefore calling Feast first would make more sense. Having KServe call Feast requires an additional RPC but it is a more intuitive architecture and makes the API call more intuitive.
from feast.
I'm going to create a new issue outlining my note above so we can discuss this with the Feast community there.
from feast.
@tokoko this is a useful blog post https://medium.com/marvelous-mlops/getting-started-with-databricks-feature-serving-bf590b318c31
from feast.
And it has this diagram:
from feast.
Related Issues (20)
- Rewrite SparkOfflineStore with ibis
- feastdev/feature-server{,-java} 0.38.0 & 0.39.0 tags missing from Docker Hub
- feast gcp notebook, getting TypeError: Client.__init__() got an unexpected keyword argument 'database' HOT 1
- Use modular fixture functions for integration test environments HOT 1
- OnDemandFeatureView.feature_transformation.infer_features does pass UDF outputs to python_type_to_feast_value_type HOT 1
- PostgresOnlineStore: Improve materialization HOT 17
- Is Feast helpful in case of the pictures and video as a features?
- FeastObject type is missing SavedDataset HOT 5
- Google cloud datastore: Client.__init__() got an unexpected keyword argument 'database' HOT 4
- Discussion: Postgres Online Store: Is connection handling done properly?
- new Dask version lock pandas to 2.0+
- Enhance the python feature server with new `list` endpoints. HOT 10
- Add Open Inference Protocol to feature servers HOT 3
- Add support for get_historical_features for vector search HOT 1
- Deprecation of distutils in python 3.12 breaking feast init HOT 1
- no image `feastdev/feast-operator:0.37.0`
- Rewrite RemoteOnlineStore to use get_online_features HOT 2
- Update CI to have a test for main feast dependency before release/deployment HOT 1
- Minor doc fix of page feature-servers
- Security fix for possible Cross-site-scripting (XSS) attack HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from feast.