Remote materialization about feast HOT 2 OPEN

The same as above but instead of creating a new component, we can reuse OfflineServer to do the request handling. This is slightly awkward from the naming perspective, but probably makes the most sense in term of usage/maintenance.

ATM the OfflineServer was designed to implement the OfflineStore interface only. Adding a method to write to the online stores would introduce an unplanned dependency and raise some concerns.

• How the actual method should be implemented on the server side, w/o a reference to the (likely remote) online_store, which would raise again the original issue?
• If we instead think to implement it in the OfflineServer, then we'd need an actual online_store configuration as well, which was not planned to have in such server (a remote online store was designed instead). Not sure about any possible side effects.

Why aren't we using the /materialize-incremental endpoint on the FeatureServer instead? (and add a new endpoint for non-incremental jobs)
This would avoid any "transport batches and batches of potentially huge datasets." as it would work on the server itself (and would use the remote offline_store to pull_latest_from_table_or_query using the flight protocol) .

Otherwise, I'd be in favour of a dedicated MaterializationServer (with remote offline_store and provided online_store), which can still be designed as a "lightweight fastapi server" if I understood the materialization flow.

from feast.