rubixml / server Goto Github PK

This ticket is to determine if Rubix Server can be used with Kubernetes to deploy and autoscale instances.

In the process we should ...

A. Determine if we need to make any changes to the library to either make it work or make it easier to work
B. Create a container for other developers to use
C. Produce documentation for the procedure to be added to the README

Related Reading:

• https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44843.pdf

Hi @andrewdalpino! I am exploring the RubixML Server currently and finding it really practical.I am curios about one aspect though. Here is the example case:

Let us say I got a REST server that accepts requests from client applications (non-php) and returns predictions.

$server = new RESTServer(‘localhost’, 8080);
$server->serve($estimator);

Suppose, I also have a Tester client (php) that might sit within the same environment (same machine server ) or remote location. This Tester client’s job is to periodically send samples to the REST server, get back predictions, check the accuracy and if the accuracy is below the certain % retrain the estimator instance. My question is how would you go about doing it with a current implementation considering the fact that you need to reach to estimator instance?
I could extend the REST server class, add a new route to a new, lets say TrainerController and retrain the model. That’s done once the request is in. But how do you deliver that request?
I see three ways: a) In Tester client create completely new Guzzle client, package the data in as json in body, set up headers and send to REST server
b) a middleware with a conditional check if RPC or REST request etc
c) Implement the existing Client interface with a slightly customized implementation logic and let this RESTclient handle all the future request to REST Server.

I think c option looks way cleaner and much reusable. So now the library will have two clients RPCClient and RESTclient. I am just curios if you are considering to add anything like this in future updates? Or would you rather leave it up to an individual developer to come up with ways of figuring this out?

thanks

Currently, the RPC client supports the Retry-After header when accompanied by a Too Many Requests (429) and SERVICE Unavailable (503) response code. This allows the server to determine how much time it needs the client to backoff before it can handle another request. This 'backpressure' mechanism allows server and client to work together to stabilize the flow of requests coming in to the server under heavy load. This enhancement is to add a new RateLimiter middleware that limits the number of requests per minute and estimates the wait time (the amount of backpressure) for a client when the threshold has been reached.