[REQUEST]: Remote execution on different architectures about compiler-explorer HOT 7 OPEN

#4315

from compiler-explorer.

#6413 merged

from compiler-explorer.

Follows that design of execution request in "message queue" (however we do that) should be at least this:

type NameValueThing = {
   name: string;
   value: string;
};

type ConfiguredTool = {
   name: string;
   options: NameValueThing[];
};

type ExecutionParams = {
   args: string[];
   stdin: string;
   runtimeTools: ConfiguredTool[];
};

Then the question is, what does the rest look like, maybe this?

type ExecutionMessage = ExecutionParams & {
   result_id: string; // unique identifier that we can lookup and store things under? maybe?
   execute_hash: string; // the hash the executable package has gotten after compilation - in the test endpoint this is a url parameter
   arch: string; // should be something like: `aarch64-linux`?
};

amd64-linux-nvidia-gpu / amd64-linux-amd-gpu / amd64-linux-intel-gpu?

from compiler-explorer.

It depends on how the selected queue would look like, but if you can only get or peek at 1 or a limited amount of messages, then you would need multiple queues? 1 per arch

from compiler-explorer.

It would be better to only use websockets and just work with DynamoDB, but there might be something something locking via queue or something. (depends on how the concurrency model of aws lambda websockets works, maybe its not needed)

from compiler-explorer.

Resources for websockets that I'll be using:

• https://docs.aws.amazon.com/apigateway/latest/developerguide/websocket-api-develop.html
• https://registry.terraform.io/providers/hashicorp/awscc/latest/docs/resources/apigatewayv2_api (awscc??)
• terraform-aws-modules/terraform-aws-apigateway-v2#8
• https://github.com/aws-samples/simple-websockets-chat-app

from compiler-explorer.

While writing a websocket I remember again why a naive websocket model is suboptimal.
But there's a solution somewhere

from compiler-explorer.