IPC communication between agent and client (CLI/GUI) about polykey HOT 17 CLOSED

Does grpc-node generate marshaling code given proto files?

from polykey.

Oh I see, grpc-node actually distributes 2 packages. One of them requires C and the other is pure JS. The C one might be able to be compiled to the nativescript platforms. But the pure JS one would be safer, but is missing only a few features. Do we need those and which one are you using?

from polykey.

from polykey.

Oh right, so that's an issue. Some classes are not well "encapsulated" since they rely on "session" side-effects. For these things I usually abstract them out with dependency injection so that they are forced to be constructed at the top-level context. Thus that main function above, you see I am forced to create the database pool object there.

from polykey.

It turns out node grpc doe not support UDS: grpc/grpc-node#258 so I am going with the nodejs net module and a proto file/protobufjs for the message format

from polykey.

@robert-cronin can you update this issue indicating that UDS does work, and how you achieved it.

from polykey.

Clarification with grpc-node https://github.com/grpc/grpc-node/blob/master/PACKAGE-COMPARISON.md it seems to mean that it's not pure JS. Which means potential issues when deploying this on nativescript.

from polykey.

Does grpc-node generate marshaling code given proto files?

protobufjs has the ability to generate JavaScript code that encodes and decodes protocol buffers. It also generates typescript definition files. These are the commands I am using for a proto folder defined at the root directory:

# Creates the JavaScript file
pbjs -t static-module -w commonjs -o proto/js/compiled.js proto/*.proto

# Generates the corresponding type definition files
pbts -o proto/js/compiled.d.ts proto/js/compiled.js

from polykey.

@robert-cronin can you update this issue indicating that UDS does work, and how you achieved it.

Yeah so it was actually quite simple. The nodejs net module allows one to specify a file path in net.createServer(...).listen(socketFilePath) and it takes care of creating the socket file. Closing the server also removes the generated socket file.

from polykey.

Oh I see, grpc-node actually distributes 2 packages. One of them requires C and the other is pure JS. The C one might be able to be compiled to the nativescript platforms. But the pure JS one would be safer, but is missing only a few features. Do we need those and which one are you using?

Yeah I was originally using grpc-js but discovered there was no way to synchronously start the server and let it dynamically choose and available port. In grpc you can do it by const port = server.bind('0.0.0.0:0') but in grpc-js the bind function doesn't return anything. The only way to do this is const port = await server.bindAsync('0.0.0.0:0') and so its impossible to follow RAII. I think the only way we will be able to use grpc-js is if we create stop and start functions for the server so they can be async. I don't mind this because there might be plenty of reasons why the user wants to control the starting and stopping of the secure server.

from polykey.

I am storing the proto files in a proto directory at the project root level and the script to generate it is stored in scripts/compile_proto.sh and is also an npm script in package.json. I will put something into the readme about this.

from polykey.

Just for future reference, it turns out the only way to import grpc-js is import * as grpc from'@grpc/grpc-js

from polykey.

from polykey.

Not sure what you mean, does loja have a dedicated Server class with a normal constructor and then a main function that acts like an init? I couldn't find anything like this.

If that's the case, I'm not sure what would be the difference to having a serverStart function?

I think there is still a way we can follow RAII, we just start the server asynchronously in the constructor. It's just not guaranteed to be started by the time the class is finished initialising.

from polykey.

Like this (index.ts):

import fs from 'fs';
import { Pool } from 'pg';
import Container from '@loja/server/bootstrap';
import router from '@loja/server/router';

(async () => {
  let db;
  try {
    db = new Pool();
    console.log(`Connected to database`);
    const container = Container(db);
    fs.mkdirSync(container['config']['LOJA_UPLOAD_DIR'], { recursive: true });
    console.log(`Upload directory set to ${container['config']['LOJA_UPLOAD_DIR']}`);
    const app = router(container);
    await app.listen(container['config']['LOJA_PORT'], container['config']['LOJA_HOST']);
    console.log(
      `Loja listening on port ${container['config']['LOJA_HOST']}:${container['config']['LOJA_PORT']}`
    );
  } catch (e) {
    if (db != null) db.end();
  }
})();

Notice the try and catch.

Also the main function is basically an IIFE.

from polykey.

oh okay, I did notice this file, but thought it wasn't relevant to PeerManager since it is supposed to be initialised as a class instance and not just a function call. I think especially from the point of view of Polykey, the sub components should be composable and not just functions to call.

from polykey.

So if we want to continue using gRPC between client and agent and not just agent to agent, then we would like to have gRPC support UDS because UDS inherits OS permissions whereas localhost can be sniffed by any program on the machine even executed by other users. However at the moment the node implementation of gRPC does not support UDS. It is possible to support if these 2 issues were to be resolved:

• grpc/grpc-node#258
• grpc/grpc#18435 - this got closed

Therefore the only solution is to use mTLS between agent and client to secure communications locally.

This can allow clients to exist on a different system to call into a PK agent. So it does have its own advantages.

from polykey.