Git Product home page Git Product logo

tgrid's Introduction

TGrid

Introduction

TGrid logo

GitHub license npm version Downloads Build Status FOSSA Status Chat on Gitter

Full name of TGrid is TypeScript Grid Computing Framework.

As its name suggests, TGrid is a useful framework for implementating Grid Computing in the TypeScript. With TGrid and its core concept Remote Funtion Call, you can make many computers to be a virtual computer.

To know more, refer below links. If you are the first comer to the TGrid, I strongly recommend you to read the Guide Documents. In article level, I Basic Concepts and Learn from Examples sections would be good choices.

1.2. Grid Computing

Grid Computing

Computers be a (virtual) computer

As its name suggests, TGrid is a useful framework for Grid Computing. However, perpective of Grid Computing in TGrid is something different. It doesn't mean just combining multiple computers uinsg network communication. TGrid insists the real Grid Computing must be possible to turning multiple computers into a virtual computer.

Therefore, within framework of the TGrid, it must be possible to develop Grid Computing System as if there has been only a computer from the beginning. A program running on a computer and a Distributed Processing System with millions, both of them must have similar program code. It's the real Grid Computing.

Do you agree with me?

1.3. Remote Function Call

TGrid realizes the Grid Computing through Remote Function Call. It literally calling remote system's functions are possible. With the Remote Function Call, you can access to objects of remote system as if they have been in my memory from the beginning.

With TGrid and Remote Function Call, it's possible to handle remote system's objects and functions as if they're mine from the beginning. Do you think what that sentence means? Right, being able to call objects and functions of the remote system, it means that current and remote system are integrated into a single virtual computer.

However, whatever Grid Computing and Remote Function Call are, you've only heard theoretical stories. Now, it's time to see the real program code. Let's see the demonstration code and feel the Remote Function Call. If you want to know more about the below demonstration code, read a section Learn from Examples wrote into the Guide Documents.

import { WebServer } from "tgrid/protocols/web";
import { CompositeCalculator } from "../../providers/Calculator";

async function main(): Promise<void>
{
    const server: WebServer<object, CompositeCalculator> = new WebServer();
    await server.open(10102, async acceptor =>
    {
        await acceptor.accept(new CompositeCalculator());
    });
}
main();
import { WebConnector } from "tgrid/protocols/web/WebConnector";
import { Driver } from "tgrid/components/Driver";

import { ICalculator } from "../../controllers/ICalculator";

async function main(): Promise<void>
{
    //----
    // CONNECTION
    //----
    const connector: WebConnector<null, null> = new WebConnector(null, null);
    await connector.connect("ws://127.0.0.1:10102");

    //----
    // CALL REMOTE FUNCTIONS
    //----
    // GET DRIVER
    const calc: Driver<ICalculator> = connector.getDriver<ICalculator>();

    // FUNCTIONS IN THE ROOT SCOPE
    console.log("1 + 6 =", await calc.plus(1, 6));
    console.log("7 * 2 =", await calc.multiplies(7, 2));

    // FUNCTIONS IN AN OBJECT (SCIENTIFIC)
    console.log("3 ^ 4 =", await calc.scientific.pow(3, 4));
    console.log("log (2, 32) =", await calc.scientific.log(2, 32));

    try
    {
        // TO CATCH EXCEPTION IS STILL POSSIBLE
        await calc.scientific.sqrt(-4);
    }
    catch (err)
    {
        console.log("SQRT (-4) -> Error:", err.message);
    }

    // FUNCTIONS IN AN OBJECT (STATISTICS)
    console.log("Mean (1, 2, 3, 4) =", await calc.statistics.mean(1, 2, 3, 4));
    console.log("Stdev. (1, 2, 3, 4) =", await calc.statistics.stdev(1, 2, 3, 4));

    //----
    // TERMINATE
    //----
    await connector.close();
}
main();
1 + 6 = 7
7 * 2 = 14
3 ^ 4 = 81
log (2, 32) = 5
SQRT (-4) -> Error: Negative value on sqaure.
Mean (1, 2, 3, 4) = 2.5
Stdev. (1, 2, 3, 4) = 1.118033988749895

2. Strengths

2.1. Easy Development

Anyone can easily make a network system.

It's difficult to make network system because many of computers are interacting together to accomplish a common task. Therefore, the word 'perfect' is inserted on every development processes; requirements must be analyzed perfectly, use-cases must be identified perfectly, data and network architectures must be designed, perfectly and mutual interaction test must be perfectly.

Something to Read

Blockchain's Network System, Steps to Hell

Difficulty Level Graph

However, with TGrid and Remote Function Call, you can come true the true Grid Computing. Many computers interacting with network communication are replaced by only one virtual computer. Even Business Logic code of the virtual computer is same with another Business Logic code running on a single physical computer.

Thus, you can make a network system very easily if you use the TGrid. Forget everything about the network; protocolcs and designing message structures, etc. You only concentrate on the Business Logic, the essence of what you want to make. Remeber that, as long as you use the TGrid, you're just making a single program running on a single (virtual) computer.

2.2. Safe Implementation

By compilation and type checking, you can make network system safe.

When developing a distributed processing system with network communication, one of the most embarrassing thing for developers is the run-time error. Whether network messages are correctly constructed and exactly parsed, all can be detected at the run-time, not the compile-time.

Let's assume a situation; There's a distributed processing system build by traditional method and there's a critical error on the system. Also, the critical error wouldn't be detected until the service starts. How terrible it is? To avoid such terrible situation, should we make a test program validating all of the network messages and all of the related scenarios? If compilation and type checking was supported, everything would be simple and clear.

TGrid provides exact solution about this compilation issue. TGrid has invented Remote Function Call to come true the real Grid Computing. What the Remote Function Call is? Calling functions remotly, isn't it a function call itself? Naturally, the function call is protected by TypeScript Compilter, therefore guarantees the Type Safety.

Thus, with TGrid and Remote Function Call, you can adapt compilation and type checking on the network system. It helps you to develop a network system safely and conveniently. Let's close this chapter with an example of Safey Implementation.

import { WebConnector } from "tgrid/protocols/web/WebConnector"
import { Driver } from "tgrid/components/Driver";

interface ICalculator
{
    plus(x: number, y: number): number;
    minus(x: number, y: number): number;

    multiplies(x: number, y: number): number;
    divides(x: number, y: number): number;
    divides(x: number, y: 0): never;
}

async function main(): Promise<void>
{
    //----
    // CONNECTION
    //----
    const connector: WebConnector<null, null> = new WebConnector(null, null);
    await connector.connect("ws://127.0.0.1:10101");

    //----
    // CALL REMOTE FUNCTIONS
    //----
    // GET DRIVER
    const calc: Driver<ICalculator> = connector.getDriver<ICalculator>();

    // CALL FUNCTIONS REMOTELY
    console.log("1 + 6 =", await calc.plus(1, 6));
    console.log("7 * 2 =", await calc.multiplies(7, 2));

    // WOULD BE COMPILE ERRORS
    console.log("1 ? 3", await calc.pliuowjhof(1, 3));
    console.log("1 - 'second'", await calc.minus(1, "second"));
    console.log("4 / 0", await calc.divides(4, 0));
}
main();
$ tsc
src/index.ts:33:37 - error TS2339: Property 'pliuowjhof' does not exist on type 'Driver<ICalculator>'.

    console.log("1 ? 3", await calc.pliuowjhof(1, 3));

src/index.ts:34:53 - error TS2345: Argument of type '"second"' is not assignable to parameter of type 'number'.

    console.log("1 - 'second'", await calc.minus(1, "second"));

src/index.ts:35:32 - error TS2349: Cannot invoke an expression whose type lacks a call signature. Type 'never' has no compatible call signatures.

    console.log("4 / 0", await calc.divides(4, 0));

2.3. Network Refactoring

Critical changes on network systems can be resolved flexibly.

In most case of developing network distributed processing system, there can be an issue that, necessary to major change on the network system. In someday, neccessary to refactoring in network level would be come, like software refactoring.

The most representative of that is the performance issue. For an example, there is a task and you estimated that the task can be done by one computer. However, when you actually started the service, the computation was so large that one computer was not enough. Thus, you should distribute the task to multiple computers. On contrary, you prepared multiple computers for a task. However, when you actually started the service, the computation was so small that just one computer is sufficient for the task. Sometimes, assigning a computer is even excessive, so you might need to merge the task into another computer.

Diagram of Composite Calculator Diagram of Hierarchical Calculator
Composite Calculator Hierarchical Calculator

I'll explain this Network Refactoring, caused by performance issue, through an example case that is very simple and clear. In a distributed processing system, there was a computer that providing a calculator. However, when this system was actually started, amount of the computations was so enormous that the single computer couldn't afford the computations. Thus, decided to separate the computer to three computers.

  • scientific: scientific calculator server
  • statistics: statistics calculator server
  • calculator: mainframe server
    • four arithmetic operations are computed by itself
    • scientific and statistics operations are shifted to another computers
    • and intermediates the computation results to client

If you solve this Network Refactoring by traditional method, it would be a hardcore duty. At first, you've to design a message protocol used for neetwork communication between those three computers. At next, you would write parsers for the designed network messges and reprocess the events according to the newly defined network architecture. Finally, you've to also prepare the verifications for those developments.

Things to be changed

  • Network Architecture
  • Message Protocol
  • Event Handling
  • Business Logic Code

However, if you use the TGrid and Remote Function Call, the issue can't be a problem. In the TGrid, each computer in the network system is just one object. Whether you implement the remote calculator in one computer or distribute operations to three computers, its Business Logic code must be the same, in always.

I also provide you the best example for this performance issue causing the Network Refactoring. The first demonstration code is an implementation of a single calculator server and the second demonstration code is an implementation of a system distributing operations to three servers. As you can see, although principle structure of network system has been changed, you don't need to worry about it if you're using the TGrid and Remote Function Call.

3. Opportunities

3.1. Blockchain

Detailed Content: Appendix > Blockchain

With TGrid, you can develop Blockchain easily.

It's a famous story that difficulty of developing blockchain is very high. Not only because of the high wages of the blockchain developers, but also from a technical point of view, blockchain is actually very difficult to implement. But, if you ask me what is such difficult, I will answer that not Business Logic* but Network System.

The Network System used by blockchain is a type of great distributed processing system, conostructed by millions of computers interacting with network communication. The great distributed processing systems like the blockchain always present us the tremendous difficulties. The word 'perfect' is inserted on every development processes; requirements must be analyzed perfectly, use-cases must be identified perfectly, data and network architectures must be designed, perfectly and mutual interaction test must be perfectly.

On contrary, Business Logic of the blockchain is not such difficult. Core elements of the blockchain are, as the name suggest, the first is 'Block' and the second is 'Chain'. The 'Block' is about defining and storing data and the 'Chain' is about policy that how to reach to an agreement when writing data to the 'Block'.

Component Conception Description
Block Data Structure Way to defining and storing data
Chain Requirements A policy for reaching to an agreement

Let's assume that you are developing the 'Block' and 'Chain' as a program running only on a single computer. In this case, you just need to design the data structure and implement code storing the data on disk. Also, you would analyze the requirements (policy) and implement them. Those skills are just the essentials for programmers. In other word, Business Logic of blockchain is something that any skilled programmers can implement.

  • To develop the Block and Chain:
    • Ability to design Data Structure
    • Ability to store Data on Device
    • Ability to analyze policy (requirements)
    • Ability to implement them

Do you remember? With TGrid and Remote Function Call, you can come true the true Grid Computing. Many computers interacting with network communication are replaced by only one virtual computer. Even Business Logic code of the virtual computer is same with another Business Logic code running on a single physical computer.

Thus, if you adapt the TGrid and Remote Function Call, difficulty of the blockchain development would be dropped to the level of [Business Logic](https://tgrid.com/en/appendix/blockchain.html rather than Network System. Forget complex Network System and just focus on the essence of what you want to develop; the [Business Logic](https://tgrid.com/en/appendix/blockchain.html.

3.2. Public Grid

Related Project: Tutorial > Projects > Grid Market

With TGrid, you can procure resources for Grid Computing from unspecified crowds, very easily and inexpensively.

When composing traditional Grid Computing, of course, many computers should be prepared. As the number of computers required increases, so does the infrastructure and costs associated with procuring them. Also, you've to install programs and configure settings for the network communication on the prepared computers. Such duties increase your efforts and let you to be tired. Is it so nature?

Name Consumer Supplier
Who Developer of Grid Computing Unspecified crowds connecting to the Internet
What Consumes resources of the Suppliers Provides resources to Consumer
How Deliver program code to Suppliers Connect to special URL by Internet Browser

However, TGrid even can economize such costs and efforts dramatically. You can procure resources for Grid Computing from the unspecified crowds. Those unspecified crowds do not need to prepare anything like installing some program or configuring some setting. The only need those unspecified crowds is just connecting to special URL by Internet Browser.

The program that each Supplier should run is provided by the Consumer as JavaScript code. Each Supplier would act its role by the JavaScript code. Of course, interactions with Supplier and Consumer (or with a third-party computer) would use the Remote Function Call, so they are just one virtual computer.

Base language of the TGrid is TypeScript and compilation result of the TypeScript is the JavaScript file. As JavaScript is a type of script language, it can be executed dinamiccaly. Therefore, the Supplier can execute the program by script code delivered by the Consumer.

Grid Market

Grid Market is one of the most typical example case for the Public Grid, a demo project for tutorial learning. In this demo project, Consumer also procures resources from the Suppliers for composing the Grid Computing system. Supplier also provides its resources just by connecting to the special URL by Internet Browser, too. Of course, in the Grid Market, the program that Supplier would run still comes from the Consumer.

However, there's a special thing about the Grid Market, it is that there is a cost for the Consumer to procure the Suppliers' resources. Also, intermediary Market exists and it charges fee for mediation between the Consumer and Supplier.

  • Market: Intermediary market for the Suppliers and Consumers.
  • Consumer: Purchase resources from the *Suppliers.
  • Supplier: Sells its own resources to the Consumer.

3.3. Market Expansions

The Grid Computing market would be grown up day by day.

The future belongs to those who prepare. Prepare the future by TGrid and Remote Function Call. Also, I hope you to hold some changes from the future.

tgrid's People

Contributors

samchon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

tgrid's Issues

Parametric value undefined be null in RFC (Remote Function Call)

Summary

  • TGrid Version: 0.1.10
  • Expected behavior: undefined be delivered as undefined in RFC
  • Actual behavior: Parametric value undefined changed to null in RFC
  • Solution: Wrap each parametric value in each object

Call remote function (1, 2, undefined) -> be (1, 2, null)

Code occuring the bug

import { WebServer } from "tgrid/protocols/web/WebServer";
import { WebConnector } from "tgrid/protocols/web/WebConnector";
import { Driver } from "tgrid/basic/Driver";

class Something
{
    public trace(...args: any[]): void
    {
        console.log(...args);
    }
}

async function main(): Promise<void>
{
    let server: WebServer = new WebServer();
    await server.open(10000, async acceptor =>
    {
        await acceptor.accept(new Something());
    });

    let connector: WebConnector = new WebConnector();
    await connector.connect("ws://127.0.0.1:10000");

    let something: Driver<Something> = connector.getDriver<Something>();
    await something.trace(1, { obj: {} }, undefined, "text");

    await connector.close();
    await server.close();
}
main();
1 { obj: {} } null 'text'

Solution

console.log(JSON.stringify([1, 2, undefined, 'text']));
'[1, 2, null, "text"]'

When insert an undefined value in an Array and stringify the Array, the undefined value is printed as a null value. That's the reason why such error has been occurred. Thus, the only way to avoid such error is to changing data structure the Invoke, message structure for RFC.

export interface IFunction<Params extends any[] = any[]>
extends IBase
{
/**
* Target function (sometimes calsuled in objects) to call.
*/
listener: string;
/**
* Parameters for the function call.
*/
parameters: Params;
}

If the Invoke.IFunction represents parametric values as not atomic values, but being capsuled into objects, the bug must be fixed.

export namespace Invoke
{
    export interface IFunction
    {
        uid: number;
        listener: string;
        parameters: IParameter[];
    }
    export interface IParameter
    {
        type: string;
        value: any;
    }
}

Support Driver type checking

The Driver should support its type checking.

import { WebConnector } from "tgrid/protocols/web/WebConnector";
import { Driver } from "tgrid/components/Driver";

async function main(): Promise<void>
{
    let connector: WebConnector;
    let driver: Driver<IController> = connector.getDriver();

    // not supported yet, but should be possible
    console.log(driver instanceof Driver); // true
}

Error in the 'Driver.d.ts' file: duplicated identifier

Summary

  • TGrid Version: 0.2.6
  • Expected behavior: There's not any problem when using the Driver
  • Actual behavior: Driver.d.ts says Duplicated identifier Driver

Implementing the #24, an ambiguous error has been occuring. There's not any error in the Driver.ts file, but its output file Driver.d.ts reports an error: Duplicated identifier Driver.

Such error occurs when a namespace, designed only for type definition, shares same name with a variable and some of type definitions in the namespace are exported and some of them are not expored. If all of the definitions in the namespace are exported, there wouldn't be any error.

It seems a bug of the TypeScript compiler. So I've published an issue reporting the bug: microsoft/TypeScript#34896

Code occuring the bug

src/components/Driver.ts

components/Driver.d.ts

export declare type Driver<Controller extends object, Parametric extends boolean = false> = Driver.Promisive<Controller, Parametric>;
export declare var Driver: {
    new (): {};
};
export declare namespace Driver {
    export type Promisive<Instance extends object, UseParametric extends boolean = false> = {
        readonly [P in keyof Instance]: Instance[P] extends Function ? Functional<Instance[P], UseParametric> : value_of<Instance[P]> extends object ? Instance[P] extends object ? Promisive<Instance[P], UseParametric> : never : never;
    } & IRemoteObject;
    export type Functional<Method extends Function, UseParametric extends boolean = false> = (Method extends (...args: infer Params) => infer Ret ? Ret extends Promise<infer PromiseRet> ? (...args: FunctionalParams<Params, UseParametric>) => Promise<Primitive<PromiseRet>> : (...args: FunctionalParams<Params, UseParametric>) => Promise<Primitive<Ret>> : (...args: any[]) => Promise<any>) & IRemoteFunction;
    type FunctionalParams<Params extends any[], UseParametric extends boolean> = UseParametric extends true ? Parametric<Params> : Params;
    export type Primitive<Instance> = value_of<Instance> extends object ? Instance extends object ? Instance extends IJsonable<infer Raw> ? value_of<Raw> extends object ? Raw extends object ? PrimitiveObject<Raw> : never : value_of<Raw> : PrimitiveObject<Instance> : never : value_of<Instance>;
    type PrimitiveObject<Instance extends object> = {
        [P in keyof Instance]: Instance[P] extends Function ? never : Primitive<Instance[P]>;
    };
    export type Parametric<Arguments extends any[]> = {
        [P in keyof Arguments]: ParametricValue<Arguments[P]>;
    };
    type ParametricValue<Instance> = value_of<Instance> | Primitive<Instance> | IJsonable<Primitive<Instance>>;
    type value_of<Instance> = is_value_of<Instance, Boolean> extends true ? boolean : is_value_of<Instance, Number> extends true ? number : is_value_of<Instance, String> extends true ? string : Instance;
    type is_value_of<Instance, Object extends IValueOf<any>> = Instance extends Object ? Object extends IValueOf<infer Primitive> ? Instance extends Primitive ? false : true : false : false;
    interface IRemoteObject {
        constructor: never;
        prototype: never;
    }
    type IRemoteFunction = {
        [P in keyof Function | "Symbol"]: never;
    };
    interface IValueOf<T> {
        valueOf(): T;
    }
    export {};
}

[RFC] Parametric types should be compatible with primitive

TGrid has adopted JSON structure on network communication.

Therefore, when calling a remote function, parametric objects always be primitive. Prototype would be JS primitive Object and all methods would be removed. I think such domain characteristic should be reflected to the Driver<Controller> through the meta programming.

Although a Controller has defined its functions to have non-primitive parameters, Driver should make the parameter type to be compatible with primitive. As you can see from the below example code, functions defined in the Feature class have objects as their parameters; Report and Member.

When you wrap the Feature class into the Driver type, those functions' parameters should be changed to compatible with their primitive object types. Methods Feature.insertMember() and Feature.publish() have object typed parameters. Those parameters should be compatible with their primitive type in the Driver<Feature>.

  • Returned type also should be primitive: #21
/* ----------------------------------------------------------------
    ORIGINAL TYPES
---------------------------------------------------------------- */
declare class Feature
{
    public count(): Number;
    public getReport(): Report;
    public getMembers(): Member[];

    public insertMember(obj: Member): void;
    public findMember(key: number | string): Member | null;

    public publish(x: Member, y: Report, pred: Number): Report;
}

declare class Report implements IReport
{
    public code: string;
    public title: string;
    public content: string;

    public archive(): void;
    public publish(): void;
}

declare class Member implements IJsonable<IMember>
{
    private id_: number;
    private name_: string;

    public login(password: string): void;
    public update(name: string): void;
    public toJSON(): IMember
}

/* ----------------------------------------------------------------
    PRIMITIVE TYPES
---------------------------------------------------------------- */
type Driver<Feature> = 
{
    public count(): Promise<number>;
    public getReport(): Promise<IReport>;
    public getMembers(): Promise<IMember[]>;

    public insertMember(obj: Member | IMember | IJsonable<IMember>): Promise<void>;
    public findMember(key: number | string): Promise<Member | null>;

    public publish
        (
            x: Member | IMember | IJsonable<IMember>, 
            y: Report | IReport | IJsonable<IReport>, 
            pred: number
        ): Promise<IReport>;
};

interface IReport
{
    code: string;
    title: string;
    content: string;
}

interface IMember
{
    id: number;
    name: string;
}

Connector.connect() with detailed options through the object type

To extendability, it would better to configure the object typed options parameter.

export class WebConnector
{
    // NOT EXTENTABLE
    public async connection(url: string, timeout?: number): Promise<void>;

    // EXTENDABLE
    public async connect(url: string, options: WebConnector.IConnectOptions): Promise<void>;
}
export namespace WebConnector
{
    export interface IConnectOptions
    {
        timeout: number
    }
}

Refactor connector classes to assign header in the constructor level.

It would better to refactor connector classes to assign header in the constructor level like below. To implement this refactoring without duplicated codes, it would better to refactor all of the connector class to be derived from an abstract class - #41. Also, the header are essential at now, but it would better to reform the header can be null.

export class WebConnector<
        Header extends object | null, 
        Provider extends object | null>
{
    public constructor(header: Header, provider: Provider);
}

Test whether all of the features are exported without any omission

TGrid is exporting many features like classes, modules and functions through the re-export statements like below:

However, the TGrid doesn't have any validation tool that testing whether all of the designed features are exported without any omission. Therefore, if I forget a newly developed feature to using the re-export statement, users of the TGrid cannot use the newly developed feature. So I should add a new testing features in the test automation program, who run by npm run test command, detecting the omission.

The new testing feature's name would be the test_exports.

Add headers in every connection method

Implementing #35, I understood that headers, it would be better to add to every connection methods. Therefore, not only WebConnector.connect() method, but also WorkerConnector.connect() and SharedWorkerConnector.connect() methods can use the headers instead of arugments.

Example

Previous WorkerConnector

export class WorkerConnector<Provider extends object>
{
    public compile(content: string, ...args: string[]): Promise<void>;
    public connect(jsFile: string, ...args: string[]): Promise<void>;
}

Future WorkerConnector

export class WorkerConnector<Provider extends object>
{
    public compile<Headers extends object>(content: string, headers: Headers): Promise<void>;
    public connect<Headers extends object>(jsFile: string, headers: Headers): Promise<void>;
}

`Worker` to support both `thread` and `process` mode

In the NodeJS, it's possible to constructing worker both by process.fork and worker-thread.

Therefore, give option to specialize the work mode in the NodeJS, by supporting the type parameter in theWorkerConnector.

In the WorkerServer case, it would find the type by itself.

export class WorkerConnector<Header, Provider extends object | null>
    extends ConnectorBase<Header, Provider>
    implements IWorkerSystem
{
    /**
     * Initializer Constructor.
     * 
     * @param header An object containing initialization data like activation.
     * @param provider An object providing features for remote system.
     * @param type You can specify the worker mode when NodeJS. Default is "thread".
     */
    public constructor
        (
            header: Header, 
            provider: Provider, 
            type: "thread" | "process" = "thread"
        )
}

Protocol - Web Socket

Outline

TGrid will support the Web Socket protocol.

The Web Socket related components would be implemented by extending and utilizing the Communicator and Driver classes.

Components

WebServer

namespace tgrid.protocols.web
{
    export class WebServer
    {
        /* ----------------------------------------------------------------
            CONSTRUCTORS
        ---------------------------------------------------------------- */
        /**
         * Default Constructor for the `ws` server.
         * 
         * Create an websocket server (`ws://`).
         */
        public constructor();

        /**
         * Initializer Constructor for the `wss` server.
         * 
         * Create a secured websocket server (`wss://`).
         * 
         * @param key Key string.
         * @param cert Certification string.
         */
        public constructor(key: string, cert: string);

        /* ----------------------------------------------------------------
            OPERATIONS
        ---------------------------------------------------------------- */
        /**
         * Open server.
         * 
         * @param port Port number to listen.
         * @param cb Callback function whenever client connects.
         */
        public open<Ret>(port: number, cb: (acceptor: WebAcceptor) => Ret): Promise<void>;

        /**
         * Close server.
         */
        public close(): Promise<void>;

        /**
         * Current state.
         */
        public get state(): WebServer.State;
    }

    export namespace WebServer
    {
        export const enum State 
        { 
            NONE = -1, 
            OPENING, 
            OPEN, 
            CLOSING, 
            CLOSED 
        }
    }
}

WebAcceptor

namespace tgrid.protocols.web
{
    export class WebAcceptor 
        extends basic.CommunicatorBase
    {
        /* ----------------------------------------------------------------
            CONSTRUCTORS
        ---------------------------------------------------------------- */
        /**
         * Hidden Constructor.
         * 
         * You can't create WebAcceptor by yourself. It would be created only by the WebServer.
         */
        private constructor(request);

        /**
         * Close connection.
         */
        public close(): Promise<void>;

        /* ----------------------------------------------------------------
            HANDSHAKES
        ---------------------------------------------------------------- */
        /**
         * Accept connection.
         * 
         * @param protocol Protocol you want to specialize.
         * @param allowOrigin If you want to restrict connection origin, then specialize it.
         * @param cookies Cookies to make client storing it.
         */
        public accept(protocol?: string, allowOrigin?: string, cookies?: ICookie[]): Promise<void>;

        /**
         * Reject connection.
         * 
         * @param status Status code.
         * @param reason Detailed reason to reject.
         * @param extraHeaders Extra heaaders if required.
         */
        public reject(status?: number, reason?: string, extraHeaders?: object): Promise<void>;

        /**
         * Start listening.
         * 
         * Start listening data (function requests) from the remote client.
         * 
         * @param provider An object to provide functions for the remote client.
         */
        public listen<Provider extends object>
            (provider: Provider): Promise<void>;

        /* ----------------------------------------------------------------
            ACCESSORS
        ---------------------------------------------------------------- */
        /**
         * Get driver for remote controller.
         *
         * @return A driver for the remote Controller.
         */
        public getDriver<Controller extends object>(): Driver<Controller>;

        /**
         * Join connection.
         */
        public join(): Promise<void>;

        //----
        // PROPERTIES
        //----
        public get state(): State;
        public get secured(): boolean;

        public get path(): string;
        public get protocol(): string;
        public get extensions(): string;
    }

    export namespace WebAcceptor
    {
        export interface ICookie { ... };

        export const enum State { ... }
    }
}

WebConnector

namespace tgrid.protocols.web
{
    export class WebConnector<Provider extends object>
        extends basic.CommunicatorBase<Provider>
    {
        /* ----------------------------------------------------------------
            CONSTRUCTORS
        ---------------------------------------------------------------- */
        /**
         * Initializer Constructor.
         * 
         * @param provider A provider for server.
         */
        public constructor(provier?: Provider);

        /**
         * Connect to remote web socket server.
         *
         * @param url URL address to connect.
         * @param protocols Candidate protocols to specialize.
         */
        public connect(url: string, protocols?: string | string[]): Promise<void>;

        /**
         * Close connection.
         * 
         * @param code Closing code.
         * @param reason Reason why.
         */
        public close(code?: number, reason?: string): Promise<void>;

        /* ----------------------------------------------------------------
            ACCCSSORS
        ---------------------------------------------------------------- */
        /**
         * Get driver for remote controller.
         *
         * @return A driver for the remote Controller.
         */
        public getDriver<Controller extends object>(): Driver<Controller>;

        /**
         * Wait server to provide.
         * 
         * Wait server to specify its `Provider`.
         */
        public wait(): Promise<void>;

        /**
         * Join connection.
         */
        public join(): Promise<void>;

        //----
        // PROPERTIES
        //----
        public get state(): WebConnector.State;
        public get url(): string;
        public get protocol(): string;
        public get extensions(): string;
    }

    export namespace WebConnector
    {
        export const enum State
        {
            NONE = -1,
            CONNECTING,
            OPEN,
            CLOSING,
            CLOSED
        }
    }
}

Sample Code

server.ts

import * as std from "tstl";
import * as tgrid from "tgrid";

import { Calculator } from ".internal/Calculator";

async function main(): Promise<void>
{
    let server: WebServer = new WebServer();
    await server.open(PORT, async acceptor =>
    {
        await acceptor.accept(); // ALLOW CONNECTION
        await acceptor.listen(/calculator/.test(acceptor.path)
            ? new Calculator()
            : new std.Vector<number>()); // SET LISTENER
    });
}

client-vector.ts

import { Vector } from "tstl/container";
import { WebConnector } from "tgrid/protocols/web";
import { Driver } from "tgrid/basic";

type IVector<T> = Pick<Vector<T>, "size" | "at" | "push_back">;

async function main(): Promise<void>
{
    let connector: WebConnector = new WebConnector();
    await connector.connect("ws://127.0.0.1:10100");

    let vec: Driver<IVector<number>> = connector.getDriver<IVector<number>>();
    for (let i: number = 0; i < 5; ++i)
        await vec.push_back(i);

    console.log("size:", await vec.size());
    for (let i: number = 0; i < await vec.size(); ++i)
        console.log("  element:", await vec.at(i));
}
main();

client-calculator.ts

import { WebConnector } from "tgrid/protocols/web";
import { Driver } from "tgrid/basic";
import { ICalculator } from "./internal/Calculator";

async function main(): Promise<void>
{
    //----
    // PREPARES
    //----
    // DO CONNECT
    let connector: WebConnector = new WebConnector();
    await connector.connect("ws://127.0.0.1:10102");

    // GET DRIVER
    let calc: Driver<ICalculator> = connector.getDriver<ICalculator>();
    
    //----
    // CALL FUNCTIONS
    //----
    // FUNCTIONS IN THE ROOT SCOPE
    console.log("1 + 6 =", await calc.plus(1, 6));
    console.log("7 * 2 =", await calc.multiplies(7, 2));

    // FUNCTIONS IN AN OBJECT (SCIENTIFIC)
    console.log("3 ^ 4 =", await calc.scientific.pow(3, 4));
    console.log("log (2, 32) =", await calc.scientific.log(2, 32));

    try
    {
        // TO CATCH EXCEPTION IS STILL POSSIBLE
        await calc.scientific.sqrt(-4);
    }
    catch (err)
    {
        console.log("SQRT (-4) -> Error:", err.message);
    }

    // FUNCTIONS IN AN OBJECT (STATISTICS)
    console.log("Mean (1, 2, 3, 4) =", await calc.statistics.mean(1, 2, 3, 4));
    console.log("Stdev. (1, 2, 3, 4) =", await calc.statistics.stdev(1, 2, 3, 4));

    //----
    // TERMINATE
    //----
    await connector.close();
}
main();

internal/Calculator.ts

/* ----------------------------------------------------------------
    INTERFACES
---------------------------------------------------------------- */
export interface ICalculator
{
    scientific: IScientific;
    statistics: IStatistics;

    plus(x: number, y: number): number;
    minus(x: number, y: number): number;
    multiplies(x: number, y: number): number;
    divides(x: number, y: number): number;
}

export interface IScientific
{
    pow(x: number, y: number): number;
    sqrt(x: number): number;
    log(x: number, y: number): number;
}

export interface IStatistics
{
    mean(...elems: number[]): number;
    stdev(...elems: number[]): number;
}

/* ----------------------------------------------------------------
    CLASSES
---------------------------------------------------------------- */
export class Calculator implements ICalculator
{
    public scientific = new Scientific();
    public statistics = new Statistics();

    public plus(x: number, y: number): number
    {
        return x + y;
    }
    public minus(x: number, y: number): number
    {
        return x - y;
    }
    
    public multiplies(x: number, y: number): number
    {
        return x * y;
    }
    public divides(x: number, y: number): number
    {
        if (y === 0)
            throw new Error("Divided by zero.");
        return x / y;
    }
}

class Scientific implements IScientific
{
    public pow(x: number, y: number): number
    {
        return Math.pow(x, y);
    }

    public log(x: number, y: number): number
    {
        if (x < 0 || y < 0)
            throw new Error("Negative value on log.");
        return Math.log(y) / Math.log(x);
    }

    public sqrt(x: number): number
    {
        if (x < 0)
            throw new Error("Negative value on sqaure.");
        return Math.sqrt(x);
    }
}

class Statistics implements IStatistics
{
    public mean(...elems: number[]): number
    {
        let ret: number = 0;
        for (let val of elems)
            ret += val;
        return ret / elems.length;
    }

    public stdev(...elems: number[]): number
    {
        let mean: number = this.mean(...elems);
        let ret: number = 0;

        for (let val of elems)
            ret += Math.pow(val - mean, 2);

        return Math.sqrt(ret / elems.length);
    }
}

Refactor API Docs

  • Add a new npm module, typedoc-plugin-exclude-references
  • Changes @typeParam to @template
  • Add @packageDocumentation before @module

WebServer.close() waits all clients to be disconnected.

Summary

  • TGrid Version: 0.6.1
  • Expected behavior: WebServer.close() closes the web socket server directly.
  • Actual behavior: WebServer.close() waits until its clients to be disconnect by themselves.

When call the WebServer.close() method and there's a client who is connecting to the WebServer, the method call would be completed until the client disconnects network connection with the server by itself. Therefore, it remained clients do not disconnect their connections by themselves, the WebServer.close() method would not be completed and fall into the forever sleep.

Code occuring the bug

const PORT = 10101;

export async function test_web_server_close(): Promise<void>
{
    let server: WebServer<null, null> = new WebServer();
    await server.open(PORT, acceptor => acceptor.accept(null));

    let connector: WebConnector<null, null> = new WebConnector(null, null);
    await connector.connect(`ws://127.0.0.1:${PORT}`);

    server.close().then(() =>
    {
        // it would be shown after 5,000 milliseconds later
        console.log("WOW");
    });
    await sleep_for(5000);
    await connector.close();
}

Use Github Actions instead of Travis-CI

The Github Actions provides not only virtual linux, but also virtual windows and virtual mac. With the virtual windows and mac, it's possible to testing the TGrid in the browser level automatically. Therefore, discard the travis-ci and adapt the new Github Actions.

Basic Component - Communicator

Outline

Communicator, a basic component, taking full charge of network communication.

sequence

The message structure for RFC (Remote Function Call) would be represented such like below and those structured data would be sent to the remote system like the top sequence diagram.

namespace tgrid.components
{
    // Message would be one of them.
    export type Invoke = IFunction | IReturn;

    export interface IFunction
    {
        uid: number; // identifier
        listener: string; // function (sometimes capsuled in objects) to call
        parameters: IParameter[]; // parameters for the function
    }
    export interface IParameter
    {
        type: string;
        value: any;
    }

    export interface IReturn
    {
        uid: number; // identifier
        success: boolean; // true -> return, false -> exception
        value: any | undefined; // returned value or thrown exception
    }
}

Communicator

The Communicator class would provide common features for RFC (Remote Function Call). Network communication of each protocol, they would be implemented in each protocol module by extending the Communicator class.

namespace tgrid.components
{
    export abstract class Communicator<Provider extends object>
    {
        /* ----------------------------------------------------------------
            CONSTRUCTORS
        ---------------------------------------------------------------- */
        /**
         * Initializer Constructor.
         * 
         * @param provider An object providing functions for the remote system.
         */
        protected constructor(provider?: Provider);

        /**
         * Destruct the communicator.
         * 
         * A destructor function should be called when the network communication has closed. 
         * It would destroy all RFC (function calls to the remote system, via the 
         * `Driver<Controller>`).
         * 
         * @param error An error instance to be thrown to the all remote functions that are 
         *              not returned yet.
         */
        protected destructor(error?: Error): Promise<void>;

        /* ----------------------------------------------------------------
            ACCESSORS
        ---------------------------------------------------------------- */
        /**
         * Get driver for remote controller.
         * 
         * The `Controller` is an interface who defines provided functions from the remote 
         system. The `Driver` is an object who makes to call remote functions, defined in the 
         * `Controller` and provided by `Provider` in the remote system, possible.
         * 
         * @return A driver for the remote controller.
         */
        public getDriver<Controller extends object>(): Driver<Controller>;

        /**
         * Join connection.
         * 
         * Wait until the connection to be closed (until `CommunicatorBase.desturctor()` called).
         */
        public join(): Promise<void>;

        /* ----------------------------------------------------------------
            COMMUNICATORS
        ---------------------------------------------------------------- */
        /**
         * Data Replier.
         * 
         * A function should be called when data has come from the remote system. 
         * 
         * When you receive a message from the remote system, then parse the message with your
         * special protocol and convert it to be an *Invoke* object. After that conversion, call 
         * this method.
         * 
         * @param invoke Structured data converted by your special protocol.
         */
        protected replyData(invoke: Invoke): void;

        /** 
         * A function sending data to the remote system.
         */
        protected abstract sendData(invoke: Invoke): void;

        /**
         * A predicator inspects whether the *network communication* is ready.
         */
        protected abstract inspectReady(): Error | null;
    }
}

Enhance package.json, "devDependencies"

To build the TGrid, users must install typescript as a global module by themselves. Also, to build the API Documents, users must install typedoc and typedoc-plugin-external-module-name globally, too. Those actions are not automated and sometimes make users annoying.

To automate those actions, it would better to write those modules in the devDependencies.

WorkerConnector.compile can cause disk | memory leak

Summary

  • TGrid Version: 0.1.0
  • Expected behavior: Disk or Memory leak should not be occured.
  • Actual behavior: Disk leak could be in NodeJS and Memory leak in Browser.

Code occuring the bug

In NodeJS

if (Compiler.remove)
{
let path: string = await Compiler.compile(content);
await this.connect(path, ...args);
await Compiler.remove(path);
}

In NodeJS environment, you can see the WorkerConnector.compile() method calls a neighbor WorkerConnector.connect() function. If the WorkerConnector.compile() method throws an exception, then Compiler.remove() cannot be called and it may cause the disk leak.

In Browser

await this.connect(<string>Compiler.compile(content), ...args);

In Web Browser, WorkerConnector.compile() compiles (creates) a JS file, but does not remove it. Of course, the JS file would be automatically removed when the browser closed, but it seems dangerous. If someone compiles (WorkerConnector.compile()) lots of JS files in a page, then they would be memory leak.

export function compile(content: string): string
{
let blob: Blob = new Blob([content], { type: "application/javascript" });
return URL.createObjectURL(blob);
}

WorkerConnector.compile() should utilize not only URL.createObject function, but also URL.removeObject function.

[RFC] Prohibit access to edge-underscored members

Prohibit Remote Function Calls on members starting or ending with the underscore (_).

import { WebConnector } from "tgrid/protocols/web/WebConnector";
import { Driver } from "tgrid/components/Driver";

interface ICalculator
{
    // Possible to access
    plus(x: number, y: number): number;
    minus(x: number, y: number): number;
    statistics: IStatistics;

    // Impossible to access
    _Divides(x: number, y: number): number; // impossible
    scientific_: IScientific; // impossible
}

async main(): Promise<void>
{
    //----
    // PREPARATION
    //----
    let connector: WebConnector = new WebConnector();
    await connector.connect("ws://127.0.0.1:8091");

    let calc: Driver<ICalculator> = connector.getDriver();

    //----
    // REMOTE FUNCTION CALLS
    //----
    // POSSIBLE
    await calc.plus(5, 6);
    await calc.minus(7, 2);
    await calc.statistics.mean(1, 2, 3, 4);

    // IMPOSSIBLE - be RuntimeError, Also should be compile error
    await calc._Divides(72, 9);
    await calc.scientific_.pow(3, 4);

    await connector.close();
}
main();

Refactor communicator classes to use abstract connector and acceptor

Refactor all of the communicator classes in the TGrid to be derived from those abstract classes.

  • class ConnectorBase
  • class AcceptorBase
export abstract class ConnectorBase<Header, Provider extends object | null>
    extends Communicator<Provider>
{
    public constructor(header: Header, provider: Provider);
    public get header(): Header;
    public get state(): ConnectorBase.State;

    protected inspectReady(method: string): Error | null;
}

export abstract class AcceptorBase<Header, Provider extends object | null>
    extends Communicator<Provider>
{
    protected constructor(header: Header);
    public get header(): Header;
    public get state(): AcceptorBase.State;

    protected inspectReady(method: string): Error | null;
}

WorkerServer.join() is not working (Also SharedWorkerAcceptor)

Summary

  • TGrid Version: 0.1.10
  • Expected behavior: WorkerServer.join() works correctly
  • Actual behavior: The program would be destroyed before

WorkerServer is a class constructed in the Worker instance. The Worker instance would be destroyed directly when the main program closes it. Thus, WorkerServer.join() cannot be run. Also, SharedWorkerAcceptor.join() does not work when the exited client was the last browser.

Code occuring the bug

index.ts

import { WorkerConnector } from "tgrid/protocols/workers/WorkerConnector";

async function main(): Promise<void>
{
    let connector: WorkerConnector();
    await connector.connect(__dirname + "/server.js");

    await connector.join();
}
main();

server.ts

import { WorkerServer } from "tgrid/protocols/workers/WorkerServer";

async function main(): Promise<void>
{
    let server: WorkerServer = new WorkerServer();
    await server.open();
    await server.join();

    console.log("Cannot reach to this code");
    console.log("The program would be terminated before");
}
main();

Protocol - Workers

Outline

TGrid will consider the worker as a remote system.

JavaScript does not support the thread. JavaScript only provides worker, which is almost same with the process. In the worker environment, sharing memory variable is not possible. Only IPC (Inter-Process Communication) is allowed.

In such restriction, I will provide RFC (Remote Function Call) for help the worker. With the RFC, it's not possible to sharing memory, however, sharing objects and functions may be possible. That's the reason why I'm planning to consider workers as remote systems.

Components

1. Dedicated Worker

1.1. WorkerServer

namespace tgrid.protocols.workers
{
    export class WorkerServer<Provider extends object>
        extends basic.CommunicatorBase<Provider>
    {
        /* ----------------------------------------------------------------
            CONSTRUCTORS
        ---------------------------------------------------------------- */
        /**
         * Initializer Constructor.
         * 
         * @param provider An object providing functions for the connector.
         */
        public constructor(provider: Provider);

        /**
         * Close connection.
         */
        public close(): Promise<void>;

        /* ----------------------------------------------------------------
            ACCESSORS
        ---------------------------------------------------------------- */
        /**
         * Get driver for remote controller.
         *
         * @return A driver for the remote Controller.
         */
        public getDriver<Controller extends object>(): components.Driver<Controller>;

        /**
         * Join connection.
         */
        public join(): Promise<void>;
    }
}

1.2. WorkerConnector

namespace tgrid.protocols.workers
{
    export class WorkerConnector<Provider extends object>
        extends basic.CommunicatorBase<Provider>
    {
        /* ----------------------------------------------------------------
            CONSTRUCTORS
        ---------------------------------------------------------------- */
        /**
         * Initializer Constructor.
         * 
         * @param provider An object providing functions for the server.
         */
        public constructor(provider: Provider);
        
        /**
         * Connect to worker server with compilation.
         * 
         * @param source JS Source Code to be server with compilation. 
         *               It would better make the source code to be bundled.
         */
        public compile(source: string): Promise<void>;

        /**
         * Connect to worker server.
         * 
         * @param jsFile JS File to be worker server.
         */
        public connect(jsFile: string): Promise<void>;

        /**
         * Close connection.
         */
        public close(): Promise<void>;

        /* ----------------------------------------------------------------
            ACCESSORS
        ---------------------------------------------------------------- */
        /**
         * Get driver for remote controller.
         *
         * @return A driver for the remote Controller.
         */
        public getDriver<Controller extends object>(): components.Driver<Controller>;

        /**
         * Join connection.
         */
        public join(): Promise<void>;
    }
}

2. Shared Worker

2.1. SharedWorkerServer

namespace tgrid.protocols.workers
{
    export class SharedWorkerServer
    {
        /**
         * Default Constructor.
         */
        public constructor();

        /**
         * Open Server.
         * 
         * @param cb Callback function called whenever client connects.
         */
        public open(cb: (acceptor: SharedWorkerAcceptor) => any): Promise<void>;

        /**
         * Close Server.
         */
        public close(): Promise<void>;
    }
}

2.2. SharedWorkerAcceptor

namespace tgrid.protocols.workers
{
    export class SharedWorkerAcceptor
        extends basic.CommunicatorBase
    {
        /* ----------------------------------------------------------------
            CONSTRUCTORS
        ---------------------------------------------------------------- */
        /**
         * Hidden Constructor.
         * 
         * You can't create WebAcceptor by yourself. It would be created only by the WebServer.
         */
        private constructor(server, port);

        /**
         * Close connection.
         */
        public close(): Promise<void>;

        /* ----------------------------------------------------------------
            HANDSHAKES
        ---------------------------------------------------------------- */
        /**
         * Accept connection.
         */
        public accept(): Promise<void>;

        /**
         * Reject connection.
         */
        public reject(): Promise<void>;

        /**
         * Start listening.
         * 
         * Start listening data (function requests) from the remote client.
         * 
         * @param provider An object to provide functions for the remote client.
         */
        public listen<Provider extends object>
            (provider: Provider): Promise<void>;

        /* ----------------------------------------------------------------
            ACCCSSORS
        ---------------------------------------------------------------- */
        /**
         * Get driver for remote controller.
         *
         * @return A driver for the remote Controller.
         */
        public getDriver<Controller extends object>(): components.Driver<Controller>;

        /**
         * Wait server to provide.
         * 
         * Wait server to specify its `Provider`.
         */
        public wait(): Promise<void>;

        /**
         * Join connection.
         */
        public join(): Promise<void>;
    }
}

2.3. SharedWorkerConnector

namespace tgrid.protocols.workers
{
    export class SharedWorkerConnector<Provider extends object>
        extends basic.CommunicatorBase<Provider>
    {
        /**
         * Initializer Constructor.
         * 
         * @param provider A provider for server.
         */
        public constructor(provider: Provider);
        
        /**
         * Connect to shared worker server.
         * 
         * @param jsFile JS File to be worker server.
         */
        public connect(jsFile: string): Promise<void>;

        /**
         * Close connection.
         */
        public close(): Promise<void>;

        /* ----------------------------------------------------------------
            ACCCSSORS
        ---------------------------------------------------------------- */
        /**
         * Get driver for remote controller.
         *
         * @return A driver for the remote Controller.
         */
        public getDriver<Controller extends object>(): components.Driver<Controller>;

        /**
         * Wait server to provide.
         * 
         * Wait server to specify its `Provider`.
         */
        public wait(): Promise<void>;

        /**
         * Join connection.
         */
        public join(): Promise<void>;

        public get state(): SharedWorkerConnector.State;
    }

    export namespace SharedWorkerConnector
    {
        export const enum State { ... }

        /**
         * Compile source to JS File.
         * 
         * @return Path of compiled JS File.
         */
        export function compile(source: string): string;
    }
}

3. Service Worker

I'm studying the Service Worker. If the Service Worker is enough good to consider as remote system, then it will also be implemented.

Sample Code

server.ts

import { WorkerServer } from "tgrid/protocols/workers";
import { Driver } from "tgrid/basic";

import { Mutex } from "tstl/thread";
import { randint } from "tstl/algorithm";

interface IController
{
    mutex: Mutex;
    print(str: string): void;
}

async function main(str: string): Promise<void>
{
    // PREPARE SERVER & DRIVER
    let server: WorkerServer = new WorkerServer();
    let driver: Driver<IController> = server.getDriver<IController>();

    // REMOTE FUNCTION CALLS
    await driver.mutex.lock();
    {
        for (let i: number = 0; i < 20; ++i)
            await driver.print(str);
        await driver.print("\n");
    }
    await driver.mutex.unlock();

    // CLOSE THE SERVER (WORKER)
    await server.close();
}
main(randint(0, 9) + "");

client.ts

import { WorkerConnector } from "tgrid/protocols/workers";

import { Vector } from "tstl/container";
import { Mutex } from "tstl/thread";

// FEATURES TO PROVIDE
namespace provider
{
    export var mutex = new Mutex();
    export function print(str: string): void
    {
        process.stdout.write(str);
    }
}

async function main(): Promise<void>
{
    let workers: Vector<WorkerConnector<typeof provider>> = new Vector();

    // CREATE WORKERS
    for (let i: number = 0; i < 4; ++i)
    {
        workers.push_back(new WorkerConnector(provider));
        workers.back().connect(__dirname + "/server.js");
    }

    // WAIT THEM TO BE CLOSED
    for (let w of workers)
        await w.join();
}
main();

Headers info when connecting to a web-socket server

Add headers info when connecting to a web-socket server composed by the TGrid. The headers may include additional information like activation and web-socket server may decide whether to accept the connection or not by parsing the headers.

When the feature adding headers has been implemented, web-socket components WebConnector, WebAcceptor and WebServer would be changed like below. Also, this update may cause a critical change on the message protocol. Therefore, when the feature has been implemented, it can't be compatible with the previous versions.

export class WebConnector<Provider extends object>
{
    public connect<Headers extends object>(url: string, headers?: Headers): Promise<void>;
}

export class WebAcceptor<Headers extends object, Provider extends object>
{
    public get path(): string;
    public get state(): string;
    public get headers(): Headers;
}

export class WebServer<Provider extends object>
{
    public open<Headers extends object>
    (
        port: number, 
        handler: (acceptor: WebAcceptor<Headers, Provider>) => any
    ): Promise<void>;
}

[RFC] Returned object type should be primitive

TGrid has adopted JSON structure on network communication.

Therefore, when calling a remote function returning object type, the returned object always be primitive. Prototype would be JS primitive Object and all methods would be removed. I think such characteristic should be reflected to the Driver<Controller> through the meta programming.

Although a Controller has defined its functions to return non-primitive object, Driver should convert the returned object type to be primitive. As you can see from the below example code, functions defined in the Feature class return object type; Report and Member.

However, when wrap the Feature class into the Driver type, those functions are all converted to return the primitive object. The interface IReport returned by Driver<Feature> does not have any method. Remote functions returning Member type are converted to return IMember type, which is derived from the Member.toJSON() method.

/* ----------------------------------------------------------------
    ORIGINAL TYPES
---------------------------------------------------------------- */
class Feature
{
    public getReport(): Report;
    public getMembers(): Member[];

    public insertMember(id: number, name: string): Member;
    public findMember(key: number | string): Member | null;
}

class Report implements IReport
{
    public code: string;
    public title: string;
    public content: string;

    public archive(): void;
    public publish(): void;
}

class Member implements IJsonable<IMember>
{
    private id_: number;
    private name_: string;

    public login(password: string): void;
    public update(name: string): void;
    public toJSON(): IMember
}

/* ----------------------------------------------------------------
    PRIMITIVE TYPES
---------------------------------------------------------------- */
type Driver<Feature> = 
{
    getReport(): Promise<IReport>;
    getMembers(): Promise<IMember[]>;

    insertMember(id: number, name: string): Promise<IMember>;
    findMember(key: number | string): Promise<IMember | null>;
}

interface IReport
{
    code: string;
    title: string;
    content: string;
}

interface IMember
{
    id: number;
    name: string;
}

Driver.Primitive converts user-defined literals

Summary

  • TGrid Version: 0.2.4
  • Expected behavior: Do not covert user-defined literals to string
  • Actual behavior: Converts user-defined literals to the string

Code occuring the bug

type MyBoolean = Driver.Pritimive<false>; // be boolean
type MyNumber = Driver.Pritimive<4>; // be number
type MyString = Driver.Pritimive<"my_literal">; // be string

Strict Mode

TGrid should support some people who are using TyeScript in Strict mode.

tsconfig.json

{
    "compilerOptions: {
        ....,
        "strict": true
    }
}

Compile error on Driver after upgrading the TypeScript to v3.9.2

Summary

  • TGrid Version: 0.4.1
  • TypeScript Version: 3.9.2
  • Expected behavior: No error on the Driver like TypeScript v3.8.3
  • Actual behavior: Be error in the TypeScript v3.9.2

After upgrading the TypeScript to v3.9.2, the Driver.Promisive causes a type of compile error like below. The error message is telling that the Driver.Promisive is referencing itself circularly, but such error had not been occurred until the update. TypeScript v3.8.3 does not generate such, recursive reference, compile error.

src/components/Driver.ts:21:101 - error TS2315: Type 'Promisive' is not generic.
src/components/Driver.ts:41:17 - error TS2456: Type alias 'Promisive' circularly references itself.
src/components/Driver.ts:47:22 - error TS2315: Type 'Promisive' is not generic.

Code occuring the bug

export type Promisive<Instance extends object, UseParametric extends boolean = false> = Readonly<
{
[P in keyof Instance]: Instance[P] extends Function
? Functional<Instance[P], UseParametric> // function, its return type would be capsuled in the Promise
: value_of<Instance[P]> extends object
? Instance[P] extends object
? Promisive<Instance[P], UseParametric> // object would be promisified
: never // cannot be
: never // atomic value
} & IRemoteObject>;

Looking at implementation code of the Driver.Promisive type, you can see that the Driver.Promisive is referencing itself in the 47th line. However, generic target of the recursive referencing is not Instance itself but Instance[P], a child instance belonged to the Instance.

Therefore, I deduct that after the upgrade of TypeScript to v3.9.2, TypeScript compiler has lost to distinguishing the target structure's hierarchical relationship when using the recursive referencing type like the Driver.Promisive.

How to do

I'll write a bug reporting issue into the TypeScript repository. Until the TypeScript developers fix the error, we need to use old version of TypeScript when using the TGrid or recursive generic type.

Change "websocket" library to another one

The websocket library is made by C++, therefore it requires user to prepare a C++ compiler when installing the npm module. It can be a great entry barrier to some developers who are using Windows and have installed NodeJS without VS C++ compiler.

Thus, it would bettet to change websocket library to anothet one who does not require thr C++ compiler

[RFC] Security Enhancements are required

Summary

  • TGrid Version: 0.1.10
  • Expected behavior: RFCs on dangerous members must be blocked.
    • Object.constructor
    • Function.toString()
    • Module.prototype
  • Actual behavior: RFCs to those dangerous members are possible.

Code occuring the bug

Until v0.1, Remote Function Call can access to dangerous members.

If you call the Function.toString() to member function who is defined in the remote object, you can read implementation code of the remote Provider. If you access to constructor if a remote object, you can access to prototype definition of the remote Provider.

It seems not good within framework of the security. I think those dangerous accesses must be blocked. The blocking must be implemented not only in the RFC level, but also the Driver (type definition) level.

import { WebConnector } from "tgrid/protocols/web/WebConnector";
import { Driver } from "tgrid/components/Driver";

interface ICalculator
{
    scientific: IScientific;
    statistics: IStatistics;

    plus(x: number, y: number): number;
    minus(x: number, y: number): number;
    multiplies(x: number, y: number): number;
    divides(x: number, y: number): number;
}

async function main(): Promise<void>
{
    let connector: WebConnector = new WebConnector();
    await connector.connect("ws://127.0.0.1:10101");

    let calc: Driver<ICalculator> = connector.getDriver();

    // CAN SEE IMPLEMENTATION CODE OF REMOTE PROVIDER
    console.log(await calc.constructor.toString()); 
    console.log(await calc.plus.toString());

    // DANGEROUS ACCESSMENTS, SHOULD BE BLOCKED
    console.log(await calc.prototype.multiplies.bind({})(3, 4));
    console.log(await calc.prototype.divides.call({}, 3, 4));

    await connector.close();
}

Enhancements

The dangerous accesses would be prohibited in RFC communication level. Also, I'm planning to block those dangerous accesses in the compiler level. The Driver type would prohibit to access to those dangerous members by assigning the never type.

type Driver<ICalculator> = 
{
    scientific: Driver<IScientific>;
    statistics: Driver<IStatistics>;

    // FUNCTIONS ARE ENHANCED BY FUNCTOR
    plus: Functor<(x: number, y: number) => Promise<number>>;
    minus: Functor<(x: number, y: number) => Promise<number>>;
    multiplies: Functor<(x: number, y: number) => Promise<number>>;
    divides: Functor<(x: number, y: number) => Promise<number>>;

    // CANNOT ACCESS TO DANGEROUS MEMBERS
    constructor: never;
    prototype: never;
};

type Functor<T extends Function> = T & 
{
    constructor: never;
    prototype: never;
    Symbol: never;
    bind: never;
    ...
};

Generate d.ts files for faye-websocket

To fix the #32 issue, I'm considering faye-websocket library. However, the faye-websocket is not made by TypeScript and even DefinitelyTyped is not supporting it.

Therefore, I should make d.ts files for the faye-websocket.

Support default import statement

Support default import statement like below:

import tgrid from "tgrid";

import components from "tgrid/components";
import protocols from "tgrid/protocols";
import web from "tgrid/protocols/web";
import workers from "tgrid/protocols/workers";
import utils from "tgrid/utils";

Basic Component - Driver

Outline

A proxy object, being a bridge between Communicator and Controller, making RFC (Remote Function Call) possible.

sequence

The Driver class would be an object who makes to call remote functions, defined in the Controller and provided by Provider in the remote system, possible. In other words, calling a functions in the Driver<Controller>, it would mean to call a matched function in the remote system's Provider object.

  • Controller: Definition only
  • Driver: Remote Function Call

Definition

To make RFC (Remote Function Call) possible, the Driver class must extend the Proxy class implementing Controller functions and delivering the function calls to the Communicator class.

namespace tgrid.components
{
    export class Driver<Controller extends object>
        extends Proxy<Promisify<Controller>>
    {
        public constructor(communicator: CommunicatorBase);
    }

    // Functions must be converted to return Promise<T> value.
    // sum(x: number, y: number): number -->> sum(x: number, y: number): Promise<number>
    type Promisify<Controller extends object>;
}

Also, calling functions in the remote system would be processed not in synchronous level. They would be processed in the asynchronous level. Thus, the Driver class should convert functions in the Controller to be return Promise typed value.

// Can be a "Controller"
interface ICalculator
{
    plus(x: number, y: number): number;
    minus(x: number, y: number): number;
    multiplies(x: number, y: number): number;
    divides(x: number, y: number): number;
}

// Return type T must be converted to the Promise<T>
type Promisify<ICalculator> = 
{
    plus(x: number, y: number): Promise<number>;
    minus(x: number, y: number): Promise<number>;
    multiplies(x: number, y: number): Promise<number>;
    divides(x: number, y: number): Promise<number>;
};

Enhance WebConnector.connect() method through Latch

Looking at implementation code of WebConnector.connect(), you can see that the method is synchronized through Promise pattern. However, the implementation seems dangerous because sequence of the synchronization are separated to many parts and those separated codes make the implementation code complicate.

To simplify the WebConnector.connect() method without damaging the synchronization, it would better to utilizing Latch class, who is a type of thread supporting component and newly added to C++20 and TSTL v2.3. Implementation code of the WebConnector.connect() method can be much simple through the Latch.wait_for() and Latch.count_down() method.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.