metacall / deploy Goto Github PK

copy of TODO

• Move the ./src/lib to a new repo so it can be reused in https://github.com/metacall/faas -> https://github.com/metacall/protocol
• Implement the deployment in case of PackageError.None in index.ts:
  • It should ask for a plan in the user (we should store this in the configuration, in a list of projects mapped to plans) or reuse an existent plan if it has been already selected.
  • Use the protocol integration test and the descriptor in order to deploy it, we can implement a function which wraps all the behavior into a simple function or implement it directly in the index.ts
• Implement command-line arguments:
  • Take a command-line argument to specify project dir (-w, --workdir, default to process.cwd())
  • Take a command-line flag to specify email (-e, --email)
  • Take a command-line flag to specify password (-p, --password)
  • Take a command-line flag to specify token (-t, --token)
  • Take a command-line flag to specify force [if there is a deploy already deployed on a existing plan, delete it and deploy again] (-f, --force)
  • Take a command-line flag to specify plan [define the plan to deploy the project: Essential, Standard, Premium, also we should implement an enum type for this and replace in in places like: https://github.com/metacall/deploy/blob/97951f907c166b856c9952ddf89b778a9fdc7fb1/src/lib/protocol.ts#L23 , https://github.com/metacall/deploy/blob/97951f907c166b856c9952ddf89b778a9fdc7fb1/src/lib/protocol.ts#L116 , ] (-P, --plan)
    • Change type string type to Plans enum.
  • Take a command-line flag to specify plan ID [same as before but indicating the plan ID which is a SHA like ID, this will need the modification of the backend because this feature is not implemented yet] (-d, --plan-id)
  • Take a command-line flag to specify use without token [avoiding auth for https://github.com/metacall/faas] (-i, --insecure)
  • Take a command-line flag to specify config dir (-d, --config-dir)
  • Take a command-line flag to specify base URL of server (-u, --server-url)
  • Take a command-line flag to specify name to deploy as (-n, --project-name, default to dirname)
  • Take a command-line flag to specify ignore pattern for the bundle (-i, --ignore-pattern, default to project type default)
• Figure out how to detect project type
  • Only detect if argument wasn't passed
• Figure out how to properly zip a folder
• Stream zip file to disk, then read off disk for sending (Can we do this better?)
• Finish the wizard in case of PackageError.JsonNotFound in index.ts:
  • Ask for environment variables (store them in a secret on the client on the configuration? or implement a way to reuse them in the deploy on the FaaS?)
  • Ask for the plan (store them in the client on the configuration? or implement a way to reuse them in the deploy on the FaaS?)
  • Store the metacall-${lang_id}.jsons once the wizard finishes so it can be reused on the next run of the deploy
• Clean code and unused comments left for implementing the rest of options
• Style all error messages and input requests everything looks uniform, clear and with the same appearence (for example, add ':' when needed, or dot / exclamations/question marks at the end of a sentence).

Stack Trace

UNKNOWN_VALUE: Unknown value: dist/test/
    at Object.commandLineArgs [as default] (/home/raj/Desktop/metacall/deploy/node_modules/command-line-args/dist/index.js:1369:21)
    at parse (/home/raj/Desktop/metacall/deploy/node_modules/ts-command-line-args/dist/parse.js:60:49)
    at Object.<anonymous> (/home/raj/Desktop/metacall/deploy/dist/cli/args.js:24:52)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/home/raj/Desktop/metacall/deploy/dist/auth.js:10:32)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/home/raj/Desktop/metacall/deploy/dist/startup.js:10:16)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/home/raj/Desktop/metacall/deploy/dist/test/protocol.integration.spec.js:10:19)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:168:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:337:24)
    at async importModuleDynamicallyWrapper (node:internal/vm/module:437:15)
    at async formattedImport (/home/raj/Desktop/metacall/deploy/node_modules/mocha/lib/nodejs/esm-utils.js:7:14)
    at async Object.exports.requireOrImport (/home/raj/Desktop/metacall/deploy/node_modules/mocha/lib/nodejs/esm-utils.js:48:32)
    at async Object.exports.loadFilesAsync (/home/raj/Desktop/metacall/deploy/node_modules/mocha/lib/nodejs/esm-utils.js:103:20)
    at async singleRun (/home/raj/Desktop/metacall/deploy/node_modules/mocha/lib/cli/run-helpers.js:125:3)
    at async Object.exports.handler (/home/raj/Desktop/metacall/deploy/node_modules/mocha/lib/cli/run.js:374:5)

Solution Proposed

1. Move protocol.integration.spec.test to metacall/protocol

Additional Context

1. This is not related to any file in the dist/test, simply means moving the file mentioned above will not solve the issue.
2. Remove cli.integration.spec.test and Remove before() from protocol.integration.spec.test and mark every test case to turn true, then test will not fail.
3. It is parsing the arguments in the wrong process and it looks like this

      [
           '/home/pc/.nvm/versions/node/v16.14.0/bin/node',
            '/home/pc/Desktop/metacall/deploy/node_modules/.bin/mocha',
            'dist/test/'
     ]

When doing metacall-deploy -f if the deploy is empty, it does not work and it should. This is a bug, it should deploy anyway.

Refer Todo, --force flag requirement.

Currently, user can pass an unknown flag to the CLI, and an error is thrown.

Error should be handled and user should be notified that the flag don't exist.

We need some enviroment variables while testing and there is a possiblity that dev won't add .env file and running tests without those env vars will be useless.

So this function will check for every enviroment var we need while testing and if succeed then we will run the tests and if not we will simply skip those tests by raising a failure with a message "Env not found or whatever".

@viferga Should we add a flag, which will list out plans available in the user's account, this will be very handy before anyone tries to deploy, wdyt?

This will also be good for testing in the beginning before we actually start testing by deployment like user must have one essential plan to test.

We don't have a feature through which user can pass enviroment vars using CLI, Implement it.

Additional things to be considered in future:-

1. Look for the .env file in user's code and fetch it so that user don't have to write all the vars again in the CLI

Suggested solution.

File to refer -> https://github.com/metacall/protocol/blob/master/src/language.ts

{
		tag: 'node',
		displayName: 'NodeJS',
		hexColor: '#3c873a',
		fileExtRegex: /^js$/,
		runnerName: 'nodejs',
		runnerFilesRegexes: [/^package\.json$/]
}

Should use displayName instead of runnerName.

The TODO.md has not been updated accordingly, so there's no proper control of what features are already done and which ones are not. Review this and update with tasks done.

We need an additional flag --logout, in order to remove the login cache.

This can help us in several ways:-

1. If the user has multiple accounts, he can switch in between.

2. In testing auth-based flags such as --email, --password, --token -> What happens when we test one of the flags from these, it saves the token and this can pass all the tests written next to them. CLI won't calculate a new token if it's already present. So, we need to remove the cache before running all auth-based tests.

Every time we attempt to deploy, we receive a list of plans; however, it is possible that a user has only one plan, in which case we should automatically deploy without providing a list of plan options.

Any interactive screen like inspect should show at the top or bottom a message like:
Press Ctrl+C for cancel.

So it becomes more intuitive.

Before, the options to choose a plan for a particular application came before the authentication procedure, which was incorrect because the proper order was authentication, followed by a check of the user account's available plans before granting access to deploy.

For installing metacall:protocol you need to install 2 extra package

• @types/ignore-walk
• @types/jsonwebtoken

these 2 packages are already devdepencies of metacall/protocol

• It should ask for a plan in the user (we should store this in the configuration, in a list of projects mapped to plans) or reuse an existent plan if it has been already selected.

• Use the protocol integration test and the descriptor in order to deploy it, we can implement a function which wraps all the behavior into a simple function or implement it directly in the index.ts

\n#################] \ Compressing and deploying... > c.txt
i Deploying F:\Project\metacall\deploy...ploying... > gauge.d.ts <--here

Error: Request failed with status code 500loying... > gauge.d.ts <--here
at createError (F:\Project\metacall\deploy\node_modules\axios\lib\core\createError.js:16:15) <--not here
at settle (F:\Project\metacall\deploy\node_modules\axios\lib\core\settle.js:17:12)
at IncomingMessage.handleStreamEnd (F:\Project\metacall\deploy\node_modules\axios\lib\adapters\http.js:269:11)

gauge.show() only flush stdout before showing progress bar but not after that
last gauge progressbar stays in process.stdout which keep on printing itself untill process.stdout is flushed

Tasks:

• Copy the CI file from here: https://github.com/metacall/protocol/blob/master/.github/workflows/ci.yml and adapt it (keep the unit tests and also the commented integration tests).
• Change the name of the package (either package.json and package-lock.json) to @metacall/deploy.
• Change the dependency of metacall-protocol from the github dependency to the npm dependency which is now @metacall/protocol.
• Update the doc, now it will be @metacall/deploy instead of metacall-deploy. We should review the cli name, how is that name specified now? I have no idea about it, we should investigate.
• Merge the PR and I will review it, and then I will add the npm auth secret and tag it for publishing.

We are using axios package only for AxiosError and this is unnecessary.

We should remove the axios package and export some type like APIError or similar from metacall/protocol wrapping axios error.

When doing deployment, node_modules appears under file selection, is not so intuitive to go up in the list to select the files that are not on node_modules.

I propose we hide folders of this nature or we simply put them at the bottom, leaving the most relevant at the top.

Is your feature request related to a problem? Please describe.
Currently, we deploy applications with environment variables but our "Deploy CLI" doesn't have an env variable caching mechanism.

Describe the solution you'd like
What I think is, we should store a particular application config at the below path:

$  .metacall/deploy/applications/projectName.ini

For Ex:

$  .metacall/deploy/applications/auth-middleware.ini

Now, Why I think it should look like that is because:-

1. We can store all application data in a single folder and it's nicely separated
2. Name of the file is like -> projectName.ini -> All the config of those applications having the same name and located in different paths can be accumulated here, one-stop solution

Let's see config file structure

[path.home_raj_desktop_authMiddleware.plan.essential.env]
ENVIROMENT=dev
PORT=5000
HOST=localhost
USER=Creatoon
MAILCHIMP_API_KEY=fssdgdfgfg565756867993ddcbbcbc

[path.home_raj_desktop_authMiddleware.plan.standard.env]
ENVIROMENT=prod
PORT=9000
HOST=https://creatoon.tech
USER=SuperUser
MAILCHIMP_API_KEY=fssdgdfgfg565756867993ddcbbcbc

[path.home_raj_desktop_authMiddleware.plan.premium.env]
ENVIROMENT=prod
PORT=8000
HOST=https://creatoon.tech
USER=SuperUser
MAILCHIMP_API_KEY=fssdgdfgfg565756867993ddcbbcbc

Why this kind of config structure?
It's simply because there may be a case when a user deploys a minimal version of an application on "Essential" Plan with some environment variables & production-grade version on "Standard and Premium" Plans with different values of env variables, that's why I figured out this structure for caching.

So after parsing it would look something like this:

{
  path: {
    home_raj_desktop_authMiddleware : {
      plan : {
        essential : {
          env: {
            ENVIROMENT: "dev",
            PORT: 5000,
            HOST: "localhost",
            USER: "Creatoon",
            MAILCHIMP_API_KEY: "fssdgdfgfg565756867993ddcbbcbc"
          }
        },
        standard : {
          env: {
            ENVIROMENT: "prod",
            PORT: 9000,
            HOST: "https://creatoon.tech",
            USER: "SuperUser",
            MAILCHIMP_API_KEY: "fssdgdfgfg565756867993ddcbbcbc"
          }
        },
        premium : {
          env: {
            ENVIROMENT: "prod",
            PORT: 8000,
            HOST: "https://creatoon.tech",
            USER: "SuperUser",
            MAILCHIMP_API_KEY: "fssdgdfgfg565756867993ddcbbcbc"
          }
        }
      }
    }
  }
  
  
}

Additional Context
We should also think about, what if the user re-deploys with some additional environment variables then we should give him an option to add more and update ones that are already cached.

@trgwii @viferga What do you think sir?
If you think it is incorrect or whatever you think about it, let me discuss it.

Thanks