mapikit / meta-system Goto Github PK

I have a function that takes a long time to be executed and not being able to change the BOP TTL is a problem to me

Description

Currently, there's a lot of doc-reading until you're at the point of having a valid Addon to work with. This Issue outlines the possibility of having a CLI command for creating an Addon boilerplate with a few parameters, so developers can more readily start developing, only adjusting what is necessary for their use-case.

Solving Suggestion

To solve this issue, It would be interesting to have these features:

• A new command, such as meta-system create-addon
• A wizard with steps to add a name, repo, and required permissions to pass to the Broker.
• Output of the code and required files in the currently running terminal directory.

Bug Description

The CLI flag --skip-prop-validation is documented, but MSYS does not recognize it.

As of 0.1.4 string related functions seem to be lacking. Two quite useful functions would be:

stringConcat

Receives two or more strings and returns one single string from the concatenated input strings. Ex.:

{ 
  strings: { aString: "Hello ", anotherString: "World", lastString: "!" } 
} => "Hello World!"

Why should this be added?: This would avoid having to put the strings into an array and then join the array, which is more time consuming and much less intuitive.

stringTemplate

Receives a string template and replacers. Then, replaces the template with all replacer values with the same name in the template. Ex.:

{ 
  template: "You thought it was ${hero}, but it was me, ${villain}!!", 
  replacers: { hero: "Dio", villain: "Jojo", notUsed: "thisIsExtra" } 
} => "You thought it was Dio, but it was me, Jojo!!"

Why should this be added?: This avoids the need for multiple string constants to be joined later (this simple example would require three constants).

Notes:

• Template terms that don't receive a replacer should be replaced with "" by default
• Extra replacers (that don't appear in the template) should just be ignored

Bug Description

As plain as it it, the -t option does not affect the application, while the --type-check does.

Description

Module alias is a package used in order to not need relative imports in the project. While using it in a final project is its designed use case, it is not recommended for npm packages, such as meta-system.

We should remove this dependency and fix the imports, as well as remove the no-relative-imports lint rule.

Bug Description

Enums are not being validated in the plain object validation function.

Steps to Reproduce

1. Write an ObjDef that contains a property with an Enum,
2. Build an object that does not follow the Enum specifications,
3. Use the validation function - it will succeed.

Description

If we need to have a feature like a chat, or sending a message in an SQS queue, the current protocols configuration for the Meta-system does not suffice our needs.

Let's say that we want to send said SQS queue message. It is better to store connections to be used instead of creating and terminating them for each message. As of version 0.1.4, we cannot do that.

A similar story goes for a a websocket: Currently, as a protocol, I can call a BOp, but a BOp cannot never "call", or use a protocol.

Proposal

If we broaden the protocols interfaces to be able to also provide their own BOps functions or information when installed, such things should no longer be a problem.

For such issues, there are two things we could do:

1. We could just have a BOps function for executing a protocol operation (like listening, or publishing a message), and the protocol, internally, does the needed logic for such feature.
2. The protocol can provide information for existing BOps functions to use.

In the BOps configuration, the first option could look like:

"configuration": [
  {
    "moduleRepo": "=sqs=Send",
    "dependencies": [...]
  }
]

The = would be analogous to the @ for Schemas BOps Functions, but for the protocols.

For the second option, the configuration could take the following formation:

"configuration": [
  {
    "moduleRepo": ":sqsSend",
    "dependencies": [
      { "origin": "protocols", "originPath": "sqs.connection", "targetPath": "connection" }
    ]
  }
]

Both features together would make the best for having a slim, yet capable, protocol interface.

Description

Before the newest version, the documentation had a search bar, but it disappeared after the latest changes. Needless to say, we need it back to make navigating around the docs a little less challenging.

Solving Suggestion

We could either bring back the old search bar (AKA fix it) or we could remove it entirely and add a different one.

Description

For medium to big systems, the configuration file size can become quite the challenge to manage, given that all the system configuration must be contained in a single file.

Solving Suggestion

The user should be able to define BOps and protocols configurations in separate files, since they tend to be the largest part of a system configuration, and are independent from one another.

The file break should be indicated having a file path string present in the place of the object.

{
  "businessOperations": [
    { /*bops1*/ },
    "/path/to/bops2",
    { /*bops3*/ },
   // And so on
  ]
}

A couple of rules should apply:

• The file path can be relative or absolute.
• Failing to find such file means the System will fail to boot.
• As the example above, not every business operation/protocol configuration needs to be split into different files.

Description

Currently, meta-system validates the dependencies to download required external packages. That validation is not working properly, as if we try to use the same module twice, with the same version, which should be okay, the validator throws an error.

Expected Behavior

Using the same version of the package twice in a same BOps should not throw an error.

Description

Meta-system is using the import statement to import JSON files. This makes node throw an ugly warning about asserting imports. 😞

Solving Suggestion

Replace JSON imports with FS operations. If there are imports that are not importing system configuration files, they should be consts instead.

Hi again,

https://mapikit.github.io/meta-system-docs/docs/api-docs/configuring/basics/

All section links are missing

Bug Description

When I try to execute a non existent SchemaFunction, MSYS does not fail to boot, additionally, it does not fail to execute the BOp either.

Steps to Reproduce

1. Create a configuration in which you have a BOp with a non existent schemaFunction.
2. Build the system (it shouldn't be possible)
3. Execute the BOp (it should at least crash with a non existent schemafunction)

Description

Functions are the base of meta-system, for everything a system needs, there is a function to do it; However, any new function requires a new repo, new definitions, and a new name we don't know if it is available on the NPM registry. This problem can be completely avoided if we use a packaging solution for multiple functions.

The package naming is already available in the version branch v0.2, so there is a base to follow and a partial solution implemented.

Also, packaging would benefit creators of functions to maintain their functions in a single repo and npm package.

Changes required

Of course, changing the meta-function-helper means changing the Meta-system as well, so let's go on details of those changes.

• Meta-function-helper will be extended and a new check for packages will be in place. There will be a new schema for defining those as well.
• Meta-System will need to have support for detecting when a package is required and pass only the needed functions of that package to the manager.

Description

Currently there is no way a newcomer can know how to contribute to Meta-System, where to look for bugs, and how the project is structured. We should have at least a .md file for specifying these aspects of the software.

Bug Description

A BOP with more than one equally numbered keys should not be valid, but MSYS stitches the functions.

Steps to Reproduce

1. Create BOP with keys with equal numbers
2. run MSYS
3. It should throw, but doesn't.

Description

Here We will describe all remaining topics to solve about MSYS v0.4 - Andromeda.

Tasks

• Add type verification for Addons Configurations values. Addons themselves already provide their configuration type through Object Definition, but this check is not being made.
• Addons are not receiving the user configuration part of their setup only. Instead, they are receiving the whole addon object.
• We should check if the done() function was called in the "configure" step of an Addon.
• MSYS launch not thrown after fatal Addon configuration. It should return an error code > 0 if running on Node, and promise rejected if running on browser.

More tasks will be added as we see fit.

Bug Description

Running the test-bop cli command exits with an error: TypeError: Cannot read properties of undefined (reading 'input')

Bug Description

Cannot launch on windows for version 0.4.5 - The path for the file is not found. There is an error pointing to a path like C:\C:\<rest of the path>

Probably a problem caused by the fix of #112 .

Description

Debugging is more than just a thing to solve bugs, it helps the person/user to understand better the flow and what happens in each step, how the data is transformed and used throughout the flow. Meta-System, currently, has no such capabilities.

Solving Suggestion

I'd say there are two main ways the current state could be improved, but the perfect solution would be to have both.

1. Have a flag for enabling extensive logging of steps and its data during a BOp execution.
2. Have another flag for emitting multiple events with data about each step of an executed BOp.

Clearly, the number 1 be best suited for running Meta-System in a standalone way, while the Number 1 would be for running it alongside another software, but both share the same usefulness.

Description

Some libraries are made to work together with other libraries, such as Express and Socket.io. Currently, MSYS has a completely isolated workspace for every added Addon, meaning no Addon can directly affect other Addons.

Solving Suggestion

We could create another Broker action that allows for Addons to export values, classes, and functions, and another action that enables their retrieval.

Description

When a BOp throws (a module inside it fails) the Addon does not return a 500 at the request, instead it finishes the request. Since it is the last step before the data exits MSYS, it should be the responsible for treating errors.

Bug Description

While trying to use an Env created in the configuration in a BOp as a module dependency, no value is resolved.

Steps to Reproduce

1. Create an environment variable in the json configuration,
2. Try to use it a BOP - It won't be resolved.

Description

In the new version 0.4 the Addons got introduced, but it did not receive the feature of file-splitting.

Solving Suggestion

Implement file-splitting similar to how the Schemas and BOps are implemented.

This is marked as a bug because it is something actually expected to work, but dumb me forgot to implement it 🤡

Description

Currently MSYS does not execute such check, so in the end we end up giving bad data to the DBProtocols to consume, or the Schemas to bind to their protocols.

Solving Suggestion

We need this verification to happen.

Description

Currently on MSYS, whenever we need to supply an array for a function to consume, we require the Array Push function to concatenate values into an array, which does not scale well when we can have 10 functions in the same BOP requiring arrays... making the process in the end quite bloated.

Solving Suggestion

As we currently have a way of creating objects keys in the "targetProp", we could also have an analogous version for creating arrays and setting an index of it, as the following:

• Object creation:
  "targetProp": "bar.newKey"
• ArrayCreation + index
  "targetProp": "foo.prop[0]" - "prop" is a key of the "foo" object, and the target prop ends up in being the "0" index in the array.

Background: This issue was created to track development for solving #50 ,

Description

Upon an architectural conversation, me and @homemmakako came to an agreement on how we should structure the new protocols capabilities. For v0.2, the protocols should be able to provide functions to be used in BOps, and also, they will be downloadable, just like a function is (and in the future, function packages #53 will be too).

This issue needs two pieces of code to be considered solved:

• Meta-Protocol-Helper (analogous to the meta-function-helper, but for protocols)
• Change to the core meta-system to be able to download and execute the new protocols schemas.

Meta-Protocol-Helper

The protocol helper should be able to do the following functions:

• Provide a CLI to check for a configuration file and the packaged functions. There should be a dependency on the meta-function-helper for the latter.
• Provides typings for meta-system to use internally and standardize the types.

Meta-System Changes

Quite some changes are needed:

• The core Meta-System must be able to download protocols.
• The current HTTP_JSONBODY protocol will be moved to an NPM library.
• Remove local types of Meta-Protocols and use the library's one. This is the heavier change, since it breaks compatibility with earlier configurations.

Description

Currently BOps do not handle throws from modules, simply quitting execution.

Solving Suggestion

BOps should gracefully handle problems within Modules, logging a good error message, and then returning an error to be handled by the caller of the BOp.

Description

Whenever making a request using the protocol, no body is shown and the request is dropped.

Expected Behavior

To not drop the body from the requests.

Description

Currently the docs are only in English. We should provide access to more people around the world to learn and use meta-system in their native language.

When a Bop A has another Bop B as its dependency but Bop B is not declared in the config, MSys just goes into a loop until it eventually breaks with a "Too much recursion" error.

We should check if bop dependencies exist beforehand to make sure this does not happen.

Description

Since V0.4, a lot changed in Meta-System internally, and because of that, many tests ended up being discarded. This lowered the overall regression-safety of PRs.

This is more of an Ongoing issue, so please, if you can improve on any Tests, PRs are welcome.

Missing url https://mapikit.github.io/meta-system-docs/docs/api-docs/getting-started/configuring/basics

Linked from last paragraph here
https://mapikit.github.io/meta-system-docs/docs/api-docs/getting-started/

Bug Description

Meta-system currently does not prevent booting when having multiple versions of the same package in a config.

I believe it should not boot given the way it installs packages via NPM, so a package install overrides another with a different version.

Description

On Meta-System, you are required to specify a MongoDB connection string whenever using schemas. The way this current module is structure makes evident the lack of flexibility of the module. We simply cannot use other databases than MongoDB.

Now, given that for 0.2 the protocols have received a "power up", we can leverage on that same structure and introduce "DBProtocols". They would be protocols but for accessing the Database, and interacting with Schemas.

With this, without any change to the schemas or any other part of the configuration, we should be able to use any different form of data persistence/caching. We could, for instance, have a DBProtocol that stores data in memory, and another that writes into a Redis.

Solving Suggestion

The plan would be to use the same structure built for v0.2 for the protocols. This means the DBProtocols should:

• Be Downloadable through NPM
• Have a verifiable interface
• Just like other pluggable parts of Meta-System, Have a library for helping develop them.

Also this would mean we would have to extract the current drivers for MongoDB from the core into a separate library, which sounds like an architectural improvement to me.