asyncapi / glee Goto Github PK

Reason/Context

To test Glee applications, developers have to either rely on 3rd party tools or write their tests. This could be hugely simplified if Glee shipped with automated testing features.

Description

Developers would write tests within a Glee project in a separate directory. While writing tests, developers should only focus on the message being sent to the server and the response provided by the server or the actions the server does. Everything else would be handled by Glee.

There could be multiple options in the command for specifying the type of test to run. Also, snapshots could be saved for future snapshot testing if specified by the user.

Reason/Context

Currently, Glee only supports creating a WebSocket server. That makes it unusable when someone wants to build a WebSocket client to connect to an existing WS server.

Description

We already have support for WebSocket but it's always assuming we want to create a server, and that's plain wrong. Instead, Glee should default to creating a WS client and only create a server when we mark the server definition in the AsyncAPI file as "local".

For instance:

asyncapi: 2.3.0
servers:
  myWebsiteWebSocketServer:
    url: 'ws://mywebsite.com/ws'
    protocol: ws
    x-kind: local
  someProductWSServer:
    url: 'ws://someproduct.com/ws'
    protocol: ws

The server myWebsiteWebSocketServer is a WS server we want to create with Glee but the server someProductWSServer is a remote server we want to send/receive information to/from. The former is what's already implemented and the latter is what we should be implementing as part of this issue. Notice the x-kind: local on the myWebsiteWebSocketServer server. Only when this is found, we should create a server, otherwise, we should default to a client. Another allowed value for x-kind is remote.

Notes

x-kind is an extension we're creating as part of this issue, this is not defined anywhere yet so we have some freedom defining how we want it to be. I'm trying to make it as similar as possible to the Remotes and local servers issue on the spec. In v3, we may end up having two root keys: servers (local servers) and remotes (remote servers or brokers). For v2.x (the current version), I think it's just fine if we add the x-kind extension but I'm happy to listen for alternatives.

Reason/Context

The current glee.config file supports only custom WebSocket servers, it would be nice if it could also help with authentication and passing in environment variables. This feature would help #319 pass in authentication details and find a better way to pass in authentication variables for the current mqtt adapter.

Description

I will try to propose some ways we can modify the current glee.config.js file.

• glee.conig.js should export an async function that would return an object containing protocol-specific configurations

export default async function () {
  return {
    websocket: {},
    mqtt: {},
    http: {}, // in future when we support http protocol
    amqp: {} 
  }
}

• Since all the protocols have different needs all protocols will have different object structures, which will be documented.
• In future we will support more protocols and the protocol objets will change too.
• Every adapter will have access to it's protocol specific config. How it uses this config is up to the adapter.
• To specify the authentication we will use security schema to define rules for authentication.

Websockets

export default function () {
  return {
    websocket: {
      server: {
        httpServer: customServer,
        adapter: 'native',
        port: process.env.PORT
      },
      client: {
        authentication: {
          bearer: process.env.TOKEN // bearer, api_key, httpApiKey as per the security scheme definition
        }
      }
    }
  }
}

mqtt

export default function () {
  return {
    mqtt: {
      authentication: {
      username: process.env.USERNAME,
      password: process.env.PASSWORD
    }
  }
}

Reason/Context

As the spec already defines all the types and schema regarding the communication, Glee misses out on a lot of benefits that typescript could provide here. After all typescript was also written with a smiling developer in mind 😄.

Description

Porting to typescript will require large changes in the codebase. Each js file will have to be touched and also the execution process has to be altered (Either using ts-node or typescripts tsc). Since no tests exist it is going to be hard to ensure the behavior does not change and no bugs are introduced.

Approach

?

Reason/Context

Currently, Glee only supports WebSocket and MQTT protocols. That makes it unusable for someone wanting to use any other protocol, like HTTP.

Description

Glee should be able to create an HTTP server (API) or a client out of an AsyncAPI definition. We can probably use the same x-kind extension defined in #259, as a way to understand when we want a client and when we want a server.

One key part of this issue is to actually work on the HTTP binding. Especially, in this issue about adding support for multiple methods.

What Dev Docs changes are you proposing?

I would propose to add some specific clarification about the asyncapi.yaml file under Application Structure section on Glee documentation.

Basically, I think It might be useful to clarify the asyncapi.yaml file name must not be changed since you can easily make the same mistake I did by replacing the file name with the name of my service (e.g. "managemet_service.yaml"), making the Glee app not working at all.

Code of Conduct

• I agree to follow this project's Code of Conduct

Reason/Context

When describing the benefits of a spec-driven framework like Glee, I'm always mentioning the fact that we could have docs and code always in sync. However, we're not generating any docs, we're leaving this responsibility to the user, which kinda breaks the purpose 😅

Description

We should provide a way to automatically generate docs. Here are the ways that come to my mind:

1. Start an HTTP static server and serve the docs there. For instance, http://localhost:3000/docs would show the docs. Fully configurable from the glee.config.js file.
2. Generate docs in a folder inside the project. Fully configurable from glee.config.js file, like the path to the folder and the Generator template to use. We can default to docs and the https://github.com/asyncapi/markdown-template/.
3. A Github Action (maybe Gitlab CI too?) that calls a webhook URL (or multiple ones?) to update external systems, like developer portals. E.g., a POST call to a URL containing the AsyncAPI definition. This should be a separate package though.

What Dev Docs changes are you proposing?

I would propose to include more specific information about the required async function under Lifecycle Events section on Glee documentation.

From my point of view, the attributes of the object that the function takes as an argument are just briefly defined but the type of each attribute or how many of them should be used is not specified, and therefore it's not clear enough how to work with them.

Here is where the info I'm missing might be included:

Code of Conduct

• I agree to follow this project's Code of Conduct

It would be awesome to have docs explaining how Glee works. These docs should be mainly targeted at contributors or potential contributors.

Describe the bug

Glee Functions should either return undefined or a well-known object containing either send or reply keys. If I return something different, say an Array —for instance—, Glee doesn't complain in any way and therefore it's hard to debug what's wrong in your code.

How to Reproduce

Go change the returned value here to an array and see nothing happens. The code will not work but it also doesn't complain.

Expected behavior

It should either fail or log a warning telling us the returned structure is not expected and that we should return either undefined or the well-known structure.

Reason/Context

Currently, Glee only supports WebSockets (server) and MQTT protocols. This means it's unusable for those using Kafka, AMQP, or any other different protocol.

Description

We should add support for AMQP 1.0. In theory, it should be just a matter of adding a new adapter in the adapters folder. Beware AMQP 1.0 and AMQP 0-9-1 are totally different protocols, despite having the same name.

#272 had the side effect of specifying server in send messages mandatory.
I mean this worked with previous versions:

{
  send: [{
    payload: 'my message'
  }]
}

but now it gives:

x Cannot read property 'protocol' of null
    at file:///Users/khudadad/Projects/asyncapi/glee/dist/lib/functions.js:107:73
    at Array.forEach (<anonymous>)
    at file:///Users/khudadad/Projects/asyncapi/glee/dist/lib/functions.js:105:111
    at Generator.next (<anonymous>)
    at fulfilled (file:///Users/khudadad/Projects/asyncapi/glee/dist/lib/functions.js:4:58)

and I have to specify the server even if there is only one server:

{
   send: [{
      server: 'websockets',
      payload: 'my message'
  }]
}

If this was intentional, we need to update the docs and show better error messages.
I think we need to implement some kind of fallback to the default server if there is only one server. 🤔

cc: @Souvikns

Reason/Context

the _connect() method in ws server adapter currently has a cognitive complexity of 35. It's really hard to understand what is going on. plus, SONAR always complains about it in PRs 😆

Description

refactor the _connect() method by grouping it's logic in separate functions for the purpose of readability and maintainability.

Reason/Context

We don't need glee init anymore since we now have create-glee-app.

Description

I think it would be a good idea to remove the glee init script.

Reason/Context

So according to https://github.com/asyncapi/glee/blob/master/docs/lifecycle-events.md we can create lifecycle events and bind them to an event, Now we do have examples where we create loops to stream a series of data, How do we stop this loop say to react to any other event since I can only subscribe to only one event I have no way terminate the loop say any other event occurs or my client connection is interrupted.

Description

Let me take a particular example to showcase the problem.

Suppose I have this Lifecycle function, which is subscribed to the event onServerConnectionOpen, which means this function starts to run when a successful connection is established, But when the connection is terminated before the loop ends the loop keeps on running, and there is no way to stop the loop.

import dummyjson from 'dummy-json'
import { Message } from '@asyncapi/glee'

export default async function ({ glee, connection }) {
  (function myLoop(i) {
    setTimeout(() => {
      glee.send(new Message({
        channel: '/travel/status',
        connection,
        payload: generateResponse()
      }))
      if (--i) myLoop(i)
    }, 1000)
  }(100))

  function generateResponse() {
    const template = `{
      "destination": "{{city}}",
      "arrival": "{{int 2 6}}h",
      "distance": "{{int 18 65}}km"
    }`
    return JSON.parse(dummyjson.parse(template))
  }
}

export const lifecycleEvent = 'onServerConnectionOpen'
export const channels = ['/travel/status']

Operating system: Windows 11
Node: 14

Describe the bug

When I try to run a project generated by glee using npm run dev command it shows this error:

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:791:11)
    at Loader.resolve (internal/modules/esm/loader.js:89:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:242:28)
    at Loader.import (internal/modules/esm/loader.js:177:28)
    at importModuleDynamically (internal/modules/esm/translators.js:114:35)
    at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:30:14)
    at file:///C:/Users/.../glee/src/lib/functions.js:27:35
    at Array.map (<anonymous>)
    at register (file:///C:/Users/.../node_modules/@asyncapi/glee/src/lib/functions.js:17:49)
    at processTicksAndRejections (internal/process/task_queues.js:95:5) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}
⚡ Server websockets is ready to accept connections.

After trying to hit the endpoint with websocat I get this error:

↙ hello was received from websockets
'Hello\r\n'
x functions[operationId].run is not a function
    at file:///C:/Users/.../node_modules/@asyncapi/glee/src/lib/runtimes/js.js:14:8
    at new Promise (<anonymous>)
    at runJS (file:///C:/Users/.../bot-the-savior/node_modules/@asyncapi/glee/src/lib/runtimes/js.js:12:10)
    at triggerFunction (file:///C:/Users/.../node_modules/@asyncapi/glee/src/lib/runtimes/index.js:26:17)
    at Function.<anonymous> (file:///C:/Users/.../node_modules/@asyncapi/glee/src/index.js:70:11)
    at file:///C:/Users/.../node_modules/@asyncapi/glee/src/lib/glee.js:155:17   
    at C:\Users\...\node_modules\async\dist\async.js:1987:20
    at C:\Users\...\node_modules\async\dist\async.js:1928:13
    at replenish (C:\Users\...\node_modules\async\dist\async.js:446:21)
    at C:\Users\...\node_modules\async\dist\async.js:451:13

How to Reproduce

1. run npx create-glee-app
2. go to the app folder.
3. run npm run dev

Expected behaviour

It shouldn't show any error and run as demonstrated.

cc: @fmvilas @jonaslagoni

Glee supports message handlers (which are invoked upon receiving a message) as well as the sending of messages (typically from within message handlers).

• When sending a message, a Glee app makes use of OutboundMessages, which can be used to send, reply, and/or broadcast a message (https://github.com/asyncapi/glee/blob/master/docs/functions.md).
• Also, OutboundMessage contains the channel property, which specifies the channel the message is sent to.
• Lastly, the message interactions performed by the Glee app are specified by an AsyncAPI document.

All the above options (send, reply, broadcast and channel) specify various aspects of sending a message, which must however always be in accordance with the AsyncAPI document of the Glee application. There is significant overlap between the information contained in these options. In other words, the current API o send messages is not DRY.

Ideally, given an AsyncAPI document, a Glee app should just need one operation to send a message to a channel. Specifically:

• The differences between send and broadcast should not matter to the Glee app.
• A reply is just a send to a reply channel specified by the sender of the incoming message, optionally with the message ID of the incoming message included in the reply message. All of that information should already be in the AsyncAPI document.

In general, every aspect of the messages sent by the Glee app should be specified in the AsyncAPI document - if at all possible - and not be repeated in Glee's message sending API. Only information that cannot be expressed in AsyncAPI should be part of Glee's message sending API (or be included in future versions of the AsyncAPI spec ;) ).

See https://asyncapi.slack.com/archives/CQVJXFNQL/p1637667052282600 for the origins of this request.

Details: https://sonarcloud.io/project/issues?id=asyncapi_glee&issues=AXqaiNMrP8p3iozZcvlP&open=AXqaiNMrP8p3iozZcvlP

Code: src/lib/glee.js

Explanation: Variables should not be shadowed. Overriding or shadowing a variable declared in an outer scope can strongly impact the readability, and therefore the maintainability, of a piece of code. Further, it could lead maintainers to introduce bugs because they think they’re using one variable but are really using another.

Details: https://sonarcloud.io/project/issues?id=asyncapi_glee&issues=AXqhLDqqAeRbydshud2H&open=AXqhLDqqAeRbydshud2H

Code: src/lib/adapter.js

Explanation: Variables should not be shadowed. Overriding or shadowing a variable declared in an outer scope can strongly impact the readability, and therefore the maintainability, of a piece of code. Further, it could lead maintainers to introduce bugs because they think they’re using one variable but are really using another.

Describe the bug

The dev logs show the "broadcasted" word when sending to a broker. That's incorrect and can lead to confusion.

How to Reproduce

Expected behavior

It should show the word "sent". For instance, "post/liked was sent to mosquitto".

However, when we're sending to a WebSockets server, it should say "was broadcasted" unless it was a reply to a specific WebSockets client.

Hints

This is the middleware in charge of logging these things: https://github.com/asyncapi/glee/blob/master/src/middlewares/logger.ts.

In #146 I introduced the first test and if it gets approved I plan to add them whenever I implement a new feature and also add them for existing code.
If the PR with the test gets merged it would be nice to include test in the CI.

Learn more about this issue here

Reason/Context

Currently, Glee only supports WebSockets (server) and MQTT protocols. This means it's unusable for those using Kafka, AMQP, or any other different protocol.

Description

We should add support for AMQP 0-9-1. In theory, it should be just a matter of adding a new adapter in the adapters folder.

Reason/Context

We're currently hosting the functions within the same repository but it could be useful to have them as serverless functions.

Description

Implement an HTTP runtime that will make an HTTP call to a FaaS (Function as a Service) provider and waits for the HTTP response.

The HTTP response may vary from one provider to another. From the top of my head, we should support AWS Lambda, Netlify functions, Vercel, Azure Functions, GCP Cloud Functions.

Reason/Context

When I want to add something to the config file currently I have to take the following steps:

1. update the lib/index.d.ts and add my config types.
2. update the docs/config-file.md to reflect the new configs that I have added/changed.
3. If no config is provided in my config file I have to hardcode the default values in adapters themselves.

When working with the codebase, it's a little hard to go back and forth between these files and make sure the are synced and the parsing logic is correct.

Description

IMO we can make some improvements to the codebase and make it easier for the new contributors to understand the codebase by having a config class that is responsible for all three of those above.

1. The class should be documented correctly with jsdoc so we can generate the doc/config-file.md from it instead of maintaining it separately.

2. all of the parsing logic should be handled by this class itself. I shouldn't have to look around for the port of my server in three different places and finally hardcode the default value in each Adapter. 😄

3. Providing default values for each configuration.

Describe the bug you found in AsyncAPI Docs.

Functions documentation shows that we can use the broadcast key in the returned object but that's not the case anymore.

Attach any resources that can help us understand the issue.

Scroll down to the table in the Functions docs where there are three options listed: send, reply, broadcast. We should get rid of broadcast.

Code of Conduct

• I agree to follow this project's Code of Conduct

Reason/Context

Currently, Glee would not behave great when spanning multiple instances. The reason is the broadcast option. As it is right now, we're storing connections in memory, so when a function responds saying it wants to broadcast a message, it goes through all the connections and sends the message. Obviously, if we have multiple instances, the message will be "broadcasted" only to a subset of clients (the ones that a given instance has in memory).

Relevant parts of the code are:

• https://github.com/asyncapi/glee/blob/master/src/lib/adapter.js#L23
• https://github.com/asyncapi/glee/blob/master/src/lib/adapter.js#L69
• https://github.com/asyncapi/glee/blob/master/src/lib/adapter.js#L82
• https://github.com/asyncapi/glee/blob/master/src/lib/adapter.js#L36

Description

We should allow for clusterization of the multiple instances. There are multiple approaches for this but I think the simplest one would be to keep things in memory and let the user choose a mechanism to communicate the different instances, e.g., Redis.

It would be something like:

1. A message with the broadcast option enabled is requested to be sent.
2. The adapter sends a message to Redis saying it wants to broadcast a message.
3. All the instances receive the message and proceed to send it.

The advantage of this approach is that it's simple but it doesn't guarantee all the messages will be sent. For instance, what happens if one of the instances doesn't receive the Redis message? That's something to think about. There are many existing techniques, we don't need to reinvent the wheel.

Reason/Context

Some people are using the JSON format of AsyncAPI or they name their files as asyncapi.yml (notice the .yml instead of .yaml).

Description

It would be awesome if Glee could auto-detect that you're using one of those files. It could also complain when you use many of them at the same time. For instance, if it finds asyncapi.yaml and asyncapi.json.

Reason/Context

Currently, Glee only supports WebSockets (server) and MQTT protocols. This means it's unusable for those using Kafka, AMQP, or any other different protocol.

Description

We should add support for Kafka. In theory, it should be just a matter of adding a new adapter in the adapters folder. However, we should also take into account that Kafka supports exactly-once semantics (unlike MQTT and WebSockets) and, therefore, it may be a good idea to address #27 first, although not required.

References

• https://www.baeldung.com/kafka-exactly-once
• https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/

Reason/Context

One of the challenges I face when using Glee is the uncertainty of the values in message payloads. Seeing the "any" type in Typescript can also be a bit intimidating. 😆

However, AsyncAPI provides a helpful feature that allows us to describe the message payload format using JSON Schema. With the use of https://github.com/asyncapi/modelina, we can conveniently convert these schemas into Typescript Types.

Where to start?

Step 1

Use Modelina and generate the types from the AsyncAPI file. Categorize the message payload type by the operationId. for example we can have types/operations/hello.d.ts for hello operation in .glee folder.
It would be great for this file to contain the whole function type so it can be easily used in hello.ts function.

Step 2

When we generate a Glee project for an AsyncAPI file, Glee should create empty blueprints for all of the operations and assign the correct types to them. this way, user can start with the blueprint and get all of the type safety and suggestions for their message payloads.

@jonaslagoni I don't have that much experience with Modelina and not sure if it's feasible. WDYT? 🤔

Describe the bug

When we receive or try to send a message that doesn't match the definition in the AsyncAPI file, Glee breaks without showing the error and instead shows an UnhandledPromiseRejectionWarning: TypeError error.

How to Reproduce

Go to social-network example and change postId to be a string instead of a number. Either do it in the code or in the AsyncAPI file but not both. We want the code to do something different than what AsyncAPI file says, just so it fails.

This error appears:

Expected behavior

It should show a nice error saying that the message we're receiving or trying to send is malformed. In essence, the errorLogger middleware should come into play.

Hints

It looks like the variable emws is not an array when it should.

Describe the bug

I noticed some inconsitencies in the formatting of the js files. While this is not critical at all, it is annoying for me as I cant configure my environment to use formatting on save as this always produces changes. I am no expert with the .eslintrc files. Is there another asnycapi project with an up-to-date version that can be reused here?
I think the linter should enforce behavior close to many standart formatters such that anyone can use their default.

1. Empty lines sometimes contain spaces and sometimes they dont (Should always be empty)
2. Between function names and the signature in braces there is sometimes a space and sometimes there is none (Should never be one)
3. Sometimes there are two instead of one space or spaces at the end of lines (Should not allow unneeded spaces)

All can be found in lib/adapter.js

1. glee/src/lib/adapter.js

   Line 20 in ae20715

2. glee/src/lib/adapter.js

   Line 14 in ae20715

   constructor (glee, serverName, server, parsedAsyncAPI) {

3. glee/src/lib/adapter.js

   Line 33 in ae20715

   this.on('message', (message, connection) => {

From slack thread

I've been looking for a tool to dynamically/programatically consume OpenAPI specs to generate javascript/typescript clients, but most tools seem to be CLIs that output the full server and/or client code after running a command line operation.

For illustration purposes, I'm thinking about an API something like the following:

const asyncApiDoc = {...}
const { client, server } = openapi.consume(asyncApiDoc)
client.receive((data) => console.log(data))
client.receive('/path/here', (data) => console.log(data))
client.send('/path/here', {...})

The required protocols can be easily parsed from the AsyncAPI document and clients/servers instantiated at runtime (assuming there is a valid JS/TS implementation for them).

Ideally, the API will abstract protocols away and allow communicating (send/receive) with the API via a single API.

It is more of an SDK rather than a CLI. Protocol bindings should ideally be external to the core implementation and thus be progressively added (e.g. for WebSockets, MQTT, HTTP, Kafka, ...).

Reason/Context

I think examples are one of the useful resources that we currently have regarding the glee project. they have helped me and I am sure they are useful to others as well.
Currently, we store the example projects that have been developed with glee under the examples folder which is not great from a maintainability point of view. besides, it locks that example in the glee repo and makes it harder to make changes to the examples.

I think there are two options that we have when it comes to handling examples.

1. Using Submodules

Each one of the contributors can create a repo for their examples and we can link them here using git submodules in the examples folder.

2. Linking them in the project's README.

creating repos for each example and linking them in the main README file. most of the other projects are doing this and I think it is the most sensible way of handling examples.

cc: @fmvilas @Souvikns

What Dev Docs changes are you proposing?

I would propose to include some information about subscribe operations under Functions section on Glee documentation.

From my point of view it's pretty clear how Glee interacts with publish operations. However, at the time of building a Glee project I've been wondering what happens then with subscribe operations and their corresponding function files, and some questions came to my mind:

• Should I include the function files from subscribe operations in the Glee functions folder as well? Or should I include only the ones from publish operations?
• Are the functions related to subscribe operations not necessary for Glee as they are not mentioned at all?
• Why some Glee examples include the operationId attribute for subscribe operations? Is it necessary at some point or just ignored by Glee? I found it a bit confusing 🤔

This is the specific part of the document where I think that info about the functions from subscribe operations might be included:

Code of Conduct

• I agree to follow this project's Code of Conduct

Learn more about this issue here

details: https://sonarcloud.io/project/issues?id=asyncapi_glee&issues=AXqaiNN0P8p3iozZcvlZ&open=AXqaiNN0P8p3iozZcvlZ

code: src/index.js

Reason/Context

In #230, I added an initial set of tests. They're enough for now but would be great to get better coverage.

Description

Add more tests. Look at test coverage results and improve them too.

Notes

Regarding good first issue label, I think it's a good one if you're learning how to test your code. However, if you've never done testing before it can be challenging. Anyhow, I'm always happy to guide you if needed.

Operating system: macOS Monterey 12.6
Node: 16

Describe the bug

When I try to run a project generated by glee using npm run dev command it shows this error:

 Glee 0.10.26 

{} Running in development mode...
↙↗ Selected server(s): websockets
────────────────────────────────────────
⚡ Server websockets is ready to accept connections.
x A client attempted to connect to channel /_next/webpack-hmr but this channel is not defined in your AsyncAPI file.
    at Server.<anonymous> (file:///Users/khudadad/test/node_modules/@asyncapi/glee/dist/adapters/ws/index.js:65:33)
    at Server.emit (node:events:513:28)
    at onParserExecuteCommon (node:_http_server:772:14)
    at onParserExecute (node:_http_server:686:3)
node:events:491
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -54,
  code: 'ECONNRESET',
  syscall: 'read'
}

How to Reproduce

1. run npx create-glee-app app-name
2. run cd app-name
3. run npm run dev

Expected behaviour

It shouldn't show any error and run as demonstrated.

Describe the bug

The example at examples/social-network is failing.

How to Reproduce

Try going to this folder, and running npm run dev on any of the Glee apps.

It fails because the lowdb package is an ES Module (those using import instead of require) and we're compiling all the Typescript code (except node_modules) to CommonJS modules (those using require instead of import). Therefore, it doesn't expect an import keyword and fails.

Expected behavior

It should support ES Modules out of the box. There will be more and more packages migrating to ES Modules over time as it's the new standard. ES modules can import CommonJS modules but CommonJS modules can't require ES Modules.

It should be easy to do. Initially, Glee supported them very well but after the Typescript migration (#212 and #213) it got broken. Changing all the code to ES Modules (and adding back .js extension to all imports) should be fine except for Jest mock of fs/promises in the test/lib/functions.test.ts file, which doesn't work yet with ESM.

I think a good and quick solution would be to just get rid of this test for now until Jest supports ESM. Check the progress (or even better, contribute) at jestjs/jest#10976.

Reason/Context

There are many points in the existing codebase that can lead to lost messages.

Description

We should review and identify which parts of the codebase can cause problems and fix them. We should be looking for:

1. Retries. What if calling a function fails? Especially when we implement the HTTP runtime. We should be retrying and letting the user configure how to do it.
2. Backpressure. What happens if a client/broker starts sending too many messages too quickly? We should be able to control the number of messages per second we want to receive, telling the client/broker to slow down. This is not always possible but definitely should be implemented for those protocols that provide a mechanism for that.
3. At-most-once, at-least-once, and exactly-once semantics. We should be able to guarantee that a message is going to be received either at most once, at least once, or exactly once. Some protocols don't have a mechanism for that so we should focus on those that have it.

Reason/Context

It would be great to know exactly what project type, glee is executing. That way it knows what runtime to run and what extra functionality it needs to perform.

Description

I suggest we add something similar to what the generator has for nunjucks and react called renderer, here maybe type makes sense? 🤔

As a side-effect, I suggest we remove the experimental flag for using the java runtime as we then explicitly tell it to use the new configuration.

Describe the bug

I'm not able to to connect to an MQTT server (located in a docker container) when trying to launch my glee app.

How to Reproduce

• Execute npm run dev in a glee project containing the asyncapi.yaml and .env files shared here.

• URLs tested in servers/development/url at asyncapi file: 0.0.0.0:1883, 127.0.0.0:1883, localhost:1883, mqtt://test.mosquitto.org:1883

• Output displayed on the terminal:

Expected behavior

The connection with the MQTT server should be established with no errors displayed on the terminal.

Learn more about this issue here

I have been trying to connect nextjs and glee together and currently having some issues, I am opening this issue to talk about my approach and the problems that I am currently facing and to discuss new ideas.

The Idea

Since nextjs already allows us to have custom servers as shown here, we should be able to use glee as a custom server. With this, we can have /api calls from nextjs handled by glee and nextjs handling react frontend.

Approach

Glee supports glee.config.js to pass in extra configurations that control the behavior of glee (more about it here), this is what I was trying to use to pass in react server object so that the glee can use that server to listen on the API calls.

import next from 'next';
import { parse } from 'url';
import { createServer } from 'http';

const dev = process.env.NODE_ENV !== 'production';

const app = next({ dev });

export default async () => {
    await app.prepare();
    const server = createServer(async (req,res) => {
        try {
           const parseUrl = parse(req.url, true);
           const {pathname, query} = parseUrl;
           if (pathname !== '/api') {
            console.log('Inside react')
            await app.render(req, res, '/', query);
           }
        } catch (error) {
            console.log(error);
        }
    })
    console.log(server);
    return {
        websocket: {
            httpServer: server,
            adapter: 'native',
            port: '3000'
        }
    }
}

So in theory all the /api calls will be handled by glee functions and the non /api calls will be handled by nextjs.

The Problem

Both glee and next creates build folders and that is not getting created and that is the issue I am facing. glee starts creating its build folder and then creates a tsconfig.tsbuildinfo and next does not even start the build process.

https://github.com/Souvikns/glee-next-example - if you want to see the whole code.

I am still working on this and will update is issue if something comes up.

Describe the bug you found in AsyncAPI Docs.

I'd like to report a bug I found in a code snippet under Lifecycle Events on Glee documentation.

Bug description

In case we want to restrict the lifecycle event by channel/topic, the variable name we are suggested to use should not be servers as we can see in the code snippet at the bottom.

Expected output

We should use the variable name channels instead, so the line to be added would be as follows:
export const channels = ['user/signedup']

Attach any resources that can help us understand the issue.

Code of Conduct

• I agree to follow this project's Code of Conduct

Reason/Context

Currently, glee only accepts glee.config.js file type for config, for IntelliSense purposes it should also accept glee.config.ts. It should give typescript file preference over javascript file.

Currently, it is hardcoded, we basically have to check which file type is present in the working directory and then load the respective file.

Describe the bug

Currently, this framework has a direct dependency on node-ipc@^10.0.1. Considering the recent developments of the node-ipc maintainer publishing malicious versions, are there any plans to switch to a fork?

Currently, the published versions now only have a harmless peacenotwar protestware dependency.

Depending on the threat model, pinning to a known-good version of node-ipc may not be suffice as there are nested, transitive dependencies which are maintained by the same person. Hence, it may be possible for the same maintainer to publish a malicious version of easy-stack, vanilla-test, ansi-colors-es6 or strong-type.

How to Reproduce

Installing the latest version of glee installs transitive dependencies managed by the @/riaevangelist.

Expected behavior

Switch to a clean fork of node-ipc.

Some available forks that I'm aware of:

• @achrinza/node-ipc - My own fork
• @node-ipc/node-ipc - Another fork with new features

Currently, as a user when we are using glee to build a WebSocket server the only way of authenticating is by writing your own authentication logic in the functions. Which is repetitive and not a good solution when you have a large application, with the addition of WebSocket and HTTP adapters, glee needs support for authentication and authorization. Right now glee needs some kind of Authentication middleware for both server and client adapters.

Currently, client adapters are able to send authentication parameters with the help of glee config, and this needs to be updated so that it can support multiple authentication and authorization process.

For starters, we need to support some popular authorization processes-

• Token-based authentication: In this method, the client requests an access token from the server, which can then be used to access protected resources. The token is sent with each websocket request and is used by the server to verify the client's identity and determine their authorization.
• Basic authentication: This method uses the same basic authentication process as the HTTP protocol, where the client sends an HTTP request with a header that contains a username and password. The server verifies the credentials and returns a response with the requested resource if the user is authenticated.
• Certificate-based authentication: In this method, the client and server use digital certificates to establish trust and verify identity. The client presents a certificate to the server, which then verifies the certificate's validity and authenticity.
• Session-based authentication: This method works similarly to session-based authentication in the HTTP protocol, where the server creates a session for the user and associates it with an identifier. The client then sends the identifier with each websocket request, allowing the server to identify the user and determine their authorization.
• OAuth 2.0 authentication: OAuth 2.0 is an open standard for authorization that provides secure access to resources by delegating access to third-party applications. The client requests access to a resource from the server, and the server returns an authorization code, which the client can then use to obtain an access token.

I've recently been thinking a lot about the scope of Glee and realized that having support for Java and other languages might be a bit overkilling at this stage. I propose we get rid of this feature now and consider it for the future. We don't lose anything, the code will still be there in the git history.

Instead, I think by moving to TypeScript (#144) we can simply encourage Java users to use TypeScript instead.

I'd still leave the "runtimes" system to handle remote code execution (lambdas, FaaS) (#25)

What do you think?