Git Product home page Git Product logo

inngest / inngest-js Goto Github PK

View Code? Open in Web Editor NEW
385.0 3.0 38.0 4 MB

The developer platform for easily building reliable workflows with zero infrastructure for TypeScript & JavaScript

Home Page: https://www.inngest.com/

License: GNU General Public License v3.0

JavaScript 1.63% TypeScript 98.24% Nix 0.13% Shell 0.01%
event-driven message-queue nodejs typescript job-queue job-scheduler serverless background queues workers

inngest-js's Introduction

Inngest

Latest release Test Status Discord Twitter Follow

Inngest is a developer platform that combines event streams, queues, and durable execution into a single reliability layer.


Build and ship durable functions and workflows in your current codebase without any additional infrastructure. Using Inngest, your entire team can ship reliable products.

  • Write durable functions in your existing codebase using an Inngest SDK
  • Run the open source Inngest Dev Server for a complete local development experience, with production parity.
  • The Inngest Platform invokes your code wherever you host it, via HTTPS. Deploy to your existing setup, and deliver products faster without managing infrastructure.

SDKs: TypeScript/JavaScriptPythonGo



The Inngest Dev Server

npx inngest-cli@latest dev

Inngest Dev Server screenshot


Overview

Inngest makes it easy to develop durable functions and workflows in your existing codebase, without any new infrastructure. Inngest Functions are triggered via events — decoupling your code within your application.

  1. You define your Inngest functions using the Inngest SDK and serve them through a simple API endpoint.
  2. Inngest automatically invokes your functions via HTTPS whenever you send events from your application.

Inngest abstracts the complex parts of building a robust, reliable, and scalable architecture away from you, so you can focus on building applications for your users.

  • Run your code anywhere - We call you via HTTPS so you can deploy your code to serverless, servers or the edge.
  • Zero-infrastructure required - No queues or workers to configure or manage — just write code and Inngest does the rest.
  • Build complex workflows with simple primitives - Our SDKs provides easy to learn step tools like run, sleep, sleepUntil, and waitForEvent that you can combine using code and patterns that you're used to create complex and robust workflows.

Read more about our vision and why Inngest exists


SDKs

Getting started

👉 Follow the full quick start guide here

A brief example

Here is an example of an Inngest function that sends a welcome email when a user signs up to an application. The function sleeps for 4 days and sends a second product tips email:

import { Inngest } from 'inngest';

const inngest = new Inngest({ id: 'my-app' });

// This function will be invoked by Inngest via HTTP any time
// the "app/user.signup" event is sent to to Inngest
export default inngest.createFunction(
  { id: 'user-onboarding-emails' },
  { event: 'app/user.signup' },
  async ({ event, step }) => {
    await step.run('send-welcome-email', async () => {
      await sendEmail({ email: event.data.email, template: 'welcome' });
    });

    await step.sleep('delay-follow-up-email', '7 days');

    await step.run('send-tips-email', async () => {
      await sendEmail({ email: event.data.email, template: 'product-tips' });
    });
  }
);

// Elsewhere in your code (e.g. in your sign up handler):
await inngest.send({
  name: 'app/user.signup',
  data: {
    email: '[email protected]',
  },
});

Some things to highlight about the above code:

  • Code within each step.run is automatically retried on error.
  • Each step.run is individually executed via HTTPS ensuring errors do not result in lost work from previous steps.
  • State from previous steps is memoized so code within steps is not re-executed on retries.
  • Functions can sleep for hours, days, or months. Inngest stops execution and continues at the exactly the right time.
  • Events can trigger one or more functions via fan-out

Learn more about writing Inngest functions in our documentation.


Project Architecture

Fundamentally, there are two core pieces to Inngest: events and functions. Functions have several subcomponents for managing complex functionality (eg. steps, edges, triggers), but high level an event triggers a function, much like you schedule a job via an RPC call to a queue. Except, in Inngest, functions are declarative. They specify which events they react to, their schedules and delays, and the steps in their sequence.


Open Source Architecture

Inngest’s architecture is made up of 6 core components:

  • Event API receives events from clients through a simple POST request, pushing them to the message queue.
  • Event Stream acts as a buffer between the API and the Runner, buffering incoming messages to ensure QoS before passing messages to be executed.
  • A Runner coordinates the execution of functions and a specific run’s State. When a new function execution is required, this schedules running the function’s steps from the trigger via the executor. Upon each step’s completion, this schedules execution of subsequent steps via iterating through the function’s Edges.
  • Executor manages executing the individual steps of a function, via drivers for each step’s runtime. It loads the specific code to execute via the DataStore. It also interfaces over the State store to save action data as each finishes or fails.
    • Drivers run the specific action code for a step, e.g. within Docker or WASM. This allows us to support a variety of runtimes.
  • State stores data about events and given function runs, including the outputs and errors of individual actions, and what’s enqueued for the future.
  • DataStore stores persisted system data including Functions and Actions version metadata.
  • Core API is the main interface for writing to the DataStore. The CLI uses this to deploy new functions and manage other key resources.

And, in this CLI:

  • The DevServer combines all the components and basic drivers for each into a single system which reads all functions from your application running on your machine, handles incoming events via the API and executes functions, all returning a readable output to the developer.

For specific information on how the DevServer works and how it compares to production read this doc.


Community

Contributing

We’re excited to embrace the community! We’re happy for any and all contributions, whether they’re feature requests, ideas, bug reports, or PRs. While we’re open source, we don’t have expectations that people do our work for us — so any contributions are indeed very much appreciated. Feel free to hack on anything and submit a PR.

Check out our contributing guide to get started.

inngest-js's People

Contributors

anasabdullahysfzai avatar brunoscheufler avatar callingmedic911 avatar cohlar avatar darwin67 avatar dependabot[bot] avatar djfarrelly avatar fritzblue avatar goodoldneon avatar inngest-release-bot avatar jacobheric avatar jessethomson avatar jpwilliams avatar khill-fbmc avatar maktouch avatar michealroberts avatar mmachatschek avatar monsterdeveloper avatar prettyirrelevant avatar renovate[bot] avatar stefanosala avatar swiecki avatar sylwiavargas avatar tonyhb avatar wtachau avatar yassilah avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

inngest-js's Issues

Allow users to create custom retry policies when defining individual steps

We need to return the retry policy on error so that the executor can update and infer the retry policy when scheduling the next attempt.

A question: what if there's an irrecoverable error (eg. segfault)? How do we know the policy at this point?

We may have to change the SDK to reply with an "intent" to run a step only if options are provided, along with additional step config (eg. retries) which are then applied to any attempt

[BUG] Could not find an event key to send events

Hello,

I opened my cmd and called this command: npx inngest-cli@latest dev

then I installed in my remix folder npm i inngest --save

if I start my server then I got everytime and every 5 or 10 seconds this:

Could not find an event key to send events; sending will throw unless an event key is added. Please pass one to the constructor, set the INNGEST_EVENT_KEY environment variable, or use inngest.setEventKey() at runtime.
No routes matched location "/x/inngest" 
Warning: A title element received an array with more than 1 element as children. In browsers title Elements can only have Text Nodes as children. If the 
children being rendered output more than a single text node in aggregate the browser will display markup and comments as text in the title and hydration 
will likely fail and fall back to client rendering
    at title
    at head
    at html
    at CatchBoundary (C:\sp\folder_bk\es\es\build\index.js:411:43)
    at RemixCatchBoundary (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\errorBoundaries.js:140:10)
    at RemixRouteError (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\components.js:84:3)
    at RenderErrorBoundary (C:\sp\folder_bk\es\es\node_modules\react-router\dist\umd\react-router.development.js:624:7)
    at Routes (C:\sp\folder_bk\es\es\node_modules\react-router\dist\umd\react-router.development.js:1174:7)
    at Router (C:\sp\folder_bk\es\es\node_modules\react-router\dist\umd\react-router.development.js:1106:17)
    at StaticRouterProvider (C:\sp\folder_bk\es\es\node_modules\react-router-dom\server.js:67:3)
    at RemixErrorBoundary (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\errorBoundaries.js:24:5)
    at RemixServer (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\server.js:47:3)
GET /x/inngest 404 - - 49.805 ms
Could not find an event key to send events; sending will throw unless an event key is added. Please pass one to the constructor, set the INNGEST_EVENT_KEY environment variable, or use inngest.setEventKey() at runtime.
No routes matched location "/.netlify/functions/inngest" 
Warning: A title element received an array with more than 1 element as children. In browsers title Elements can only have Text Nodes as children. If the 
children being rendered output more than a single text node in aggregate the browser will display markup and comments as text in the title and hydration 
will likely fail and fall back to client rendering
    at title
    at head
    at html
    at CatchBoundary (C:\sp\folder_bk\es\es\build\index.js:411:43)
    at RemixCatchBoundary (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\errorBoundaries.js:140:10)
    at RemixRouteError (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\components.js:84:3)
    at RenderErrorBoundary (C:\sp\folder_bk\es\es\node_modules\react-router\dist\umd\react-router.development.js:624:7)
    at Routes (C:\sp\folder_bk\es\es\node_modules\react-router\dist\umd\react-router.development.js:1174:7)
    at Router (C:\sp\folder_bk\es\es\node_modules\react-router\dist\umd\react-router.development.js:1106:17)
    at StaticRouterProvider (C:\sp\folder_bk\es\es\node_modules\react-router-dom\server.js:67:3)
    at RemixErrorBoundary (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\errorBoundaries.js:24:5)
    at RemixServer (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\server.js:47:3)
GET /.netlify/functions/inngest 404 - - 51.139 ms
Could not find an event key to send events; sending will throw unless an event key is added. Please pass one to the constructor, set the INNGEST_EVENT_KEY environment variable, or use inngest.setEventKey() at runtime.
No routes matched location "/.redwood/functions/inngest" 
Warning: A title element received an array with more than 1 element as children. In browsers title Elements can only have Text Nodes as children. If the 
children being rendered output more than a single text node in aggregate the browser will display markup and comments as text in the title and hydration 
will likely fail and fall back to client rendering
    at title
    at head
    at html
    at CatchBoundary (C:\sp\folder_bk\es\es\build\index.js:411:43)
    at RemixCatchBoundary (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\errorBoundaries.js:140:10)
    at RemixRouteError (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\components.js:84:3)
    at RenderErrorBoundary (C:\sp\folder_bk\es\es\node_modules\react-router\dist\umd\react-router.development.js:624:7)
    at Routes (C:\sp\folder_bk\es\es\node_modules\react-router\dist\umd\react-router.development.js:1174:7)
    at Router (C:\sp\folder_bk\es\es\node_modules\react-router\dist\umd\react-router.development.js:1106:17)
    at StaticRouterProvider (C:\sp\folder_bk\es\es\node_modules\react-router-dom\server.js:67:3)
    at RemixErrorBoundary (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\errorBoundaries.js:24:5)
    at RemixServer (C:\sp\folder_bk\es\es\node_modules\@remix-run\react\dist\server.js:47:3)
GET /.redwood/functions/inngest 404 - - 54.012 ms
Could not find an event key to send events; sending will throw unless an event key is added. Please pass one to the constructor, set the INNGEST_EVENT_KEY environment variable, or use inngest.setEventKey() at runtime.

Why I am getting this error ?

[BUG] Property does not exist on serve()

Describe the bug

When upgrading to v3.0.0, we're getting a typing error:

Property 'PUT' does not exist on type '(req: Either<NextApiRequest, NextRequest>, res: NextApiResponse) => Promise<Response | undefined>'.ts(2339)

This is how we have our client defined:

import { serve } from "inngest/next";
import { functions, inngest } from "@/inngest";

export const { GET, POST, PUT } = serve({
  client: inngest,
  functions,
});

To Reproduce
Steps to reproduce the behavior:

  1. Install [email protected] in a TypeScript NextJS 13 project.
  2. Initialize the client (see above).
  • Note: You can just use an empty functions: [] definition, same thing.

Expected behavior
Should not have a typing error.

Code snippets / Logs / Screenshots
image

System info (please complete the following information):

  • OS: Man/Linux
  • npm package Version: 3.0.0
  • Framework: Next.js
  • Platform: Local, Vercel

Sending events in local dev environment should not fail if the local devserver is not running

Is your feature request related to a problem? Please describe.
When developing e.g. a CMS it's not always essential to have the local inngest dev server running in order for the app to work.

When sending an event, most of the time you'll have hooks in your CMS that call inngest.send() to trigger events on the system. If the local dev server isn't running, the Inngest SDK will try to send the event to the official https://inn.gs/ which fails with an 401 Event key Not Found error.

I want an option locally to make the inngest SDK silently fail instead throwing an error if the inngest event could not be sent.

Describe the solution you'd like
A flag that will allow allowSendingEventsWithoutDevServer

Describe alternatives you've considered

The workaround I implemented looks like this:

export const inngest = new Inngest({
  name: 'app',
  inngestBaseUrl: process.env.NODE_ENV === 'development' ? 'http://localhost:8288/' : undefined,
  fetch:
    process.env.NODE_ENV === 'development'
      ? async (input: RequestInfo | URL, init?: RequestInit) => {
          try {
            const response = await fetch(input, init);

            return response;
          } catch (e) {
            console.warn(
              'Local inngest dev server not running. Events not being processed. Start the dev server with "npx inngest-cli@latest dev"'
            );
            return {
              status: 200,
            } as Response;
          }
        }
      : undefined,
});

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Pending Approval

These branches will be created by Renovate only once you click their checkbox below.

  • Update Node.js to v18.20.3
  • Update dependency @changesets/changelog-github to ^0.5.0
  • Update dependency npm to v9.9.3
  • Update dependency zod to ~3.23.0
  • Update GitHub Artifact Actions to v4 (major) (actions/download-artifact, actions/upload-artifact)
  • Update Node.js to v20 (node, @types/node)
  • Update actions/checkout action to v4
  • Update actions/setup-node action to v4
  • Update dependency @sveltejs/kit to v2
  • Update dependency @types/jest to v29
  • Update dependency @vercel/node to v3
  • Update dependency canonicalize to v2
  • Update dependency chalk to v5
  • Update dependency concurrently to v8
  • Update dependency next to v14
  • Update dependency nodemon to v3
  • Update dependency npm to v10
  • Update dependency strip-ansi to v7
  • Update dependency tsx to v4
  • Update edumserrano/find-create-or-update-comment action to v3
  • 🔐 Create all pending approval PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

  • Update Linting (major) (@typescript-eslint/eslint-plugin, @typescript-eslint/parser, @typescript-eslint/rule-tester, @typescript-eslint/utils, eslint)

Detected dependencies

devcontainer
.devcontainer/devcontainer.json
  • mcr.microsoft.com/devcontainers/universal 2
  • ghcr.io/enricosecondulfo/devcontainer-features/volta 1
github-actions
.github/actions/setup-and-build/action.yml
  • pnpm/action-setup v2
  • actions/setup-node v3
  • oven-sh/setup-bun v1
.github/workflows/backport.yml
  • tibdex/backport v2
.github/workflows/label-prs.yml
  • actions/checkout v2
.github/workflows/pr-task-list-checker.yml
.github/workflows/pr.yml
  • actions/checkout v3
  • actions/setup-node v3
  • actions/checkout v3
  • actions/checkout v3
  • actions/checkout v3
  • actions/checkout v3
  • actions/checkout v3
  • actions/checkout v3
  • actions/checkout v3
  • actions/checkout v3
  • actions/upload-artifact v3
  • actions/checkout v3
  • actions/checkout v3
  • actions/download-artifact v3
.github/workflows/prerelease.yml
  • actions/checkout v3
  • edumserrano/find-create-or-update-comment v1
  • actions/checkout v3
  • edumserrano/find-create-or-update-comment v1
.github/workflows/release.yml
  • actions/checkout v3
  • softprops/turnstyle v1
  • changesets/action v1
npm
package.json
  • @actions/core ^1.10.0
  • @actions/exec ^1.1.1
  • @actions/github ^6.0.0
  • @changesets/changelog-github ^0.4.8
  • @changesets/cli ^2.26.2
  • cross-env ^7.0.3
packages/eslint-plugin-internal/package.json
  • eslint ^8.23.1
packages/eslint-plugin/package.json
  • @typescript-eslint/utils ^6.11.0
  • typescript ~5.4.0
  • @typescript-eslint/rule-tester ^6.11.0
  • jest ^29.3.1
  • ts-jest ^29.1.0
packages/inngest/package.json
  • @types/debug ^4.1.12
  • canonicalize ^1.0.8
  • chalk ^4.1.2
  • cross-fetch ^4.0.0
  • debug ^4.3.4
  • hash.js ^1.1.7
  • json-stringify-safe ^5.0.1
  • ms ^2.1.3
  • serialize-error-cjs ^0.1.3
  • strip-ansi ^5.2.0
  • zod ~3.22.3
  • @actions/core ^1.10.0
  • @actions/exec ^1.1.1
  • @jest/globals ^29.5.0
  • @shopify/jest-koa-mocks ^5.1.1
  • @sveltejs/kit ^1.27.3
  • @total-typescript/shoehorn ^0.1.1
  • @types/aws-lambda ^8.10.108
  • @types/express ^4.17.13
  • @types/inquirer ^9.0.3
  • @types/jest ^27.4.1
  • @types/json-stringify-safe ^5.0.0
  • @types/koa ^2.13.11
  • @types/minimist ^1.2.2
  • @types/ms ^0.7.31
  • @types/node ^18.16.16
  • @types/sha.js ^2.4.0
  • @typescript-eslint/eslint-plugin ^6.0.0
  • @typescript-eslint/parser ^6.0.0
  • @vercel/node ^2.15.9
  • aws-lambda ^1.0.7
  • callsites ^4.1.0
  • concurrently ^7.4.0
  • eslint ^8.30.0
  • eslint-config-prettier ^9.0.0
  • eslint-plugin-import ^2.27.5
  • eslint-plugin-prettier ^5.0.0
  • express ^4.19.2
  • fastify ^4.21.0
  • genversion ^3.1.1
  • glob ^10.3.10
  • h3 ^1.8.1
  • hono ^4.2.7
  • inquirer ^9.2.10
  • jest ^29.3.1
  • jest-fetch-mock ^3.0.3
  • koa ^2.14.2
  • minimist ^1.2.8
  • mitata ^0.1.11
  • next ^13.5.4
  • nock ^13.2.9
  • node-mocks-http ^1.11.0
  • nodemon ^2.0.20
  • prettier ^3.1.0
  • shx ^0.3.4
  • ts-jest ^29.1.0
  • tsx ^3.12.7
  • typescript ~5.4.0
  • @sveltejs/kit >=1.27.3
  • @vercel/node >=2.15.9
  • aws-lambda >=1.0.7
  • express >=4.19.2
  • fastify >=4.21.0
  • h3 >=1.8.1
  • hono >=4.2.7
  • koa >=2.14.2
  • next >=12.0.0
  • typescript >=4.7.2
  • node >=14
  • node 18.12.1
  • npm 9.6.4
packages/middleware-encryption/package.json
  • crypto-js ^4.2.0
  • @types/crypto-js ^4.2.1
  • inngest ^3.19.7
  • typescript ~5.4.0
  • inngest >=3.0.0

  • Check this box to trigger a request for Renovate to run again on this repository

[BUG] Background Job Function Not Executing in Production

Describe the bug
I recently started using Inngest for my server needs and encountered an issue with background jobs. The setup works perfectly on local development, but upon deploying to production, the event triggers without calling the associated function. Despite the event being triggered as expected, the associated function remains unexecuted in the production setup.

To Reproduce
Steps to reproduce the behavior:

Expected behavior
After triggering an event it should call the associated function

Code snippets / Logs / Screenshots

Check these on first image events are fired

image

Here its associated function not being called

image

System info (please complete the following information):

  • OS: Windows
  • npm package Version [2.4.0]
  • Framework - Express
  • Platform - Railway.app

Additional context
I have setup both INNGEST_SIGNING_KEY and INNGEST_EVENT_KEY on railway.app platform

[BUG] NonRetriableError triggers retry

Describe the bug
The step is retried when the NonRetriableError was thrown.

To Reproduce

  1. Create a function, which throws NonRetriableError inside the step.run:
const test = inngest.createFunction(
  { name: "Test" },
  { event: "test" },
  async ({ step }) => {
    await step.run("test", () => {
      throw new NonRetriableError("Should not retry");
    });
  }
);
  1. Send test event:
curl -v -d '{"name":"test","data":{}}' http://localhost:8288/e/test

Expected behavior
The step is not retried.

Code snippets / Logs / Screenshots
Repo with reproduction case: https://github.com/dddenis/inngest-non-retriable-error

System info (please complete the following information):

  • OS: Mac
  • npm package Version: 1.9.3
  • Framework: Next.js
  • Platform: Vercel

[Feature] Add middleware callback with step info before each step is ran

Is your feature request related to a problem? Please describe.
I'm frustrated when I'm trying to debug my inngest functions, because I have to manually add logs before each step.run call to get an overview of the progress (when not using the inngest devtools)

Describe the solution you'd like
The middleware gives me information about the function run, but not which step is currently being ran. I would like to be able to log each step run from my own middleware, e.g.

beforeExecution({ step }) {
  ctx.logger.info(`Running step ${step.name}`)
} 

Where the logger is the same proxy logger instance

Describe alternatives you've considered

  • Using transformOutput, which has the context of the step. This runs after, ideally I'd be able to log before and after (then I could also measure run timings)

I'm concious I may be misunderstanding part of the execution flow, but I wasn't able to find the step name that's about to be executed in all the middleware callbacks/context

Additional context
Given a function run with a body like the below:

    await step.run("FIRST STEP", async () => {
      logger.info("--> FIRST STEP");
      return new Promise((resolve) => {
        logger.info("Waiting 1s");
        setTimeout(() => {
          logger.info("<-- FIRST STEP");
          resolve(null);
        }, 300);
      });
    });

    logger.info("Between steps");

    await step.run("SECOND STEP", async () => {
      // same as above
    });

I get roughly these logs

myapp:dev: steps []
myapp:dev: [17:41:23.147] INFO (19703): --> FIRST STEP
myapp:dev: [17:41:23.147] INFO (19703): Waiting 1s
myapp:dev: [17:41:23.449] INFO (19703): <-- FIRST STEP
myapp:dev: step {
myapp:dev:   name: 'FIRST STEP',
myapp:dev: }
myapp:dev: steps [ { id: 'c4dff821e3b2828e75decb728e9b96e3cdece1ac', data: null } ]
myapp:dev: beforeExecution
myapp:dev: [17:41:24.773] INFO (19703): Between steps
myapp:dev: [17:41:24.773] INFO (19703): --> SECOND STEP
myapp:dev: [17:41:24.773] INFO (19703): Waiting 1s
myapp:dev: [17:41:25.074] INFO (19703): <-- SECOND STEP
myapp:dev: step {
myapp:dev:   name: 'SECOND STEP',
myapp:dev: }

p.s. Some of your diagrams about execution are great, I think the middleware docs could really do with a diagram showing when they run in relation to each stage of a function/step

[BUG] Did you mean to import inngest/nuxt.js?

Describe the bug

So, this should hopefully be a simple bug to squash.

Essentilly, within Nuxt / nitro environments, the import as described here in the documentation, does not work.

We get faced with this error:

[nitro] [request error] [unhandled] Cannot find module '/var/task/node_modules/inngest/nuxt' imported from /var/task/chunks/inngest.mjs
Did you mean to import inngest/nuxt.js?

Perhaps we could supply individual "namespaces" exports in the project's package.json?


To Reproduce

Setup the

  1. Setup the serve eventHandler as described in the documentation for Nuxt
  2. Watch the errors unfold when you navigate to "/api/inngest".

[Feature] Replace zod with typeschema for a less opinionated api

Is your feature request related to a problem? Please describe.
I don’t use zod but I do use a schema library. It shouldn’t matter which one.

Describe the solution you'd like
I want to get typeschema into the inngest api before the potential changes proposed by Theo bake zod in further. (I like what I saw in that video btw)
https://github.com/decs/typeschema

Describe alternatives you've considered
Just using the types with inngest EventSchema - works for now but…

Prevent SDK debug landing page from showing in production

Describe the bug
/api/inngest should not show in production.

To Reproduce

  1. Use the latest SDK, ship it to production on Vercel.
  2. Visit /api/inngest

Expected behavior
It should not show.

Ideally, it should show some sort of message to the user that the page has been disabled in production for security reasons.

Code snippets / Logs / Screenshots
n/a

System info (please complete the following information):

  • npm package Version: 0.9.0
  • Framework: Next.js
  • Platform: Vercel

[BUG] Inngest SDK causes Docker container to exit unexpectedly when registering with Inngest dev server

This is a strange one which I think boils down to the default fetch implementation used by the SDK and possibly the native node implementation as it somehow relates to Docker 🤷‍♂️? (pure speculation here)

I encountered an issue where the Inngest SDK seemed to cause my Docker container to unexpectedly exit without any additional output. This occurred while attempting to register with the Inngest CLI development server, which is running separately on my host machine.

This problem specifically arises in Docker Desktop for Mac versions 4.19.0 and above. Interestingly, it does not occur in version 4.18.0.

When running the server directly from the host machine, outside of a Docker container, there is no problem.

Upon conducting some initial debugging, it appears that the issue is related to the default fetch implementation used by the SDK. However, when I supplied the fetch implementation from the node-fetch package, the problem was resolved. Curiously, during my debugging process, the behavior was not consistently reproducible. While stepping through the SDK's code, the unexpected container exit did not always occur, and the API would sometimes receive a successful response from the register endpoint.

I've created a sample repo which reproduces the issue. The README has everything you should need.

I have kept the line in the src/server.ts file that passes a different fetch implementation. You are welcome to uncomment it to observe the differences described above.

  • Inngest SDK: v2.1.0
  • Node: v18.16.1
  • MacOS: v12.6.8
  • Docker Desktop for Mac (with intel chip): v4.19.0+
  • Docker Engine: v23.0.5
  • Docker Settings (Resources)
    • CPUs: 4
    • Memory: 8 GB
    • Swap: 1 GB
    • Virtual disk limit: 64 GB

EDIT

I'm not sure this is actually an Inngest SDK issue but it may be of concern if it affects enough developers looking to use Inngest who also may run into this. My thought it is that it might be worth confirming and potentially mitigating either though documentation or by changing the default fetch implementation (or both).

[BUG] Installing 2.5.0 on Vercel fails

Describe the bug
Just started out using inngest so i'm not that familar with it yet. So could be i'm missing something here... But my first deploys to Vercel are failing and I don't really know why that is.

When I switch to using 2.4.1 it works fine.

When I install and build using 2.5.0 locally, it all works fine.

See Vercel logs below.

To Reproduce
Steps to reproduce the behavior:

  1. Use 2.5.0
  2. Deploy to Vercel, error
  3. Use 2.4.1
  4. Deploy to Vercel, no error

Expected behavior
No errors during build on Vercel on 2.5.0

Code snippets / Logs / Screenshots
Happens during npm install:

01:18:29.223 | npm ERR! code 254
01:18:29.223 | npm ERR! path /vercel/path0/node_modules/inngest
01:18:29.224 | npm ERR! command failed
01:18:29.224 | npm ERR! command sh -c npx only-allow pnpm
01:18:29.224 | npm ERR! npm ERR! code ENOENT
01:18:29.224 | npm ERR! npm ERR! syscall lstat
01:18:29.224 | npm ERR! npm ERR! path /vercel/path0/lib
01:18:29.224 | npm ERR! npm ERR! errno -2
01:18:29.224 | npm ERR! npm ERR! enoent ENOENT: no such file or directory, lstat '/vercel/path0/lib'
01:18:29.224 | npm ERR! npm ERR! enoent This is related to npm not being able to find a file.
01:18:29.224 | npm ERR! npm ERR! enoent
01:18:29.225 | npm ERR!
01:18:29.225 | npm ERR! npm ERR! A complete log of this run can be found in:
01:18:29.225 | npm ERR! npm ERR!     /vercel/.npm/_logs/2023-08-19T23_18_27_175Z-debug-0.log
01:18:29.225 |  
01:18:29.225 | npm ERR! A complete log of this run can be found in:
01:18:29.225 | npm ERR!     /vercel/.npm/_logs/2023-08-19T23_17_59_611Z-debug-0.log
01:18:29.266 | Error: Command "npm install --prefix=../.." exited with 254

System info (please complete the following information):

  • OS: Mac
  • npm package Version: 2.5.0
  • Framework: Next.JS
  • Platform: Vercel

Additional context

Documentation: Add documentation surrounding step.sendEvent and other async operations

Hi! Just had a small request for a documentation update:

https://www.inngest.com/docs/guides/fan-out-jobs In the fan out jobs documentation, it isn't stated that to use step.sendEvent, all other asynchronous operations must be wrapped in a step.run. Would love to just see some clear wording around this so others don't get confused when step.send doesn't work in their jobs- I saw another user have the same problem after I did on Discord, so think this would be worthwhile:

My discord thread:
https://discord.com/channels/842170679536517141/845000011040555018/1090805866187468862

Other user having the same issue:
https://discord.com/channels/842170679536517141/1093066335023550467/1093066335023550467

This might not be the right place to open a doc issue, please feel free to triage accordingly. I simply opened this here since I'm using the JS SDK

Type errors in inngest types and in dependencies

Describe the bug
Typescript (v4.4.3) gives type errors. The errors are both related to inngest/types.d.ts and Inngest's type-fest dependency module.

To Reproduce
Steps to reproduce the behavior:

  1. Create an instance of Inngest:
import { Inngest } from "inngest";

const client = new Inngest({ name: "My app" });
  1. Run tsc.

Expected behavior
I expected the code to be compiled without any errors.

Code snippets / Logs / Screenshots
Temporarily fix:
Setting typescript's skipLibCheck compiler option to true will solve the problem temporarily as it skips the type checking on imported packages. tsconfig.json:

  ...
  "compilerOptions": {
    "skipLibCheck": true,
  },
  ...

Error logs:

node_modules/inngest/node_modules/type-fest/source/camel-case.d.ts:25:32 - error TS1005: '?' expected.

25  infer FirstWord extends string,
                                  ~

node_modules/inngest/node_modules/type-fest/source/camel-case.d.ts:26:42 - error TS1005: '?' expected.

26  ...infer RemainingWords extends string[],
                                            ~

node_modules/inngest/node_modules/type-fest/source/join.d.ts:32:56 - error TS1005: '?' expected.

32    ...infer Rest extends ReadonlyArray<string | number>,
                                                          ~

node_modules/inngest/types.d.ts:137:45 - error TS1005: '>' expected.

137     tools: ReturnType<typeof createStepTools<TEvents, TTrigger>>[0];
                                                ~

node_modules/inngest/types.d.ts:137:64 - error TS1005: '(' expected.

137     tools: ReturnType<typeof createStepTools<TEvents, TTrigger>>[0];
                                                                   ~

node_modules/inngest/types.d.ts:174:1 - error TS1128: Declaration or statement expected.

174 };
    ~


Found 6 errors.

error Command failed with exit code 2.

System info:

  • OS: Mac
  • npm package Version 1.5.1
  • Framework Express
  • Platform Netlify

Redirect users to dev server UI for sending events

Summary

Sending events via the SDK UI a) requires the dev server to be connected and b) is a worse experience than the dev server.

Let's replace the SDK UI event sending dropdown with a banner directing users to the dev server or instructing how to start the dev server if it's not detected.

Related

Simplification of defining event types

Is your feature request related to a problem? Please describe.
I don't like that in the example on the site, the event name is repeated 3 times.

Describe the solution you'd like
I would love utility types or helper functions to reduce the repetition.

Describe alternatives you've considered
I tried making a type as such to reduce the overhead some:

type InngestNamedEventRecord<T extends { name: string }> = Record<T["name"], T>;

That worked fine, defining the event like so:

export type SendEmailEvent = InngestNamedEventRecord<{
  name: "app/email.send";
  data: {
    mailTo: string;
    subject: string;
    message: string;
  };
}>;

You can then & merge that type with all the others to reduce the repetition in the main event record provided to the client.

Make the address of the Dev Server a clickable link

Is your feature request related to a problem? Please describe.
It would be nice if the address for the inngest dev server was clickable.

Describe the solution you'd like
Wrap the address in an <a> tag

Describe alternatives you've considered
It seems like a simple feature.

Additional context
image

[BUG] `No INNGEST_ENV branch name found`

Describe the bug
Despite setting the env var INNGEST_ENV and even explicitly setting the env via the constructor options, I continue to get the message No INNGEST_ENV branch name found. I'm seemingly doing everything right but tbh the dashboard overhaul which now makes the existing docs pretty obsolete is making integrating inngest very difficult.

To Reproduce
Steps to reproduce the behavior:

  1. new Inngest({ name: "...", env: process.env.INNGEST_ENV })
  2. $ curl -X PUT http://local.grossr.co.nz:3001/api/inngest
  3. Logs: {"message":"No INNGEST_ENV branch name found"}

Expected behavior

Code snippets / Logs / Screenshots
If applicable, add screenshots to help explain your problem.

System info (please complete the following information):

  • OS: [e.g. Mac, Windows]
  • npm package Version [e.g. 0.7.0]
  • Framework [e.g. Next.js, Express]
  • Platform [e.g. Vercel, AWS Lambda]

Additional context
Add any other context about the problem here.

Switch `inngestRegisterUrl` to just `inngestAPI`

Is your feature request related to a problem? Please describe.
Requiring a user to add the /fn/register should be unnecessary as every Inngest environment should have this same url.

inngest-js/src/types.ts

Lines 188 to 192 in fdea3ef

/**
* The URL used to register functions with Inngest.
* Defaults to https://api.inngest.com/fn/register
*/
inngestRegisterUrl?: string;

Describe the solution you'd like
We should make this simpler by simple passing the URL of the API itself and the SDK can append the path.

Describe alternatives you've considered
n/a - this is mostly an internal use case right now, but may grow with more advanced dev server usage and preview environment support.

Additional context
n/a

`Inngest#send()` error messages are vague

Summary

We have a list of errors thrown by Inngest#send() to inform the user that Inngest did not successfully receive their event payload. See src/components/Inngest.ts#L151.

None of the errors provides helpful information, requiring a user to guess what's wrong.

Let's adjust so that every error message tells a user:

  • What happened
  • To the best of our ability, why
  • Help the user fix the error by mentioning a likely cause
  • Provide further steps (relevant docs link) should the likely cause not be the issue

Add a transformer like trpc

Is your feature request related to a problem? Please describe.
Date objects get converted to string

Describe the solution you'd like
Use superjson as a transformer (or any other alternative)

Describe alternatives you've considered
For the moment you need to manually convert to superjson and then send the event / parse back on the handler

Additional context
some api like serve(inngest, [fns], { transformer: superjson })

[Feature] Autodetect functions

Autodetect created functions in inngest.

For now, I've created a utility command that's working fine for me but we'd love not to register the functions manually.

const fs = require("fs");
const BASE_PATH = "src/lib/inngest/functions";

// Read all the files and folders recursively in the given path and create a list of all the files
const getFilesRecursively = (path) => {
  const files = fs.readdirSync(path);
  let fileList = [];

  files.forEach((file) => {
    if (fs.statSync(`${path}/${file}`).isDirectory()) {
      fileList = [...fileList, ...getFilesRecursively(`${path}/${file}`)];
    } else {
      fileList.push(`${path}/${file}`);
    }
  });

  return fileList;
};

const isFileValid = (filePath) => {
  // Check if file contains inngest.createFunction
  const fileContent = fs.readFileSync(filePath, "utf8");
  if (!fileContent.includes("inngest.createFunction")) {
    return false;
  }
  return true;
};

// Remove base path from the file path
const files = getFilesRecursively(BASE_PATH)
  .filter((file) => isFileValid(file))
  .map((file) => file.replace(`${BASE_PATH}/`, ""));

// Open BASE_PATH/../index.ts and write the import statements for all the files
const content = `
// Generated file. Do not edit manually. Run 'node scripts/register-inngest-functions.js' to regenerate this file.
const functions = [
${files
      .map(
        (file) => `    require("./functions/${file.replace(".ts", "")}").default`
      )
      .join(",\n")}
];
export default functions;
`;

fs.writeFileSync(`${BASE_PATH}/../index.ts`, content);

// Print warning message for invalid files in yellow color
const invalidFiles = getFilesRecursively(BASE_PATH).filter(
  (file) => !isFileValid(file)
);
if (invalidFiles.length > 0) {
  console.log(
    `\x1b[33m${invalidFiles.length} files are invalid and not registered.\x1b[0m`
  );
  invalidFiles.forEach((file, index) => {
    console.log(`\x1b[33m${index + 1}. ${file}\x1b[0m`);
  }
  );
}

// Print success message with number of functions registered in green color
console.log(
  `\x1b[32m${files.length} functions registered successfully.\x1b[0m`
);

In route.ts

import { serve } from "inngest/next";
import { inngest } from "@/lib/inngest/client";
import functions from "@/lib/inngest";

// Create an API that serves zero functions
export const { GET, POST, PUT } = serve(inngest, functions);

serve() - Unsafe assignment of an `any` value

Describe the bug

serve() return value is untyped throwing off eslint and/or causing trouble with tsc compilation.

To Reproduce

  1. Following the quick start guide
  2. Given a configured client
  3. We have an api route handler (copy-pasta from quickstart):
import { serve } from "inngest/next";
import { inngest } from "../../../inngest/client";

// Create an API that serves zero functions
export const { GET, POST, PUT } = serve(inngest, [
  /* your functions will be passed here later! */
]);

Expected behavior

No eslint errors, valid return type.

Actual

export const { GET, POST, PUT } = serve(inngest, [
Unsafe assignment of an `any` value.eslint[@typescript-eslint/no-unsafe-assignment](https://typescript-eslint.io/rules/no-unsafe-assignment)

Screenshots / Stack trace dump
image

System info

  • inngest-cli version: 0.13.6
  • OS: macOS
  • Version 13.4
  • TS 5.0.4
  • Node 18.15
  • Next.js 13.3.1

Additional context

The explanation in source code is debatable - I'm importing the serve function directly from "inngest/next" so I was expecting the types to be relevant to Next.js route handlers.

// InngestCommHandler.d.ts
/**
 * This `any` return is appropriate.
 *
 * While we can infer the signature of the returned value, we cannot guarantee
 * that we have used the same types as the framework we are integrating with,
 * which sometimes can cause frustrating collisions for a user that result in
 * `as unknown as X` casts.
 *
 * Instead, we will use `any` here and have the user be able to place it
 * anywhere they need.
 */
) => any;

Support Vercel Edge Functions

When using NextJS * Vercel Edge functions fetch is not correctly detected.

ReferenceError: XMLHttpRequest is not defined

Looking at the code https://github.com/inngest/inngest-js/blob/main/src/components/Inngest.ts#L129 it looks like if you don't provide a fetch in the constructor it falls back to using cross-fetch, which I guess thinks the edge functions are a browser and tries to use XMLHttpRequest polyfil.

As a workaround supplying fetch seems to work and I'm able to send events.

new Inngest({fetch})

[BUG] Module not found: can't resolve 'encoding'

Describe the bug
I see this warning when integrating Inngest into a new Next.js project:

 ⚠ ./node_modules/cross-fetch/node_modules/node-fetch/lib/index.js
Module not found: Can't resolve 'encoding' in '/Users/ian/Code/promptmetrix/node_modules/cross-fetch/node_modules/node-fetch/lib'

Import trace for requested module:
./node_modules/cross-fetch/node_modules/node-fetch/lib/index.js
./node_modules/cross-fetch/dist/node-ponyfill.js
./node_modules/inngest/helpers/env.js
./node_modules/inngest/components/InngestCommHandler.js
./node_modules/inngest/next.js
./src/app/api/inngest/route.ts

Supabase had the same warning in the past. See the following issue: supabase/supabase-js#612

To Reproduce
Steps to reproduce the behavior:

  1. Follow instructions on https://www.inngest.com/docs/quick-start for the "App Router" variant.

Expected behavior
No warnings.

Code snippets / Logs / Screenshots

System info (please complete the following information):

  • OS: macOS
  • npm package Version: 3.1.1
  • Framework: Next.js
  • Platform: N/A

[BUG] unhandled promise rejected on remix with vercel

Hello,

I use inngest with remix and vercel and have successfull deployed my app but if I send data with inngest I get this:

Unhandled Promise Rejection 	{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"TypeError: First parameter has member 'readable' that is not a ReadableStream.","reason":{"errorType":"TypeError","errorMessage":"First parameter has member 'readable' that is not a ReadableStream.","stack":["TypeError: First parameter has member 'readable' that is not a ReadableStream.","    at assertReadableStream (/var/task/node_modules/web-streams-polyfill/dist/ponyfill.js:362:19)","    at convertReadableWritablePair (/var/task/node_modules/web-streams-polyfill/dist/ponyfill.js:3524:9)","    at ReadableStream.pipeThrough (/var/task/node_modules/web-streams-polyfill/dist/ponyfill.js:3608:29)","    at fetchFinale (node:internal/deps/undici/undici:10965:56)","    at mainFetch (node:internal/deps/undici/undici:10857:9)","    at processTicksAndRejections (node:internal/process/task_queues:95:5)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: TypeError: First parameter has member 'readable' that is not a ReadableStream.","    at process.<anonymous> (file:///var/runtime/index.mjs:1186:17)","    at process.emit (node:events:525:35)","    at process.emit (/var/task/node_modules/source-map-support/source-map-support.js:516:21)","    at emit (node:internal/process/promises:149:20)","    at processPromiseRejections (node:internal/process/promises:283:27)","    at processTicksAndRejections (node:internal/process/task_queues:96:32)"]}
Unknown application error occurred
Runtime.Unknown

But it send successfull the data and all works and I recieve my data after inngest is. finish but idk why I got this error.

code:

        if(typeof id === 'number') {

          const buffer = await getPDFVerify({db_name: connectorDB, id: id as number});
          const email = await getClientMailByID({db_name: connectorDB, p: { client_id: obj.client_id }})
        
          if(typeof email === 'string' && endObj.send_mail === '1') {
            console.log('INNGGEST');
            console.log(email);
            const saaa = await inngest.send({
              name: 'Default ingest key',
              data: {
                email,
                // @ts-ignore
                buffer
              }
            });
            console.log(saaa);
          }
        }

I use the latest inngest version

[BUG] Documentation Error

I believe there is a misleading mistake in the documentation of step.waitForEvent().

The below example from the documentation should have single quotes ' rather than double quotes ".

// Based on my experience, this does NOT work, waitForEvents times out
const topicSelected = await step.waitForEvent("ai/post.topic.selected", {
   timeout: "5m",
   // "async" is the "ai/post.topic.selected" event here:
   if: `async.data.completionId == "${generatedTopics.completionId}"`
});

// Based on my experience, this works
const topicSelected = await step.waitForEvent("ai/post.topic.selected", {
   timeout: "5m",
   // "async" is the "ai/post.topic.selected" event here:
   if: `async.data.completionId == '${generatedTopics.completionId}'`
});

System info (please complete the following information):

  • Framework: Vercel serverless functions (@vercel/node)
  • Platform: Vercel

Mapped object type returns from `step.run()` are oversimplified

Describe the bug

Mapped object types with property overrides lose those overrides when output from step.run().

This affects a user using the plaid/plaid-node package, where many of the types returned from client methods are these mapped object types with overrides.

It looks like this project uses OpenAPITools/openapi-generator to create these types, so it's likely that any project using that will suffer the same fate.

To Reproduce

// Create a mapped object type with overrides
interface Foo {
  [x: string]: any;
  foo: boolean;
}

// Within a step function, return the object type
const result = await step.run("Issue", () => undefined as unknown as Foo);

// Observe that `result` has lost the `foo: boolean;` property.

Expected behavior

The object type should be maintained.

System info (please complete the following information):

  • OS: any
  • npm package Version: 1.0.0
  • Framework: any
  • Platform: any

Additional context

The issue is caused because we use a Jsonify<> type from sindresorhus/type-fest to simulate the (de)serialization of data as it moves to/from Inngest, which is stripping the extra properties.

Need to investigate whether this is a conscious choice and, if not, whether the behaviour of the object type being maintained is reasonable.

Add a Nuxt 3 handler

Is your feature request related to a problem? Please describe.
I'm using Nuxt 3 to build an app and deploy it on Vercel, so I just made a custom handler to serve my inngest functions.

Describe the solution you'd like
I'll make a PR with my suggested custom handler.

Describe alternatives you've considered

Additional context

[Feature] Allow creating child loggers, or modifying the parent logger state

Is your feature request related to a problem? Please describe.
Inngest's automatic child logger creation is great, but I find myself wanting to pass additional info with each log line, so that they can be grouped together later.

Describe the solution you'd like
I'd like to either be able to

a) Extend the logger from the context by calling .child on it and passing my own context.
It seems like passing a generic inferred from the logger instance passed to the Inngest constructor down to the function could help tell if a logger supports child logging, from the context of the function

  async ({ event, step, logger }) => {
    const transactionLogger = logger.child({ transactionId });

    const model = new SomeModel(deps, transactionLogger);

    transactionLogger.info(`Doing thing for transaction`);

b) Somehow attach more metadata to the function, and/or it's logs (rough sketch as an example, not particularly a suggested API)

  async ({ event, step, logger, setMeta }) => {
    setMeta({ transactionId })
    // all calls to `logger`, if it supports child loggers, will now have the meta applied

Option b is with #330 in mind, although since the middleware run before user code is executed that wouldn't really work.

c) If b doesn't work because of execution order, perhaps being able to call .child from a function-specific middleware

const loggingMiddlewarenew InngestMiddleware({
// ...
          transformInput(ctx) {
            const logger = ctx.logger.child({ transactionId: fromEventDataSomehow })
            return { ctx: { logger } };
          },
});

inngest.createFunction(
  { name: "Example", middleware: [loggingMiddleware] },
  { event: "app/example" },
  async ({ event, step, ctx }) => {
    ctx.logger.info(`I have transactionId in my context`)

Describe alternatives you've considered

  • I can just log the transactionId once, and from that find the inngest provided runID and use that to filter logs
  • I've used pnpm patch to patch the inngest SDK for our app to add hacky support for a child logger. This works for us for now, but it feels very hacky
--- a/middleware/logger.js
+++ b/middleware/logger.js
@@ -26,6 +26,9 @@ class DefaultLogger {
+    child(...args) {
+        throw new Error("logger.child not implemented for DefaultLogger")
+    }
@@ -62,6 +65,9 @@ class ProxyLogger {
+    child(...args) {
+        return __classPrivateFieldGet(this, _ProxyLogger_logger, "f").child(...args);
+    }

Additional context
I found this part of the docs slightly misleading, as it led me to believe I'd be able to bring my own child-compatable logger. Re-reading it I can see it doesn't actually state that, but it tripped me up.

child logger
If the logger library supports a child logger .child() implementation, the built-in middleware will utilize it to add function runtime metadata for you.

Allow specifying all function.cue config options when defining functions

When defining functions using the object pattern createFunction({ name: "", ... }), we should allow every function.cue parameter to be specified.

The missing fields now are:

  • idempotency
  • throttle.

This allows people to configure functions to run at most N times per X period, or control idempotency for functions overall. In the future I'm sure we'll be adding more here, such as alerting mechanisms etc. :)

SolidStart support?

Is your feature request related to a problem? Please describe.

N/A

Describe the solution you'd like

Would like to use Inngest in conjunction with solid-start app.

Describe alternatives you've considered

N/A

Additional context

N/A

`inngest/next` `serve` Type instantiation is excessively deep and possibly infinite. (ts-2589)

I am working on an integration in a TypeScript Next.js app.

I've started to see the following error on the inngest/next serve function:

Screen Shot 2023-03-21 at 15 23 55 PM

I tried removing the functions one-by-one and the error only goes away when I remove all of them.

Using:

https://github.com/inngest/inngest/releases/tag/v0.11.0
https://github.com/inngest/inngest-js/releases/tag/v1.4.1

I tried reverting to 1.4.0 and still same result. Now using latest 1.4.1

At the moment I have to rely on //@ts-expect-error to allow me to compile.

Thoughts?

Next.js `basePath` breaks registration

Describe the bug

When using basePath in a next.config.js file, the "inngest/next" serve handler doesn't detect the correct URL.

To Reproduce

  1. Add basePath: "/example" to examples/framework-nextjs/next.config.js
  2. Run pnpm run dev:example --example framework-nextjs
  3. Run npx inngest-cli@latest dev
  4. In the dev server UI, add http://localhost:3000/example/api/inngest

Expected behavior

The app should correctly register, but instead fails.

The reported URL from the app is still http://localhost:3000/api/inngest (missing the /example prefix), so the dev server also erroneously displays both that and the prefixed URL.

It looks as though Next.js gives us little

Code snippets / Logs / Screenshots

[inngest] reporting URL as: {
  href: 'http://localhost:3000/api/inngest',
  __NEXT_ROUTER_BASEPATH: undefined,
  __NEXT_MANUAL_CLIENT_BASE_PATH: undefined
}

image

System info:

System:
  OS: Linux 5.15 Ubuntu 20.04.6 LTS (Focal Fossa)
  CPU: (16) x64 12th Gen Intel(R) Core(TM) i5-12600K
  Memory: 10.87 GB / 15.53 GB
  Container: Yes
  Shell: 5.0.17 - /bin/bash
Binaries:
  Node: 18.12.1 - ~/.volta/tools/image/node/18.12.1/bin/node
  Yarn: 1.22.19 - ~/.volta/tools/image/yarn/1.22.19/bin/yarn
  npm: 9.6.4 - ~/.volta/tools/image/npm/9.6.4/bin/npm

Local server throws Invalid signature error after updating to 1.3.1 from 1.2.0

After updating from 1.2.0 to 1.3.1 today the local dev server stopped processing functions with this error (nothing else in the application was changed):

4:06PM INF output > step output attempt=2 edge={"incoming":"step","outgoing":"$trigger"} fn_id=app-push-to-channel fn_name="Push to channel" generator=null output={"body":{"__serialized":true,"message":"Invalid signature","name":"Error","stack":"Error: Invalid signature\n    at RequestSignature.verifySignature (.../nextjs/node_modules/inngest/components/InngestCommHandler.js:627:19)\n    at InngestCommHandler.validateSignature (.../nextjs/node_modules/inngest/components/InngestCommHandler.js:556:35)\n    at InngestCommHandler.handleAction (.../nextjs/node_modules/inngest/components/InngestCommHandler.js:257:22)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async ServerTiming.wrap (.../nextjs/node_modules/inngest/helpers/ServerTiming.js:70:20)\n    at async .../nextjs/node_modules/inngest/components/InngestCommHandler.js:232:31\n    at async Object.apiResolver (.../nextjs/node_modules/next/dist/server/api-utils/node.js:372:9)\n    at async DevServer.runApi (.../nextjs/node_modules/next/dist/server/next-server.js:513:9)\n    at async Object.fn (.../nextjs/node_modules/next/dist/server/next-server.js:815:35)\n    at async Router.execute (.../nextjs/node_modules/next/dist/server/router.js:243:32)\n    at async DevServer.runImpl (.../nextjs/node_modules/next/dist/server/base-server.js:432:29)\n    at async DevServer.run (.../nextjs/node_modules/next/dist/server/dev/next-dev-server.js:814:20)\n    at async DevServer.handleRequestImpl (.../nextjs/node_modules/next/dist/server/base-server.js:375:20)\n    at async .../nextjs/node_modules/next/dist/server/base-server.js:157:99","type":"internal"},"status":500} run_id=01GTC8Y2TR17DQFJK0KS6CDVWA step=step

My INNGEST_SIGNING_KEY environment variable is set to the test key value.

The event trigger:

await inngest.send({ name: 'app/vote.created', data: { vote } });

The function:

export default inngest.createFunction(
  {
    name: 'Push to channel',
    throttle: {
      key: 'rank-{{ event.data.vote.channelId }}',
      count: 1,
      period: '1s',
    },
  },
  { event: 'app/vote.created' },
  async ({ event }) => {
    // ...
  }
);

Add logger parameter to onFailure

With normal functions I'm using the logger this way:

 async ({ event, step, logger }) => {
    await step.run("Trigger task", async () => {
      try {
        return await task()
      } catch (err) {
        logger.error("Step Trigger upload event failed: ", {
          userId: event.data.userId
        });
        throw err;
      }
    });
}

Nevertheless, I don't get a logger on the onFailure section as it's not available, so I would need to do my error handling differently in different places, which is not ideal.

Is there a reason it was designed this way? Would be great to have it!

Add `step.log()` tool for logging

Logging is a common need. We should provide this via step tooling.

We should repliate common methods of console for ease of use.

console should also always proxy to the raw methods, meaning console.table() still works locally, even if Inngest doesn't support it.

async ({ step, console }) => {
  console.log("Info", obj, etc);
  console.debug("Debug", obj, etc);
  console.error("Error", obj, etc);
  console.info("Info", obj, etc);
  console.trace("Trace", obj, etc);
  console.warn("Warn", obj, etc);
}

How does this work within a step.run()? We want it to.

Environment not detected in Vercel Edge

When deploying on Vercel Edge the INNGEST_EVENT_KEY is not detected from the environment correctly.

Error: Could not find an event key to send events. Please pass one to the constructor, set the INNGEST_EVENT_KEY environment variable, or use inngest.setEventKey() at runtime.

Workaround

export default new Inngest<Events>({
    name: "b2b.store",
    eventKey: process.env["INNGEST_EVENT_KEY"],
    fetch,
});

Better Typescript support please!

Current state

I love the project and the service. Just wired up our apps and implemented first functions.
I'm on the 2.0.0 branch.

The Typescript support is a bit of a hurdle at the moment.

  • Docs around it are outdated (still on the 1.* branch) or are confusing.
  • There seems to be an undocumented cli method for "generating types", but it only works with the cloud service - it's impossible to use it to generate local types as I'm writing my new functions.
  • I'm not sure what the thinking behind inngest types was, but it requires functions to be published. I can't publish functions before I write them, and project won't compile if the types are invalid (tsc or eslint will fail), so I can't get them published to generate the types (circular problem)
  • The Inngest main class requires providing types in EventSchemas.
  • The fromUnion and fromRecord require hand-crafted types, which is what I'm forced to do for now.
  • fromZod would be useful, however it consumes a map of string->ZodObject, so I can't just go with zod definitions.
  • fromZod would be VERY useful, if it would actually perform runtime validation, but it sadly doesn't.
  • the Inngest.eventSchemas is expecting the types for functions, which means that the file requires importing all the functions' files. One could put the types separate from the functions (instead of co-locating) but then it's harder to reason and maintain the functions.
  • Even with outright manually crafted function types, TS still gets confused (doesn't discriminate the union) -- see the validated function example below

Here's an example of how I worked around those limitations in my project:

Functions index

inngest/functions/index.ts

import { NonRetriableError } from "inngest";
import { type z } from "zod";
import { inngest } from "../client";

// I have to import all the functions and their types
import {
  crmDealStageChange,
  type crmDealStageChange,
} from "./crm/deal.stage.change";
import {
  inngestFunctionFailed,
  type InngestFunctionFailed,
} from "./inngest/function.failed";
import { healthPing, type HealthPing } from "./health/ping";
import { healthPong, type HealthPong } from "./health/pong";

// This is consumed by api/inngest/route.ts
export const allFunctions = [
  crmDealStageChange,
  inngestFunctionFailed,
  healthPing,
  healthPong,
];

// This is consumed by the client
export type AllFunctions =
  | CrmDealStageChange
  | InngestFunctionFailed
  | HealthPing
  | HealthPong;


// This is a wrapper to create a validated function. I found it 
// easier to maintain and control than a middleware
type CreateFunctionArgs = Parameters<typeof inngest.createFunction>;
export function createValidatedFunction<
  T extends z.AnyZodObject = z.AnyZodObject
>(
  validator: T,
  opts: CreateFunctionArgs[0],
  trigger: CreateFunctionArgs[1],
  handler: CreateFunctionArgs[2]
) {
  return inngest.createFunction(opts, trigger, async (handlerArgs) => {
    // let data: ReturnType<typeof validator.parse>;
    const data = validator.safeParse(handlerArgs.event.data);
    if (!data.success) {
      const errorsString = data.error.errors
        .map((e) => `${e.path.join(".")}: ${e.message}`)
        .join(", ");
      throw new NonRetriableError(`Invalid input - ${errorsString}`);
    }

    handlerArgs.event.data = data.data as never;
    return await handler(handlerArgs);
  });
}

The client

inngest/client.ts

import { EventSchemas, Inngest } from "inngest";
import { logger } from "../utils/logger";
import { type AllFunctions } from "./functions";
import { env, envName, isDev, isStaging } from "../env.mjs";

export const inngest = new Inngest({
  name: "our system",
  // inngest doesn't support manually adding envs, so we have to override here
  env: ["production", "staging", "development"].includes(envName)
    ? envName
    : env.VERCEL_GIT_COMMIT_REF,
  logger,

  // This is how we pull in the types
  schemas: new EventSchemas().fromUnion<AllFunctions>(),
});

Example non-validated function:

inngest/functions/health/ping.ts

import { inngest } from "inngest/client";

export type HealthPing = {
  name: "health/ping";
  data: {
    id: number;
  };
};

export const healthPing = inngest.createFunction(
  { name: "Ping" },
  { event: "health/ping" },
  async ({ step, event }) => {
    await step.sendEvent({
      name: "health/pong",
      data: {
        id: event.data.id,
      },
    });
    return true;
  }
);

Example validated function

inngest/functions/crm/deal.stage.change.ts

import { z } from "zod";
import { createValidatedFunction } from "..";

const input = z.object({
  dealId: z.string(),
  orderStatus: z.enum('ACTIVE', 'DELETED'),
});

export type CrmDealStageChange = {
  name: "crm/deal.stage.change";
  data: z.infer<typeof input>;
};

export const crmDealStageChange = createValidatedFunction(
  input,
  { name: "CRM deal stage changed" },
  { event: "crm/deal.stage.change" },
  async ({ event, step }) => {
    // This was quite disappointing - TS can't discriminate the union from the 
    // function name, so this cast is necessary, otherwise TS fails with
   //   "ts2339: property dealId doesn't exist on type ..."
    const { dealId, orderStatus } =
      event.data as CrmDealStageChange["data"];

    return await step.run("Upserting orders from deal", async () => {
      // do some work
    });
  }
);

Some thoughts

  • I understand the need to separate consumers from functions.
  • I've tried to solve it in a few ways, but the biggest issue was circular references due to how inference is set up in inngest createFunction and other places.
  • I think it'd be more ergonomic if the types were derived from the function itself, in dev.
  • Code-gen is fine with me... we use it heaps with stuff like ts-gql, prisma etc.
  • I don't mind writing a function, then running a cli to generate types based on schema, or vice-versa.
  • There is a chance that I'm completely missing the plot here, and my source-code scanning skills failed me ... if I'm doing it wrong, please shine some light on what the recommended project structure is? (before the docs cover it)
  • Thanks for reading! 💚

[BUG] thread is blocked when inngest run function on REMIX

www_screencapture_com_2023-3-22_01_34.mp4

Hello,

I run a test function but when the function is called I cannot visit my page after the function is finish. Do I have understand inngest wrong or what I am doing wrong ?

I named the function pdfCreate but I only run a for loop with numbers.

testApi.ts

export default aync function Hello() {
  return (
    <p>Hello</p>
  )
}

export async function loader() {
  const test = await inngest.send({
    name: "app/pdf.download",
    data: {
      email: '[email protected]'
    },
    user: {
      id: '1'
    }
  });
  console.log(test);
  return json({});
}

pdfCreate:

export default function PDFCreate() {
  let total = 0;

  for(let i = 0; i < 10_000_000_000; i++) {
    total++;
  }
}

Inngest function

export default inngest.createFunction(
  { name: "Download PDF File" },
  { event: "app/pdf.download" }, // The event that will trigger this function

  // This function will be called every time an event payload is received
  async ({ event }) => {
    const d = await PDFCreate();
    return d;
  }
);

[BUG]: Old Secrets Link Used in Types documentation comments

Describe the bug

Related to 164fd5c

See: "@link https://app.inngest.com/secrets}" is still old link

export interface RegisterOptions {
  /**
   * A key used to sign requests to and from Inngest in order to prove that the
   * source is legitimate.
   *
   * You must provide a signing key to communicate securely with Inngest. If
   * your key is not provided here, we'll try to retrieve it from the
   * `INNGEST_SIGNING_KEY` environment variable.
   *
   * You can retrieve your signing key from the Inngest UI inside the "Secrets"
   * section at {@link https://app.inngest.com/secrets}. We highly recommend
   * that you add this to your platform's available environment variables as
   * `INNGEST_SIGNING_KEY`.
   *
   * If no key can be found, you will not be able to register your functions or
   * receive events from Inngest.
   */
  signingKey?: string;

Should be:

"https://app.inngest.com/env/production/manage/signing-key"

Expected behavior

The links should match.

The dev server was updated, but the code docs still have old link.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.