CloudFront returns "null" for dynamic routes in Node 20 and NextJs about ion HOT 23 CLOSED

Could you try enabling streaming in open-next.config.ts, it seems that there is an issue with next not properly awaiting for the end before returning.
It would help if you could provide a sample repro of this issue as well.

// This is an example `open-next.config.ts` with streaming
import type { OpenNextConfig } from 'open-next/types/open-next.d.ts';

const config = {
  default: {
    override: {
      wrapper: "aws-lambda-streaming"
    }
  },
} satisfies OpenNextConfig;

export default config;
export type Config = typeof config;

from ion.

@sethcarlton If you still have issues with the middleware in V3 you can open a new issue directly in the open-next repo.

@Hazmatyre At the moment there is no place to check for this apart from a thread in the discord. I'll start posting releases directly on the open-next channel in the sst discord.

@fwang I think you can go back to Node 20 for the construct and set the OpenNext version to 3.0.0-rc.11. The issue can probably be closed as well.

from ion.

@sethcarlton can you also try using Node.js 18 in v0.0.188, and see if that helps?

from ion.

@conico974 this works! I copied your config into my project root.

For context, I setup an existing NextJS project with Ion v0.0.189 and considering all my routes were dynamic, all my pages on sst deploy were returning null. Dev mode works fine.

from ion.

0.0.189, Node 20, No Streaming > returns null
0.0.189, Node 18, No Streaming > works as expected
0.0.189, Node 20, With Streaming > no longer shows null, but the behavior is odd. It seems like the page server component is throwing an exception before middleware runs, but this might be NextAuth working poorly. I've been having some issues with it.

from ion.

Temporarily reverted the server function to use Node 18 until this is resolved.

@sethcarlton Can you give 0.0.194 a try?

from ion.

@fwang

When on 0.0.193, the issue is happening when I have a conditional component linked inside a layout file. Removing that component gets rid of the issue, however, I still see the following in the logs no matter how simple the middleware is in Node 20:

Routing failed. TypeError: Cannot set property crypto of #<Object> which has only a getter
    at file:///var/task/middleware.mjs:18:2060

When moving to 0.0.194, initially the issue, but another exception was being thrown in the middleware and it was masking it. When I simplified the middleware down to the minimum, I am getting the "null" response (but there is another error in the logs)

I know this isn't necessarily concrete or helpful info yet, but it seems to be some combination of middleware exceptions, dynamic components, and potentially streaming. In any one of my tests that had issues there was an error in middleware.mjs so that seems to be the ultimate culprit.

Still working on getting a stripped down example to reproduce it, maybe @Hazmatyre can share theirs?

from ion.

@sethcarlton
Pretty sure that it has nothing to do with the middleware, this is a different issue and it should be fixed in [email protected] and node 20.

I haven't been able to reproduce the issue exactly (but i've managed to recreate a kind of similar issue), so everything that i say here must be taken with a grain of salt.

From what i've understood, it looks like the issue might be in node itself. Next use some detached promise to await for the finish event of the response.
In node 20 it seems that on this finish event, node is silently crashing. Everything is under a try catch so it should have returned an Internal server error and print some logs on the server but it doesn't seem to be the case.
If node is silently crashing it explains everything that's happening here.

There is a bunch of similar issue in the node repo itself.

from ion.

@conico974 I'd buy that, thanks. I was just coming here to say that the middleware was a red herring. Also, my issue only occurs when navigating to a problem page directly. If it gets pre rendered by next and navigated to via a Link then it works fine.

from ion.

@conico974 Updating my findings since I'm facing the same issue.

Version: 0.0.194

I found that when using fetch to fetch data on the server there are no issues with returning null.

async function getData() {
  const res = await fetch('https://api.example.com/...')
  // The return value is *not* serialized
  // You can return Date, Map, Set, etc.
 
  if (!res.ok) {
    // This will activate the closest `error.js` Error Boundary
    throw new Error('Failed to fetch data')
  }
 
  return res.json()
}
 
export default async function Page() {
  const data = await getData()
 
  return <main></main>
}

If I do the same thing using a query from my drizzle database like below. The cloud front url returns null. Note this is only happens on refresh, If I route using to a page that has sever side fetching with data api there are no issue.

export const db = drizzle(new RDSDataClient({}), {
  database: Resource.MyDatabase.database,
  secretArn: Resource.MyDatabase.secretArn,
  resourceArn: Resource.MyDatabase.clusterArn,
})

from ion.

I tested with [email protected] and Node 20 and still have the same issue unfortunately

from ion.

Update:

Being on 0.0.194 and adding the open-next.config.ts mentioned above resolved the issue for me

Note: Not recommended to use streaming in production.
https://open-next.js.org/inner_workings/streaming

from ion.

Not recommended to use streaming in production

This is for V2, for V3 it is safe to use

from ion.

Tossing some info here before I'll be busy for a couple days. The issue occurs in both Node 18 and 20 for me so I think the default can probably be set back to 20? I currently have other issues when enabling streaming so that hasn't been a resolution for me (i'm using [email protected])

I can get around the issue at times but it's hard to predict. Here are some scenarios that appear to cause it in my case (on page refresh only):

• I had a client component consuming a component which used an inline 'use server' action. No build errors, but caused the runtime null
• Fetching an avatar image in a nested component as part of a layout
• If a page fails to find a valid route when prefetching

from ion.

@sethcarlton What are your issues with streaming?

from ion.

I've finally managed to reproduce the issue by using a client component consuming a server component as a child which used an inline use server wrapped inside a <Suspense>. Without the suspense it was working just fine.

Anyway i've found a fix for it in [email protected], hopefully it fixes things for all of you. I've tested it with my reproduction both in node 18 or 20.

@sethcarlton @danielstevenberger @Hazmatyre If you could give it a try, so that we can confirm that it is fixed

from ion.

I've made a silly mistake, i forgot to check the status code, everything returned 500 even though it was properly returning the html.

It's fixed in [email protected]

from ion.

@conico974 just tried with [email protected] and it works! I still have streaming disabled. When I try to re-enable it again I'll let you know if I run into any issues.

The scenario you described definitely sounds like one of the places where I was having issues. Thank you for the fix.

from ion.

Thanks @conico974 I've confirmed it's resolved as well in [email protected]

from ion.

@conico974 as for my streaming issues, they appear to just be with next-auth (and probably unrelated to this issue as a whole). It has been giving me fits and doesn't feel stable, so I'm just going to migrate to Lucia or SST Auth.

Login/cookies don't work appropriately with streaming, it's not sending the PKCE code_verifier cookie properly.

And when I'm already logged in, if I have next-auth in middleware I get this for certain routes that deal with auth

Routing failed. TypeError: RequestInit: duplex option is required when sending a body.

from ion.

@conico974 thanks buddy, all working with rc11.

Added openNextVersion: "3.0.0-rc.11" to my SST config and removed the open-next config with the override.

Really silly question, but where do I check for OpenNext release candidate updates? I can't find it under tags or releases.

from ion.

@fwang tell me when you upgrade? I need to update the quick start.

from ion.

Release v0.0.226 has the update.

@jayair assigning this back to you.

from ion.