Http Outbound request's header Host value comes with port number. about spin HOT 5 OPEN

@lann I suspect this is where the problem is:

from spin.

I'm not sure where this behavior is coming from but in general it should be fine to include a default port number in an http host header.

If you manually set the host header on the request with just the hostname does it get replaced?

from spin.

If I force to add my own host header value with this: .header("Host", "seungjin.s3.amazonaws.com"),
I got a following message from Amzaon.
It is not overwriting but sending duplicated host header value:

<?xml version="1.0" encoding="UTF-8"?><Error><Code>DuplicateHeaderName</Code><Message>Your request contains duplicate headers.</Message>

from spin.

Okay. In this case, I am signing its request with the Host header value. Having a port or not is a BIG difference. The HTTP spec says it is okay to have it but also okay not to have it. But that's probably an old standard nobody needed to think about when signing a request with the hostname.

I sign the request with just the hostname (without the port), but AWS is using its host with the port because that's what AWS is getting. So the verification never happens. (Spin ships with a port, but curl and my Rust code with reqwest do not ship it.) To solve this for now (temporarily...), I also signed with the port number (forcefully adding {host}:{port}) and sent it to AWS. Of course, verification passed, and my image was up online with the trick.

The AWS API is a simpler case because my target is only one (AWS). But when I deal with Mastodon, it is not that simple. Some webservers (proxy, nginx, caddy, apache...) pass the port, and some don't. That's a configuration matter.

Anyway, I found a way I can work around it, but I don't think it is a permanent solution. Based on my search, Go/Java SDK developers have encountered similar cases.

I will get back and bash later when this issue becomes more critical. :-) Please don't close it until we clearly know what's going on. :-)

from spin.

Thanks for the update.

It is not overwriting but sending duplicated host header value:

This I think may be a bug, but will require some research to understand why its happening and how best to fix it.

from spin.