Incorrect "Valid HTTPS" results for amazonaws.com? about pshtt HOT 5 CLOSED

This is intended behavior, so perhaps we should make the README more clear. The README right now says:

A domain has "valid HTTPS" if it responds on port 443 at its canonical hostname with an unexpired valid certificate for the hostname.

The canonical hostname only refers to the canonical endpoint on the original domain. So if you're scanning amazonaws.com, the canonical hostname can only be amazonaws.com or www.amazonaws.com.

One of the goals of measuring Valid HTTPS, for an SLD or any of its subdomains, is to identify "is there anything preventing preloading?" As it stands, if amazonaws.com were preloaded, the redirect to aws.amazon.com would break and not function. So, I believe Valid HTTPS should be false.

from pshtt.

Ah, yes, I was confused by:

Canonical URL - A judgment call based on the observed redirect logic of the domain.

But now I see the difference between "URL" vs "hostname." Maybe just delete "at its canonical hostname" from the "Valid HTTPS" definition?

from pshtt.

It's the other way around - Canonical URL is a judgment call based on the observed internal redirect logic of the domain.

For example.com, the Canonical URL can have one of four values:

• http://example.com
• https://example.com
• http://www.example.com
• https://www.example.com

It can get slightly confusing when talking about canonical endpoints, because there are actually two calculations: which is the canonical hostname (www or the root), and what is the canonical protocol (http: or https:)? The "Canonical URL" is effectively the union of those two calculations.

Sometimes only one of those calculations is relevant for a given field. For example, whether a domain has "Valid HTTPS" means looking at the validity (e.g. open on port 443, cert is valid for the hostname) of either https://example.com or https://www.example.com. So the code identifies the canonical hostname and then examines HTTPS at that hostname.

So it could be the case that "Valid HTTPS" is based on a scan of https://www.example.com, but that http://www.example.com is the "Canonical URL".

Is this making sense? It's definitely more complicated than I was originally imagining when I first wrote some of this logic, but it is designed for extreme resilience to arguments made by people undergoing compliance audits informed by this tool's criteria.

from pshtt.

Looking at it again, I realize I was misunderstanding "Canonical URL." Intuitively I think the canonical URL for amazonaws.com is https://aws.amazon.com. But since Canonical URL can only encompass www/non-www varianets, that's not the Canonical URL and instead it's the domain itself. Maybe this:

Canonical URL - A judgment call based on the observed redirect logic of the domain.
Valid HTTPS - A domain has "valid HTTPS" if it responds on port 443 at its canonical hostname with an unexpired valid certificate for the hostname.

Should say:

Canonical URL - One of the four endpoints described above; a judgment call based on the observed redirect logic of the domain.
Valid HTTPS - A domain has "valid HTTPS" if it responds on port 443 at the hostname in its Canonical URL with an unexpired valid certificate for the hostname. This can be true even if the Canonical URL uses HTTP.

from pshtt.

Those are good edits!

And I can understand why the definition might not feel intuitive. The best way I can frame the rationale is -

• Valid HTTPS - "If I preloaded the domain, would this break?"
• Canonical URL - "If I were to hyperlink example.com, what should the href attribute of the link be?"

from pshtt.