ERR_HTTP2_HEADER_SINGLE_VALUE & Certificate signed by unknown authority about secret-agent HOT 3 CLOSED

Hi @dmzebrov. Thanks for the report and feedback!

I can replicate these. I have a fix for the first one.

The second one has a workaround you can use, but I need to dig in to see why Chrome and Firefox are ok with this certificate authority, but the Go runtime (which is what makes the TLS connection) does not...

To get around Error 2, add MITM_ALLOW_INSECURE=true to your env (either via nodejs code: process.env.MITM_ALLOW_INSECURE='true'; or via cli: MITM_ALLOW_INSECURE=true node code/street-beat.ru). Please note though, this isn't an official flag, just to get you through for now!

from secret-agent.

Cool! The fix for error 1 will be in the next release. Should be mid-week.

I'm not actually sure I can fix Error 2 right now without patching Go. It appears to be a Mac issue they're still working through, and in particular there's a Commodo certificate authority that had a cert expire in may. No one was quite sure how to handle it. It sounds like one problem people run into is the expired authority remaining in your Mac keychain. When Go verifies the new certificate, it gets tripped up with the expired cert (at least, that's what I could make of the Go issues I was poking through).

For now, we're going to simply turn off certificate validation as the default setting. It's less valuable for scraping than when you're putting your sensitive information into a website while browsing. We'll wait for Go to resolve this internally and toggle back on. The net result is likely not noticeable for scraping.

from secret-agent.

That's worked, thanks for the quick response! Will fix for the Error 1 be included in the next release? I would like to help you with fixing Error 2, but I don't know Go at all, so I hope for you ☺️

from secret-agent.