Block outgoing connections other than to ports 80 & 443 about morph HOT 15 OPEN

@henare I've assigned this ticket to you because this should now be pretty easy for you to do. If you look at https://github.com/openaustralia/morph/blob/master/provisioning/roles/morph-app/files/iptables.rules that sets up the iptables rules for forwarding outgoing docker traffic destined for http(s) to mitmproxy.

I'm thinking it's a matter of adding the correct rules here.

I think we want to block all outgoing tcp/udp traffic except for http(s) and DNS and responses from existing incoming connections. Is there anything else?

from morph.

I think you've covered all we need in that list.

I'm holding off this until we've resolved #328 and re-enabled the firewall rules.

from morph.

@henare is this something that you might be able to pick up again at some stage now that #328 looks to be fixed?

from morph.

Reopening this because of #499 (comment)

from morph.

@henare is this something you will have a chance to look at soon or shall I have a go at fixing it?

from morph.

@mlandauer I was planning to work on it this afternoon but it could use a different set of eyes if you're up for it?

from morph.

@henare I really don't mind either way. Whatever suits you. I, of course, would be happy if you took this on and I'm also happy to have a crack at it myself.

from morph.

I'm taking this on. Here's the timeline as I understand it:

• Merged changes on Friday at 1
• Deployed changes on Friday at 1:30
• Matthew and Henare checked changes seemed to be doing what they should
• Overnight on Friday scrapers started to fail because of timeouts
• On Saturday Matthew updated packages and restarted the server. Docker did not start on boot and had to be started manually. The firewall rules were not in place after the restart.
• On Sunday scrapers again failed
• On Monday morning Matthew discovered that mitmproxy had hung and restarted it
• On Monday afternoon Matthew found scrapers failing again. He restarted the server and Docker did not start. Disabling the firewall rules allowed Docker to start but there were still errors. Eventually clearing out /var/lib/docker and pulling down base images was what fixed the problem.

from morph.

I can reproduce the issue of the Docker service not starting when the firewall rules are enabled.

It seems we're not alone in having issues getting iptables set up at boot with Docker.

from morph.

Sounds like excellent progress :)
On 30 Mar 2015 18:29, "Henare Degan" [email protected] wrote:

I can reproduce the issue of the Docker service not starting when the
firewall rules are enabled.

It seems we're not alone
https://forums.docker.com/t/docker-and-iptables-configuration-startup/904
in having issues getting iptables set up at boot with Docker.

—
Reply to this email directly or view it on GitHub
#19 (comment).

from morph.

Docker is started after network interfaces come up:

start on (local-filesystems and net-device-up IFACE!=lo)

And our iptables rules are currently applied post-up so I think it's safe to say that the iptables rules get applied before Docker starts and therefore before Docker has had the chance to created the DOCKER iptables chain.

There are two ways I can currently think of resolving this:

1. find a different technique of managing this traffic (i.e. not using the DOCKER chain)
2. working out a clever way of applying the rules at a different time, e.g. after the Docker service starts or as part of the Morph.io application starting

I'm looking at 2. right now.

from morph.

Is there any reason that you can't apply the rules in a similar way to the redirect rules for mitmproxy? They're just using the interface that docker sets up rather than the filter chain.

from morph.

Is there any reason that you can't apply the rules in a similar way to the redirect rules for mitmproxy? They're just using the interface that docker sets up rather than the filter chain.

That's what I tried initially but I couldn't for the life of my get it working. Then I noticed that the first rule in the default FORWARD chain sends it to the DOCKER chain so it seemed like The Right Way of doing things.

I'm feeling really stuck right now so I'm going to put this aside.

from morph.

Cool, thought you would have tried that first. No point banging heads over this one.

from morph.

Okay. I think I'm going to put this on hold for a bit. Trying to understand how docker is using iptables. It's not completely clear to me how things are supposed to work. With a little break I'm sure I can come back to this and sort it out

from morph.