This repository contains two components
- API Server proxy sidecar
- API Server proxy pod webhook
This component consists of apiserver-proxy-sidecar which runs on every Node in a Shoot cluster.
It does the following:
-
adds the IP Address (
--ip-addressflag) to the loopback interface (--interfaceflag). -
[optionally] executes the following
iptablesrules (e.g with--ip-address=10.96.0.2):-A PREROUTING -t raw -d 10.96.0.2/32 -p tcp -m tcp --dport 443 -j NOTRACK -A OUTPUT -t raw -d 10.96.0.2/32 -p tcp -m tcp --dport 443 -j NOTRACK -A OUTPUT -t raw -s 10.96.0.2/32 -p tcp -m tcp --sport 443 -j NOTRACK -A INPUT -t filter -d 10.96.0.2/32 -p tcp -m tcp --dport 443 -j ACCEPT -A OUTPUT -t filter -s 10.96.0.2/32 -p tcp -m tcp --sport 443 -j ACCEPTThose rules allow traffic to this IP address and disable conntrack as the IP address is local to the machine.
-
Every 1 min repeats the process and starts from
1.
After this, the actual apiserver-proxy can listen on this IP address (10.96.0.2) and send traffic to the correct kube-apiserver.
The implementation of that proxy is fully transparent and can be replaced at any given moment without any modifications to the apiserver-proxy-sidecar.
bazel run //cmd/apiserver-proxy-sidecar -- --help
--add_dir_header If true, adds the file directory to the header
--alsologtostderr log to standard error as well as files
--cleanup [optional] indicates whether created interface and iptables should be removed on exit.
--daemon [optional] indicates if the sidecar should run as a daemon (default true)
--interface string [optional] name of the interface to add address to. (default "lo")
--ip-address string ip-address on which the proxy is listening.
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
--log_dir string If non-empty, write log files in this directory
--log_file string If non-empty, use this log file
--log_file_max_size uint Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
--logtostderr log to standard error instead of files (default true)
--port string [optional] port on which the proxy is listening. (default "443")
--setup-iptables [optional] indicates whether iptables rules should be setup.
--skip_headers If true, avoid header prefixes in the log messages
--skip_log_headers If true, avoid headers when opening log files
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
--sync-interval duration [optional] interval to check for iptables rules. (default 1m0s)
-v, --v Level number for the log level verbosity
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered loggingThe API Server proxy pod webhook server is a simple mutating admission webhook which adds the Fully Qualified Domain Name of the Kube API Server to Pods as environment variable KUBERNETES_SERVICE_HOST. The value is set with --apiserver-fqdn flag.
Test it with:
bazel run //cmd/apiserver-proxy-pod-webhook -- --apiserver-fqdn=foo.bar. --cert-dir ${PWD}/internal/admission/testdataAnd in another terminal:
curl -k -XPOST --silent \
-H "Content-Type: application/json" \
-d "@internal/admission/testdata/admission.json" \
-H "Accept: application/json" \
https://localhost:9443/webhook/pod-apiserver-env | jq -r '.response.patch' | base64 -d | jq -r '.'Output:
[
{
"op": "add",
"path": "/spec/initContainers/0/env",
"value": [
{
"name": "KUBERNETES_SERVICE_HOST",
"value": "foo.bar."
}
]
},
{
"op": "add",
"path": "/spec/containers/0/env",
"value": [
{
"name": "KUBERNETES_SERVICE_HOST",
"value": "foo.bar."
}
]
}
]bazel run //cmd/apiserver-proxy-pod-webhook -- --help
--apiserver-fqdn string apiserver-fqdn is the fully qualified domain name of the Kube-API Server e.g. example.com.
--cert-dir string cert-dir is the directory that contains the server key and certificate. The server key and certificate.
--cert-name string [optional] cert-name is the server certificate name. (default "tls.crt")
--client-ca-name string [optional] client-ca-name is the CA certificate name which server used to verify remote(client)'s certificate. Defaults to "", which means server does not verify client's certificate.
--host string [optional] host is the address that the server will listen on. Defaults to "" - all addresses.
--key-name string [optional] key-name is the server key name. (default "tls.key")
--port int [optional] port is the port number that the server will serve. (default 9443)bazel is used for building and testing. Optional Dockerfile is provided.
go mod tidy
go mod vendor
bazel run //:gazellebazel run --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 //cmd/apiserver-proxy-sidecar:go_image -- --norun
bazel run --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 //cmd/apiserver-proxy-pod-webhook:go_image -- --norunNote: It requires a running Docker.
bazel test "//..."
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent