cisco-open / camblet-driver Goto Github PK

Problem Statement

By default, the driver is designed to be compiled with GCC, as is typical for most Linux kernels. However, in some cases, the kernel may be compiled with clang. In such instances, we should ensure that our driver can also be compiled using clang. It's generally recommended to avoid using a different compiler than the one used for the kernel, but providing compatibility with clang ensures flexibility in various environments.

Proposed Solution

When the kernel is compiled with clang, our driver can be compiled just as easily by executing the following commands:

sudo apt install -y clang llvm lld
make LLVM=1

If you need to cross-compile the driver, ensure you have the necessary dependencies installed:

sudo apt install -y clang llvm lld

Additionally, you'll need to update the build commands inside both the Makefile and the BearSSL Kbuild file to reflect the use of clang. Replace the existing build command with the following:

$(MAKE) -C $(KBUILD) M=$(PWD) V=$(VERBOSE) CC=$(CLANG) CONFIG_CC_IS_CLANG=y CONFIG_FTRACE_MCOUNT_USE_CC='' CONFIG_RETHUNK='' CONFIG_CC_IS_GCC='' modules
```
At the end:
```
make LLVM=1
```

Problem Statement

Please describe the problem to be addressed by the proposed feature.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Detect if the traffic is already TLS encrypted on the user space, and except on a handshake (and policy evaluation) don't do anything else (e.g.: double record encryption can be skipped entirely, which is a huge performance win).

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Description

Most memory allocation calls aren't checked to see if the memory allocation fails, potentially leading to null pointer dereferences throughout the codebase.

Expected Behavior

Every time kzalloc() and other functions are called, the return value needs to be checked against NULL to ensure that memory allocation was successful, and errors need to be handled both locally and up the stack.

Actual Behavior

Memory allocation is only very rarely checked to see if it succeeds. This can lead to kernel oops at the very least, leading to inconsistent kernel states, or kernel panics.

Affected Version

0.6.0, and probably others

Steps to Reproduce

https://github.com/search?q=repo%3Acisco-open/camblet-driver%20kzalloc&type=code

Checklist

• I have read the contributing guidelines
• I have verified this does not duplicate an existing issue

With PR #205 the connect uses the camblet_poll for polling, we should do the same with the accept side as well.

Problem Statement

BearSSL received it's last commit on 2023 02 so we can say it is no longer maintained.
We need features like tls 1.3, dtls support, arm64 optimization, maintained project, CSR generation.
It seems these features may not be available.

Proposed Solution

Replace bearSSL with a more mature library WolfSSL.

Alternatives Considered

• s2n-tls(https://github.com/aws/s2n-tls)
• Tempesta TLS(https://github.com/tempesta-tech/tempesta/tree/master/tls)
• Kernel crypto API

Also create a few other test cases:

• using a go http client (heavily concurrent)
• TLS passthrough
• one using BearSSL only (disable kTLS)
• ...

According to the wolfSSL documentation Secure Enclave could be leveraged https://www.wolfssl.com/difference-hsm-tpm-secure-enclave-secure-element-hardware-root-trust/

Problem Statement

https://docs.kernel.org/networking/tls.html#send-tls-control-messages

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Currently only the ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 is the supported cipher with kTLS.
The reason for this is partly bearSSL.
With other ciphers, we are unable to determine the required fields to set all necessary information for kTLS.

Proposed Solution

Either try to find out how to gather the related information from bearSSL or since we are changing the SSL library to WolfSSL we can wait until the support.

Additional Context

bearSSL seems abandoned.

Problem Statement

Only RSA is supported currently.

Proposed Solution

ECDSA should be supported as well.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Please describe the problem to be addressed by the proposed feature.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Zerocopy TLS is implemented in #49 but it is not covering non-kTLS connections, for example where the sender is using BearSSL.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

This problem doesn't affect Kernel version 6.5 and above, since the sendpage method has been removed and now is replaced by sendmsg which is already covered.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

https://elixir.bootlin.com/linux/latest/source/net/ipv6/tcp_ipv6.c#L2136

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Please describe the problem to be addressed by the proposed feature.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Google Cloud Platform (GCP) defaults to using Container-Optimized OS (COS), which features a kernel compiled with clang and lacks a package manager. Additionally, according to its documentation, COS does not permit the installation of third-party kernel modules.

To compile the driver within a Docker container using the running COS version, the following commands are necessary:

Docker container with the proper rights:

docker run --name ubuntu --rm --privileged --cap-add=ALL -it -v /lib/modules:/lib/modules ubuntu

Requirements:

apt  install -y clang llvm lld bison flex libssl-dev libelf-dev bc dwarves
# To get the build ID: 
cat /etc/os-release
mkdir /root/cos
cd /root/cos
mkdir /root/cos/kernel-src
mkdir /root/cos/kernel-headers
curl -sL https://storage.googleapis.com/cos-tools/$BUILD_ID/kernel-src.tar.gz --output kernel-src.tar.gz
tar -xf kernel-src.tar.gz -C /root/cos/kernel-src
curl -sL https://storage.googleapis.com/cos-tools/$BUILD_ID/kernel-headers.tgz --output kernel-headers.tgz
tar -xf kernel-headers.tgz -C /root/cos/kernel-headers
# Generate config for the kernel to build
mkdir -p /root/cos/kernel-src/build-base
cp /root/cos/kernel-headers/usr/src/linux-headers-*/.config /root/cos/kernel-src/build-base/
cd /root/cos/kernel-src
make LLVM=1  lakitu_defconfig O=build-base

Make the kernel-headers:

make LLVM=1 O=build-base -j$(nproc) headers 

Problem Statement

We need to decide which connections to intercept with our filters, we might need to use OPA here.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Please describe the problem to be addressed by the proposed feature.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

It would be ideal to develop a generic build platform capable of building for the most common Linux distributions and kernel versions. This approach would enable us to build the driver locally only when a build does not already exist in our repository.

Problem Statement

Create a self-contained docker image with compiler and headers and things that can build and install the kernel module on a system the most frictionless way on machines that have a container runtime installed with a single command.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Currently the agent assigns the workload certificates by signing the workload CSRs coming from the kernel.

Proposed Solution

A better solution would be to move that functionality into the kernel and the agent will only be responsible the sign the intermediate CA CSRs.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Please describe the problem to be addressed by the proposed feature.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Description

wget hangs when hitting an mTLS policy covered connection when using BearSSL transport (when file-server is the server, with python3 -m http.server), on Linux 5.15:

wget http://localhost:8000 -O -
--2024-03-05 11:08:27--  http://localhost:8000/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 615 [text/html]
Saving to: ‘STDOUT’

-                                                0%[                                                                                                    ]       0  --.-KB/s  

Expected Behavior

It should exit with 0.

Actual Behavior

Please describe what happened instead.

Affected Version

Only with ktls_available=0

Steps to Reproduce

1. First step
2. Second step
3. etc.

Checklist

• I have read the contributing guidelines
• I have verified this does not duplicate an existing issue

Problem Statement

Otherwise the clients couldn't detect socket close events for example.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

https://github.com/spiffe/spiffe/blob/main/standards/JWT-SVID.md

Problem Statement

Curl vs Nginx setup, after nginx sends back the http response headers it wants to sendfile the index.html, on the the curl side this happens after this, when it tries to read it:

ppoll([{fd=5, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, {tv_sec=0, tv_nsec=0}, NULL, 0) = 1 ([{fd=5, revents=POLLIN|POLLERR|POLLRDNORM}], left {tv_sec=0, tv_nsec=0})
recvfrom(5, 0xaaaaf7346650, 615, 0, NULL, NULL) = -1 EMSGSIZE (Message too long)

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

As an alternative we can turn off sendfile in Nginx until it gets resolved, with sendfile off;.

Additional Context

The issue probably lies there that Nginx sendfile doesn't encrypt the file content when using the module:

T 127.0.0.1:8000 -> 127.0.0.1:53272 [AP] #19
  ..........8.Z.?.o..K...sl..;...cA~......*...\..$....Z..o.b.Rd.(3.oX..D..e..tgx..l...=....^..c..Y.Y...X..5.......u.}....n.A(...}..7Y.F......x..]..*MR).....YR.=.N'p..b..23.PWu.g..]...<
  ~b.*\@CSK.lLm.*...iL.m..)..`x}).+..M..._.cp,6q..W...S..W%Fn.,..?.N....M._R......Du....u.a..'^.R.O..+..o./z.....=.............6>[email protected].........!A.FG...O
  ...;.e~..(..bZ...*o.\u-6.@N.}..^...?....|..KWf..ye.......i....aK......"...f.yr+&...._....e..%.XH%..x...\..#.^......=.KO..:.G...Q=2y4HH..MBL..s.^....*....vf.<s._0.[..(..\.......^.ds..
  B.h.....H..O..$............}t..Zs..3wM.....z.x[x].I.R#...".L+......|..p.o1..lU..{.?-...^.lr....C]}.g.qP.Hx.^y..'..e.t........h........`b(..."..4..].%..!...K..Z%.G<!DOCTYPE html>.<htm
  l>.<head>.<title>Welcome to nginx!</title>.<style>.html { color-scheme: light dark; }.body { width: 35em; margin: 0 auto;.font-family: Tahoma, Verdana, Arial, sans-serif; }.</style>.
  </head>.<body>.<h1>Welcome to nginx!</h1>.<p>If you see this page, the nginx web server is successfully installed and.working. Further configuration is required.</p>..<p>For online d
  ocumentation and support please refer to.<a href="http://nginx.org/">nginx.org</a>.<br/>.Commercial support is available at.<a href="http://nginx.com/">nginx.com</a>.</p>..<p><em>Tha
  nk you for using nginx.</em></p>.</body>.</html>.                                                                                                                                     
#^Cexit

The issue is that sendpage is not covered.

Problem Statement

Please describe the problem to be addressed by the proposed feature.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Description

The test named "Test downloading a bigger file" fails on ubuntu 20.04 with kernel 5.15.

See run: https://github.com/cisco-open/camblet-driver/actions/runs/8186473439/job/22385010335

Expected Behavior

Please describe what you expected would happen.

Actual Behavior

Please describe what happened instead.

Affected Version

Please provide the version number where this issue was encountered.

Steps to Reproduce

1. First step
2. Second step
3. etc.

Checklist

• I have read the contributing guidelines
• I have verified this does not duplicate an existing issue

Problem Statement

Please describe the problem to be addressed by the proposed feature.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Investigate whether Unix Domain Sockets can be used for communication between the Driver and the Agent. Currently, a character device is being utilized.

Proposed Solution

Replace the current implementation with Unix Domain Socket

Alternatives Considered

Via TCP.

Problem Statement

Now the code generates a new certificate for every connection. It is inefficient and unnecessary. We should cache these certificates based on SAN fields and expiration time.

Proposed Solution

Create a linked list or hashmap that stores these certificates. Reissue certs only if expired.

Problem Statement

CSR gen module now statically uses parameters to generate CSR request, we must introduce new parameters to overwrite this.

Proposed Solution

Introduce new parameters to the csr gen module:
subject, dns, uri, email, ips.

Problem Statement

It would be nice to analyze application level traffic and authenticate, make decisions based on that.

Proposed Solution

A nice and simple (compatible license, 2 plain C files) HTTP parser is https://github.com/h2o/picohttpparser

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Investigate how can we support Mac Os within our driver.

Thanks for your great talk here at KubeCon!
eBPF has some programming restrictions that makes it safe for kernel use cases (eg no infinite loops).

if I got it correctly the eBPF module calls into a wasm module via a kfunc. Is this a blocking process? Could the wasm module potentially execute an infinite loop and circumvent the security restrictions for eBPF Modules?

Problem Statement

Please describe the problem to be addressed by the proposed feature.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Please describe the problem to be addressed by the proposed feature.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

• Trust bundle for federation
• Different trust domains necessitate different CA certificates
• Ability to accept other trust stores, e.g., J Corp accepts G Corp.
• Standardized method for connecting trust domains

Problem Statement

The driver should parse and load the common CA certificate bundles of OS distributions, so we can validate certificates from well-known signers.

Proposed Solution

Read and parse https://packages.ubuntu.com/lunar/all/ca-certificates/filelist

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Enable the retrieval of data from the opposite side using socket options. Valuable data includes the SPIFFE ID.

Problem Statement

Currently the filter return values are not handled, this means we are in Continue always mode.

Proposed Solution

Please describe what you envision the solution to this problem would look like.

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Like in #50 we use Bash in CI to test such features as big file upload/download, but we should create a programmable test framework, which is capable of:

• building the kernel module
• loading/unloading the kernel module
• control the kernel module
• load wasm modules (proxywasm, rego, etc.)
• start test severs/clients and make interactions with them to test features and correctness

Proposed Solution

bats-core

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

From performance and maintainability point of view this is a must.

Proposed Solution

Check out https://github.com/lxin/tls_hs

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

Problem Statement

Creating multiple policies becomes necessary when multiple users require identical access rights to a service. This process is not only error-prone but can also be tiresome

Proposed Solution

Introduce templating functionality to streamline the creation of policies.

E.g.:

- selectors:
  - process:uid: [501, 1001]
    process:name: [curl, wrk]
    destination:port: [8000, 8080]
  certificate:
    workloadID: curl
  egress:
  - selectors:
    - app:label: traefik
    certificate:
      workloadID: specific-workload-id/[[process:uid]]/[[process:name]]

Description

[Wed Dec 13 10:43:00 2023] ================================================================================
[Wed Dec 13 10:43:00 2023] UBSAN: array-index-out-of-bounds in /Users/nandork/Code/src/github.com/cisco-open/nasp-kernel-module/third-party/wasm3/source/m3_code.c:102:5
[Wed Dec 13 10:43:00 2023] index 2 is out of range for type 'void *[1]'

Expected Behavior

We shouldn't get this message, no freezes.

Actual Behavior

Please describe what happened instead.

Affected Version

Please provide the version number where this issue was encountered.

Steps to Reproduce

1. First step
2. Second step
3. etc.

Checklist

• I have read the contributing guidelines
• I have verified this does not duplicate an existing issue