Make it easy to install on the Mac.

Right now when a Handshake response is received (in reply to an Initiation message), a Keep-Alive message is sent. Instead it should try to send a queued message when available (which is what others implementations do).

Relevant code in src/noise/handshake.rs:

            2 => {
                self.log(Verbosity::Debug, "Received handhsake_response");
                let mut handshake = self.handshake.lock();
                match handshake.receive_handshake_response(src) {
                    Ok(session) => {
                        let keepalive_packet = session.format_packet_data(&[], dst);
                        let index = session.local_index();
                        *self.sessions[index % N_SESSIONS].write() = Some(session);
                        // Make session the current session
                        self.current.store(index, Ordering::SeqCst);
                        self.timer_tick_session_established(true); // New session established, we are the initiator
                        self.timer_tick(TimerName::TimeLastPacketReceived);
                        TunnResult::WriteToNetwork(keepalive_packet) // Send a keepalive as a response
                    }

we'll want it to do so when we open source this

There was a similar feature request in WireGuard mailing list1, and Tunsafe (another implementation of the WireGuard protocol in the userspace) has already supported this feature in their beta version.

Though Wireguard kernel version may not want to support this feature for various reason, but I think this is a good idea to support this feature in the userspace Wireguard implementation.

Various ISP providers and corporate drop/block UDP packets, so it is useful to have this feature in these situations. Also most VPN traffic obfuscation techniques require a TCP connection instead of UDP. VPN traffic obfuscation is useful in countries where Internet censorship is applied (quote from 2).
Though the BoringTun over TCP's performance can be much worse than the UDP based one, I think these users can also accept this fact.

So I want to proposal BoringTun

1. Support for BoringTun over TCP.
2. Support for masquerading TCP connections as HTTPS traffic.
3. Support for encrypted SNI and also letting user custom the SNI URLs (unencrypted) on the above HTTPS traffic obfuscation mode.

TunSafe has a documentation for their TCP implementation3, I think this is a good reference. It's a good thing If BoringTun can collaborate with TunSafe author to improve this spec. You can see TunSafe's TCP support details in their changelog4 (TunSafe v1.5-rc2).

Thanks.

is an h2 but should be h3

Right now cookie replies to initiation handshake messages are accepted.
Cookie replies to response messages are ignored.

No cookie replies are generated at all.

Reading this post from one of the Wireguard maintainers: https://lists.zx2c4.com/pipermail/wireguard/2019-March/004048.html

Are there any specific reasons for not sharing changes upstream?
It's great that a company like Cloudflare uses an opensource project to work with, but refusing to join in on the project seems counterproductive.

Is there an official stance on this?

I've got Ubuntu 16.04 running in an OpenVZ VPS instance, kernel:

Linux 2.6.32-042stab136.1 #1 SMP Wed Feb 27 09:04:24 MSK 2019 x86_64 x86_64 x86_64 GNU/Linux

TUN is enabled:

# cat /dev/net/tun
cat: /dev/net/tun: File descriptor in bad state

Here's what I get:

# boringtun -f --disable-multi-queue --disable-drop-priviliges wgnet0
Timer("Invalid argument")
# boringtun -f --disable-multi-queue wgnet0
Timer("Invalid argument")
# boringtun -f wgnet0
Timer("Invalid argument")

I've done this already:

# setcap cap_net_admin+epi boringtun

And it didn't throw any errors. Here's the current state:

# getcap ~/.cargo/bin/boringtun
/root/.cargo/bin/boringtun = cap_net_admin+eip

Any advice appreciated!

I have successfully created client and Network Extension with boringtun for iOS

• XCode 10.2 (10E125), iOS 12.2)
• rustc 1.34.0 (91856ed52 2019-04-10)
• cargo-lipo 3.0.1
• cargo 1.34.0 (6789d8a0a 2019-04-01)

Tunnel to linux kernel implementation works.

However about 1/2 to 1/3 times when calling new_tunnel(..) it returns NULL.

Do you have any suggestions how to debug the issue? Or is it something trivial?

When running boringtun on a system where IPv6 is disabled, I get:
Socket("Address family not supported by protocol")
and the process exits.

Strace indicates that the reason is indeed IPv6 socket creation:
socket(AF_INET6, SOCK_DGRAM, IPPROTO_IP) = -1 EAFNOSUPPORT (Address family not supported by protocol)

Currently the only way to modify a peer is to remove it, and then add again with new config.

I am trying to assess whether the library will be functional enough to implement a Wireguard client in iOS, Android, and Windows, based on boringtun.

I am wondering, however, what exactly the following sentence (specified in the README), means:

"It implements the underlying WireGuard protocol, without the network or tunnel stacks, those can be implemented in a platform idiomatic way."

When I run cargo fmt I get a parse error:

$ sudo WG_QUICK_USERSPACE_IMPLEMENTATION=boringtun wg-quick up wg0
sudo: sorry, you are not allowed to set the following environment variables: WG_QUICK_USERSPACE_IMPLEMENTATION

Works fine without sudo(asks a password) and from root.
Arch Linux, was installed from AUR(it's pretty much just cargo build --release)

WireGuard protocol requirements for per-peer session lifetime based on the whitepaper:

1. Sessions must no longer be used REJECT-AFTER-TIME (180 seconds). Data must not be sent when a session is expired and received data must be ignored.
2. If an initiator sends data and session age >= REKEY-AFTER-TIME (120 seconds), it should attempt to start a new handshake.
3. If a responder receives data and the session age is >= REJECT-AFTER-TIME - KEEPALIVE-TIMEOUT - REKEY-TIMEOUT (180 - 10 - 5 seconds), it should try to initiate a new handshake once for this event.
4. After REJECT-AFTER-TIME * 3, all existing sessions must be destroyed.
5. During key rotation, one must be able to decrypt data using the current and previous session (subject to the above REJECT-AFTER-TIME requirement), but previous sessions must be destroyed.

boringtun implements session retrieval and management in src/noise/timers.rs and src/noise/mod.rs.

Sessions are cleared only when:

• Tunn::set_static_private: called when a new private key is set.
• Tunn::clear_all: called when either:
  • The most recent established session is older than REJECT-AFTER-TIME * 3 (established defined as receipt of an initiator/response message, not the 1.5-RTT data message for the responder!)
  • A new handshake was started by this device more than REKEY-ATTEMPT-TIME ago, but was not answered (handshake timeout).

Sessions are created on receipt of handshake messages in network_to_tunnel:

• On receipt of a valid initiation message. Note that this session cannot be used yet since the current index has not yet been updated.
• On receipt of a valid response handshake message that matches an existing in-progress handshake. Note that this session cannot be used yet since the current index has not yet been updated.

Sessions are used in:

• time_since_last_handshake - uses the current session.
• tunnel_to_network - uses the current session. Called when a locally-originated data packet is about to be sent.
• receive_data_packet - uses the session based on the receiver index which is chosen by the remote peer.

The "current" session is updated in:

• network_to_tunnel - when an initiator receives a handshake response.
• receive_data_packet - when a data packet for an existing session is received and is successfully decrypted with a valid counter that is not reused.

Problems:

1. The "current session" can be changed by the remote peer (or attacker) who sends a valid data packet. The "current session" should only be replaced when the a session is newer. Otherwise, consider this attack against the boringtun as initiator (a similar issue applies in the responder role):
   1. A honest boringtun initiator establishes a session with a responder and sends data. An attacker intercepts and withholds some data packets from the responder.
   2. The initiator (or responder) starts a new session due to key rotation and switches over to the new session.
   3. The attacker sends one of the data packets to the initiator, possibly after REJECT-AFTER-TIME has passed for the first session.
   4. The initiator switches the session secrets back to the old session, data sent by the initiator will not be protected with newer session secrets.
   5. This basically defeats key rotation and violates Requirement 1.
2. Requirement 2 seems not fully implemented. If an initiator has sent data at some point, and after REKEY_AFTER_TIME, it should not automatically send an initiation message. Instead it should only trigger a new handshake right after sending a data message (including Keep-Alive). Right now it unconditionally sends an initiation message after REKEY-AFTER-TIME while it should have remained silent.
3. Requirement 3 seems not fully implemented. REJECT_AFTER_TIME - KEEPALIVE_TIMEOUT - REKEY_TIMEOUT must be checked by the responder, but boringtun only checks it for the initiator.
4. Requirement 4 seems satisfied but with one potential issue. As sessions are considered "established" right after receiving the Initiation message, the following attack is possible against a boringtun responder:
   1. Initiator starts handshake by sending an Initiation message. An attacker prevents these messages from arriving, so the initiator repeatedly sends new initiation messages.
   2. The attacker forwards the very first initiation message and thus establishes a session.
   3. Assume that the initiator no longer has data to send, so after REJECT-AFTER-TIME * 3, all sessions must be cleared.
   4. Just before it expires though, the attacker forwards the second initiation message. As this is not a replay, the boringtun responder assumes that a new session is established.
   5. This process can be repeated by an attacker using the remaining initiation messages.
   6. The result is that the session lifetimes can be prolonged.
5. Requirement 5 is not satisfied because previous sessions are not cleared.
6. (minor observation) Tunn:tick_handshake_started appears to be dead code, Tunn::format_handshake_initiation calls self.timer_tick(TimerName::TimeLastHandshakeStarted) directly.

Summary: Key rotation is not properly implemented, keys should have been cleared but are actual maintained for potentially a very long time. This might defeat forward secrecy when the boringtun client/server is raided since all keys are still in memory.

Note: this is based on a manual code review, I did not actually try to test the actual implementation.

Analysis applies to current master (commit e765e12).

In src/main.rs, there are some code paths that ignore errors. Among the possibly problematic ones are:

• When run as root, any failure to change user/group IDs will not result in a fatal error. This might might increase the risk of running with unnecessary privileged. (btw, the option name is misspelled, #43 was incomplete.)

    if !matches.is_present("disable-drop-priviliges") {
        drop_privileges().ok();
    }

• When running backgrounded (without --foreground), any errors will result in the application being foregrounded anyway. This may be contrary to user expectations.

        match daemonize.start() {
            Ok(_) => println!("Success, daemonized"),
            Err(e) => eprintln!("Error, {}", e),
        }

We should link to the CI badge on the README.

Is there any sample of integrating boringtun with Android VPN api available, or are we currently at the level of jni bindings? Thanks

As an API user, I would appreciate an error rather than killing the whole application:

    fn update_peer(
        &mut self,
        pub_key: X25519PublicKey,
        remove: bool,
        _replace_ips: bool,
        endpoint: Option<SocketAddr>,
        allowed_ips: Vec<AllowedIP>,
        keepalive: Option<u16>,
        preshared_key: Option<[u8; 32]>,
    ) {
// ...
        // Update an existing peer
        if self.peers.get(&pub_key).is_some() {
            // We already have a peer, we need to merge the existing config into the newly created one
            panic!("Modifying existing peers is not yet supported. Remove and add again instead.");
        }

The protocol defines the following message limits:

• Rekey-After-Messages 2^64 * 2^16 - 1
• Reject-After-Messages 2^64 − 2^4 − 1

Although unlikely in practice, the implementation must check that these message limits are not reached and avoid a counter wraparound. The implementation of ReceivingKeyCounterValidator in src/noise/session.rs must be updated to check this.

Right now a malicious implementation could produce the following messages sequence which is accepted by boringtun:

• counter=0 (sets next to 1 in mark_did_receive)
• counter=2^64 - 1 (sets next to 0 due to integer overflow)
• counter=0 (oops! counter reuse - replay accepted!)

Additionally, going from counter 0 to counter 2^64 will also result in a lot of unnecessary executions which facilitates a DoS:

        // Packets where dropped, or maybe reordered, skip them and mark unused
        let mut i = self.next; // 1
        while i < counter { // i = 1 .. 2^64 - 1
            self.clear_bit(i);
            i += 1;
        }

If the gap is too big, perhaps the whole bitmap should be cleared.

FWIW, the WireGuard kernel implementation uses a window of 2048 bits whereas this one uses 1024.

Another related issue is that sending_key_counter and sending_key_counter are an AtomicUsize which may be 32-bit on 32-bit architectures whereas the counter in the protocol is 64-bit.

examples in rust are tested in tests (easy to maintain) and and are a good tool for supporting users. since we won't have full docs for release, i think we should offer 2-3 examples of using the libary. @vkrasnov - do you think this is something you could do today?

• https://github.com/cloudflare/boringtun/blob/master/src/device/api.rs#L19
• https://github.com/cloudflare/boringtun/blob/master/src/device/api.rs#L43

Both of these seem to be trying to use it as an assertion, however Result::is_ok() simply returns a bool. I think both of these should be using expect() instead.

When started from root, it fails:

# boringtun -f wg0
Failed to drop privileges: DropPrivileges("NULL from getlogin")

With --disable-drop-privileges it goes through.

getlogin is said to be a non-recommended way of determining user.

Hello, there're many ARM routers which lacks hard float (for example Netgear R7000/R6300v2, running bcm 4708/4709, Cortex A9) , and they're running old kernel (can not use kernel implementation), it will be very helpful if boringtun can provide soft float build for these devices.

currently we are depending on a fork of a crate that exists only on github. we should definitely upstream our patch to the crate but in the meantime, we don't want this to block us publishing. i'd recommend that we publish a crate with the change and then swap it out when we get the changes upstreamed :)

Cryptographic calculations for the Noise-based handshake should probably be separated from WireGuard-specific additions, like Type, Receiver/Sender ID, MAC1, MAC2. Advantages would be:

• Able to verify the Noise-specific implementation without distraction from "unimportant parts" (MAC1, etc.)
• Able to verify the WireGuard-specific parts with the ability to abstract away the Noise computations.

Ideally the Noise protocol state/computations are put in a separate structure/impl (say, "NoiseState") which could then be embedded in the "Handshake" structure.

I cloned down the project and ran cargo test. It compiled, and then began running, and prompted me for a password:

I'm not entirely sure what's happening here but I imagine contributors will expect cargo test to not behave like this. We may consider changing the current behavior or documenting how to best run the tests locally.

Either use Google's ( https://github.com/google/oss-fuzz ) or crate our own.

I'm trying to use BoringTun in a Docker container in order to provide a low-overhead skyhook-style VPN in a container scheduler.

My Docker compose looks like this:

---
version: '3'
services:
  missioncontrol:
    build: .
    image: naftulikay/missioncontrol:latest
    container_name: missioncontrol
    privileged: true
    sysctls:
      - net.ipv4.ip_forward=1
      - net.ipv4.conf.all.rp_filter=2
    cap_add:
      - net_admin
    ports:
      - '51820:51820'

I'm using Ubuntu 18.04 as a base image and I'm installing the WireGuard PPA and the wireguard and wireguard-tools packages.

Within this container, if I run boringtun --foreground wgnet0, I get the following output:

root@bf45d7330b3d:/# boringtun --foreground --verbosity debug wgnet0
Socket("no such file or directory")

If I try to wg-quick, I get the following output:

root@bf45d7330b3d:/# WG_QUICK_USERSPACE_IMPLEMENTATION=boringtun wg-quick up wgnet0
[#] ip link add wgnet0 type wireguard
RTNETLINK answers: Operation not supported
[!] Missing WireGuard kernel module. Falling back to slow userspace implementation.
[#] boringtun wgnet0
boringtun failed to start
[#] wg setconf wgnet0 /dev/fd/63
Unable to modify interface: Protocol not supported
Unable to access interface: Protocol not supported
[#] ip link delete dev wgnet0
Cannot find device "wgnet0"

/etc/wireguard/wgnet0.conf does in fact exist:

[Interface]
Address = 0.0.0.0/0
SaveConfig = true
ListenPort = 51820

PostUp   = iptables -A FORWARD -i wgnet0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wgnet0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

PrivateKey = REDACTED

I am running these commands as root and as you can see, I'm adding the NET_ADMIN capability so it should be able to do things properly.

Do I need a kernel module in order to setup the interface? I had assumed that since this was a userspace implementation, I wouldn't need to insert modules into the kernel.

https://crates.io/crates/byteorder

I'd love to run this application inside a docker container.
I ran wireguard-go inside a docker container in the past, i believe this is possible with this project as well.
In an example use case, the only additional requirements are adding the required capabilites via docker run --cap-add=NET_ADMIN --device=/dev/net/tun [...] for wireguard to create the tunnel interface.

Dockerfile from my wireguard-go docker image:

# Build container
FROM golang:alpine AS builder

WORKDIR /src
COPY ./wireguard-go /src
RUN apk add --no-cache git make && make

# Runtime container
FROM alpine:latest

WORKDIR /app
COPY --from=builder /src/wireguard-go /app

ENV WG_I_PREFER_BUGGY_USERSPACE_TO_POLISHED_KMOD=1 \
    LOG_LEVEL=info \
    WG_INTERFACE=wg0

ENTRYPOINT /app/wireguard-go --foreground $WG_INTERFACE

this way we can publish the library and the binary separately. https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html

Right now the timestamp field of the initiation message is too precise due (TimeStamper::stamp). It should be truncated to avoid acting as a time oracle which could aid in other attacks.

The Linux kernel implementation choses to truncate the value based on the possible number of initiations per second, see WireGuard/wireguard-monolithic-historical@c62836ef83bf.

Wireguard is agnostic to dropped packets (it's the job of applications reliant on UDP transmissions or just TCP to handle retransmits) but this results in abysmal performance in environments where dropped packets are common.

There are multiple solutions to this on github, basically socat-ing an error correcting (reed solomon) tunnel to go through Wireguard and use that instead. ex: https://github.com/wangyu-/UDPspeeder

Have you considered or would you be willing to consider augmenting Wireguard to keep track of dropped packets and engaging Reed Solomon coding for tunnels depending on how much packet loss they get. It can probably be made backward compatible with the kernel version or perhaps they can be convinced to cooperate.

The benefits are not marginal. A simple experiment with one of the aforementioned tunnels and a packet loss of just 5-10% can result in absurd improvements. ex: https://raw.githubusercontent.com/wangyu-/UDPspeeder/branch_libev/images/en/scp_compare2.PNG

If a two peers are added with 0.0.0.0/0, then that should probably be rejected with an error. Based on $insert in allowed_ips.rs and the code below, it looks like the current behavior is to silently overwrite existing peers. Not sure if that is desirable.

    fn update_peer(
        // ...
        allowed_ips: Vec<AllowedIP>,
        // ...
    ) {
        // ...
        for AllowedIP { addr, cidr } in allowed_ips {
            self.peers_by_ip.insert(addr, cidr as _, Arc::clone(&peer));
        }

It would be nice to have tests as well. One thing I am not entirely sure about is how to handle overlapping ranges, e.g. 10.1.2.0/24 and 10.1.2.100/32.

A performance issue with message counters in #54 was closed via #59. However an explicit check for large message counters is still missing. To reiterate from #54:

The protocol defines the following message limits:

• Rekey-After-Messages 2^64 * 2^16 - 1
• Reject-After-Messages 2^64 − 2^4 − 1

Although unlikely in practice, the implementation must check that these message limits are not reached and avoid a counter wraparound. The implementation of ReceivingKeyCounterValidator in src/noise/session.rs must be updated to check this.

Note that it is practically very, very unlikely to reach these limits. However, it is possible for a malicious implementation to send packets with arbitrary high counter values. It would simplify code analysis if integer overflows can be ruled out by an explicit check.

maybe even just 0.0.1 or so. would help us packagers a bit :)

Hi there,
I'm trying to run boringtun on a vServer that uses virtuozzo as its hypervisor and it exits immediately with the following message:

~> sudo boringtun -f -v debug wg0 
Timer("Invalid argument")
~> 

The system:

~> uname -a
Linux **** 4.15.0 #1 SMP Thu Feb 21 17:35:43 MSK 2019 x86_64 x86_64 x86_64 GNU/Linux
~> cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"
~> rustup show
Default host: x86_64-unknown-linux-gnu

stable-x86_64-unknown-linux-gnu (default)
rustc 1.33.0 (2aa4c46cf 2019-02-28)
~> 

Have you got any clue what could be causing this?

Regards,
Markus

The main one left is CFJNI

Peer indices (sender/receiver IDs) seem to start with zero and are incremented by one:

impl Device {
    fn next_index(&mut self) -> u32 {
        let next_index = self.next_index;
        self.next_index += 1;
        assert!(next_index < (1 << 24), "Too many peers created");
        next_index
    }
// ...
    fn update_peer(...) {
// ...
        let next_index = self.next_index();
// ...
        let mut tunn = Tunn::new(
// ...
            next_index,
        )
        .unwrap();
// ...
        let peer = Peer::new(tunn, next_index, endpoint, &allowed_ips, preshared_key);
// ...
        self.peers_by_idx.insert(next_index, Arc::clone(&peer));

Which are used right here:

    fn register_udp_handler(&self, udp: Arc<UDPSocket>) -> Result<(), Error> {
                // ...
                while let Ok((addr, packet)) = udp.recvfrom(&mut t.src_buf[..]) {
                    // ...
                    let peer = match (packet[0], packet.len()) {
                        // ...
                        HANDSHAKE_RESPONSE => {
                            let peer_idx = u32::from_le_bytes(make_array(&packet[8..])) >> 8;
                            peers_by_idx.get(&peer_idx)
                        }
                        COOKIE_REPLY => {
                            let peer_idx = u32::from_le_bytes(make_array(&packet[4..])) >> 8;
                            peers_by_idx.get(&peer_idx)
                        }
                        (DATA, DATA_OVERHEAD_SZ...std::usize::MAX) => {
                            // A data packet, with at least a header
                            let peer_idx = u32::from_le_bytes(make_array(&packet[4..])) >> 8;
                            peers_by_idx.get(&peer_idx)
                        }

Problems:

• The receiver/sender ID should be random and unpredictable. Otherwise it could give an attacker an advantage in pulling off a denial of service attack.
• A monotonically increasing counter from 0 could leak activity and reveal whether a peer has actually connected or not. This kind of side-channel information could potentially be used in another attack.
• Two sessions (after key rotation) should not have the same sender/receiver ID. That would make them linkable.