Voice Channel support about chorus HOT 12 OPEN

Good luck :)

from chorus.

At least this is well documented

Some relevant info:

• Encryption is done using xsalsa (specifically xsalsa20_poly1305, xsalsa20_poly1305_suffix or xsalsa20_poly1305_lite), I've found the easiest way to implement this is probably the RustCrypto secretbox crate (which already combines salsa20 with poly1305)
  (Also, Discord says they themselves use libsodium and secretbox is part of NaCl-compat, which is a rust implementation of libsodium's API)

• Audio is encoded with the Opus open audio codec (stereo, @ 48kHz). For opus in rust, we can use opus-rs, which provides high level bindings for libopus, so we can just consult the official opus documentation

(for encryption, see this part of discord's docs)

The encryption and encoding seems to be the hardest part of implementing this, the actual communication is very similar to the gateway

from chorus.

Oh and we also have no info about video, so we should probably implement voice first

from chorus.

Doing some research, video uses VP8 and VP9, maybe env-libvpx-sys in rust?

{
	"codecs": [
		{
			"name": "opus",
			"type": "audio",
			"priority": 1000,
			"payload_type": 109,
			"rtx_payload_type": null
		},
		{
			"name": "VP8",
			"type": "video",
			"priority": 2000,
			"payload_type": 120,
			"rtx_payload_type": 124
		},
		{
			"name": "VP9",
			"type": "video",
			"priority": 3000,
			"payload_type": 121,
			"rtx_payload_type": 125
		}
	]
}

from chorus.

Webrtc-rs may also be worth looking into, since it seems to support all the needed codecs

from chorus.

Potentially maybe perhaps looking into this on feature/webrtc
I probably won't manage to implement it though lol

from chorus.

Also looking at how Serenity implemented this, they seem to have developed their own rtp parser and even their own opus bindings

from chorus.

I wonder if we could... yoink this :D

from chorus.

yup :D

from chorus.

Slight change of plans, raw UDP first then webrtc
because webrtc is really complex and raw udp seems to be way easier to implement for now

from chorus.

wasm cannot support udp.

from chorus.

See https://github.com/webrtc-rs/webrtc and https://github.com/wasm-peers/wasm-peers

from chorus.