Git Product home page Git Product logo

roc-toolkit's Introduction

Roc Toolkit: real-time audio streaming

build release matrix

Roc is a toolkit for real-time audio streaming over the network.

You can read about the project here:

Summary

The toolkit consists of a C library and a set of command-line tools.

Key features:

  • real-time streaming with guaranteed latency;
  • restoring lost packets using Forward Erasure Correction codes;
  • converting between the sender and receiver clock domains;
  • CD-quality audio;
  • multiple profiles for different CPU and latency requirements;
  • portability;
  • relying on open, standard protocols.

Besides library and tools, toolkit provides:

Documentation

Documentation for the latest release is available here.

Additionally, there is also Doxygen-generated documentation for internal modules.

Build status

See details on continuous integration here.

Branch Status
master build
develop build

Versioning

See details here:

Platforms

See details here.

If you want to run Roc on an embedded system or a single-board computer, see:

Plans

See here:

Donations

If you would like to support the project financially, please refer to this page. This project is developed by volunteers in their free time, and your donations will help to spend more time on the project and keep it growing.

Thank you!

Donate using Liberapay

Community

We have a forum, mailing list, and Matrix chat room. See details here.

Contributing

Contributions in any form are always welcome! Please check out contribution guidelines.

Tasks needing help are listed here.

Licensing

See details on licenses here.

Authors

See a list of maintainers and contributors here.

roc-toolkit's People

Contributors

adrianopol avatar alexandremgo avatar angelthorns avatar anurag10 avatar baranovmv avatar cristobalm avatar dshil avatar enigmaro avatar foreverasilver avatar fusuiyi123 avatar garethmarlow avatar gavv avatar hrick87 avatar hrishikeshsuresh avatar ivanmiklec avatar kcy1019 avatar khansamad47 avatar matteoarella avatar nolan-veed avatar ortex avatar pekureda avatar pvimal816 avatar rohank99 avatar ssfuen avatar stevenzhu94 avatar sthibaul avatar swapnil-gaur avatar veronikaro avatar villaquiranm avatar zenuo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

roc-toolkit's Issues

Refactor output latency in server

Move output latency from SampleBufferQueue to Server.

  • remove start threshold from SampleBufferQueue
  • generate output_latency / samples_per_tick zero buffers before first tick in server
  • cleanup config

See also #44.

Configurable sample rate

Three components are aware of sample rate:

  • sndio::Reader
  • sndio::Writer
  • audio::TimedWriter, used in pipeline::Server and pipeline::Client

What we need:

  • add --rate option for roc-send and roc-recv
  • add sample rate setter and getter to IPacket
  • set sample rate in Splitter
  • drop packets with mismatching sample rate in Watchdog

See also #51.

Refactor roc_pipeline

  • extract session management from server
  • merge base and derived classes (client, server, session)
  • remove session composer (use pool directly)
  • remove audio::IRenderer
  • merge audio::Renderer into pipeline::Session

Session negotiation

Implement some sort of session negotiation.

At a minimum, we need to be able to transfer session parameters from the sender to the receiver. The parameters should include rtpmap (dynamic payload types) and fssi (FEC-scheme specific information).

Why we need this:

  • to fully confirm FECFRAME and make FEC scheme parameters configurable;
  • to be able to add more encodings, which require dynamic payload types.

The session description should be in the SDP format because this is required by various RTP-related RFCs, e.g. FECFRAME or Opus for RTP.

As for the control protocol, we can use RTSP/1.0 or SIP.

We should also add a session control port to the API, command-line tools, and PulseAudio modules.

RTCP support

RFC 3550 requires conforming implementations to provide RTCP support. In fact, roc can function without it, though RTCP provides some benefits:

  • without RTCP, sender has no feedback from receiver; it even doesn't know, if receiver was able to receive traffic, if receiver was able to process the traffic and didn't reject it for some reason, and if quality of service is acceptable;
  • RTCP allows receiver to detect associated streams from single sender in a standard-conforming way, e.g. audio stream and FEC stream (we're now using an ad-hoc implementation that conforms to no standard);
  • RTCP provides timing synchronisation for sender and receiver and for several streams of single sender (e.g. audio and video);
  • RTCP can be used to implement dynamic adjustment of latency and FEC code rate (using RFC 5725);
  • RTCP can be used to implement retransmission using RFC 4588 and RFC 4585;
  • RTCP allows multiple senders to share the same source transport address (e.g. in case of RTP translator);
  • RTCP allows seamless sender restart or address change (this also allows sender roaming);
  • RTCP allows receiver to terminate sender's session immediately without waiting for timeout;
  • RTCP is used in RTSP, which we may want to support in future;
  • with RTCP, receiver sends to sender various statistics that may be useful for troubleshooting, especially when receiver is some embedded device which is harder to debug;
  • some senders may reject connection if there is no RTCP feedback from receiver (I'm not aware of such senders however);
  • finally, RTCP also allows to exchange arbitrary application-specific data.

RTCP consists of the following message types:

  • SR (sender report): statistics sent from sender to receiver;
  • RR (receiver report): statistics and feedback send from receiver to sender;
  • SDES (source description items): sender identification sent from sender to receiver;
  • BYE: session termination, initiated by sender or receiver;
  • APP: arbitrary application-specific data.

IPv6 support

Current implementation unconditionally uses IPv4:

  • uses 4 bytes for IP address;
  • calls ipv4 functions from libuv to parse and bind address.

We can add support for IPv6, activated if it's available at build time and if user provided an IPv6 address.

Improve scons options

  • doxygen support
  • check external dependencies and their versions
  • use pkg-config if available
  • configurable paths for external dependencies
  • option to download and build external dependencies (--with-3rdparty)
  • option to select enabled targets manually (--with-targets)
  • check clang-format version
  • configurable compiler and tools paths

Check alignment in LDPC_BlockEncoder

Determine alignment requirements for OpenFEC and check that buffers are properly aligned.

When alignment requirements are not met, OpenFEC encoder segfaults. (Misaligned packets were fixed in 1977737).

Reduce output latency

  • understand why higher output latency is currently required for ALSA backend compared to pulseaudio backend;
  • determine minimum possible latency for pulseaudio and ALSA (bare and over pulseaudio);
  • reduce output latency:
    • tune SoX parameters;
    • or, alternatively, tune ALSA parameters directly.

See alsa_play_tuned.cpp for full list of ALSA parameters that may affect latency.

See also #44 and #46.

Document tools usage

  • network options
  • reading/writing files
  • reading/writing devices (ALSA, pulseaudio)

Get rid of hardcoded resampler frame size?

Resampler frame size is configured at compile time. Does it indirectly depend on other parameters, like packet size and player frame size?

If so, using non-default values for those parameters may break resampler.

Fix ALSA/pulseaudio capturing

Example:

pactl list sources | grep Name: | grep monitor
	Name: alsa_output.pci-0000_01_00.1.hdmi-stereo.monitor
	Name: alsa_output.pci-0000_00_14.2.analog-stereo.monitor

roc-send -vv -t pulseaudio -i alsa_output.pci-0000_01_00.1.hdmi-stereo.monitor \
    -s 127.0.0.1:10001 -r 127.0.0.1:10002

Tricky case with Depacketizer and fec::Reader

It seems that in current implementation there is a case when we're unnecessary dropping received packet(s).

Assume sender generates three packets:

+----+----+----+
| P1 | P2 | P3 |
+----+----+----+

Then they're reordered by interleaver or network:

+----+----+----+
| P3 | P1 | P2 |
+----+----+----+

and P1 and P2 are delayed a bit:

+----+-------+----+----+
| P3 | delay | P1 | P2 |
+----+-------+----+----+

When Depacketizer wants to read P1 from fec::Reader, only P3 is already received:

+----+-------+
| P3 | delay |
+----+-------+

Thus, fec::Reader considers that P1 and P2 are lost and returns P3. Depacketizer notes that P1 and P2 are lost. Depacketizer renders zeros instead of P1.

Then, P1 and P2 are received:

+----+----+
| P1 | P2 |
+----+----+

After this, Depacketizer is going to render samples for P2, previously marked as lost.

Here is the problem:

  • P2 is received and it's time to extract samples from P2
  • however, Depacketizer renders zeros instead of P2, because it already got P3 from fec::Reader and thinks that P2 is lost
  • anyway, fec::Reader drops P2 because it never returns late packets

However, technically nothing prevents us from rendering P2 in this situation.

Note that with fec::Reader, the situation when queue is empty or almost empty is not rare in practice, because network latency jitter value may be close to aim_queue_size - fec_block_size (7 packets currently).

Is this a problem?

Add INSTALL.md

  • external dependencies
  • building
  • installing
  • supported platforms

Get rid of hardcoded parameters in roc_fec

In current implementation, the following encoder/decoder parameters are configured at compile time:

  • block size (number of data and FEC packets in block);
  • packet size (number of bytes in audio packet).

They are:

  • ROC_CONFIG_DEFAULT_FEC_BLOCK_DATA_PACKETS
  • ROC_CONFIG_DEFAULT_FEC_BLOCK_REDUNDANT_PACKETS
  • ROC_CONFIG_DEFAULT_PACKET_SIZE

Moreover, if server and client are built with different parameters, server may behave incorrectly.

What we need instead:

  • configure parameters at runtime (when creating encoder/decoder), and use ROC_CONFIG_ values just as defaults;
  • handle possibility of parameters mismatch;
  • provide corresponding runtime configuration options in ServerConfig and ClientConfig in roc_pipeline;
  • use smaller packet size in client/server tests to make them faster.

Refactor Chanalyzer

  • Implement per-channel instances of IPacketReader instead if single IAudioPacketReader
  • Remove IAudioPacketReader interface
  • Cast packets to IAudioPacket in Streamer

Configure CI for Linux/x86_64

First, we need https://travis-ci.org/ for various linux distros and mac os.

See also #38 and #42.

Docker:

  • create docker hub automated builds for supported distros
  • use pre-built images from docker hub instead of configuring them every build

Linux distros:

  • fedora latest
  • debian stable
  • ubuntu 14.04, 16.04
  • centos 5, 7

Platforms:

  • x86_64

Compilers:

  • gcc 4.1 (centos 5)
  • gcc 4.4, 4.6 (ubuntu 14.04)
  • gcc 4.8, 4.9, 5.x (ubuntu 16.04)
  • clang 3.4, 3.6, 3.7 (ubuntu 16.04)

Variants:

  • debug
  • release
  • --with-3rdparty=all

Later, we can also use travis for MacOS and http://www.appveyor.com/ for windows builds.

Add support for Mac OS X

  • Adapt SConstruct for MacOS
  • Adapt 3rdparty building for MacOS (#88)
  • Don't use version script on MacOS or maybe don't use it at all (#89)
  • Disable _POSIX_C_SOURCE on MacOS (doesn't work as expected)
  • Remove core::SpinMutex (we don't really need it) (#91)
  • Add target_posixtime and target_darwin to SConstruct (#99)
  • Port time functions to MacOS (#99)
  • Add Travis build for MacOS
  • Update wiki

Resampler tool

Add roc-resample tool:

  • read input samples from wav file
  • pass to resampler
  • measure resampler performance (min/max/avg real seconds per virtual second)
  • write result samples to wav file

Add Opus support

Opus is a lossy streaming audio codec.

Specs:

Adding Opus support introduces a few challenges:

  • currently RTP payload decoder is stateless; Opus decoder is stateful;
  • currently RTP payload decoder and FEC decoder are separate pipeline steps; Opus decoder combines them.

Our fec::Decoder is packet-format-independent and works before decoding audio packet payload. Thus, it theoretically may be used in conjunction with Opus decoder, if it makes sense.

Improve RTP support

  • add tests for RTP parser/composer
  • handle packet's SSRC (keep RTCP (#14) and SDP (#34) in mind)
    • add SSRC support to RTP packet
    • fill SSRC in Splitter and fec::Encoder
    • check SSRC and seqnum in fec::Decoder
    • parse packets before routing to session
    • identify session by destination address + source address + SSRC list
    • identify packet queue inside session by SSRC
    • auto-detect SSRC for every packet queue inside session

Replace ad-hoc FEC packets with something standard-compliant

See also #6 and #13.

I've found two alternative groups of RFCs:

  1. FEC Framework (FECFRAME)

    There is also Raptor(Q) support for FEC Framework:

  2. Generic FEC

Finally, these RFCs are mentioned by both FEC Framework and Generic FEC:

Improve roc-send and roc-recv tools

  • roc-recv: add --oneshot option; if it's set, exit when first client disconnects
  • roc-send: fix EOF handling when sending files
  • roc-send: produce readable error message when sox can't open input file

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.