antoinechalifour / talkie.lol Goto Github PK

View Code? Open in Web Editor NEW

6.0 3.0 2.0 1.81 MB

A webapp for experimenting with WebRTC

Home Page: https://www.talkie.lol

TypeScript 98.37% HTML 0.24% CSS 1.10% Dockerfile 0.05% JavaScript 0.23%

webrtc typescript videos react graphql apollo heroku redis github-actions media

talkie.lol's Introduction

Talkie.LOL

Talkie is a free, open-source, secure, peer-to-peer video chat app that you can use to talk to your friends.

Try it here!

Quick Overview

Talkie is a side-project that I use to play with WebRTC and brand new Web APIs. Talkie uses :

Category	API	Used for...
WebRTC	RtcPeerconnection	Creating peer to peer connections
WebRTC	Data Channels	Sending messages and images
WebRTC	MediaDevices / enumerateDevices	Selecting audio and video sources
WebRTC	MediaDevices/ getUserMedia	Stream the user audio / video
WebRTC	MediaDevices / getDisplayMedia	Share the user screen
PictureInPicture	Video / requestPictureInPicture	Enable picture in picture for a user
Clipboard	writeText	Copy the space link
Clipboard	write	Copy the space QR code
Clipboard	`paste` event	Send images from the clipboard over data channels
Canvas	captureStream	(Experiment) Merge all streams to a canvas
Canvas	toBlob	To copy a canvas to the clipboard
WebAudio	AnalyzerNode	Detect sound and silence
Navigator	Connection	Detect connection type
PWA	Service Workers	Installing Talkie on your device!

Stack

Signaling server

The source code can be found here.

The signaling server is a GraphQL API, written using :

✅ TypeScript
🚀 Apollo server as a GraphQL server
🄺 Koa
♻️ Dependency Injection using Awilix
📦 Redis as a database / pubsub

Web app

The source code can be found here.

The frontend is a React app using :

This project extensively uses new APIs such as CSS Grid or CSS custom properties.

Development environment

Requirements

To run this project locally, you will need

Docker and docker-compose
Yarn
A version of Node supported by create-react-app.

Running locally

To start the app :

Install dependencies using yarn in the project root
Run docker-compose up in the project root (this will start Redis and the signaling server in watch mode using ts-node)
Run yarn start in the packages/web directory (this will start the front-end)

Deployment

# Configure the remote machine (install Nginx and Docker)
ansible-playbook infra/playbooks/install.yml

# Configure the reverse proxy
ansible-playbook infra/playbooks/reverse_proxy.yml

# Deploy the app
ansible-playbook infra/playbooks/talkie.yml

talkie.lol's People

Contributors

Stargazers

Watchers

Forkers

todokku bahachammakhi

talkie.lol's Issues

Limit the number of people in a space

Limit the number of people in a room to 4. This requires handling SPACE_LEFT events on socket disconnections.

Firefox compatibility

Right now the app relies on the connectionState to determine which user is connected (see the onPeerConnectionConnected function). Unfortunately, this field is not supported in firefox.

Make STUN servers list configurable from the API

Ideally the environment would be served from the server. I guess we could add the STUN conf to the Login mutation to only redeploy the backend when we'll update these servers.

Use "polite negotiation"

There could be some race condition in the current production code. Read this really interesting article for more info : https://blog.mozilla.org/webrtc/perfect-negotiation-in-webrtc/

Use proper STUN servers

Right now I just picked some random server from a gist. I guess I could deploy a STUN server to Heroku or use a dedicated service like Twilio.

Take a picture of a user

Use Web Audio Analyzer

We could do some crazy stuff https://codepen.io/gpyne/pen/WjYwrd

SpeechRecognition API

Use SpeechRecognition API to send subtitles to other peers

Make data channel connections more robust

Current implementation crashes when sending large images (state is closed)

Todo

split large files using chunks
re-open data-channels on crash

Add TURN servers

See https://bloggeek.me/google-free-turn-server/.

...but it cost a lot of moneyyy 💰💰💰💰

Feature detection

WebRTC

Use window.RTCConnection to detect whether WebRTC is available
Redirect to an error page when the browser does not support WebRTC

Clipboard API

Clipboard API is part of progressive enhancement

Use navigator.clipboard to detech whether the clipboard API is available
Hide the "share button" when this feature is not supported

Picture in Picture

PinP is part of progressive enhancement.

Use document.pictureInPictureEnabled to detect whether picture in picture can be used
Hide the picture in picture button when PinP is not supported

Add a chat

Add a chat using Data Channels, a WebRTC feature for sharing binary data.

Add "reactions"

Add "reactions" (like clap, thumbs up, ...)

Maybe custom reactions as well ?

Chat is not visible in light mode

Video filters

This is very experimental and I'm not really sure what to implement, but using the same trick as the picture-in-picture task, we can manipulate the stream in a canvas.

So we could add things like :

black & white video
sepia video
green screen integration ?

Let's just be creative

We can actually render multiple videos in a canvas element, and use canvas.streamCapture(fps) to get the stream and use it in a single video element that will be used for PIP.

A POC is available in the branche picture-in-picture