As we go along we would like to record the meetings hold on meething app.

Upon a new room creation it would be useful to save the room features and parameters (create date, participants counts, allowed features, etc) to a Gun object for using in logic and display purposes such as:

• display session timer since beginning
• owner(s) of the conference
• permissions
• limits of participants, duration, etc

// Edit 'Added Bert Nerd' Persona
Hi Team,

It’s Week 1 of the Mozilla Spring MVP Lab and that means we are gathering ideas, making sketches, thinking about and researching possible feature sets.

State so far:

• Pretty Stable Video Conferences for small groups (@alexios is working on quantifying how stable and how many people)
• We can mute mic, we can turn video off and I think we can turn them back on 😉
• We have a list of active users and can display their id’s (usernames to come) in the video stream and in a list of users
• We can lock a room so no new person can join and implementing unlock in the next couple days (4/21)

Where are we going:

This week is for our team to gather features, assign work, do some research on what technologies (libraries, functionalities, webRTC) to use and to setup calls to get to know each other and share ideas.

Hopefully that will result in a backlog of features we can develop and work on in the remaining 7 weeks.
There is lots of activity between @QVDev , @lmangani and @jabis. We don’t want to interrupt that, but we do want ideas from all sides so we can agree on an outlook we all can commit to and feel we can contribute to.

Personas

To help us in ideation for the Product I recommend using Personas to help guide our work to be focused on specific possible users of our system.

• Granny Smith - She is of the older generation and will mainly receive invites from her family to join calls. Her eyesight might not be perfect, she needs a bit of help making sure she isn’t on mute or that others can see her.
• Joe Clark - Corporate CTO who wants his remote teams to work together better. He’s looking at making sure people can focus and aren’t distracted, but also that information about his companies secrets aren’t stolen in the process.
• Ari Obianwu - Citizen Journalist in an oppressive country. He needs a secure way to communicate with sources and news papers around the globe, without having to reveal his real identity. He needs a way to share information aka files, screenshots or just screen share without compromising on his own safety.
• Sandy Teen - She loves hanging out with friends. She needs a way to quickly connect with a set of friends to organize parties or just hang out. The less hassle the better. She does want to use this system to communicate with her Granny and doesn’t want to get frustrated by having to say “Push the red button on the bottom” 50 times.
• Arthur PHD Student - He wants to quickly connect with researchers across the globe to discuss his findings. He needs to share his screen, flip to the camera and share files.
• Bert Nerd - Loves stats and seeing how the network works. Wants to see his internet and browser work the magic. Wants to know packets lost and bandwith and connection .... everything he can get to see.

Personas are a good way to keep the people who would use this product in mind. These aren’t set in stone of course and can be further detailed as we go. They should at least partially reflect someone we know and can sympathize with. Especially on UI.

Major Features that come to mind looking at these:

• Secure Video and Audio Transmission (Encryption ? / Peer-to-Peer without interceptors ?)
• Good UX that has visible indicators when you are on mute or when your camera is off.
• A shared whiteboard for collaboration (stretch goal?)
• Screen Share with “Entire Screen” and “Only select Window” mode
• One Click Meeting Start, Default Name remembered
• Calling Granny Mode, gives you control to unmute and turn camera on (stretch goal? Can also be host permissions instead of Granny Mode)
• File Sharing (Mozilla already has a service to send files, maybe it has an API -> Research; stretch goal?)
• Auto Arranging Multi-Party Videos, freedom to move them around etc.

Kickoff / Conclusion

Let’s discuss here. I think the idea is not to merely build a clone of zoom or other already existing products.
Let’s build something unique on the technology @QVDev and @lmangani have worked so hard to punch out over the last year (or two), and let’s make it our own.

Let’s get to work!!

When @QVDev and I were dogfooding, I noticed his image would shift to the right when I didn't quite use the right widget location in grid.

Looked like canvas is left aligned and gets cut off on the right, making it appear out of center. If I stretched to use 2 colums, it usually was only 1 row high, which cut some of his face off at top and bottom.

Will take a picture and add here if I see it again.

@QVDev disagrees with this 👍

@lmangani agrees but think not possible in 8 weeks 👍 .

Mark is excited by this, haha 👎

Check out:

• https://www.intercom.com/
• https://www.userlike.com/en/features

Goal:

• Drop in 1 script tag on a website, optionally set some configs "autoconnect: false" etc.
• Bottom right bubble shows current person who is talking, so it switches between people as talking.
• Hover over the bubble and it spreads all the bubbles out across the screen so you can see all talking people.
• Click on any person and they become full screen background ontop of the website but below the other bubbles
• Drag any bubble around the screen to re-arrange.

Can answer any other Qs or fill in more info.

We would like to write connection errors to debug where things go wrong instead of using the console log we want to write all this data to a specific gun instance and build a simple front end displaying all the messages so we can easily look trough the same data

Hard to explain, but for some reason, the user is accustomed to seeing their video flipped horizontally. Perhaps this is because most selfie cameras show the user video this way.

Right now in the app, the video presents the user video the other way.

I need to review most recent issues. Comment here when you dont have a task assigned.

platform: macOs
browser: chrome
problem: I started the server and joined a room, but nothing happened. No video, no sound. There was no meeting stream created when joining the room.

Describe the bug
Double-click on video stream, make it go full-screen instead of PiP.
Firefox on latest, Mac as well as Firefox on Windows 10.

To Reproduce
Open a new meeting,
Talk to someone else or a second tab
Double-click on their video stream

Expected behavior
PiP pops out draggable.

Screenshots
N/A

Desktop (please complete the following information):

• OS: Macos Catalina
• Browser Firefox Dev Edition v76.05b, Firefox v75

Additional context
https://meething.hepic.tel/
latest

When people start recording or off/on grid mode people should be able to get informed see:

https://www.w3schools.com/howto/howto_js_snackbar.asp

Discovery Topic.

Adding a local TUN/STUN service to the stack might improve the discovery and peering experience and potentially allow us to later move the same logic to be served by Gun (which already knows the public identity of a remote peer through its sockets) directly through a plugin.

resolution: 720,
maxFps: 10,
constraints: {
video: {
frameRate: 15,
height: {
ideal: 720,
max: 720,
min: 240
}
}
}

Is a suggestion from @lmangani

see https://blog.addpipe.com/getusermedia-video-constraints/

Tried an informal qa test half an hour ago first with 5-11 users (not sure how many actually entered the room since i saw only up to 2 empty video frames at a time) and then with 3 users and had the same issue with empty frames. Will do better in future with bug reporting documentation but i attached the log from both the big chaotic call (test 1) and the second call/room with 3 people sharing video and audio. Will try to do this again with better logging and 2-3 users tomorrow morning (EST so 9 hrs from now)

First Test 5-11 Users
test1_5-11Users.log

Second Test With Different Room and 3 users
test2_3Users.log

A couple unedited screenshots with some shrank unfortunately by receiving over messenger (Unordered)

My own

unique issue with no other users shown on ipad i guess?

Branch: Master
OS: Windows 10 64-bit
Browsers Tested: Firefox(latest) - Transparent video feed, Chrome(latest)- Blacked out video feed.
Observed on HEPIC

Description
Participants video feed are either transparent or blacked out.
Audio too does not work.

It works with multiple tabs from the same browser.

Error Logs and screenshots

Error Log

Event Log

Multi Party signaling is buggy in terms of notifying all parties about new users joining the room.
The room should keep the list of active participants and their connections. Once the room participants are tracked in Gun, the signaling logic needs to be redesigned to allow dispatching new subscribes to all participants individually.

Rather than reinventing the wheel, it would be great if this app could join a distributed Gun Chat leveraging all community features/ideas or integrating with existing networks. If anyone's willing to come in help us design and implement this subsystem please stand up :)

Mockup for picture in picture UI

Currently purely static built using vanilla HTML and CSS.

Demo on glitch

TODO:

• Add JS logic to enable button functionality.
• Integrate into main codebase.
• Modify design based on feedback.
• Port to SCSS (if required)

OS: Windows 10 PRO
Browser: Microsoft Edge 44.18362.449.0

Peer: meething-qvdev.herokuapp
Date: 4/22 1100 pst

Opened 2 tabs to test audio recording. Navbar appeared, no round circle on bottom left

Error messages:

Comment them here

Describe the bug
NB; Tests were done on same LAN.

To Reproduce
Steps to reproduce the behavior:

1. Open multiple instances using Chrome and Firefox (2+ instances EACH) with half on Chrome and half on Firefox.
2. Close some of the instances

Expected behavior
When an instance/user leaves, user ID and video frame should disappear on other instances.

Screenshots

Desktop (please complete the following information):

• OS: Win 10
• Browser: chrome to firefox
• Version: latest

Additional context

• Link you used? REDACTED
• Branch you used? (Such as fix_audio) Master/hepic

Add any other context about the problem here.

Metric

• Perfomance / Benchmarking
• Latency
• Cross site scripting

When one presses offGrid() everyone should go offgrid some for onGrid()

Branch: Master
OS: Linux 64 bit (Manjaro)
Browsers Tested: Firefox and Firefox Dev
Networks Tested: Broadband, and mobile data

Observed Behaviour:
Upon creating and entering a room, there's no popup asking for mic and camera. Only the chat window and user widget appear. This was observed when running it locally as well as on the web.

Link: http://meething-webrtc-gun.glitch.me/?room=room1_st51wyukt7p

Console error, warning logs, and screenshot are attached below.

Console_log.txt
Console_log_errors.txt
no_video_issue

Note: Other meeting clients work without any issue.

Potential candidate for a (powerful-browser-based) MCU for videowalling over a single stream. Just exploring and leaving note here for others to check out:

https://github.com/t-mullen/video-stream-merger

https://t-mullen.github.io/video-stream-merger/demo/p2p.html

Discovery Topic.

Adding a local TUN/STUN service to the stack might improve the discovery and peering experience and potentially allow us to later move the same logic to be served by Gun (which already knows the public identity of a remote peer through its sockets) directly through a plugin.

Integrating node-turn seems to be quite trivial and the service provides a valid responds to basic browser requests

Config

var Turn = require('node-turn');
const PORT = process.env.PORT || 19302;
console.log("App Port: " + PORT);
var server = new Turn({
  // set options
  authMech: 'none',
  debugLevel: 'DEBUG',
  listeningIps: ['0.0.0.0'],
  listeningPort: `${PORT}`,
  log: function(a,b){ console.log('LOG',a,b)}
});
server.start();

Response

Receiving UDP: from IPV4://xx.xx.xx.xx:33000 to IPV4://yy.yy.yy.yy:3478 binding request TransactionID: 787444776d4b30534f65784c
Sending UDP: from IPV4://yy.yy.yy.yy:3478 to IPV4://xx.xx.xx.xx:33000 binding success TransactionID: 787444776d4b30534f65784c
  xor-mapped-address: IPV4://xx.xx.xx.xx:33000

The local node can be added to the logic here

We would like to be able to disconnect from super peers so all people in the room are locked in to a private mode and nobody new can join. There is already some example code in presence.js and iris.to has also example code

https://github.com/irislib/iris-messenger/blob/04a03d93dd4c1cc2c5be1f8e591269e1e2be908c/src/js/app.js#L111

also need a new button for this on top

https://jitsi.org services are awesome and I think they can be made more awesome with gunDB

They have mature implementations of video conferencing software which can be retrofitted to work with GUN I believe

Related: #56
I suspect the stream handling with MediaStream constructor is of issue here (unsupported in Safari/Safari on iOS)

Will need people with iPhones/iPads to test, I have none ;(

My friends and I tried https://meething-webrtc-gun.glitch.me/ today but we found that the video chat cannot be enabled due to an exception
Uncaught TypeError: myStream.getVideoTracks is not a function at HTMLButtonElement.<anonymous> (rtc.js:308)

You probably need to update the deployed version or did we miss anything?

We are using Chrome Version 80.0.3987.149 (Official Build) (64-bit)

When other users create a new room, the DAM inbound blocker reveals the room name.

This is obviously because it's experimental. But this is a reminder to remove the room name before going production.

We need a master flag to detect mobile clients and their screen size/ratio to automatically disable certain interactive features (grid, etc) and optimize the screen estate.

https://github.com/meething/webrtc-gun/blob/master/src/index.html#L13

Replacing bootstrap with plain CSS grid and styles would make the app more lightweight. It looks like Bootstrap is mainly just used for column and row layout too.

Would need to test if this would affect "dragable" css though. Unsure of what that plugin does today in the app.

There is a branch to kick it off

https://github.com/meething/webrtc-gun/tree/bandwidth_limit

• D3 render graph
• Move sim to webworker
• integrate into a class that consumes network msg from presence.js

This is to show who we are connected with in the room.
Future addon would be to show all connected people in the lobby across the global network.
Although not power hungry at all, webworker will make sure we don't bog down the main thread

Im excited!!

reconnect almost works see the new client coming in but no video or audio just empty socket id in the grid

Sometimes it seems the local stream start black and changes back when I mute unmute cameras.

this is what had put in submission cause we needed to say something, haha, of course can be revised:

Splitting into 2 sub teams, Product & Infrastructure.

Week -1: Lorenzo built a working video conference prototype with GUN as experiment. https://github.com/meething/webrtc-gun

Week 0: Organized team, do calls, assign roles, submit this application. (Now)

Week 1: Kshitij, Cale, Alex get https://github.com/justadudewhohacks/face-api.js up & running on a simple test page to familiarize themselves with API. Mark, Lorenzo, Dirk familiarize with intercepting Video streams while still feeding it to WebRTC & SFU.

Week 2: Product team makes a PR for ugly integration of face tracker into Meething repo. Infra team connects video streams to it.

Week 3: Product team tests that it works & gets user feedback on UX improvements. Infra team tests using GUN to control SFU relays.

Week 4: Product team cleans up UX. Infra team deals with networking disasters & delays.

Week 5: Product team polishes experience. Infra team debugs dropped connections & hiccups.

Week 6: Test, test test.

Week 7: Product team refactors code for better browser/device compatibility. Infra team fine-tunes default down-sampling rates on video streams to match real world bandwidth limits.

Week 8: Prepare splash page, presentation, pitch, and demo. Fix last minute bugs/features.

We might want to consider adopting this to improve interoperability between FF and Chrome SDP
sdp-interop

It should be quite trivial (and fun?) migrating the current video layout into gridster and let users arrange and modify video streams as they see fit. Should only need basic grid, and a couple functions to add/remove the blocks containing remote user video.

https://gridstackjs.com/#demo

Up for grabs if anyone likes the idea and feels like it!

When I make one video full height and drag it over another one the other one dissapears probably it puts it underneath. Videos should be auto arrange when the user does something that is not fitting correctly or maybe move to original place

Love what you're doing here!!

I'd like to request a feature: screen sharing, something like muaz-khan has acchieved, would be super useful feature :)

I hope I am able to contribute in your repo soon-ish, but a bit busy ATM.

You can ping me at GUN discord/gitter tho :)

BR,
Jabis

An idea from my wife.

"What if you just missed something and you wanted to take a quick saving of the last 30 seconds you missed. You should have a button for that."