Real-time screen sharing of video data sent through any protocol that would work well.

This feature is essential to the status of the project itself, making the project useable or not, it's a project milestone without a doubt.

Streaming

I'm unsure what's the ideal protocol for this project, WebSockets? I'd think so, providing an always online central server would not be preferable.

Because of how a VPS would be UNDER heavy pressure even with a small amount of connections, also costly I'd think, having high usage of memory, CPU, bandwidth, even if only would be having e.g 10 sessions concurrently.

Serverless?

Would it be possible to develop a serverless solution using any protocols, that would be very cost effective, saving a lot of money.

If you do know the right way of implementation for this, please contribute with a a comment describing how-to!

Running a 24/7 websocket server, able to receive and then re-transmit real-time data would be the easy way to connect two parts.

Priorities

Security is essential — no matter protocol.

Encrypting the stream should be done by default, keeping sensitive information secure for all end users.

Generally, privacy is what's it should be about, all features should be developed with security and privacy in mind.

Multiple viewers

I see no point in limiting viewer connection count to just one remote client?

It shouldn't be any problem allowing a group of clients to watch someone sharing their screen, sessions where 1 or more watch simultaneous.