Increased data use with websockets about cgm-remote-monitor HOT 18 CLOSED

Odd. I will take a look.

from cgm-remote-monitor.

FYI, the captures start after the browser sessions had stabilized and all objects had loaded so I would see a baseline throughput.

from cgm-remote-monitor.

Looks like because the client and server are very inefficient (they aren't sending deltas, but instead sending 2.5 days of data every minute!) then at the very least I think Free Azure Websites users should not use WebSockets. I propose we update the docs.

from cgm-remote-monitor.

That has always been true. Why is WebSockets so much more overhead than
sending the same data over the old method?

-Scott

On Sun, Nov 9, 2014 at 9:14 PM, Scott Hanselman [email protected]
wrote:

Looks like because the client and server are very inefficient (they aren't
sending deltas, but instead sending 2.5 days of data every minute!) then at
the very least I think Free Azure Websites users should not use WebSockets.
I propose we update the docs.

—
Reply to this email directly or view it on GitHub
#235 (comment)
.

from cgm-remote-monitor.

I agree. I will update the docs, but we also need to see how we can shorten the negotiation timeout to something more acceptable.

from cgm-remote-monitor.

I took a close look at the packet captures for CM with websockets compared with both Brownie and CM without websockets. The data use has the same pattern of keepalive chatter and site refresh data. The websocket chatter is more efficient by ~50% than the xhr chatter... less packets, less data.

The problem is that the site refresh data is literally scaled 5x. Is there something in the code that is sending the refresh data through a loop when using websockets? I find it hard to believe that the web socket protocol would add an order of magnitude in data overhead for the typical 5k/10k site refreshes.

Here's my data to back it up. http://bit.ly/1zFSYUB.

@bewest, @scottleibrand, @shanselman @jasoncalabrese, @rnpenguin, @ktind, what do you think?

from cgm-remote-monitor.

@scottleibrand's PR for the mono sound file improves initial load times and reduces cached data, but the bulk of the data is still related to redrawing the website every minute. How big of a job is it to switch to sending deltas? Does a delta approach introduce new variability to the enduser experience due to browser preferences, default browser cache behavior, etc?

from cgm-remote-monitor.

Maybe we should turn websockets off, until we find a solution. A 5x data increase is going to push people over their limits.

from cgm-remote-monitor.

That was our feeling too. Web sockets should remain off but we will have to explain to people why there is a 10-15 second load delay on their site. Wondering if we should wait on encouraging them to upgrade to CM.

On Nov 10, 2014, at 8:43 AM, Jason Calabrese [email protected] wrote:

Maybe we should turn websockets off, until we find a solution. A 5x data increase is going to push people over their limits.

—
Reply to this email directly or view it on GitHub #235 (comment).

from cgm-remote-monitor.

The websocket change is probably the biggest change on the web monitor side and it was added late in the release. My vote is to pull it out for now.

from cgm-remote-monitor.

I created #239 to disable websockets for now

from cgm-remote-monitor.

0.5.1 has fixed the excessive data out usage as well as the websocket negotiation timeout when disabled in Azure.

Any idea what causes the increased payload size for websocket site refreshes?

from cgm-remote-monitor.

Switching to 'deltas' is on roadmap, but is part of a big change.
FWIW, nothing about polling behavior has been changed in a long time. It could be changed from 1 minute to 4 minutes pretty easily, though.

from cgm-remote-monitor.

After the connection is upgraded to websockets, the 2.5 days of BG data is sent clear-text rather than compressed using gzip/deflate. See this example: http://bit.ly/1xnDNMg

Based on the following link, websocket data compression is out there, but I'm not sure how universally it's supported beyond the newest browsers. If that's a question mark, maybe we shelve websockets until MQTT is ready.

https://www.igvita.com/2013/11/27/configuring-and-optimizing-websocket-compression/

from cgm-remote-monitor.

It looks like that's still an Internet-Draft, and likely won't be supported
in older browsers right away even once it's standardized. So I agree that
we probably don't want to have people on Azure free instances do websockets
for now.

On Mon, Nov 10, 2014 at 5:36 PM, Jim Sifferle [email protected]
wrote:

After the connection is upgraded to websockets, the 2.5 days of BG data is
sent clear-text rather than compressed using gzip/deflate. See this
example: http://bit.ly/1xnDNMg

Based on the following link, I'm not sure whether all current browsers
support websocket data compression.

https://www.igvita.com/2013/11/27/configuring-and-optimizing-websocket-compression/

If that's a question mark, maybe we shelve websockets until MQTT is ready.

—
Reply to this email directly or view it on GitHub
#235 (comment)
.

from cgm-remote-monitor.

Interesting side effect, nice find @jimsiff !

from cgm-remote-monitor.

Nice research @jimsiff. One we can stopping sending all the data each update maybe we could send the initial data via xhr and the rest via websockets

from cgm-remote-monitor.

fixed way back with 0.5.1

from cgm-remote-monitor.