replayio / gecko-dev Goto Github PK

The timeline is useful for seeking to messages on it and giving users a sense of where they are in the recording, but a lot more improvements are possible. One that seems especially important is to allow users to use the timeline to see screenshots from anywhere in the recording, just by mousing over that part of the timeline. Clicking on the timeline would take users to that point of the recording to inspect DOM/CSS, and maybe the buttons on the side of the timeline could be used to move through the recording frame-by-frame. I think this would be very handy in a variety of situations, but especially so for users viewing someone else's recording (they can see everything that happened on the page by mousing over the timeline) and for inspector users wanting to inspect state during a user interaction (which I find to be very difficult with existing tools).

If a critical error occurs and replay must stop, we remove the recording/replaying tab and replace it with an about:webreplay tab that describes the error. This logic only works if the replaying tab is selected when the error occurs. It would be better if this was more robust.

about:webreplay shows recordings which the user has saved to the cloud, but if they get a recording from someone else and view it then it would be nice to put that in about:webreplay as well in a 'recently viewed recordings' section.

If a breakpoint is hit while recording, we have to bring a replaying process to the same point and pause it instead, which can be a delay of several seconds even on simple recordings. This is a major area where performance with web replay is worse than with a normal debugger, and it would be good to investigate fixing this. We could pause the recording process at the point instead, which would happen immediately. Because the pause can interact with the recording we would need to pause any replaying processes in the same place and perform the same actions at that point, but that shouldn't be super complicated.

Replaying tabs have a title like "Replaying: UUID", which isn't localized but should be.

Trying to start a recording when viewing a file:// URL turns off recording and goes to a blank page.

Style sheets are treated by the inspector in similar ways to how sources are treated by the debugger: they are added once and persist between pauses. Right now proxies representing style sheets can only be accessed while paused at the place where they were generated. This should be fixed so we have a persistent representation of style sheets which will interoperate better with the inspector code. The stack below shows a failure from this issue.

[Control 19.78] ReplayInspector Server Error: Error: Assertion Failed! Stack: ThrowError@resource://devtools/server/actors/replay/module.js line 7 > Function:99:17
assert@resource://devtools/server/actors/replay/module.js line 7 > Function:88:15
getProxyData@resource://devtools/server/actors/replay/module.js line 7 > Function:3054:9
hasRulesModifiedByCSSOM@resource://devtools/server/actors/replay/module.js line 7 > Function:3124:55
hasRulesModifiedByCSSOM@resource://devtools/server/actors/stylesheets.js:368:27
get canSetRuleText@resource://devtools/server/actors/styles.js:1226:26
getAuthoredCssText@resource://devtools/server/actors/styles.js:1584:9
getApplied@resource://devtools/server/actors/styles.js:602:18
asynchandler@resource://devtools/shared/protocol/Actor.js:154:37
onPacket@resource://devtools/server/debugger-server-connection.js:378:58
receiveMessage@resource://devtools/shared/transport/child-transport.js:66:16
MessageListener.receiveMessage_addListener@resource://devtools/shared/transport/child-transport.js:40:14
ready@resource://devtools/shared/transport/child-transport.js:57:10
_onConnection@resource://devtools/server/debugger-server.js:450:15
connectToParent@resource://devtools/server/debugger-server.js:331:17
onConnect<@resource://devtools/server/startup/frame.js:62:35
exports.makeInfallible/<@resource://devtools/shared/ThreadSafeDevToolsUtils.js:111:22
MessageListener.receiveMessage*@resource://devtools/server/startup/frame.js:92:23
@resource://devtools/server/startup/frame.js:174:5

If the viewport which was used to make a recording is smaller than the viewport which is replaying the recording, the recorded content is shown in the upper left with white margins around it. It would be cleaner if the recorded content was centered in the viewport, and that the margins are shaded to make it clearer that they do not contain web content.

Errors that occur in parent and middleman processes should be included in logs, to help with identifying and fixing errors. Content never runs in either of these processes, so these errors will not contain user information.

Currently we only log data from the UI, middleman, and replaying processes. Recording processes can have important log data as well, in particular errors in the replay JS pulled down from the cloud. Plumbing should be added to send out this data in the same manner as the UI/middleman data.

It might be nice to color code errors, console messages, logpoints, and other kinds of execution points that can be shown on the timeline. It's definitely possible to go overboard here --- allowing users to change the color used for different logpoints might help, or might make things confusing. Regardless, some experimentation here would be nice to see if this is a good approach.

Sometimes there is an unhandled recording divergence when viewing TodoMVC recordings in the inspector, which breaks the UI.

For several technical reasons the inspector does not show anything when unpaused. This can be confusing for users used to seeing the DOM in the inspector update as the page executes. To help with this it would be nice if a screen was shown in the inspector when not paused, which makes it clear why the DOM isn't being shown and how to fix that (press the pause button next to the timeline, or maybe have a big pause button in the inspector itself).

The about page (Web Replay -> About Web Replay) needs to be updated to use Web Replay imagery and content.

The browser toolbar is the primary UI surface area that users will interact with

• left navigation should feel comfortable
• right actions should be as usable as possible
  • it should be easy to start a recording
  • it should be easy to upload a recording

figma

When installing the Web Replay app the screen where the user drags the app to the applications folder is pretty rough, and should be clean and easy to use like other macOS apps.

I think this might be possible with the current UI but I don't know how to do it, and whether it is currently possible or not it would be nice if there were elements of the UI which could be used to zoom in to parts of the timeline, or zoom back out. This would make it a lot easier to deal with the timeline when the recording is long or when there are a lot of messages on it.

here are 2 crash reports that i hit while recording/replaying https://firefox-dev.tools/debugger-examples/examples/todomvc/

https://drive.google.com/drive/u/0/folders/1oLdw6DQESOBLQfGZEjkFsYgui70R-Hbh

I've already removed some toolbar buttons that are shown in a normal firefox browser, but the toolbar still has too much clutter. The following buttons should be removed:

Home button - This doesn't seem so important these days (I never click on it) and distracts from the recording button next to it.

Bookmarks/history button - This is pretty minor and I don't think people using this browser will be super interested in it.

Sidebar button - Ditto.

A startup crash at an apparently null IP was seen while reviewing logs. Backtrace info:

Backtrace #0: __ZN7mozilla12recordreplay6Thread4WaitEv+474
Backtrace #1: __ZN7mozilla12recordreplay17HitEndOfRecordingEv+191
Backtrace #2: __ZN7mozilla12recordreplay21RecordingEventSectionC2EPNS0_6ThreadE+173
Backtrace #3: _RecordReplayInterface_InternalRecordReplayValue+149
Backtrace #4: __ZN7mozilla12recordreplay6Thread13BindToCurrentEv+299
Backtrace #5: __ZN7mozilla12recordreplay6Thread10ThreadMainEPv+41

It should be possible to modify the page's CSS and see the effect on the screen, even when paused somewhere in the recording. This is an important feature of the regular inspector, and when using web replay this should work similarly well. Any changes made to the page layout won't carry over when unpausing and repausing somewhere else, but if the CSS changes could be packed up (there might be functionality for this already) and then reapplied when repausing we could get something similar, and even apply those changes at all paints in the recording so that users can see the result of the changes they make when using the screenshot scrubber on the timeline.

Timeline messages show the script location when moused over, but it would be nice if there was more information shown, including the message text. This might be better done with a doorhanger than a tooltip.

A logo for web replay should be ready soon, and the icons and other imagery in the browser need to be updated to not use the default unbranded nightly images.

STR:

1. add a logpoint
2. warp to a log
3. edit the log

ER: separator is in the right spot
AR: separator is at the bottom

It would be nice to be able to deliver updates to users very quickly. If changes are limited to cloud software (including the control/replay JS downloaded to the user's machine from the cloud) then we can already do this, but it would be nice to have a way to deliver updates to the user's binaries or other files in the downloaded DMG without forcing them to redownload another DMG. If we could use this to ensure that users viewing recordings are on the latest and greatest build then it will simplify versioning issues a lot in the interaction between the user and the cloud.

I don't think firefox can deliver updates via binary patches but I think that chromium can. Since we need our own update channel anyways it seems like delivering updates as binary patches could be part of that.

After the changes in cd5cf44, when an external call is being performed and there isn't information in the recording about how to do it, dummy values (generally zero, though not necessarily) are filled in for the outputs. This can cause the page to render improperly, and while this should be rare, it would be good to show an indicator on the tab when it is the case so that users will not be nearly as confused by the unexpected output.

The initial version of web replay compiled against mozilla-central has a default homepage of google.com and shows blank pages on new tab. The new gecko-dev branch is showing the default firefox new tab page, which has firefox specific stuff in it we don't want. The latter needs to be fixed to match the former.

Right now there is no concept of a user to the web replay browser. Anyone is allowed to upload recordings, and anyone is allowed to view a recording if they have the link (recording links have a UUID which the user would need to know, but those are fairly easy to leak).

Going forward, we need more options in this space. Users need to be able to login in order to view recordings (maybe we can just tie this to a github account, which is I believe what Pernosco does). Users will not need to login in order to make and submit recordings, but if they are logged in we can use that info to let users optionally limit the set of users allowed to view created recordings.

Dependent issues for various things here will be created once the plan is more fleshed out.

My local build cannot upload recordings.

This is because the build is not in the cloud, which is understandable, but worth filing

The payment strategy is yet to be determined, but except for a free trial users will need to pay for a subscription in order to view recordings (the plan for now is to allow submitting recordings for free, which doesn't require a login). Some sort of payment processing system will be needed for this.

There is already a way to jump between different statements in a frame using the mouse (right click -> Continue To Here, maybe some other way too, not sure). This isn't very discoverable and it would be nice to improve the source tab UI when paused in a frame so this navigation is easier. We should be trying to lure users away from the stepping buttons (and regular breakpoints, but that's a separate issue) when possible. There are a couple things here:

• Statements in the frame which are executed should be shown differently from statements which don't execute. I think that graying out code which doesn't execute and/or making it non-interactive to hover actions would work well.

• Hovering over a statement which does execute should show a jump icon somewhere, like we do with console messages. I think the icon would work well in any whitespace to the left of the executed statement. We probably can't show it when there isn't any such whitespace, but that should be pretty rare.

There have been frequent failures when style sheets are being fetched while the debugger isn't paused. Fetching style sheets requires information about the DOM, and we only allow the DOM to be fetched when the debugger is properly paused (it has paused and isn't about to resume). Looking through the source, I think this is happening because the source map service is fetching the style sheets when it starts up, presumably so it can perform source mapping on CSS files.

We do want to support source maps, but our handling of them isn't great right now --- the service only fetches the most recent source map, while we want to be able to retain/fetch source maps for the version of the code that was running while replaying. Until our handling here improves I think the best thing here is to disable fetching style sheets in the source map service.

Crash reports are either not being sent out, or are being sent to a mozilla server. The browser needs to be updated to send crash reports to a web replay address, and we need to be able to process them in the backend.

Users who are doing QA will generally want to always record certain sites they are visiting. If there was a way in about:webreplay to set filters for sites which are always recorded, this workflow would be a lot more efficient vs. having to manually start recording every time.

loading about:webreplay?recording={976ef176-79db-9045-a837-871449242288} takes ~9 seconds for me.

It would be good to bring that time down to ~1 second or introduce a loading animation similar to the profiler

Stopping and restarting recording on a tab sometimes leaves us in a state where the tab is blank, even when it's supposed to be loading content.

Seen while looking at logs, with libevent code on the stack:

FatalError: Event Mismatch: Recorded sendmsg Replayed kevent

This probably indicates that an IPDL message is being sent differently while replaying than while recording. This isn't enough by itself to see what's going on, but we can add diagnostic record/replay asserts and start tracking the problem down. We'll want to leave the diagnostics in, this isn't the first time that non-deterministic behavior has caused messages to be sent differently in the replay.

Links to recordings currently look like about:webreplay?recording=some-uuid. These load correctly when using the web replay browser, but there are some remaining issues:

• These don't look like URLs to pages or apps that can automatically convert URLs into links, which makes them harder to work with.

• If the user opens one of these links in another browser, it won't recognize the link and will take them to some random google search page or something, which probably won't be helpful for the user in figuring out the problem.

I think that both of these problems could be fixed by having a page on webreplay.org or webreplayaws.org and including it in recording links, like https://webreplay.org/view-recording?id=UUID. This page would need to determine whether the user is using a web replay browser (maybe by sniffing the UA string, though we make that look like a regular firefox UA for now). If so it rewrites the URL to the about:webreplay one (which has the chrome logic needed to connect the tab to the cloud server), and if not it shows a help page describing what the user needs to do to view the recording.

There is already functionality for creating recordings of all content processes using a command line option, and saving them to the local disk. This should be extended to allow automatically submitting and logging cloud recordings as well, which should make it easier to integrate with CI services.

When growing the window the tickmarks shown on the timeline aren't updated to fill the new space.

The new recording button UX is pretty good and easy to use I think, but if a user starts up the browser for the first time and doesn't notice it then they'll be pretty lost. This button needs to be easy to use for anyone, including non-technical users who downloaded web replay to reproduce a bug for a developer. It would be good to add a doorhanger to the button to make it really obvious when first starting up the browser.

A few inspector features like the grid inspector are disabled when replaying.

It would be nice to incorporate some of the landing page elements from https://firefox-replay.com/wip.html

• the sections with copy + GIFs do a good job of describing the functionality

The frame timeline should always reflect the current point in the executing frame.

Blank strings and null values are shown as undefined in the watch expressions panel and inline preview.

When a replaying process has diverged from the recording and wants to make a call which we don't know how to resolve, we try to perform that call on the user's machine instead. This is problematic for several reasons:

• If the user is running a different OS than the one the recording was made on, their machine can't perform the call. This will be a headache when we start supporting other OS'es besides macOS. We could use a cloud server running the right OS, but that is a lot of bother and expense without much payback.

• External calls are performed synchronously, and having to perform many of them over a network connection could be incredibly slow even in non-pathological cases.

• Bugs in the external call system can crash the middleman process on the user's machine, bringing down the replay without any chance for crash recovery.

The external call improvements from december should help greatly in reducing the number of external calls made, and I'm hoping we can eliminate the need for them in almost all cases. In places where they are still needed (the user is rendering gfx while replaying that never occurred while recording) we might end up showing some black boxes or something for text that couldn't be rendered, but that seems like it will be ok.

This is with my one local build, so might be something on my machine

There have been a number of startup crashes seen recently. While it would be nice if these were all fixed, it would be better if the crash recovery mechanism could deal with replaying processes that crash before they fork. If we just start up another remote process it probably won't crash.

Similar to other screen capturing tools, it would be nice if the original mouse position was shown on the screen when replaying recordings, with an indicator when it was recently clicked. The click indicator should work well both when scrubbing the timeline or when paused at a scripted location, which might require a little finesse.

After the changes in cd5cf44, we use dummy (generally zero) values for the outputs of external calls whose outputs we couldn't find in the recording. External calls are mainly used for rendering fonts and widgets, and this can lead to text or widgets missing in the rendered output or being rendered as black boxes. After #49 we'll show an indicator in this case, but it would be good if this wasn't so distracting and we showed something very similar at least to what the real output would be. We could look through the recording for similar calls whose outputs we could use, or we could perform some limited emulation of these calls on the host linux system.