tungs / timeweb Goto Github PK

Is your feature request related to a problem? Please describe.
In the future, it'd be useful to be able to configure timeweb

Describe the solution you'd like
Either or both of:

• A user-definable variable, something like: <script> var timewebConfig = {...}</script> that is occurs before the inclusion of timeweb.js
• A method in timeweb to configure these values

Describe alternatives you've considered
None

Additional context
Some possibilities of configuration:

• When to increment videos (preanimate/postanimate or while iterating through time)
• Incrementing time on every user call that references it, to avoid infinite loops that assume time progression

Is your feature request related to a problem? Please describe.
Certain webpages and libraries run loops based off of time. For instance, breathe.js will loop until 14 milliseconds have passed. If virtual time is not incremented at all, then this condition is never reached, creating unintended behavior and sometimes infinite loops.

Describe the solution you'd like
One way to mitigate this is to automatically increment time a fractional amount each time virtual time is retrieved. This time increment can be customized by the user. This is potentially complicated because timeweb has a virtual timeline of future code to be run that is possibly contingent on virtual time not incrementing. This should be examined of whether this change would possibly break the virtual timeline.

Describe alternatives you've considered
The client code can also be altered to instead have additional conditions. For instance, breathe.js has a throttling mode where after a certain number of iterations, it will relinquish, in addition to checking time.

Is your feature request related to a problem? Please describe.
In some cases, it's desirable to use real-time functions and elements instead of virtual time functions and elements. Currently the way to access the overwritten time-handling functions and code use undocumented, potentially changeable features.

Describe the solution you'd like
timeweb should explicity export these functions to the timeweb object in some way.

Describe alternatives you've considered
The current way of using undocumented features works, but isn't ideal.

Tasks for this feature:

• Figuring out export property name(s) for the old functions. Update: It's realtime. For instance, timeweb.realtime.setTimeout versus document._timeweb_oldSetTimeout.
• Referencing or moving the old objects from those found in overwrite-time to those properties. If the prefixed _timeweb_old globals are removed, this could potentially create some breaking changes. Update: They were removed as of v0.2.0.
• Those calling the old document.createElement and document.createElementNS should also add the property node._timeweb_customized = true when initializing the node and returning it, so it doesn't get processed by timeweb's mutation observer. Update: element._timeweb_customized is renamed/split into two properties element._timeweb_processed and element._timeweb_realtime, for when timeweb has already processed the element or whether it shouldn't be processed at all, respectively.
• (Potentially) Check for a custom attribute in HTML elements in case if the user doesn't want timeweb to overwrite it. Applicable in cases where raw HTML is used and added to the DOM. Maybe something like data-timeweb-realtime.

Is your feature request related to a problem? Please describe.
Web Animations operate a lot like videos. It would be interesting to investigate whether the API can be overwritten to allow for a virtual timeline. Also look into element.animate.

Describe the solution you'd like
Add virtual timeline support for the Web Animations API.

Describe alternatives you've considered
There is a polyfill that may possibly be adopted or adapted to automatically use the already supported JavaScript time handling functions.

Additional context
Originally mentioned by @laggingreflex in tungs/timesnap#21

Is your feature request related to a problem? Please describe.
Currently microtasks aren't considered when iterating through time. Timeweb currently iterates through time in a loop, which doesn't allow for queued microtasks to run.

Describe the solution you'd like
It may be useful if processNextBlock used recursive calls to queueMicrotask, so if any of the functions run with setTimeout/setInterval that queue microtasks would run before the next iteration timeouts/intervals. Also it's worth considering whether other sequential parts of timeweb should relinquish to microtasks (events, requestAnimationFrame)

Describe alternatives you've considered
None.

More Considerations
Microtasks may also be useful when trying to save webgl canvas data before it becomes stale, though I'm not sure about hte lifecycle of canvas data.

Additional context
queueMicrotask API
Guide to Microtasks

This issue is meant to house references to real time currently handled and unhandled by timeweb. The handled features may eventually be moved to the README or wiki.

Scope

timeweb is intended only to cover cases where JavaScript can overwrite time-handling functions and elements in browser. Anything outside of it are considered out of scope.

Handled Cases

• setTimeout(), cancelTimeout()
• setInterval(), cancelInterval()
• requestAnimationFrame(), cancelAnimationFrame()
• Date.now(), performance.now()
• new Date()
• new CustomEvent().timeStamp

Partially Handled Cases

• video elements -- video playing is aimed to be handled by pausing the video and seeking to the frame as needed. Some event dispatching is handled, though there is still unintended event dispatching that isn't entirely handled.
  • video.currentTime should be handled

Unhandled Cases

Probably Possible

• requestIdleCallback Seems like the normal implementation might work in most cases, though the real time timeout may not interact with other timeouts as intended. May want to move this to the virtual timeline anyway, for more deterministic behavior

Might Be Possible, with Some Work

• Web Animations, longer discussion in #4 Also see element.animate, document.timeline, and DocumentTimeline
• SMIL Animations -- Also see tungs/timecut#39

Probably Out of Scope

• CSS Transitions -- Unless there's a library/polyfill that disables CSS Transitions and reimplements them via JavaScript, it's most likely outside of scope
• CSS Animations -- This might be possible by stopping animations and then providing a time-offset, though this might require CSS and selector parsing which would be a significant challenge
• Real-time processors of Audio and Video
• Event Dispatching -- This is pretty low level, so I wouldn't be surprised if it is out of scope. It may be possible to add an event listener that prevents dispatching (this already happens with videos) to every element that tries to add a listener, though this would likely be expensive. There'd also be a need to use a MutationObserver to detatch event listeners declared in DOM and then reattach them after the new dispatcher is attached.
• event.timeStamp -- it looks like the the property .timeStamp can be overwritten via Object.defineProperty(Event.prototype, 'timeStamp', { get: ... }) (see the implementation of CustomEvent), but actually setting the creation time at creation requires overwriting or hooking into the constructor. Making a new constructor for Event does not seem to overwrite normally dispatched events that inherit from Event. It may be possible to hook into other methods that would set a virtual creation time, though I can imagine cases where no other method or object is accessed between creation and usage (for instance, if one is using event dispatch and retrieval it in lieu of a setTimeout).

Is your feature request related to a problem? Please describe.
For certain time-based programs, it'd be useful to be able to set the virtual time so that new Date() and Date.now() can return predictable and setable values.

Describe the solution you'd like
Have a function to set the time, like timeweb.setVirtualTime or timeweb.virtual.setTime (if using the virtual namespace)

Describe alternatives you've considered
None

Additional context
timeweb currently sets the initial virtual time used for Dates based off of the realtime time. Initial running page time is set to zero.

Currently, video nodes still dispatches many events (canplay, etc), and timeweb only prevents some events (play and pause). Since video nodes that timeweb processes are always paused, and then seeked through to "play" them, this can create some unintended consequences. See tungs/timecut#55 for an example, where a canplay event is dispatched every single frame.

Video behavior may be further customizable by adding event listeners in addMediaNode and calling event.stopImmediatePropagation (or event.stopPropagation). That behavior is currently customized for play and pause events.

List of media events can be seen here. Of the list (abort, canplay, canplaythrough, durationchange, emptied, ended, error, loadeddata, loadedmetadata, loadstart, pause, play, playing, progress, ratechange, seeked, seeking, stalled, suspend, timeupdate, volumechange, waiting), only some of them are relevant. For each relevant one, it's likely that custom behavior needs to be defined and custom events need to be dispatched to restore intended behavior.

Is your feature request related to a problem? Please describe.
Currently when timeweb goes to a certain virtual time, it creates a blocking loop (see processUntilTime). This can create performance issues and poor user experience when run inside a browser.

Additionally, the user may want to stop iterating based off of some logic of dynamic page conditions. In this case, they would want to be able to exit the blocking loop too (see tungs/timesnap#63 for an example).

Describe the solution you'd like
It would be nice to be able to set some sort of condition to either relinquish that loop, either based on real time (so as not to freeze the browser), or some user defined criteria.

It would be good to be able to pass a custom function that serves as a condition to end the blocking loop in processUntilTime. It'd be good if the user could distinguish between relinquishing (where the loop is returned to), versus ending.

Describe alternatives you've considered
This could be another function besides goTo, like relinquishingGoTo.

Additional context
Note that some of the capturing of screenshots relies on the blocking nature of timeweb (for instance WebGL canvases can become stale if not captured immediately after rendering). This should be considered when implementing this feature.

Is your feature request related to a problem? Please describe.
There are some situations where I'd like to be able to engage or disengage the virtual timeline.

Describe the solution you'd like
I'd like there to be a function to call that switches from a virtual timeline to a realtime timeline. Events queued in the virtual timeline should then be transferred over to the realtime timeline and vice versa.

Describe alternatives you've considered
An alternative would be to simulate a realtime timeline by keeping the overwritten functions, but then also using a realtime requestAnimationFrame loop to measure an elapsed time, and use that time with timeweb.goTo.

Additional context
In addition to overwriting a window's and document's time handling functions back and forth, video elements would need to be significantly changed.

Currently, when timeweb processes a media node, if the node is playing, it immediately dispatches a play event. If a listener is attached after the node is processed, that event isn't received. This is particularly relevant when nodes are defined in HTML, so the MutationObserver immediately processes them before JavaScript has a chance of attaching a listener.

There's an underlying question of when the play event should be dispatched on autoplayed media. It would be interesting to investigate what spec says, and how current browsers implement this.

The event should maybe be dispatched via a setTimeout (probably in virtual time, though real time may work too).

For a use case, see tungs/timecut#54