What happens when Float32Array set at AudioWorkletProcessor output is less than 128 sample frames? about web-audio-api-v2 HOT 6 CLOSED

We are not responsible for the lack of tutorials. The spec is explicitly not a tutorial; it is for those sufficiently "skilled in the art", as they say.

While I consider myself "skilled in the art", I have also made mistakes in process(), like accidentally replacing all the <sequence<Float32Array>>s with 0 when I wanted to zero out the outputs. This is one reason we wanted some clarification on what happens if you do silly things like that.

If the latencyHint affects the number of calls to process() such that the average number of calls per second (or minute) is not consistent for "interactive", "balanced", and "playback", file an issue on the browser with a simple test case. It is known that the calls can be very bursty depending on the latency and the audio hardware parameters. This is what getOutputTimestamp() is for.

from web-audio-api-v2.

Apparently outputs at process() is a const variable, immutable. Therefore output = new Float32Array(64) is not possible.

When an output is set with a Float32Array having length less than 128 the rest of the outputs Float32Array remains filled with 0's. That results in observable glitches or gaps in audio output.

However, that part of the graph algorithm does not appear to be clearly specified.

from web-audio-api-v2.

The behavior when you modify the arrays to process() is being discussed in issue #1933.

The processing algorithm may not explicitly say so, but if you read between the lines, everything must process 128 frames at a time. I suppose this could be improved, but I've never heard of anyone else only partially filling the output array and expecting it not to glitch.

from web-audio-api-v2.

One problem is even after the user discovers (not by reading the specification, but through trial and error) that 128 length Float32Arrays are required for there to not be silence (glitches, gaps) in playback, and adjusts code to only pass 128 length Float32Arrays to process() there is still the potential for gaps and glitches in audio output - outside of the Linux/PulseAudio issue (demonstration https://plnkr.co/edit/nECtUZ?preview).

from web-audio-api-v2.

We're not responsible to people who do things without reading the spec. The spec may be rather opaque if you're not familiar, but the spec is not a tutorial and should not be. There are other resources for that.

Gaps and glitches in the audio output are bugs in the implementation. I don't believe there is anything in the spec that could possibly improve that situation.

from web-audio-api-v2.

There are other resources for that.

AFAICT there are no tutorials for AudioWorklet other than https://github.com/GoogleChromeLabs/web-audio-samples/tree/master/audio-worklet, which appear to be more basic examples rather than tutorials.

There is no tutorial that states latencyHint directly impacts the number process() calls. There is no tutorial to parse and provide Float32Arrays having exactly 128 length to process(). Found that for self, experimenting. Without the latencyHint experiment AudioWorklet would probably still be glitching more than it already is.

Started experimenting with AudioWorklet due to the oft-cited claim that somehome WebCodecs proposal would solve Web Audio API issues, though after a month of experimentation have found plausible way that can occur, specifically not using MediaStreamTrack (suggested existing API proposed to be used by WebCodecs), where the Chromium implementers suggest to not use MediaStream or MediaStreamTrack due to the challenge to differentiating known MediaStreamTrack bug from known AudioWorklet bugs. Meaning the suggesting that somehow WebCodecs is a viable solution is pure theory, without any examples of that being possible at all. Nonetheless, more than one Web Audio API issue has been closed citing that until now un-vetted claim.

Perhaps you are mistaken about the intent of filing these issues. Am not asking for help or a tutorial. Can write own code and find the flaws in the specification and implementation for self, and solve own use cases. Am pointing out the deficiencies in the specification and implementation by performing stress tests on what is claimed, not claimed, and implemented re AudioWorklet. For the purpose of overall improvement of each. Or, if the field tests thus far disclosed are ignored, then AudioWorklet will remain broken where it is broken now, either by act, omission, or unintended consequences.

It is obvious not many users in the field have thoroughly tested AudioWorklet in the field. One reason for that is lack of a complete demonstration and tear-down tests. Again, to improve the overall result.

One aspect of the specification that can definitely be improved is clarity.

The specification can clearly state

1. If the Float32Array set at outputs within process() does not have length 128 glitches will occur. That will save time for users in the field who cannot rely on the specification to state that
2. latencyHint directly affects the audio output of AudioWorkletProcessor due to the number of process() calls the which correlate directly to the value set, reproducible by changing the value at https://plnkr.co/edit/vLwcze?preview ("balanced" and "interactive" output glitches when used)

Those are two areas where the specification can be improved, regardless of the implementation.

from web-audio-api-v2.