Comments (9)
Different AudioWorkletProcessor.process()
callback intervals can be observed when setting different AudioContent.latencyHint
values (https://plnkr.co/edit/qBUEPiBWZyvRxD65F1ws?p=preview), with "balanced"
always having the least number of process()
calls, yet inconsistent results when comaring 0.5
, 1.0
, and "playback"
.
Could not locate language in the current specification that unequivocally states that latencyHint
SHOULD or MUST result in different callback intervals for process()
function, nor that the results should be consistent relevant to the value set.
Is the result working as intended?
from web-audio-api-v2.
It's not really clear how you compute the callback interval. But the latency value can affect when process()
is called. This is also dependent on the sample rate of the audio as well as the operating system.
You might be interested in getOutputTimestamp. See also WebAudio/web-audio-api#12 and related issues.
from web-audio-api-v2.
It's a hint, by definition it is valid, for example, to implement it and do nothing. This is spelled out. The actual latency must however be reported, but doesn't have to match. it can be absolutely backward and still be compliant: an implementation cannot always control the actual latency, regardless of what is being asked by authors.
The timings at which process
are called are not written in the spec, because a performant implementation doesn't control when it's being called, this is explained in section 2.1 (non normative of course).
from web-audio-api-v2.
It's not really clear how you compute the callback interval.
Count the number of times process()
is executed within 1 second with different valuesset for latencyHint
.
There is an observable relationship between the value set at latencyHint
and the number of times process()
is executed within a given time span. However that relationship is not clearly specified (AFAICT not specified at all).
from web-audio-api-v2.
The timings at which process are called are not written in the spec, because a performant implementation doesn't control when it's being called
Well, that appears to be what is occurring in the only implementation of AudioWorklet
that am aware of (Chromium). It is currently possible to observe process()
being executed N number of times directly corresponding the AudioContext.latencyHint
.
from web-audio-api-v2.
For a set of 10 tests, where the value is the number of process()
calls per 1 second.
[
[
{
"balanced": 303
},
{
"interactive": 344
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
],
[
{
"balanced": 302
},
{
"interactive": 347
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
],
[
{
"balanced": 311
},
{
"interactive": 348
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
],
[
{
"balanced": 300
},
{
"interactive": 344
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
],
[
{
"balanced": 308
},
{
"interactive": 348
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
],
[
{
"balanced": 313
},
{
"interactive": 340
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
],
[
{
"balanced": 305
},
{
"interactive": 345
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
],
[
{
"balanced": 302
},
{
"interactive": 348
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
],
[
{
"balanced": 312
},
{
"interactive": 344
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
],
[
{
"balanced": 308
},
{
"interactive": 344
},
{
"playback": 352
},
{
"0.5": 384
},
{
"1": 384
}
]
]
Code to reproduce
main.js
class WorkletNode extends AudioWorkletNode {
constructor(ctx) {
super(ctx, "worklet-processor", {
numberOfInputs: 1,
numberOfOutputs: 1,
channelCount: 1,
});
}
}
const latencyHints = [{
latencyHint: "interactive"
}, {
latencyHint: "balanced"
}, {
latencyHint: "playback"
}, {
latencyHint: 0.5
}, {
latencyHint: 1
}];
async function main(
latencyHint
) {
const processorCallbackCount = [];
const ctx = new AudioContext({
sampleRate: 44100,
latencyHint: latencyHint
});
console.log(ctx.baseLatency);
const osc = new OscillatorNode(ctx, {
type: "sine",
frequency: 400
});
osc.start();
await ctx.audioWorklet.addModule("./processor.js");
const worklet = new WorkletNode(ctx);
worklet.port.onmessage = e => processorCallbackCount.push(e.data);
worklet.onprocessorerror = e => {
console.error(e);
console.trace();
};
osc.connect(worklet).connect(ctx.destination);
await new Promise(resolve => setTimeout(resolve, 1000));
osc.stop();
worklet.port.close();
osc.disconnect(worklet);
worklet.disconnect();
await ctx.suspend();
await ctx.close();
return processorCallbackCount.length;
}
async function getAudioWorkletProcessorLatency() {
const result = [];
for (const {
latencyHint
}
of latencyHints) {
result.push({
[latencyHint]: await main(latencyHint)
});
}
return result.sort((a, b) => Object.values(a)[0] < Object.values(b)[0] ? -1 : 0);
}
async function getAudioWorkletProcessorLatencyStatistics(length) {
const result = [];
for (const _ of Array.from({
length
})) {
result.push(await getAudioWorkletProcessorLatency());
}
return result;
}
getAudioWorkletProcessorLatencyStatistics(10)
.then(result => {
const json = JSON.stringify(result, null, 2);
console.dir(result);
document.querySelector("pre").textContent = json;
}, console.error);
main();
processor.js
class WorkletProcessor extends AudioWorkletProcessor {
process(inputs, outputs, parameters) {
// Output DC.
//console.log(globalThis.setTimeout);
//return;
const input = inputs[0][0];
const output = outputs[0][0];
// console.log(output);
output.set(input);
this.port.postMessage({currentTime});
return true;
}
}
registerProcessor("worklet-processor", WorkletProcessor)
from web-audio-api-v2.
Since you have chosen a sample rate of 44.1 kHz, there should be 44100/128 calls to process, or about 344 calls per sec. If you're not getting this, file a bug against Chrome.
BTW, I think your process method is more complicated than it needs to be and generates too much garbage. Just add a counter and increment it each time it's called. Inspect the currentTime and post a message when 1 sec (or 10 sec or whatever) have passed. No garbage generated.
Anyway, I think the spec is pretty clear: process gets called about 344 times per sec in this case. They may be bursty, but the average should be that.
from web-audio-api-v2.
BTW, I think your process method is more complicated than it needs to be and generates too much garbage.
The code originates from a Chromium bug and is only used to demonstrate that AudioContext.latencyHint
value has a concrete effect on the number of AudioWorkletProcessor.process()
calls - though cannot locate any reference to such a symbiotic relationship between the two at the specification. Is that behaviour intended?
@rtoy The point of this issue is that different values set at AudioContext.latencyHint
directly impact the number of times AudioWorkletProcessor.process()
is executed.
From the responses to this issue thus far it is not clear that is the intended behaviour.
What is clear is that behaviour is not described in Web Audio API specification.
Therefore, the issue is not resolved by any definitive answer from contributors to the specification.
from web-audio-api-v2.
The processing algorithm is described here: https://webaudio.github.io/web-audio-api/#processing-model. The latencyHint
isn't involved.
Anyway this issue is closed. You've already filed a bug for Chrome. Further discussions will happen there. I don't think there's any inconsistency in the spec, but the processing algorithm is pretty complicated.
from web-audio-api-v2.
Related Issues (20)
- V2 documentation logistics HOT 2
- interaction between audio elements and the Web Audio API HOT 2
- Add ability to pause/resume AudioBufferSourceNode HOT 11
- AudioOutputContext HOT 21
- AnalyserNode: provide access to complex FFT result HOT 15
- AnalyserNode: efficiency improvements to FFT post-processing HOT 3
- Alternatives for module loading of AudioWorklet HOT 5
- Should 'Atomics.wait' be available in AudioWorklets associated with an OfflineAudioContext? HOT 8
- Request to Expose AudioBuffer to DedicatedWorker HOT 29
- Rename master branch to main HOT 2
- Set up to display the spec. HOT 10
- AudioWorklet should have a 'oversample' and a 'bufferSize' property HOT 5
- cancelScheduledValues and cancelAndHoldAtTime throws RangeError for non-finite
- Switching between 2 Microphones HOT 1
- No way to convert data from WebCodecs AudioData to AudioBuffer HOT 7
- Add frequency of one oscillator to be based on another HOT 1
- Define TextEncoderStream(), and TextDecoderStream() in AudioWorkletGlobalScope HOT 1
- Add Travis CI HOT 1
- RawPCMStreamNode (StreamNode, or InputStreamNode) HOT 8
- Set up PR review HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from web-audio-api-v2.