Git Product home page Git Product logo

superpoweredsdk / web-audio-javascript-webassembly-sdk-interactive-audio Goto Github PK

View Code? Open in Web Editor NEW
143.0 22.0 16.0 39.21 MB

๐ŸŒ Superpowered Web Audio JavaScript and WebAssembly SDK for modern web browsers. Allows developers to implement low-latency interactive audio features into web sites and web apps with a friendly Javascript API. https://superpowered.com

audio webassembly spatialization time-stretch wasm javascript javascript-library web-audio audioworklet web-audio-module

web-audio-javascript-webassembly-sdk-interactive-audio's Introduction

Superpowered Inc develops the Superpowered Web Audio JavaScript and WebAssembly SDK ("JS/WASM SDK") for modern web browsers, websites, progressive web apps and more.

Developers can use Superpowered interactive audio features in JavaScript without the need of building, initializing or even touching WebAssembly or C++.

For the most up-to-date information, see: https://superpowered.com/js-wasm-overview

JavaScript + WebAssembly

The JS/WASM SDK is contained in this repository. For C++ SDKs for native apps, we offer Superpowered C++ Audio SDK, C++ Networking SDK, and C++ Crypto SDK featuring low-power and real-time latency. They can be found here: https://github.com/superpoweredSDK/Low-Latency-Android-iOS-Linux-Windows-tvOS-macOS-Interactive-Audio-Platform/

Installation and Usage

npm install @superpoweredsdk/web

See usage guide over at https://docs.superpowered.com/getting-started/how-to-integrate?lang=js

Supported Functionality

  • Effects: echo, delay, bitcrusher, flanger, gate, roll, reverb, whoosh, compressor, clipper, limiter, 3 band EQ
  • Filters: resonant low-pass, resonant high-pass, low-shelf, high-shelf, bandpass, notch, parametric
  • Music Analysis: bpm detection, key detection, beatgrid detection, audio waveform, filter bank analysis
  • Object-based 3D Audio Spatializer
  • Mixing: stereo mixer, mono mixer, crossfading, mixing, volume, peak
  • Format conversion (32 bit, 24 bit, 16 bit)
  • Audio Resampler
  • Time domain to frequency domain, frequency domain to time domain
  • Time Stretching, Pitch Shifting
  • FFT: complex, real, real-polar
  • Web Audio I/O, support for ScriptProcessorNode, Workers, Worklets and Audio Worklet

Demos

Real-time (NOT RENDERED), low-latency time-stretching in the browser:
https://superpowered.com/js-wasm-sdk/example_timestretching/

Real-time low-latency reverb and filter in the browser:
https://superpowered.com/js-wasm-sdk/example_effects/

Real-time low-latency guitar distortion in the browser:
https://superpowered.com/js-wasm-sdk/example_guitardistortion/

Supported Web Browsers

The Superpowered Web Audio JavaScript and WebAssembly SDK supports the following web browsers: official public stable versions of all major web browsers, including desktop and mobile variants (iOS, Android), such as Chrome, Safari, Firefox and Opera. The only exception is Microsoft Edge, that requires developer build version 74 minimum.

Support

Superpowered offers multiple support options.

Developer Documentation (C++ and JavaScript): https://docs.superpowered.com

Email: [email protected]

Knowledge base: https://superpowered.zendesk.com/hc/en-us

StackOverflow: https://stackoverflow.com/search?tab=newest&q=superpowered

YouTube: https://www.youtube.com/playlist?list=PLtRKsB6a4xFMXJrZ9wjscOow3nASBoEbU

Paid support options: https://superpowered.com/support

Licensing

JS/WASM SDK is licensed separately on a case-by-case basis. Parties interested in using JS/WASM SDK must contact [email protected]. Free license may be available at our sole discretion. Parties are encouraged to experiment and create private applications with the JS/WASM SDK, but may not launch publicly and/or without a license, which we shall grant at our sole discretion. Please note that any unauthorized use of JS/WASM SDK may result in interruption of service without notice.

For details, please see: https://superpowered.com/licensing

For licensing inquiries, please email [email protected].

Custom Application Development Services

Superpowered offers custom development services focusing on low-latency, interactive audio applications for mobile, web, desktop and embedded.

For development inquiries, please email [email protected].

Contact

If you want to be informed about new code releases, bug fixes, general news and information about Superpowered, please email [email protected].

For licensing inquiries, please email [email protected].

Notes

Superpowered FFT benefits from ideas in Construction of a High-Performance FFT by Eric Postpischil (http://edp.org/resume.htm).

The Superpowered MP3 and AAC decoder benefits from optimizations by Ken Cooke.

web-audio-javascript-webassembly-sdk-interactive-audio's People

Contributors

dodds-cc avatar gaborszanto avatar jsantell avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

web-audio-javascript-webassembly-sdk-interactive-audio's Issues

JS WASM can't process more than 12 AdvancedAudioPlayers

Good day,

I noticed that when processing more than 12 AdvancedAudioPlayers in WASM, some players stop playing and eventually nothing plays. This does not happen on the native platforms I tested (Android, iOS and OSX).

I did extensive testing in these sample apps:

I know that:

  • When it happens, processStereo stops creating an output and returns false, and getPositionMs doesn't advance.
  • With:
    • 10 simultaneous AAPs work fine,
    • 20 AAPs, sound stops after 9515 processStereo()
    • 30 APPs, sound stops after 3695 processStereo()
    • 40 AAPs, sound stops after 151 processStereo()
  • It's not a performance issue. SP takes about 100ms to process over 1000s of audio;
  • It's not a timing issue. I tested processing the audio outside the process() callbacks. Even with all the time in the world, it fails to process all the players;
  • Not a memory issue. Doesn't matter if the WASM is given 32mb or 256mb;
  • It's about the number of playing AAPlayers that are in the playing state at a given time. I can instantiate 40 AAPlayers but if it only plays 5 of them, everything goes fine.

I hope this helps and it's an easy fix. I am out of ideas. Tell me if you want me to try something.
Tiago

Multiple AdvancedAudioPlayers will stop when setPosition is called

I'm having issues when using more that 5 AdvancedAudioPlayers. More specifically, they all stop playing after calling setPosition multiple times.

In this example we have 6 AdvancedAudioPlayers running, and if you click the "random seek" button 4 times, the playback will stop. Check your browser console for following logs:

image

It seem that the first initialized AdvancedAudioPlayers constantly gets out of sync and playback will stop? While things seem to be stopped player.isPlaying() keeps reporting that is playing?

You can play with the number of initialized AdvancedAudioPlayers by changing /processor.js:15

Syncing two advanced players doesn't work

Sorry this still isn't resolved and there are no examples or other issues for me to refer to for help.

There are two players A and B. Both are playing, I know this because I can hear them. Both are playing the same track. The BPM and firstBeatMs are accurate, I know this because the phase of each player individually is reported correctly and is correctly synced to the bar. Both have:

player.outputSamplerate = 44100;
player.timeStretching = false;
player.formantCorrection = 0.5;
player.fixDoubleOrHalfBPM = true;
player.syncMode = this.Superpowered.AdvancedAudioPlayer.SyncMode_TempoAndBeat;
player.originalBPM = trackTempo;
player.firstBeatMs = trackFirstBeatMs;
player.syncToBpm = 120;
player.syncToQuantum = 4;
I'm trying to sync B to A.
Currently I'm calling playerB.syncToPhase = playerA.getPhase(); every call of processAudio but it's not changing the phase of playerB. I've confirmed this by sending the phases back from the audio engine to the UI and showing the phase difference. It should be 0 but it's a random amount that depends on when I start playback of player B.

What are the exact steps I need to take?

Thank you

Trouble getting time stretching example working

Let me preface by saying I'm not too versed in JS so perhaps this is super obvious...

I copied the contents of example_timestretching and superpowered.js to my server, and I get the following error upon pressing the 'Start' button:


superpowered.js:8 Uncaught TypeError: Cannot read property 'addModule' of undefined
    at Object.Module.createAudioNode (superpowered.js:8)
    at HTMLButtonElement.start (main.js:87)
Module.createAudioNode @ superpowered.js:8

It appears the audioContext object does not have a audioWorklet property while running on my server, while if I debug on the running example on the Superpowered server, it does, and the example works.

This is desktop Chrome.

Multiple songs aren't playing due to memory

I tried to load around eight audios in 8 players with average song duration of 3minutes. But the audio stops in between.
This should be something to do with the memory since i tried same with same 8 players but with fewer duration. it worked fine.

how to implement StereoMixer?

Hi there.

I'm implementing superpowered on moises, and looking on the examples you guys made I was able to implement my own time stretching app, however I need to use the StereoMixer feature to synchronously play up to 6 mp3 files, and just by studying the docs I couldn't make it work.

I have a few things that are not clear to me:

  1. ok, I have all my decoded audio data buffers ready to be passed to my Superpowered processor, how should I send it?
    1.1 can it be just the raw buffer or do I need to break it down in left/right channels like we do on the timestretching example?

  2. how should I handle it on processAudio function? I tried to pass all my 4 buffers to the mixer.process(buffer0, buffer1, buffer2, buffer3, output, 128) but it didn't worked.

  3. can I change the pitch and speed rate when I'm using the StereoMixer? or is it only possible using TimeStretching?

  4. would you happen to have an example that implements the StereoMixer? it would be very helpful to see it working live

  5. Is it possible to programmatically seek forward/backward the playback using stereo mixer? In our case we may have up to 6 tracks playing in parallel, since stereo mixer only allows up to 4 inputs, we'll need to use 2 stereo mixers as the docs suggests. Will they keep sync as the user seeks different parts of the audio?

To make it easier to understand our scenario, here is a screenshot of our mixer. We currently use tone.js to handle our audio buffers and control the timeline/pan/mute/solo/volume/seek
image

I couldn't find a solution to my issue on stackoverflow nor on github (in fact, all the content I found was from people using the Android/iOS sdk's ๐Ÿ˜„ ), hopefully we'll see an adoption increase of wasm in the future.

Thankyou!

react js example to get access a micrphone

./src/superpowered/SuperpoweredTrackLoaderModule.js
Line 41:42: 'define' is not defined no-undef
Line 41:57: Expected imports instead of AMD define() import/no-amd
Line 41:57: 'define' is not defined no-undef

Hi there, I got this error, maybe I did smth incorrectly, do u have an example for react.js?

Consider publishing to npm

It would be very helpful if you could publish this library to npm so that we can install it as a dependency instead of having to download it and include the raw files in our project.

If nothing else, you could include a package.json file in the project, which would allow people to install directly from GitHub.

Thanks for the consideration!

The node created callback is called before the onReady() function in the AudioWorkletProcessor

Superpowered.createAudioNode(audioContext, currentPath + '/processor.js', 'MyProcessor',
                function (newNode) {
                    console.log("node");
                });
class MyProcessor extends SuperpoweredModule.AudioWorkletProcessor {

    // runs after the constructor
    onReady() { console.log("ready");}
}

"node" prints before "ready'. This causes problem in using onReady() for initialization since on very short audio files the decoding can finish before onReady() is called.

Analyzing in chunks

I'm trying to analyze some audio in chunks to avoid the OOM errors of doing the whole thing at once. However, the waveforms are always 0 after the first hundred or so elements. Does this look ok?

analyseTrack() {
    const blockLength = Math.min(
      this.channelData[0].length,
      maxAnalysisBlockLength
    );
    let interleavedBuffer = Superpowered.createFloatArray(blockLength * 2);

    const channelBuffers = [
      Superpowered.createFloatArray(blockLength),
      Superpowered.createFloatArray(blockLength)
    ];

    let analyzer = Superpowered.new(
      "Analyzer",
      this.trackSampleRate,
      this.trackDuration
    );

    for (
      let blockStart = 0;
      blockStart < this.channelData[0].length - blockLength;
      blockStart += blockLength
    ) {
      for (let i = 0; i < blockLength; i++) {
        channelBuffers[0].array[i] = this.channelData[0][blockStart + i];
        channelBuffers[1].array[i] = this.channelData[1][blockStart + i];
      }

      Superpowered.Interleave(
        channelBuffers[0].pointer,
        channelBuffers[1].pointer,
        interleavedBuffer.pointer,
        blockLength
      );

      analyzer.process(interleavedBuffer.pointer, blockLength, -1);

      this.sendMessageToMainScope({
        type: "analysisProgress",
        data: blockStart / this.channelData[0].length
      });
    }

    analyzer.makeResults(
      60, // Detected bpm will be more than or equal to this. Recommended value: 60.
      200, // Detected bpm will be less than or equal to this. Recommended value: 200.
      0, // If you know the bpm set it here. Use 0 otherwise.
      0, // Provides a "hint" for the analyzer with this. Use 0 otherwise.
      true, // True: calculate beatgridStartMs. False: save some CPU with not calculating it.
      0, // Provides a "hint" for the analyzer with this. Use 0 otherwise.
      true, // True: make overviewWaveform. False: save some CPU and memory with not making it.
      false, // True: make the low/mid/high waveforms. False: save some CPU and memory with not making them.
      false // True: calculate keyIndex. False: save some CPU with not calculating it.
    );

    const results = {
      bpm: analyzer.bpm,
      beatgridStartMs: analyzer.beatgridStartMs,
      overviewWaveform: analyzer.overviewWaveform,
      peakWaveform: analyzer.peakWaveform
    };

    analyzer.destruct();

    Superpowered.destroyFloatArray(interleavedBuffer);
    for (let buffer of channelBuffers) Superpowered.destroyFloatArray(buffer);

    return results;
  }

Firefox using ScriptProcessorNode

Firefox on MacOS (haven't verified other OS yet) is using ScriptProcessorNode instead of AudioWorklet. Is this intended behavior?

Live analyzing audio input

Hey! I'm playing around with the JS SDK to doing a live analysis of audio input, unfortunately I cannot make Superpowered.Analyzer work. Loading the wasm code, initializing an audio node and connecting to the web audio user input is working fine:

Superpowered.value = await SuperpoweredGlue.fetch('/superpowered.wasm');
Superpowered.value.Initialize({ ... });

WebAudioManager.value = new SuperpoweredWebAudio(44100, Superpowered.value);
const audioInputStream = await WebAudioManager.value.getUserMediaForAudioAsync({ 'fastAndTransparentAudio': true });
const audioInput = WebAudioManager.value.audioContext.createMediaStreamSource(audioInputStream);
AudioNode.value = await WebAudioManager.value.createAudioNodeAsync(ProcessorUrl, 'AudioProcessor', (message) => {});
audioInput.connect(AudioNode.value);
WebAudioManager.value.audioContext.resume();

However inside the audio processor, the analyser will not produce any data:

import { SuperpoweredWebAudio } from '../superpowered/SuperpoweredWebAudio.js';

class AudioProcessor extends SuperpoweredWebAudio.AudioWorkletProcessor {

    onReady() {
        this.analyzer = new this.Superpowered.Analyzer(44100, 60);
    }

    processAudio(inputBuffer, outputBuffer, buffersize, parameters) {
        if(this.analyzing) {
            this.analyzer.process(inputBuffer, buffersize, -1);
            this.analyzer.makeResults(...);
            this.sendMessageToMainScope({ command: 'ANALYSE', value: {
                bpm: this.analyzer.bpm,
                notes: this.analyzer.notes,
            } });
        }
    }
}

if (typeof AudioWorkletProcessor === 'function') registerProcessor('AudioProcessor', AudioProcessor);
export default AudioProcessor;

bpm is always 0 and notes undefined.

Thank you for your help!

Instancing Superpowered objects from custom WASM library

Hi,

I've been having success building a custom WASM library and linking in the superpowered.bc file in a manner similar to the example in statictest however I've hit an issue when trying to instance Superpowered objects/processing units.

The most minimal example of the issue can be seen by modifying main.cpp file in the statictest example as follows:

#include "Superpowered.h"
#include "SuperpoweredSimple.h"
#include "SuperpoweredWhoosh.h"
#include <emscripten.h>
#include <emscripten/bind.h>

static unsigned int testFunction() {
    Superpowered::Initialize("ExampleLicenseKey-WillExpire-OnNextUpdate", false, false, false, false, false, false, false);
    Superpowered::Whoosh *woosh = new Superpowered::Whoosh(44100);
    return Superpowered::Version();
}

EMSCRIPTEN_BINDINGS (TEST) {
    emscripten::function("testFunction", &testFunction);
}

This builds fine using build.sh but testFunction no longer returns a value, and instead I get Aborted() in the console when trying to instantiate a Whoosh.

I've tried with Emscripten 2.0.31/30, Sp SDK version 20400 and 20300, and have tested running ./build.sh on macOS and on Linux.

Peter

TimeStretching advancedProcess implementation example

Although implementation examples for the TimeStretching class are provided for usage with the addInput and getOutput methods, examples of the optimised usage with the advancedProcess method are not provided.

It would be great to see an example of the alternative use of The TImeStretching class with usage of advancedProcess, outputList and the associated logic with AudiopointerlistElement and AudiopointerList.

Syncing two advanced audio players

I'm having a lot of difficulty syncing two AdvancedAudioPlayers. The documentation is confusing and there is only one example.
Each player has the sync mode SyncMode_TempoAndBeat and original bpm and first beat ms set correctly.

The way my code works is that it automatically switches between two players as tracks are loaded. The problem is that they just aren't synced. I'm calling syncToMsElapsedSinceLastBeat to sync each player to the other but it just doesn't work. What am I missing?

Ideally I'd like to sync the tracks on a 4 bars quantisation, rather than beat. But the syncToQuantum thing is confusing.

class Player extends SuperpoweredWebAudio.AudioWorkletProcessor {
  tempo = 120;
  isLoaded = [false, false];

  onReady() {
    this.players = [
      new this.Superpowered.AdvancedAudioPlayer(
        this.samplerate,
        2,
        2,
        0,
        0.501,
        2,
        false
      ),
      new this.Superpowered.AdvancedAudioPlayer(
        this.samplerate,
        2,
        2,
        0,
        0.501,
        2,
        false
      ),
    ];

    this.players.forEach((player) => {
      player.outputSamplerate = 44100; // The player output sample rate in Hz.
      player.timeStretching = false; // Enable/disable time-stretching. Default: true.
      player.formantCorrection = 0.5; // Amount of formant correction, between 0 (none) and 1 (full). Default: 0.
      player.fixDoubleOrHalfBPM = true; // If true and playbackRate is above 1.4f or below 0.6f, it will sync the tempo as half or double. Default: false.
      player.defaultQuantum = 16; // Sets the quantum for quantized synchronization. Example: 4 means 4 beats.
      player.pitchShiftCents = 0; // Pitch shift cents, from -2400 (two octaves down) to 2400 (two octaves up). Use values representing notes (multiply of 100), between -1200 and 1200 for low CPU load. Default: 0 (no pitch shift).
      player.loopOnEOF = false; // If true, jumps back and continues playback. If false, playback stops. Default: false.
      player.reverseToForwardAtLoopStart = false; // If this is true with playing backwards and looping, then reaching the beginning of the loop will change playback direction to forwards. Default: false.
      player.timeStretchingSound = 1; // The sound parameter of the internal TimeStretching instance.
      player.syncMode =
        this.Superpowered.AdvancedAudioPlayer.SyncMode_TempoAndBeat;
    });

    this.nextPlayerIndex = 0;
    this.currPlayerIndex = 1;
  }

  onDestruct() {
    this.players.forEach((player) => player.destruct());
  }

  onMessageFromMainScope(message) {
    if (message.SuperpoweredLoaded) {
      const currPlayer = this.players[this.currPlayerIndex];

      const nextPlayer = this.players[this.nextPlayerIndex];
      this.isLoaded[this.nextPlayerIndex] = false;

      nextPlayer.openMemory(
        this.Superpowered.arrayBufferToWASM(message.SuperpoweredLoaded.buffer),
        false,
        false
      );

      nextPlayer.originalBPM = this.nextTrackData.tempo;
      nextPlayer.firstBeatMs = this.nextTrackData.firstBeatMs;
      nextPlayer.syncToBpm = this.tempo;
    } else if (message.type === "loadTrack") {
      SuperpoweredTrackLoader.downloadAndDecode(
        `/test.mp3`,
        this
      );
      this.nextTrackData = message.data;
    } else if (message.type === "isPlaying") {
      if (!this.players[0].isPlaying())
        this.players.forEach((player) => player.playSynchronized());
      else this.players.forEach((player) => player.pause());
    } else if (message.type === "tempo") {
      this.tempo = message.data;
      this.players.forEach((player) => (player.syncToBpm = this.tempo));
    }
  }

  processAudio(inputBuffer, outputBuffer, buffersize, parameters) {
    const currPlayer = this.players[this.currPlayerIndex];

    const nextPlayer = this.players[this.nextPlayerIndex];
    if (
      nextPlayer.getLatestEvent() ===
      this.Superpowered.AdvancedAudioPlayer.PlayerEvent_Opened
    ) {
      this.isLoaded[this.nextPlayerIndex] = true;
      nextPlayer.syncToMsElapsedSinceLastBeat =
        currPlayer.getMsElapsedSinceLastBeat();
      nextPlayer.setPosition(
        this.nextTrackData.firstBeatMs,
        false,
        false,
        false,
        false
      );
      nextPlayer.playSynchronized();

      this.sendMessageToMainScope({
        type: "onLoaded",
        data: this.nextPlayerIndex,
      });

      const temp = this.nextPlayerIndex;
      this.nextPlayerIndex = this.currPlayerIndex;
      this.currPlayerIndex = temp;
    }

    let isSilence = true;

    // this.players[0].syncToPhase = this.players[1].getPhase();
    // this.players[1].syncToPhase = this.players[0].getPhase();
    this.players[0].syncToMsElapsedSinceLastBeat =
      this.players[1].getMsElapsedSinceLastBeat();
    this.players[1].syncToMsElapsedSinceLastBeat =
      this.players[0].getMsElapsedSinceLastBeat();
    for (let p = 0; p < 2; p++) {
      if (
        this.players[p].processStereo(
          outputBuffer.pointer,
          p > 0,
          buffersize,
          1
        )
      )
        isSilence = false;
    }

    if (isSilence) {
      this.Superpowered.memorySet(outputBuffer.pointer, 0, buffersize * 8);
    }
  }
}

How do you properly cleanup AudioContext and AudioWorkletProcessor? (possible memory leak)

If I try to close the AudioContext and make a new one, along with a new AudioNode, a memory leak happens - it looks like all the raw audio data from the last node is retained in the AudioNode.

See example with slightly modified example_timestretching at https://github.com/svenoaks/superpowered-js-memory-leak.git. Press Start button and let it do it's thing. You'll see the memory from a Google Chrome Helper (or FF equivalent) continually rise.

I can prevent this by never using a new AudioNode, but I think it would be better if I was able to release them properly and create new ones at will.

Missing bitcode (Superpowered.bc) to link against

Previous versions of Superpowered have included Emscripten bitcode file for linking custom WASM builds.

Documentation states:
The library is also available in Emscripten Bitcode format for custom WASM builds.

Will that not be supported going forward?

_free function sometimes throws memory access errors

Depending on what happens before, freeing memory can cause

wasm-001654a2-26:1 Uncaught (in promise) RuntimeError: memory access out of bounds
    at wasm-function[26]:0xb03
    at Object.Module._free (http://localhost:63343/MusicSpeedChangerWeb/superpowered-no-module.js:8:557177)
    at start (http://localhost:63343/MusicSpeedChangerWeb/file-saving-worker.js:163:22)
    at Object.onReady (http://localhost:63343/MusicSpeedChangerWeb/file-saving-worker.js:7:24)
    at Object.Module.onRuntimeInitialized (http://localhost:63343/MusicSpeedChangerWeb/superpowered-no-module.js:8:2531)
    at doRun (http://localhost:63343/MusicSpeedChangerWeb/superpowered-no-module.js:8:563492)
    at run (http://localhost:63343/MusicSpeedChangerWeb/superpowered-no-module.js:8:563652)
    at runCaller (http://localhost:63343/MusicSpeedChangerWeb/superpowered-no-module.js:8:563176)
    at removeRunDependency (http://localhost:63343/MusicSpeedChangerWeb/superpowered-no-module.js:8:19887)
    at receiveInstance (http://localhost:63343/MusicSpeedChangerWeb/superpowered-no-module.js:8:508939)

This is from memory allocated with:

let timeStretching = Superpowered.new('TimeStretching', samplerate, 0.25, 1);
let pcm = Superpowered.createFloatArray(blockSizeFrames * 8);
        let outputBuffer = Superpowered.createFloatArray(blockSizeFrames * 2);

        let pointerL = Superpowered._malloc(blockSizeFrames * 2);
        let pointerR = Superpowered._malloc(blockSizeFrames * 2);

        let leftInts = new Int16Array(
            Superpowered.HEAP16.buffer,
            pointerL,
            blockSizeFrames
        );

        let rightInts = new Int16Array(
            Superpowered.HEAP16.buffer,
            pointerR,
            blockSizeFrames
        );

        let floatL = Superpowered.createFloatArray(blockSizeFrames);
        let floatR = Superpowered.createFloatArray(blockSizeFrames);

and then freed after use:

Superpowered.destroyFloatArray(pcm);
        Superpowered.destroyFloatArray(outputBuffer);
        Superpowered._free(pointerL);
        Superpowered._free(pointerR);
        Superpowered.destroyFloatArray(floatL);
        Superpowered.destroyFloatArray(floatR);
        timeStretching.destruct();

When I introduced a library (https://github.com/egoroof/browser-id3-writer) for writing mp3 tags is when the errors starting happening with some files on Superpowered._free(pointerR);.

Exporting Audio to a File

In my web application, I need to be able to export audio to a file.
The docs says that AdvancedAudioPlayer "Can be used in a real-time audio processing context, but can not be used for offline processing."
Is there any other way to export processed audio using Superpowered? If not, is there some workaround?

superpowered.js can't be loaded in Worklet (without experimental flag)

The only way I could get Superpowered to load in a Worklet script is to enable "Experimental Web Platform features" in Chrome, use
const fileSaveWorker = new Worker("file-saving-worker.js", { type : 'module' }); //that's what requires the flag

when creating the Worker, and add

import SuperpoweredModule from './superpowered.js'

at top of file-saving-worker-js

WIthout the flag, worker can't be loaded as module and can't load modules. Using importScripts() in the worker gives error, as does trying to pass in Superpowered instance as a message.

So, it doesn't look like Superpowered can be used in a Worker yet?

Possibility to embed Superpowered in emscripten build without exposing all Superpowered methods on WASM instance

Hi,

I've been building a C++ -> WASM library using your provided superpowered.bc library.

I can't seem to avoid exporting all of the superpowered methods etc to the JS-accessible interface of the instantiated WASM code (under __functions__) when using your library. This means that the WASM binary contains the entire Superpowered library implementation, regardless of whether or not it is all used.

I'm guessing this is because all of the methods are decorated with EMSCRIPTEN_KEEPALIVE or similar during the build process of the .bc. I've asked on the emscripten GH but there don't seem to be any options to undo the effect of EMSCRIPTEN_KEEPALIVE downstream in the build process.

Do you have any suggestions of how this might be achieved? Or alternatively is it possible to get a build of superpowered (either .bc or .a) without having Superpowered:: namespace methods flagged with EMSCRIPTEN_KEEPALIVE.

I did try converting the .bc to .a using emar, as I thought this might strip the effect of the decoration, but the result of including it in my build was unchanged.

This is not urgent as it isn't blocking my development, but would be nice to have if it was possible to keep the binary and interface as compact as possible.

Peter

ERR_TIMED_OUT when trying to fetch the license

Hello ๐Ÿ‘‹๐Ÿป

While instantiating the superpowered instance, it fails to fetch the license https://superpowered.com/license/.../wasm.txt. We get an ERR_TIMED_OUT error.

Before having this problem, the license used to, sometimes, fail to fetch with a ERR_CONNECTION_REFUSED. Since this morning, however, the license fails to fetch completely.

Thanks

Value for player.firstBeatMs gets reset after after calling player.openMemory

After creating a player inside onReady() I set the value for player.firstBeatMs. I then download and decode an audio file and inside the SuperpoweredLoaded message log the value for player.firstBeatMs before and after calling player.openMemory(). The value for firstBeatMs gets reset back to 0 after calling openMemory. Is this expected behaviour? Other parameters do not get reset. Do we just need to set firstBeatMs in the SuperpoweredLoaded message after calling openMemory?

import { SuperpoweredWebAudio, SuperpoweredTrackLoader } from '/superpowered/SuperpoweredWebAudio.js';

class MyProcessor extends SuperpoweredWebAudio.AudioWorkletProcessor {
    // runs after the constructor
    onReady() {
        this.player = new this.Superpowered.AdvancedAudioPlayer(this.samplerate, 2, 2, 0, 0.501, 2, false);
        this.player.syncMode = this.Superpowered.AdvancedAudioPlayer.SyncMode_TempoAndBeat;
        this.player.firstBeatMs = 1000.24;
        this.bufferCount = 0;
        console.log(this.player.firstBeatMs) // logs 1000.24
        SuperpoweredTrackLoader.downloadAndDecode('example_track.mp3', this);
    }

    onMessageFromMainScope(message) {
        if (message.SuperpoweredLoaded) {
            const { buffer, url } = message.SuperpoweredLoaded;
            console.log(this.player.firstBeatMs) // logs 1000.24
            this.player.openMemory(this.Superpowered.arrayBufferToWASM(buffer), false, false);
            console.log(this.player.firstBeatMs) // logs 0
            this.player.play();
            this.sendMessageToMainScope({ loaded: true });
        }
    }

    processAudio(inputBuffer, outputBuffer, buffersize, parameters) {
        if (this.bufferCount % 5000 === 0) console.log(this.player.firstBeatMs) // logs 0
        
        if (!this.player.processStereo(outputBuffer.pointer, false, buffersize, 1)) {
            for (let n = 0; n < buffersize * 2; n++) outputBuffer.array[n] = 0;
        };
        this.bufferCount++;        
    }
}

if (typeof AudioWorkletProcessor === 'function') registerProcessor('MyProcessor', MyProcessor);
export default MyProcessor;

Cannot reach bottom of TOC in the docs without scrolling to bottom

It's not possible to see the entire contents of Superpowered Javascript SDK without scrolling to the bottom of the entire page.

Steps to reproduce:

  1. Use a screen that's 15" or less
  2. Visit https://superpowered.com/js-wasm-sdk/docs.html
  3. Move pointer to the sidebar with the headings, try to scroll to bottom.

Expected results:

Sidebar is scrolled, revealing the headings further down

Actual results:

Whole page is scrolled, except the sidebar.

OOM error creating a buffer

Creating a float buffer for analysing a 6 minute track causes an OOM error. Is there any way to increase the maximum memory allowance?

Superpowered.createFloatArray(14684006);

Thanks

Throw error instead of showing alert

It appears the superpowered module and superpowered-worker modules call alert when license keys fail. This is a very agressive user interaction, and a preferable pattern would be to throw an error instead so developers can choose how to handle the failure.

If there are other areas in the library where alerts are invoked, they should ideally also be changed to errors. Thank you!

Superpowered.Analyzer and .Waveform are completely broken?

I am trying to evaluate the SDK for a large web audio project, but it is giving me a hard time to get anything donw.
Nothing the analyzer calculates makes any sense.

I have a mono wave file containing a 1kHz sine wave tone for 1s at -3dBFS which I use to check results of the analyzer.

This is my code (running at 44.1 KHz sample rate and it lives inside a worker):

const leftChannel = new Superpowered.Float32Buffer(e.data.inputSabLeftChannel.length);
const rightChannel = new Superpowered.Float32Buffer(e.data.inputSabRightChannel.length);

leftChannel.array = e.data.inputSabLeftChannel.slice();
rightChannel.array = e.data.inputSabRightChannel.slice();

const interleavedArray = new Superpowered.Float32Buffer(e.data.inputSabLeftChannel.length * 2);

Superpowered.Interleave(leftChannel.pointer, rightChannel.pointer, interleavedArray.pointer, leftChannel.length);

const waveform = new Superpowered.Waveform(
  e.data.sampleRate,
  e.data.duration,
);

waveform.process(
  interleavedArray.pointer,
  128,
  e.data.duration,
);

waveform.makeResult();

const peakWaveform = waveform.getPeakWaveform();

console.log('peaks', peakWaveform.array);

First of all the documentation is wrong. The docs tell to access the peak waveform via waveform.peakWaveform which returns undefined. I had to return all the keys of the waveform object to see that I have to call waveform.getPeakWaveform(). But this also returns nonsense.

Because even though I process only 128 frames of the 44100 available frames, the peaksWaveform array has a length of 47312.

The first values are:

[0, 0, 0, 0, 169, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 153, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...]

Which also makes no sense.

Also there is no documentation on what the third parameter on waveform.process actually does. It doesn't seem to matter, because putting in -1, 0, or 1 returns the same peaks array.

Am I doing something wrong here? For context, this is my init code inside the worker:

importScripts('../superpowered/SuperpoweredGlue.js');

// Create a SuperpoweredGlue instance and load WebAssembly:
let Superpowered;
let isReady = false;

SuperpoweredGlue.fetch('../superpowered/superpowered.wasm').then((res) => {
  Superpowered = res;

  // Initialize Superpowered.
  Superpowered.Initialize({
    licenseKey: 'ExampleLicenseKey-WillExpire-OnNextUpdate',
    enableAudioAnalysis: true,
    enableFFTAndFrequencyDomain: true,
    enableAudioTimeStretching: false,
    enableAudioEffects: false,
    enableAudioPlayerAndDecoder: true,
    enableCryptographics: false,
    enableNetworking: false
  });

  isReady = true;
});

Example for playing multiple sounds and/or tracks

Hello,

Is there any example you could provide for playing multiple simultaneous sounds and/or tracks? Something that looks like a mini piano would be really helpful!

I'm trying to have one global audio context, then connecting multiple audio nodes to it. Each process has its own player object. I get no errors, but also no sound.

If you're curious about where I'm doing something wrong, here's the gist of it (pun intended): https://gist.github.com/designbyadrian/947d4090bea34b4aff498cf09f0d7711

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.