googlechrome / omnitone Goto Github PK
View Code? Open in Web Editor NEWSpatial Audio Rendering on the web.
Home Page: https://googlechrome.github.io/omnitone
License: Apache License 2.0
Spatial Audio Rendering on the web.
Home Page: https://googlechrome.github.io/omnitone
License: Apache License 2.0
The current HOA is missing the SOA support. Add HRIR data for SOA.
I am unable to run the npm test functionality as described in the readme. I'm running on Windows 10 using PowerShell. Here is the input and return I received.
PS D:\Downloads\omnitone-master\omnitone-master> npm test
[email protected] test D:\Downloads\omnitone-master\omnitone-master
node_modules/karma/bin/karma start
'node_modules' is not recognized as an internal or external command,
operable program or batch file.
npm ERR! Test failed. See above for more details.
I opened up the node_modules/karma/bin/karma file and found it pretty empty other than:
#!/usr/bin/env node
require('../lib/cli').run();
I followed up and open the cli file, and did a text search for node_modules, and came up with zero results. Anyone else having this problem? I'd be happy to debug further if I'm not alone on this.
The ConvolverNode has a parameter normalize which is set to true by default. This adjusts the level of the impulse response based on the RMS value in an attempt to normlize the level of various impulse responses that developers use. It could cause undesirable effects in your spatial audio rendering with multiple HRIRs since they will adjusted by different amounts by this algorithm. Normalize should be set to false.
https://webaudio.github.io/web-audio-api/#attributes-23
Hi nice work with the changes. I have finally updated my helper modules which I will report back soon. It will handle both renderers and handle the correct matrix4 rotation from the three camera now.
However the package filesize has increased. Is there a way to reduce and optimise this ?
There is some uneccessary code I can see I would like to somehow remove ie
https://github.com/GoogleChrome/omnitone/blob/master/src/omnitone.js#L194
I do not need this polyfill
https://github.com/GoogleChrome/omnitone/blob/master/src/polyfill.js#L67
I obtain an audio context from three.js code which provides a polyfill ie
getContext: function () {
if ( context === undefined ) {
context = new ( window.AudioContext || window.webkitAudioContext )();
}
return context;
},
Is there anything else that could be stripped out ? Should I make a fork to do these myself ?
Remove the followings:
src/audiobuffer-manager.js
src/foa-decoder.js
src/foa-phase-matched-filter.js
src/foa-speaker-data.js
src/foa-virtual-speaker.js
test/foa-phasematchedfilter.js
Hi !
Really nice work there.
I'm trying to use this in order to make a web player audio and video 360.
I was trying to use Omnitone with my audio files and there was no problem with that.
Then, I added a html video tag to my html page in order to connect the video sound to Omnitone and spatialize it. The sound is effectively spatialized, but I can hear like it's "crackling", a little disturbing and irregular noise made its apparition.
I tried to treat a local audio file with Omnitone, adding a video tag with a local video file on the same page, and I can still hear the sound crackling.
Moreover, even in a page with just a button to play an audio local file and an audiocontext (just like the foarenderer example), if I have to load another webpage in a new tab or window, there will be more crackling during the loading time. But with the FoaRenderer example online, I don't notice that.
Is this a known issue ? If not, there is a special way to treat the video element sound withtout any lost of quality ??
Thanks a lot !
I get a JS error caused by accessing platform[0]
in Chrome OS cause ua.match
doesnt' check for cros
. See
function getBrowserInfo () {
var ua = navigator.userAgent;
// Check the mobile platform first.
var platform = ua.match(/android|ipad|iphone/i);
if (!platform)
platform = ua.match(/linux|mac os x|win64/i);
// Check Edge first. If fails, check the other major browsers.
var client = ua.match(/(edge)\/(\d+)/i);
if (!client)
client = ua.match(/(chrome|safari|firefox|opera)\/(\d+)/i);
return {
brand: client[1],
version: client[2],
platform: platform[0]
};
}
Remove the followings:
build/resources/E*.wav
src/resource/E*.wav
src/resource/cube.config
src/audiobuffer-manager.js (replaced by buffer-list.js)
I want to pair 360 video with ambisonics, and stream live it on the internet (delay is acceptable for stitching and pairing). Can omnitone be a solution for this issue? What are the possibilities for live VR reproduction?
As I come from the Audio and computer but not from a recent programming or web development environment, I am finding it very difficult to find some information that I consider basic.
It probably is but still it is hard to find written down.
As having recorded Ambisonics during 10 years my technical/programming questions are these,:
is it mandatory for all audio only HOA media to by split in to no more than 8 channel files?
My only indication that it is so is that omnitone enclosed files are split in that way!
If so is there a function / module available that will hide this from a media creator?
When using omnitone in Firefox or Chrome what audio formats are then supported?
I am mostly thinking of formats like Opus, vorbis-ogg and other compressed formats in addition to wav.
As a additional question, what is triggering the activation of HOA decoding in omnitone?
I have found that 16 channel HOA files in wav, ogg and Opus files only decode the first 4 FOA channels.
Demos on https://googlechrome.github.io/omnitone/#gallery don't load.
Seems like an App Engine over quota issue. When trying to load https://storage.googleapis.com/omnitone-demo.google.com.a.appspot.com/resonance-1080p-h264-aac.mp4 I get InsufficientQuota
error.
On Chrome on both Mac and PC clicking the play button on the simple demo.html example gives me:
"VM39 demo.html:44 Uncaught (in promise) DOMException: The element has no supported sources."
line 44: audioElement.play();
Demos on the omnitone website work fine, though....
Hello,
Is it possible to integrate Omnitone with ES6 ?
My issue :
[!] Error: 'omnitone' is not exported by node_modules\omnitone\build\omnitone.js
https://github.com/rollup/rollup/wiki/Troubleshooting#name-is-not-exported-by-module
src\js\linkAudio360.js (2:9)
1: import videojs from 'video.js';
2: import { omnitone } from 'omnitone';
^
also tried : import { * as Omnitone } from 'omnitone';
and : import Omnitone from 'omnitone';
But doesn't seems to work..
Any help?
Thanks in advance
[edit] : Last one seems to work but only like this : Omnitone.Omnitone.createFOARenderer...
Can it be written in another way ?
The library contains the keyword "let" and that broke uglifyjs which does not understand es6. Maybe transpile with babel?
What can be done to support the Three.js integration better? We can consider adding new APIs for this purpose.
FYI: This topic stemmed from this PR.
Currently generating monolithic HRIR data set has to be done manually. Unify the process, so it can be a part of the "build-all" script.
generate-hrir-base64.js
generate-hrir-wave.sh
Currently Omnitone's HRIR loading is asynchronous, mainly because it makes loading/decoding process non-blocking. Using BASE64 data for synchronous loading is reasonable considering the data size is not significantly large, but it's still a blocking process.
cc: @drewbitllama @mrdoob -
Would this be better for Songbird or the other Three.js-based WebVR apps? I know Songbird's construction/initialization is not asynchronous, so I knew this could be a problem.
On the other hand, resource loading in many web apps is asynchronous, so perhaps Songbird can implement to support asynchronous initialization. (construction => wait for resource loading => fire event when ready to use)
README.md contains a link to the HRIRs which appears to be broken. The link under questions is THIS, which is under the FOADeocoder (Fully-configurable) heading.
I noticed this hard link in a few other implementations of omnitone across the web and it always pops up in the web dev consoles as missing files. HERE has some of the same files, but not all that I see called for (e.g. E35.26_A135_D1.4.wav).
The link for THIS still works, but it doesn't look like I can go digging around the directory structure to verify if the other files are there or not though.
The spatialization seems to be incorrect in Version 10.0.3 (12602.4.8).
Seems the old one HRTF filters are not available anymore.
Sample : https://raw.githubusercontent.com/google/spatial-media/master/support/hrtfs/cube//E-35.26_A-135_D1.4.wav
cf. https://github.com/google/spatial-media/tree/master/spatial-audio and this pull request : google/spatial-media#162
The significant code of HOA, SOA support is actually splitting and merging HRIR files. Perhaps we should consider split them all into a pack of stereo files. This will reduce the setup time and the buffer duplication. For example:
Also we do want to keep the information below in src/resources
:
@drewbitllama WDYT?
Is it possible to pass the output through another AudioNode
? I like to use a single GainNode
for muting and adjusting volume of the entire experience in one place.
When trying out HOA rendering with Songbird examples on Safari, we hit the following error:
TypeError: i.copyToChannel is not a function. (In 'i.copyToChannel(t.getChannelData(r),s+r)', 'i.copyToChannel' is undefined)
The latest version is 0.1.2
Eventually HRIR files will be served via: https://www.gstatic.com/external_hosted
If the sound has a loop property, the sound file is playing but there is no sound output.
var audioElement = document.createElement('audio');
audioElement.loop = true;
This issue is also verified when calling audioElement.pause();
when the end of sound is reached then
audioElement.currentTime = 0;
audioElement.play();
Can be verified on this sample : http://labs.plan8.se/ambisonics-webplayer/ with 0.1.8 version.
the "main"
property in this projects package.json file points to src/main.js
. It should point to build/omnitone.js
or build/omnitone.min.js
so the module can be required correctly. Do you have a preference to which file in the build/
directory that should be the "main" entry to the module?
cc @hoch
Just putting this here as a reminder :]
Here is the code using it. I will add a sample online soon (tested with Omnitone 1.0.6 and 1.2.1) :
https://github.com/gopro/forgejs/tree/alps-700-omnitone-fix
https://github.com/gopro/forgejs-samples/tree/alps-700-omnitone-fix
Hi,
There is a problem with this link ?
https://raw.githubusercontent.com/google/spatial-media/master/support/hrtfs/cube/
I had to replace with this (line 1050 in omnitone.js)
var HRTFSET_URL = 'https://raw.githubusercontent.com/GoogleChrome/omnitone/master/src/resources/';
I'm really impressed by the Omnitone examples, except with the audio being out of sync with the video.
The video appears to be going slower that the audio. If I press pause, when I start playing the video jumps forward to where it is supposed to be. I thought that it may be an issue with it streaming over the net, so I've tried running the demo off my own via a localhost and still has the same problem.
Is anybody also getting this problem?
I thought that it may be because of ambisonic decoding but you would expect the audio to be behind the video, not the other way round.
Hi everyone,
I'm a sound designer for video games and 360 videos and I'm really interested in ambisonic audio. I'm trying to make a simple 360 video player that can also play ambix mixing.
So i tried to make a video player in a html page by using the code that omnitone provide. But I'm really bad at coding so it doesn't work.
And i also tried to copy the code from the demo page but it doesn't work locally.
Is someone have already worked on a simple html page with a video ambix player ? Or can someone give me some advice to do this ?
Sorry if I'm not clear enough about my request. Thanks a lot.
On Mac OS 10_11 (El Capitan), safari 9 HRIR does not lead properly, but gives this error:
[Error] [Omnitone] HOARenderer: HRIR loading/decoding failed. (@658.19ms)
This is for both HOA and FOA, this in itself might not be an issue for this project, but I am unable to catch the error in any way.
The error is thrown during the initialize() method, since that is a promise, the .then(success, error) or .catch(error) should work. It does not.
When adding a .then(function() {console.log('success'); }, function() {console.log('error'); });
to the initialize call, it prints nothing and the exception is thrown anyway.
When trying to provoke this behavior on another machine I called the renderer with hrirPathList that would fail, this resulted in another exception being called. The same behavior in not being able to catch the exception persists though.
Recreate this issue this way:
var foaRenderer = Omnitone.createFOARenderer(this.audioCtx, {hrirPathList: ['hello', 'path']}); foaRenderer.initialize().then(function() { console.log('Successful'); }, function(error) { console.log('error', error); });
Expected result:
[error] [Omnitone] BufferList: XHR error while loading "hello(Not Found) [log] error false
Actual result:
[error] [Omnitone] BufferList: XHR error while loading "hello(Not Found)
I'm using this setup to generate my ambisonics files, and pairing this with this viewer
Initially I thought things were working fine, but after playing around with the Ambix plugins within Reaper, and messing around with various locations on the sphere, I noticed some problems.
Position the mono source on either the left or right hand side of the sphere using the Ambix plugins in Reaper (exactly pointing to the middle of the left ear)
Render to .wav as specified in google tutorial
Load into the Photo-Sphere-Viewer
Rotates well and auralizes in the expected location to the left or right as specified.
Position the mono source on the exact front or back of the sphere using the Ambix plugins in Reaper
Render to .wav as specified in google tutorial
Load into the Photo-Sphere-Viewer
Does not rotate or spatalize at all
The rotation and spatalization within Reaper is working fine so I know its not that setup. Also if I import my exported Ambix file back into Reaper, I can control the yaw just fine, so it doesn't seem to be a problem with how I'm rendering the Ambix file either.
Here is my omnitone init code:
/**** Omnitone Code ****/
// Set up an audio element to feed the ambisonic source audio feed.
var audioElement = document.createElement('audio');
audioElement.src = 'audio/AmbisonicOcean.wav';
// Create AudioContext, MediaElementSourceNode and FOARenderer.
var audioContext = new AudioContext();
var audioElementSource =
audioContext.createMediaElementSource(audioElement);
var foaRenderer = Omnitone.createFOARenderer(audioContext, {
HRIRUrl: 'HRTFs/sh_hrir_o_1.wav'
});
// Make connection and start play.
foaRenderer.initialize().then(function () {
audioElementSource.connect(foaRenderer.input);
foaRenderer.output.connect(audioContext.destination);
audioElement.play();
});
viewer.on('ready', function() {
var camera = viewer.camera;
// Rotate the sound field by passing Three.js camera object. (4x4 matrix)
foaRenderer.setRotationMatrixFromCamera(camera.matrix);
// foaDecoder.setRotationMatrixFromCamera(camera.matrix);
});
viewer.on('render', function() {
foaRenderer.setRotationMatrixFromCamera(viewer.camera.matrix)
});
Any thoughts on what is going wrong? As it stands, I can't spatalize anything directly in front or behind the viewer.
There are references to Omnitone.LOG()
all over the place. This implies that Omnitone
should be global. This causes a reference error to be thrown when this library is bundled with tools like browserify.
omnitone is ~50mb in npm. We need to reduce this footprint asap
du -h
reveals the very large size of the repository
√ du -h
301M .
301M total
with a more verbose output:
√ find * | xargs du | sort -nr | awk '{print $2}' | xargs du -h
50M examples/resources
50M examples
50M examples/resources
50M examples/resources
28M examples/resources/jlwarbirds3.amb
22M examples/resources/4ch_B_FuMaNorm_FuMaOrd_speech.wav
52K src/resources
108K src
52K build/resources
104K build
68K doc/diagram-omnitone.png
68K doc
52K src/resources
52K src/resources
52K build/resources
52K build/resources
36K build/omnitone.js
24K test
16K build/omnitone.min.js
12K src/resources/LICENSE
12K build/resources/LICENSE
12K README.md
12K LICENSE
8.0K test/test-setup.js
8.0K src/foa-rotator.js
8.0K src/foa-phase-matched-filter.js
8.0K src/foa-decoder.js
4.0K webpack.config.js
4.0K webpack.config.all.js
4.0K test/test-router.js
4.0K test/test-rotator.js
4.0K test/test-phasematchedfilter.js
4.0K test/index.html
4.0K src/version.js
4.0K src/utils.js
4.0K src/resources/cube.config
4.0K src/resources/README.md
4.0K src/resources/E35.26_A45_D1.4.wav
4.0K src/resources/E35.26_A135_D1.4.wav
4.0K src/resources/E35.26_A-45_D1.4.wav
4.0K src/resources/E35.26_A-135_D1.4.wav
4.0K src/resources/E-35.26_A45_D1.4.wav
4.0K src/resources/E-35.26_A135_D1.4.wav
4.0K src/resources/E-35.26_A-45_D1.4.wav
4.0K src/resources/E-35.26_A-135_D1.4.wav
4.0K src/omnitone.js
4.0K src/main.js
4.0K src/foa-virtual-speaker.js
4.0K src/foa-speaker-data.js
4.0K src/foa-router.js
4.0K src/audiobuffer-manager.js
4.0K package.json
4.0K examples/resources/LICENSE
4.0K examples/demo.html
4.0K build/resources/cube.config
4.0K build/resources/README.md
4.0K build/resources/E35.26_A45_D1.4.wav
4.0K build/resources/E35.26_A135_D1.4.wav
4.0K build/resources/E35.26_A-45_D1.4.wav
4.0K build/resources/E35.26_A-135_D1.4.wav
4.0K build/resources/E-35.26_A45_D1.4.wav
4.0K build/resources/E-35.26_A135_D1.4.wav
4.0K build/resources/E-35.26_A-45_D1.4.wav
4.0K build/resources/E-35.26_A-135_D1.4.wav
4.0K bower.json
4.0K CONTRIBUTING.md
this can be solved with .npmignore
or .files
directive in the package.json
file. I'll have a PR coming shortly
I'm not sure if this is correct but I believe something like this could help dynamically fix the safari channel order from the original channel map config.
This Is what I have implemented according to the documentation.
If this works would this be of interest to add as a global static util ?
OmnitoneUtils.channelMapSafari = function (channels) {
channels.splice(0, 0, channels.splice(2, 1)[0]);
return channels;
};
This package includes an old version of fsevents as an optional dependency. I'm unable to install that package on my version of OSX with the latest Node engine. It's not absolutely necessary, but it does screw things up. The issue I reference seems to be fixed with newer versions of fsevents. But it's an exact version match, so it won't upgrade automatically.
I can't figure out why this dependency is in here, since I don't see it referenced in any other code, unless it's vestigial and not really necessary.
Please upgrade the dependency version or remove it. Thank you.
Demos in the landing page does not work well on mobile phone.
It would be helpful to provide a way to enable debug mode from within the browser. e.g. Add a "try to play anyway" button to the toast message or enable the debug mode state to be specified in the URL.
The information regarding mobile browser support is currently fragmented and confusing. The main source seems to be #2 which specifies:
- Chrome for Android M51 does not support the multichannel audio decoding in MediaElement yet. However, the 4-channel audio stream can be decoded properly with the UnifiedMediaPipeline flag enabled.
- Chrome for Android M53 (which is the dev channel ATM) supports the multichannel audio decoding by default, thus Omnitone can be used there without fiddling flags.
The Beta is currently M52 and the only reference to Unified Media Pipeline is a flag confusingly named "Disable Unified Media Pipeline" (with the option available to "enable" the disabling).
But after attaching a desktop debugger I was able to manually change the value of the player debug
value to 'true' and the Resonance demo played successfully. (Highlight for me: following the sound of the clucking chickens. :) )
Well done on this project. I was wondering if there is a possibility for mediasource support as in Mpeg Dash packaged audio and video ?
If the audio is packaged with the spatial audio tool and then packaged to Mpeg Dash. Would this work ?
According to this it would need to append to a seperate audio tag, then the audiocontext correct ?
https://developer.mozilla.org/en-US/docs/Web/API/AudioContext/createMediaElementSource
https://github.com/mdn/media-source-buffer
The goal is to not have to include seperate audio from the video that becomes out of sync ?
This is a chunked audio demo appending to audio api it seems ? It however is appending two buffers as in assumes stereo channels.
Rather than video, is it possible to show a static 360º image (say via VR View) and play an Ambisonic file that is linked to the viewpoint of the 360º VR View photograph?
Besides the two demos included here, does anyone have their own examples of using Omnitone on the web?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.