otalk / hark Goto Github PK

Sometimes UI becomes unresponsive so user is not able take action.

I've noticed a somewhat lack of activity on issues and PRs, just checking if otalk will be active to help shepard PRs if I start contributing.

I suspect the current max(freq) strategy for is somewhat unstable, since it just takes the maximum and ignores the frequency.

getByteTimeDomainData may enable us to calculate root mean square according to http://tools.ietf.org/html/rfc6465#appendix-A.1
If that doesn't work... we'll have to figure out something with the FFT data.

I get the stream from a user, pass it to hark, and it never says "talking".

• Safari Technology Preview in OSX High Sierra.
• Mute the mic volume.
• Check the dBs value in "volume_change" event:

It decreases until ~-500 and, after that, it becomes -100 (constant). It never reaches -Infinity.
In the other side, in Chrome/Firefox in behaves differently: it decreases gradually to -Infinity.

Not sure if hark may do something to "normalize" it. Currently it's so hard to detect "muted microphone".

Hi!

For projects that don't use browserify is very inconvenient having to download the bundle and have it checked in in git rather that as a external dependency like other libraries. Have you considering adding bower support? Would you be interested in a pull-request with a bower.json to support bower install?

Cheers!

The demo link in README points to http://otalk.github.io/hark/example/. Since getUserMedia() requires now HTTPS, it should be https://otalk.github.io/hark/example/.

at least on chrome/linux. on windows, speaking / not speaking events are still generated.
Firefox doesnt seem to generate those events either.

deleted

Adding 2 mb to a lib just for an example seems kind of crazy. The lib otherwise would be well under 50kb and seems much preferable to 2mb.

The bundle wrapper is doing something wrong. hark.bundle.js is broken. Take a look at the source and you immediately see that getMaxVolume() is not being called anywhere else in the source. I went back to the unbundled version and modernized the variable scope prefixes while I was at it, and it works now. Note: My version is formatted to be used as a module and I am importing wildemitted.js from a a sibling events directory, also as a module.

Here is my "unbundled" version. Tested and it works fine:

// Import dependencies (ensure WildEmitter is available as an ES6 module)

import WildEmitter from '../events/wildemitter.mjs';

export function getMaxVolume(analyser, fftBins) {
  let maxVolume = -Infinity;
  analyser.getFloatFrequencyData(fftBins);

  for (let i = 4, ii = fftBins.length; i < ii; i++) {
    if (fftBins[i] > maxVolume && fftBins[i] < 0) {
      maxVolume = fftBins[i];
    }
  }
  return maxVolume;
}

let audioContextType;
if (typeof window !== 'undefined') {
  audioContextType = window.AudioContext || window.webkitAudioContext;
}

let audioContext = null;

export function hark(stream, options) {
  let harker = new WildEmitter(); // Ensure WildEmitter is correctly imported or implemented

  if (!audioContextType) return harker;

  options = options || {};
  let smoothing = options.smoothing || 0.1,
    interval = options.interval || 50,
    threshold = options.threshold,
    play = options.play,
    history = options.history || 10;

  audioContext = options.audioContext || audioContext || new audioContextType();

  let analyser = audioContext.createAnalyser();
  analyser.fftSize = 512;
  analyser.smoothingTimeConstant = smoothing;
  let fftBins = new Float32Array(analyser.frequencyBinCount);

  let sourceNode;
  if (stream.jquery) stream = stream[0];
  if (stream instanceof HTMLAudioElement || stream instanceof HTMLVideoElement) {
    sourceNode = audioContext.createMediaElementSource(stream);
    if (typeof play === 'undefined') play = true;
    threshold = threshold || -50;
  } else {
    sourceNode = audioContext.createMediaStreamSource(stream);
    threshold = threshold || -50;
  }

  sourceNode.connect(analyser);
  if (play) analyser.connect(audioContext.destination);

  // Implement harker logic...

  return harker;
}

Seems to just stop emitting events if the page is in the background.

In Chromium 62:

[Deprecation] GainNode.gain.value setter smoothing is deprecated and will be removed in M64, around January 2018. Please use setTargetAtTime() instead if smoothing is needed. See https://www.chromestatus.com/features/5287995770929152 for more details.

[Deprecation] AudioParam value setter will become equivalent to AudioParam.setValueAtTime() in M65, around March 2018 See https://webaudio.github.io/web-audio-api/#dom-audioparam-value for more details.

Hey, I noticed that you only had support for audio elements as inputs, would it be possible to also add support for <video> tags? createMediaElementSource supposedly also works with <video> tags

Works perfectly fine for my local audio stream retrieved with getUserMedia(). However when putting in a remote WebRTC stream it does not work. I found this bug: https://code.google.com/p/chromium/issues/detail?id=112367 Do you have any idea how to get this working?

Browser is: Chrome Version 33.0.1750.117

I did an app with hark work great but suddenly the webpage stop working and then i have to reopen (in less than one minute).
i dont know what is happening. Is this normal in the new versions of chrome?

Currently there is no way to start() hark monitoring. When the exposed function hark(stream, options) is triggered, it automatically starts the poll.
A better way would be to:

1. support an api to start the poll: hark.start()
2. on hark.stop(), clear the setTimeout, disconnect the analyser and free associated resources
3. Also, a way to update the media stream at runtime: hark.setStream() - this would first do hark.stop() to teardown the current existing pipeline, and then re-create it using the new stream.

Is there a reason why the latest npm version is on 1.1.6 while the latest release here is 1.2.0?

Can you publish an update to npm?

Sometimes, not sure why and hard to reproduce (but it happens often,) I get this error in Safari OSX:

TypeError: null is not an object (evaluating 'audioContext.createAnalyser')

Obviously it happens here, although I cannot understand why that happens. Looking at the code it's clear that audioContext is never null...

My usage is very simple, I just do this:

this._hark = hark(stream, { play: false });

where stream is always a proper MediaStream instance which always has an audio track.

We've been using AudioContext object to analyze media stream. Chrome plans to change audio policies December 2018 and they'll require user gesture for AudioContext to work:

https://developers.google.com/web/updates/2017/09/autoplay-policy-changes#webaudio

Key Point: If an AudioContext is created prior to the document receiving a user gesture, it will be created in the "suspended" state, and you will need to call resume() after a user gesture is received.

It means, that even if user granted access to mic for given website, audio analysis wont work if i.e. user fired the page and mic is being captured.

Do you have an alternative way to detect current mic level indication?

It might be useful to add a clear note to the readme that this lib currently only works with local mediastreams and not remote media streams that are received through a webrtc peer connection.

example link is 404. //cc @latentflip

i have fixed this in my fork. pull request is #8

A squelch circuit turns off the sound when no one is talking. You'll hear it in police TV shows, when the cops talk to each other over walkie-talkies.

Well, using

var options = {};
  var speechEvents = hark(localStream, options);
  speechEvents.on('speaking', function() {
      stream_part();
    });
    speechEvents.on('stopped_speaking', function() {
      stream_recorder.stopRecording(function() {
        io_chunk(stream_recorder.getBlob());
    });
    });
/* */
function stream_part() {
  stream_recorder = new RecordRTC(localStream, { 
      type: 'audio', mimeType: 'audio/wav',
      recorderType: StereoAudioRecorder, numberOfAudioChannels: 1,
      desiredSampRate: 16000
  });
  stream_recorder.startRecording();
  }
  function io_chunk(part) {
    socket.emit('stream', part);
  }

All ok in chrome, but in Safari records corrupts somehow, how to prevent this, any ideas? Thx!

Typo: the README.md states in the section "Usage" that an event "current_volume" is fired on every poll event. The event that is fired is called "volume_change".

A good addition to the library could be adding an API like getVolume(), which would return the current volume level. Currently, the library does a poll for the speaking events, as well as the volume. Having a poll for speaking events does make sense, but for volume, user might want to run the volume meter poll on the client side, and just query the volume when needed.

Hello,

i am working on a media app that built by react native, and based on my research i come up to hark to listen and ideally convert speech to audio waves like the one in the whatsapp audio message waves.

is it possible to use this library with react native and convert speech to audio waves?

best wishes,

I tried with the latest browser.
https://otalk.github.io/hark/example/
nothing haoppoens both with webcam both with video

I have tried using setInterval() function and options.interval but it's not working.