Git Product home page Git Product logo

Comments (71)

marionleborgne avatar marionleborgne commented on July 16, 2024 3

@stephenplusplus Thanks for helping out with the issue. In case this helps with defining the scope of the use case:

I am experiencing the same issue as @jmillan [1] and my use case is:

  1. Capture audio from the browser's microphone.
  2. Use streamingRecognize to transcribe audio being streamed in real-time.

No buffering / saving to file / uploading audio to GCS.

Thanks for your help.


[1] I.e. I am getting the same error after transcribing audio for a while (~5m but duration isn't consistent):

events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: 14 UNAVAILABLE: 408:Request Time-out
    at Object.exports.createStatusError (/node_modules/grpc/src/common.js:87:15)
    at ClientDuplexStream._emitStatusIfDone (/node_modules/grpc/src/client.js:235:26)
    at ClientDuplexStream._receiveStatus (/node_modules/grpc/src/client.js:213:8)
    at Object.onReceiveStatus (/node_modules/grpc/src/client_interceptors.js:1290:15)
    at InterceptingListener._callNext (/node_modules/grpc/src/client_interceptors.js:564:42)
    at InterceptingListener.onReceiveStatus (/node_modules/grpc/src/client_interceptors.js:614:8)
    at /node_modules/grpc/src/client_interceptors.js:1110:18

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024 3

Hi,

Bug is reproducible now.

(node:6) DeprecationWarning: grpc.load: Use the @grpc/proto-loader module with grpc.loadPackageDefinition instead
streamingRecognize() "data" event error [code:11,message:Audio Timeout Error: Long duration elapsed without audio. Audio should be sent close to real time.]:

events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: 14 UNAVAILABLE: 408:Request Time-out
    at Object.exports.createStatusError (/service/node_modules/grpc/src/common.js:87:15)
    at ClientDuplexStream._emitStatusIfDone (/service/node_modules/grpc/src/client.js:235:26)
    at ClientDuplexStream._receiveStatus (/service/node_modules/grpc/src/client.js:213:8)
    at Object.onReceiveStatus (/service/node_modules/grpc/src/client_interceptors.js:1290:15)
    at InterceptingListener._callNext (/service/node_modules/grpc/src/client_interceptors.js:564:42)
    at InterceptingListener.onReceiveStatus (/service/node_modules/grpc/src/client_interceptors.js:614:8)
    at /service/node_modules/grpc/src/client_interceptors.js:1110:18

How get the test environment

git clone https://github.com/jmillan/NodeJsSpeechIssue62.git

Follow the instructions in the client and server folders.

Test environment description

The project has two parts: client and server.

Client

Takes your microphone as input audio if your platform is MacOS or Linux, otherwise it generates random audio. The audio is encoded into RTP and sent to the server.

Server

Runs in a Docker, hence you need to have it installed. Receives the RTP from the client, decodes it and passes the raw audio to Google speech.

How reproduce the crash

  1. Run the client.
  2. Run the server.
  3. Kill the client (Ctrl-C)
  4. Wait and check the server output. The crash will appear within a couple or three minutes.

from nodejs-speech.

ibc avatar ibc commented on July 16, 2024 3

@jerjou: nobody is telling that it should not fail. We just say that it should NOT crash the Node process. A library should never crash the whole Node process, but node-speech (or its dependencies gax or grpc-node) are crashing the Node process. Why?

Because, in some circumstances, nodejs-speech is emitting an "error" event that is not handled by any event listener (nor internally and neither possible to listen to it from the app given that no API is exposed for that). And it happens that, in Node, when an "error" event is emitted and there is no event listener for it, the whole Node process exits with error:

https://nodejs.org/api/events.html

This is the bug, and not whether the lib expects continuous audio stream or not. The app may do a "bad" usage of the library, but that should never imply a whole Node process crash.

Is that one reproducible?

Yes, just please check https://github.com/jmillan/NodeJsSpeechIssue62, an environment to reproduce the error.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024 3

Hi,

I'm about to test the fix.

The issue was opened for v1.4.0. The CHANGELOG.md file indicates that this has been fixed for v2.1.0.

I've updated to v2.1.0 and I'm getting the following error (error event in streamingRecognize() returned instance:

[3 INVALID_ARGUMENT: Malordered Data Received. Send exactly one config, followed by audio data.]

Would you be so kind to respond these questions,

  • How has the API changed for this error to happen?

  • Is v2.1.0 stable or beta?

In summary. I just would like to upgrade to the fixed version and make it work as before.

Thanks

from nodejs-speech.

stephenplusplus avatar stephenplusplus commented on July 16, 2024 2

Sorry for the trouble. I have a feeling that is a new issue, so I moved it over to here: #170

from nodejs-speech.

atifsajjad avatar atifsajjad commented on July 16, 2024 1

Similar error for me as well:

events.js:183
throw er; // Unhandled 'error' event
^

Error: 14 UNAVAILABLE: 408:Request Time-out
at Object.exports.createStatusError (\node_modules\grpc\src\common.js:87:15)
at ClientDuplexStream._emitStatusIfDone (\node_modules\grpc\src\client.js:235:26)
at ClientDuplexStream._receiveStatus (\node_modules\grpc\src\client.js:213:8)
at Object.onReceiveStatus (\node_modules\grpc\src\client_interceptors.js:1316:15)
at InterceptingListener._callNext (\node_modules\grpc\src\client_interceptors.js:590:42)
at InterceptingListener.onReceiveStatus (\node_modules\grpc\src\client_interceptors.js:640:8)
at \node_modules\grpc\src\client_interceptors.js:1136:18

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024 1

Hi @atifsajjad,

You can make use of Node's uncaughtException event. Beware that by that time you will only be able to log or reload your service.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024 1

Sure, I'll prepare it.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024 1

@stephenplusplus,

This issue relates to streamingRecognize being used to transcribe live audio streams. No buffering.
I'm preparing the test code by using an audio file and streamingRecognize in order make it easier to automate and test, which I already did in the past.

@marionleborgne,

That's exactly the issue.

from nodejs-speech.

stephenplusplus avatar stephenplusplus commented on July 16, 2024 1

Thanks. I believe my question from #62 (comment) is still accurate, then.

@JustinBeckwith / @kinwa91 could you forward this question to someone from the Speech team? Currently, it seems the behavior users are expecting is not possible.

from nodejs-speech.

niravpatel2008 avatar niravpatel2008 commented on July 16, 2024

Hi @jmillan

I was also facing this issue and cause of that my application get crashed.

I just get fixed in tricky way Create new recognition object after every 50 sec and unpipe previous one
And use try cache while creating new object of recognition and in cache recall init method like below code.

var recognizeStream;
var initSpeechToText = function(){
		try
		{
			return self.speech_to_text.streamingRecognize(self.recognizeStreamOpt)
				.on('error', (e)=>onError)
				.on('data', data => onData)	
		}
		catch (e)
		{
			console.log("ERROR BIG", e);
			return initSpeechToText();
		}
}

var startStt = function(flag){
	if (flag)
	{
		bufferStream.unpipe(recognizeStream);
		console.log("re init after 60 sec");
	}
	
	recognizeStream = initSpeechToText();
	bufferStream.pipe(recognizeStream);
}

startStt();
sstInterval = setInterval(() => { startStt(true) } , 60000);

will fix bad gateway 502 error.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Hi @niravpatel2008,

I just get fixed in tricky way Create new recolonization object after every 50 sec

I guess you mean recognition when saying recolonization. Do you refer to the write stream returned by streamingRecognize()?

and unpipe previous one will fix bad gateway 502 error.

Let me try to understand what you say. Lets consider that we already have a read stream piped to a write steam and we are getting the transcripts:

  1. Create new write stream out of streamingRecognize()
  2. Unpipe read stream from current write stream.
  3. Once unpiped, pipe read stream with new write stream, which is from now on the current one.
  4. Wait 50 seconds
  5. Goto 1

from nodejs-speech.

niravpatel2008 avatar niravpatel2008 commented on July 16, 2024

@jmillan

Sorry its my mistake due to auto correct(its recognition not recolonization).

Yes you are on right track please implement your steps and hope it will also works for you.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Already testing,

Thanks for you comments @niravpatel2008

from nodejs-speech.

niravpatel2008 avatar niravpatel2008 commented on July 16, 2024

@jmillan

Please have look at update comment

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Sill experiencing the crash within this code:

	_start()
	{
		logger.debug('_start()');

		this._recognizeStream = this._createRecognizeStream();

		// Pipe the audio stream into the Speech API.
		this._audioStream.pipe(this._recognizeStream);

		// Substitute current recognizeStream with a new one before timeout expiration.
		setTimeout(() =>
		{
			this._audioStream.unpipe(this._recognizeStream);
		}, 50 * 1000);
	}

	_createRecognizeStream()
	{
		logger.debug('_createRecognizeStream()');

		const recognizeStream = this._client.streamingRecognize(this._request);

		recognizeStream
			.on('error', (error) =>
			{
				logger.error('streamingRecognize() "error" event [%s]', error.message);
			})
			.on('data', (data) =>
			{
				if (data.error)
					logger.error('streamingRecognize() "data" event error [%s]', data.error);

				else
					logger.error(data.results[0].alternatives[0].transcript);
			})
			.on('unpipe', () =>
			{
				logger.debug('unpiped, ending old recognizeStream');

				recognizeStream.end();

				this._start();
			});

		return recognizeStream;
	}

After some time:

events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: 14 UNAVAILABLE: 502:Bad Gateway
    at createStatusError (/service/node_modules/grpc/src/client.js:64:15)
    at ClientDuplexStream._emitStatusIfDone (/service/node_modules/grpc/src/client.js:270:19)
    at ClientDuplexStream._receiveStatus (/service/node_modules/grpc/src/client.js:248:8)
    at /service/node_modules/grpc/src/client.js:804:12

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Please have look at update comment

I just have seen this comment. Checking right now.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Not even with the minimum code possible

	_start()
	{
		logger.debug('_start()');

		this._recognizeStream = this._createRecognizeStream();

		// Pipe the audio stream into the Speech API.
		this._audioStream.pipe(this._recognizeStream);

		// Substitute current recognizeStream with a new one before timeout expiration.
		setTimeout(() =>
		{
			this._audioStream.unpipe(this._recognizeStream);

			this._start();
		}, 60 * 1000);
	}

	_createRecognizeStream()
	{
		logger.debug('_createRecognizeStream()');

		const recognizeStream = this._client.streamingRecognize(this._request);

		recognizeStream
			.on('error', (error) =>
			{
				logger.error('streamingRecognize() "error" event [%s]', error.message);
			})
			.on('data', (data) =>
			{
				if (data.error)
					logger.error('streamingRecognize() "data" event error [%s]', data.error);

				else
					logger.error(data.results[0].alternatives[0].transcript);
			});

		return recognizeStream;
	}

The issue must be somewhere else releated to grpc itself more than within the use of the speech library.

from nodejs-speech.

niravpatel2008 avatar niravpatel2008 commented on July 16, 2024

@jmillan Please add try cache like below. 502 Bad Gateway is generic error code here is the link for [causes] (https://www.lifewire.com/502-bad-gateway-error-explained-2622939).

So just to fix this issue lets add try catch block in your code which will fix this issue. until grpc lib dev fix in library.

_start()
{
	logger.debug('_start()');

	this._recognizeStream = this._createRecognizeStream();

	// Pipe the audio stream into the Speech API.
	this._audioStream.pipe(this._recognizeStream);

	// Substitute current recognizeStream with a new one before timeout expiration.
	setTimeout(() =>
	{
		this._audioStream.unpipe(this._recognizeStream);

		this._start();
	}, 60 * 1000);
}

_createRecognizeStream()
{
	try
	{
		logger.debug('_createRecognizeStream()');
		
		const recognizeStream = this._client.streamingRecognize(this._request);

		recognizeStream
			.on('error', (error) =>
			{
				logger.error('streamingRecognize() "error" event [%s]', error.message);
			})
			.on('data', (data) =>
			{
				if (data.error)
					logger.error('streamingRecognize() "data" event error [%s]', data.error);

				else
					logger.error(data.results[0].alternatives[0].transcript);
			});

		return recognizeStream;
	}
	catch(e)
	{
		return this._createRecognizeStream();
	}
}

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

I used to have try/catch there originally (before opening this issue) and I was getting the error. Anyway I've just put it and test is ongoing.

Thanks. Will update here.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Exception happened again.

There's no alternative left but going down to grpc.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

In case this brings some light to someone who knows about grpc:

This is result of setting an uncaughtException event handler in the process:

  uncaughtException:  { Error: 14 UNAVAILABLE: 502:Bad Gateway
    at createStatusError (/service/node_modules/grpc/src/client.js:64:15)
    at ClientDuplexStream._emitStatusIfDone (/service/node_modules/grpc/src/client.js:270:19)
    at ClientDuplexStream._receiveStatus (/service/node_modules/grpc/src/client.js:248:8)
    at /service/node_modules/grpc/src/client.js:804:12
  code: 14,
  metadata:
   Metadata {
     _internal_repr: { 'content-length': [Array], date: [Array], 'alt-svc': [Array] } },
  details: '502:Bad Gateway' } +680ms

  uncaughtException metadata:  Metadata {
  _internal_repr:
   { 'content-length': [ '0' ],
     date: [ 'Tue, 24 Apr 2018 15:29:41 GMT' ],
     'alt-svc':
      [ 'hq=":443"; ma=2592000; quic=51303433; quic=51303432; quic=51303431; quic=51303339; quic=51303335,quic=":443"; ma=2592000; v="43,42,41,39,35"' ] } } +15ms

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Reproducible

The 5th time it creates a new write stream via streamingRecognize() it crashes.

from nodejs-speech.

niravpatel2008 avatar niravpatel2008 commented on July 16, 2024

@jmillan i was also facing this issue but by changing in flow as i mentioned above try catch block and timer it fixed for me least its working for me.

from nodejs-speech.

marcelgoya avatar marcelgoya commented on July 16, 2024

@niravpatel2008 I've just tried your try/catch fix and it also didn't work for me.

from nodejs-speech.

marcelgoya avatar marcelgoya commented on July 16, 2024

For the folks from Google, is there any way to catch this exception or does it need to be fixed inside the library?

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

The limits docs clearly state that there is a maximum streaming request time of 5 minutes, and that requests and/or attempts at audio processing in excess of these limits will produce an error.

Could anyone please confirm that what we are trying to achieve is currently not doable with google-speech?

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

niravpatel2008, could you please share some details of your installation?

  • OS
  • Node.js version
  • npm version
  • @google-cloud/speech version
  • Parameters provided to streamRecognize()
  • Audio input type

Thanks in advance.

from nodejs-speech.

niravpatel2008 avatar niravpatel2008 commented on July 16, 2024

@jmillan Sure below are the installation details.

OS : centos 6.5
Node.js version : v6.12.0
npm version: 5.5.1
@google-cloud/speech version : ^1.1.0
Parameters provided to streamRecognize() :

{
			config: {
			  encoding: "LINEAR16",
			  sampleRateHertz: "16000",
			  languageCode: "en-US",
			  profanity_filter: true,
			  model: "phone_call"
			},
			interimResults: true,
		}

Audio input type: wav (16k rate)

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Thanks @niravpatel2008,

How large is the wav file? Does it exceed 5 or even 10 minutes?

from nodejs-speech.

niravpatel2008 avatar niravpatel2008 commented on July 16, 2024

@jmillan i am using wav streaming speech to text. and it wont be predefined but we have tested up to 20 mins of live streaming.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Same scenario right now the problem is not reproduced...

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Ended up crashing again

from nodejs-speech.

phsultan avatar phsultan commented on July 16, 2024

Hi José Luis,

You may want to give a try to a nodejs-speech wrapper I wrote for the same purpose, that is, get a continuous transcription for an audio file which size is not known in advance: https://github.com/phsultan/real-time-audio-transcription

It is currently limited to handling .wav files, as the sampling frequency is actually taken from the .wav header but that can be modified in the future to process raw audio samples. Basically, we aim to take recorded tracks from VoIP servers like Asterisk and FreeSWITCH and process them in real time by sending (synchronized only) transcription requests over the nodejs-speech NPM package.

I'd love to get your feedback for improvements if you find any interest in that program and have some time to try it out, thanks !

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Hi @phsultan,

You are not using streamingRecognize() as I do but embedding the audio in each request. I think our scenarios are not the same.

from nodejs-speech.

niravpatel2008 avatar niravpatel2008 commented on July 16, 2024

@jmillan lets do one thing have one complete example which crashes for you and provide here. so i can download and test in my machine.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Hi @niravpatel2008, the code is the one I've already exposed in this issue. It's a copy-paste literally.

from nodejs-speech.

manwe avatar manwe commented on July 16, 2024

Same problem here. I used similar code as I also want realtime transcription of audio streams during a telephony call.

from nodejs-speech.

phsultan avatar phsultan commented on July 16, 2024

It looks like the reading pace for the Readable Stream attached to the audio needs to be controlled not to pipe audio data too fast, at least from my various tests.

To do so, I set the highWaterMark option on the Readable Stream and use setTimeout in the data listener like so:

let readStream = fs.createReadStream(filename, { highWaterMark: sampleRateHertz*2*0.2 });
  readStream.on('data', (chunk) => {
    readStream.pause();
    setTimeout(() => {
      readStream.resume();
    }, 200);
  })

This would simply read audio data every 200 msec (arbitrarly set and can be obviously changed).

Using this gist on a wav file of unknown size worked in my case (up ti 10 minutes of audio were tested). Be careful not to provide a 2 channels (stereo) file, and adjust the sampling frequency if needed (and the language of course :-)):

node streamingRecognize.js audio.wav --sampleRateHertz=8000 --languageCode=fr-FR

Based on the snippets provided by @jmillan and @niravpatel2008 , I added another gist that tries to match with you guys code but does what's described above.

All my testing have been done on LINEAR16, mono, and 8000Hz as sampling frequency, so make sure to adjust this value.

from nodejs-speech.

manwe avatar manwe commented on July 16, 2024

@phsultan during my tests, when you try to pipe audio too fast while using streamrecognize, you get a message about sending audio in realtime.
My readstream is a live audio recording that it's being generated in realtime. Can't send audio faster than it's being recorded. I send linear mono 8KHz.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Please note that either way we are facing a bug in grpc as show in this comment #62 (comment)

Sooner or later (@phsultan, sometimes it crashes after 80+ minutes) it ends up crashing.

We are talking about a bug here.

from nodejs-speech.

phsultan avatar phsultan commented on July 16, 2024

@manwe sorry I meant not to feed nodejs-speech too fast. In my tests, I start a (recorded) SIP call with FreeSWITCH, then around 5 seconds after the recording (.wav) file has been created, I start the transcription. The code above just makes sure we don't send audio bytes too fast to Google, and than we don't reach the continuously updated end of the audio record.

@jmillan yep, I was actually able to reproduce the crash you reported, but adjusting the audio feed to Google the way I did it in the code examples fixed this crash (calls were lasting at most 10 minutes though). Sorry if it didn't help.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Thanks for your comments @phsultan.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

I was hit by another error. (After two hours of streaming)

It is always the same error 'Unhandled "error" event'. It happens because there is no 'error' event handler in the instance that is emitting the 'error' event.

I am setting such event handler in the instances I create, so this is not happening due to a misuse of the API. Someone in the nodejs-speech team could please check or scale?

events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: 14 UNAVAILABLE: EOF
    at createStatusError (/service/node_modules/grpc/src/client.js:64:15)
    at ClientDuplexStream._emitStatusIfDone (/service/node_modules/grpc/src/client.js:270:19)
    at ClientDuplexStream._receiveStatus (/service/node_modules/grpc/src/client.js:248:8)
    at /service/node_modules/grpc/src/client.js:804:12

from nodejs-speech.

enlaichu avatar enlaichu commented on July 16, 2024

We're seeing this error as well, after about 4 mins of use. Doesn't happen all the time but it's happened a few times for sure.

1|server   | Error: 14 UNAVAILABLE: 502:Bad Gateway
1|server   |     at createStatusError (/speech-recognition/node_modules/grpc/src/client.js:64:15)
1|server   |     at ClientDuplexStream._emitStatusIfDone (/speech-recognition/node_modules/grpc/src/client.js:270:19)
1|server   |     at ClientDuplexStream._receiveStatus (/speech-recognition/node_modules/grpc/src/client.js:248:8)
1|server   |     at /speech-recognition/node_modules/grpc/src/client.js:804:12

from nodejs-speech.

pvanickova avatar pvanickova commented on July 16, 2024

We are facing the same issue - streaming audio, 502 and node.js crash:

Error: 14 UNAVAILABLE: 502:Bad Gateway
    at createStatusError (/app/node_modules/grpc/src/client.js:64:15)
    at ClientDuplexStream._emitStatusIfDone (/app/node_modules/grpc/src/client.js:270:19)
    at ClientDuplexStream._receiveStatus (/app/node_modules/grpc/src/client.js:248:8)
    at /app/node_modules/grpc/src/client.js:804:12
Emitted 'error' event at:
    at StreamProxy.Duplexify._destroy (/app/node_modules/duplexify/index.js:197:15)
    at /app/node_modules/duplexify/index.js:181:10
    at process._tickCallback (internal/process/next_tick.js:176:11)

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Hi,

Anything I could do for helping just let me know. My current approach is restarting the service on every crash.

from nodejs-speech.

pvanickova avatar pvanickova commented on July 16, 2024

Hello,
I was just wondering if there has been any progress with this issue or if there's anything I could do to help with the fix.

from nodejs-speech.

jkwlui avatar jkwlui commented on July 16, 2024

As I'm not familiar with the code base, can you help look into this issue? Thanks @stephenplusplus

from nodejs-speech.

atifsajjad avatar atifsajjad commented on July 16, 2024

@jmillan I'm also trying to do the same but having some trouble with it. Would you be kind enough to share how you catch the unhandled error event without knowing what exact object is throwing the error? I can't seem to figure it out.

from nodejs-speech.

atifsajjad avatar atifsajjad commented on July 16, 2024

Thank you very much :)

from nodejs-speech.

jkwlui avatar jkwlui commented on July 16, 2024

ping-ping @stephenplusplus - are you able to help out? If not I can reassign :)

from nodejs-speech.

stephenplusplus avatar stephenplusplus commented on July 16, 2024

@jmillan thanks for your patience thus far. Would you be able to include a full reproduction sample, including instantiating your wrapper with a provided audio file? Then, we will all have the same materials to go from.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Hi,

I need to troubleshoot why I'm not getting any answer to the initial google api HTTP request. I'm using the same credentials json file as in the server.

This is the reproduction sample code: https://github.com/jmillan/NodeJsSpeechIssue62

I'll come back once fixed.

from nodejs-speech.

stephenplusplus avatar stephenplusplus commented on July 16, 2024

@kinwa91 @JustinBeckwith There's a general question here that might be best for a Speech representative:

My main question is: Is it actually possible to achieve continuous transcriptions of undefined audio lengths by using StreamingRecognize or any other ways provided by this service?

I'm going to rephrase-- please let me know if the question is still accurate @jmillan:

Is there a way to stream long audio inputs for processing by the Speech API? For example, if you have a 30 minute file locally, you currently have 2 options, AFAIK:

  1. Buffer it into memory, and provide in configuration to longRunningRecognize(config)
  2. Upload it to the internet (publicly-accessible URL or GCS), and provide URL in configuration to longRunningRecognize(config)

I believe another use case would be live input from a microphone. I think ultimately what we're looking for is something that won't require uploading to the internet or buffering into memory.


@jmillan - regarding the uncaught error, I'm still investigating. I think chasing down answers to the above is more important for now. Again, please let me know if I missed any details.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Locally taking the microphone as the audio input and streaming it with streamingRecognize does not seem to crash. Indeed it does not even fire any error or data event with error due to session time expiration, which it does when the audio source is remote.

Code is available here

from nodejs-speech.

jkwlui avatar jkwlui commented on July 16, 2024

Forwarded the issue with the Speech team - will update here when we know more!

from nodejs-speech.

jerjou avatar jerjou commented on July 16, 2024

...man, javascript has changed a lot since I last touched it. Or maybe it's just that I can't read typescript.
¯\(ツ)

Okay, so my first guess is that you're running into a grpc client deadline. ie the nodejs grpc client's makeBidiStreamRequest takes a client CallOptions object that has a deadline parameter. Presumably, this is the method that's being called behind the scenes by gax or something, but I don't know enough about it (and I can't seem to find it). Presumably that is set to a default value, which I also can't find, but it looks like you might be able to set it.

I'll keep digging, but you might try setting the timeout as mentioned above and see if that fixes it.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Hi,

I'll work on providing you a reproducible environment so you can test and make sure the test is fixed.

from nodejs-speech.

ibc avatar ibc commented on July 16, 2024

@jerjou: Even if setting some parameter to certain value avoided the problem, the bug does exist. And the bug does exist because, somewhere, grpc-node library emits an "error" event that it not handled by gax or by nodejs-speech, and hence the Node process exists with error.

from nodejs-speech.

jerjou avatar jerjou commented on July 16, 2024

This is a different thing - notice that the error message is different:

streamingRecognize() "data" event error [code:11,message:Audio Timeout Error: Long duration elapsed without audio. Audio should be sent close to real time.]:

This happens in the streaming api when I think 5 seconds elapses between audio data being sent to the API. Since it's a streaming API, the API expects audio data to be continuously sent (in near-realtime) until the client signals that it's done (via a halfclose on the underlying grpc connection. For python that corresponds to a StopIteration on the request stream. Not sure for nodejs), and if there's too long a delay between audio chunks being sent, you get this error.

I believe the original error from this bug was:

Error: 14 UNAVAILABLE: 502:Bad Gateway

Is that one reproducible?

from nodejs-speech.

jerjou avatar jerjou commented on July 16, 2024

Ah - sounds like I'm not the appropriate person to address this, then. I'll jump off the thread and leave it to @stephenplusplus and @kinwa91 to provide error-handling hooks in the client lib.

from nodejs-speech.

ibc avatar ibc commented on July 16, 2024

Thanks

from nodejs-speech.

stephenplusplus avatar stephenplusplus commented on July 16, 2024

With the current example (https://github.com/jmillan/NodeJsSpeechIssue62), it's difficult to debug because:

  1. It's trying to do something unsupported
  2. It's a complex app to jump into

Could someone whip up a simpler reproduction case, or streamline the existing one?

from nodejs-speech.

ibc avatar ibc commented on July 16, 2024

Stephen, the issue is at JS level somewhere in nodejs-speech, gax or grpc-node. Some of them (probably the formers) are not setting a needed "error" event handler. If we remove nodejs-speech and gax, obviously there won't be any issue.

I cannot figure out any easier way to reproduce the problem. It crashes 100% of times when running the given steps.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

@stephenplusplus,

With the current example (https://github.com/jmillan/NodeJsSpeechIssue62), it's difficult to debug because:

It's trying to do something unsupported
It's a complex app to jump into

Give this a try: https://github.com/jmillan/NodeJsSpeechIssue62Simplified

0 depedencies, single file, no audio, crash 100% reproductivity.

from nodejs-speech.

stephenplusplus avatar stephenplusplus commented on July 16, 2024

Thank you, @jmillan! Fix sent in #156.

from nodejs-speech.

jmillan avatar jmillan commented on July 16, 2024

Awesome :-)

from nodejs-speech.

timaschew avatar timaschew commented on July 16, 2024

Will the MR also fix errors like this

events.js:165
      throw er; // Unhandled 'error' event
      ^

Error: 14 UNAVAILABLE: TCP Read failed
    at Object.exports.createStatusError (/Users/timaschew/dev/my-mirror/node_modules/grpc/src/common.js:87:15)
    at ClientDuplexStream._emitStatusIfDone (/Users/timaschew/dev/my-mirror/node_modules/grpc/src/client.js:235:26)
    at ClientDuplexStream._receiveStatus (/Users/timaschew/dev/my-mirror/node_modules/grpc/src/client.js:213:8)
    at Object.onReceiveStatus (/Users/timaschew/dev/my-mirror/node_modules/grpc/src/client_interceptors.js:1290:15)
    at InterceptingListener._callNext (/Users/timaschew/dev/my-mirror/node_modules/grpc/src/client_interceptors.js:564:42)
    at InterceptingListener.onReceiveStatus (/Users/timaschew/dev/my-mirror/node_modules/grpc/src/client_interceptors.js:614:8)
    at /Users/timaschew/dev/my-mirror/node_modules/grpc/src/client_interceptors.js:1110:18
Emitted 'error' event at:
    at StreamProxy.Duplexify._destroy (/Users/timaschew/dev/my-mirror/node_modules/duplexify/index.js:191:15)
    at /Users/timaschew/dev/my-mirror/node_modules/duplexify/index.js:182:10
    at process._tickCallback (internal/process/next_tick.js:112:11)

sometimes the log contains this prefix error { streamingError

I'm using @google-cloud/[email protected]

from nodejs-speech.

fatihyasar avatar fatihyasar commented on July 16, 2024

+1

from nodejs-speech.

MallammaHR avatar MallammaHR commented on July 16, 2024

RESTART: /home/pi/Desktop/mallu/python-sdk/examples/microphone-speech-to-text.py
Enter CTRL+C to end recording...
Error received: Handshake status 502 Bad Gateway
Connection closed
sir I am getting this error how to solv ethis problem .please can you help me..

from nodejs-speech.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.