Unsupported Operation Exception about streampack HOT 30 CLOSED

I closed this issue because the timestamp bug should probably be filed as a separate issue.

from streampack.

If you look at the code for Android's CameraX project, it might help you. Specifically, lines 348 to 387 of https://github.com/google/mediapipe/blob/master/mediapipe/java/com/google/mediapipe/components/CameraXPreviewHelper.java.

from streampack.

Hi!
I'm also super excited even if I don't have enough time to work on it.

Which Android version do you have?

from streampack.

I'm on version 11.

from streampack.

In CameraExtensions.kt, in Context.getCameraOrientation, can you remove:

val displayRotation = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
        this.display!!.rotation
    } else 

from streampack.

That fixed the issue!

However, I can't get the stream to play. The server is receiving it, but ffplay is unable to display the stream. After maybe 10-20 seconds, ffplay displays the following messages:

12:41:18.339593/T0x700008c41000!W:SRT.qr: @898072221:No room to store incoming packet: offset=0 avail=0 ack.seq=1646609462 pkt.seq=1646609462 rcv-remain=8191 drift=1545
12:41:18.339743/T0x700008c41000!W:SRT.qr: @898072221:No room to store incoming packet: offset=1 avail=0 ack.seq=1646609462 pkt.seq=1646609463 rcv-remain=8191 drift=1545
12:41:19.324344/T0x700008c41000!W:SRT.qr: @898072221:No room to store incoming packet: offset=8 avail=0 ack.seq=1646609462 pkt.seq=1646609470 rcv-remain=8191 drift=1545
12:41:19.324490/T0x700008c41000!W:SRT.qr: @898072221:No room to store incoming packet: offset=9 avail=0 ack.seq=1646609462 pkt.seq=1646609471 rcv-remain=8191 drift=1545
12:41:19.340980/T0x700008c41000!W:SRT.qr: @898072221:No room to store incoming packet: offset=12 avail=0 ack.seq=1646609462 pkt.seq=1646609474 rcv-remain=8191 drift=1545
12:41:19.341026/T0x700008c41000!W:SRT.qr: @898072221:No room to store incoming packet: offset=13 avail=0 ack.seq=1646609462 pkt.seq=1646609475 rcv-remain=8191 drift=1545
12:41:19.372917/T0x700008c41000!W:SRT.qr: @898072221:No room to store incoming packet: offset=17 avail=0 ack.seq=1646609462 pkt.seq=1646609479 rcv-remain=8191 drift=1545

from streampack.

Ok, then I will temporary rollback 87b0f3d

About your second issue, I have never seen it. It seems that ffmpeg can't demux.
I will have a look in ffmpeg.

from streampack.

Have you change the settings?
Do you have any log in your phone logcat?

from streampack.

Here are my settings:

from streampack.

Do you have specific settings for ffplay?
Which version of srt do you use in ffplay?

from streampack.

Do you have specific settings for ffplay?

I'm using the command you provided.

Which version of srt do you use in ffplay?

I'm on version 1.4.3

from streampack.

I found this: Haivision/srt#409
"no room to store" means:

• either ffplay is to slow to read
• either ffplay read buffer are too small.

But in both case, I don't see why I don't have the same issue.

from streampack.

Can you switch to srt 1.4.4 in ffplay so we get a better log message?

from streampack.

I got the stream to play, but there's a delay of about 1-2 minutes before it starts playing.

What version of SRT are you on? Isn't 1.4.3 the latest?

from streampack.

You are right, 1.4.3 is the latest.

from streampack.

Do you have any issue in logcat?
You can also try to play with rcvbuf in ffplay command line.

from streampack.

Here's my logcat output:

2021-06-03 14:43:43.255 21219-21219/com.github.thibaultbee.streampack.sample I/PreviewViewModel: Connection succeeded
2021-06-03 14:43:43.272 21219-21437/com.github.thibaultbee.streampack.sample W/Codec2Client: query -- param skipped: index = 1342179345.
2021-06-03 14:43:43.272 21219-21437/com.github.thibaultbee.streampack.sample W/Codec2Client: query -- param skipped: index = 2415921170.
2021-06-03 14:43:43.277 21219-21437/com.github.thibaultbee.streampack.sample E/FMQ: grantorIdx must be less than 3
2021-06-03 14:43:43.277 21219-21437/com.github.thibaultbee.streampack.sample E/FMQ: grantorIdx must be less than 3
2021-06-03 14:43:43.277 21219-21437/com.github.thibaultbee.streampack.sample D/CCodecBufferChannel: [c2.android.aac.encoder#865] Created input block pool with allocatorID 16 => poolID 17 - OK (0)
2021-06-03 14:43:43.278 21219-21437/com.github.thibaultbee.streampack.sample I/CCodecBufferChannel: [c2.android.aac.encoder#865] Created output block pool with allocatorID 16 => poolID 310 - OK
2021-06-03 14:43:43.278 21219-21437/com.github.thibaultbee.streampack.sample D/CCodecBufferChannel: [c2.android.aac.encoder#865] Configured output block pool ids 310 => OK
2021-06-03 14:43:43.280 21219-21437/com.github.thibaultbee.streampack.sample E/ion: ioctl c0044901 failed with code -1: Inappropriate ioctl for device
2021-06-03 14:43:43.283 21219-21219/com.github.thibaultbee.streampack.sample E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2021-06-03 14:43:43.287 21219-21437/com.github.thibaultbee.streampack.sample E/FMQ: grantorIdx must be less than 3
2021-06-03 14:43:43.290 21219-21237/com.github.thibaultbee.streampack.sample W/Gralloc4: allocator 3.x is not supported
2021-06-03 14:43:43.291 21219-21271/com.github.thibaultbee.streampack.sample E/FMQ: grantorIdx must be less than 3
2021-06-03 14:43:43.291 21219-21444/com.github.thibaultbee.streampack.sample W/Codec2Client: query -- param skipped: index = 1342179345.
2021-06-03 14:43:43.291 21219-21444/com.github.thibaultbee.streampack.sample W/Codec2Client: query -- param skipped: index = 2415921170.
2021-06-03 14:43:43.291 21219-21444/com.github.thibaultbee.streampack.sample W/Codec2Client: query -- param skipped: index = 1610614798.
2021-06-03 14:43:43.291 21219-21444/com.github.thibaultbee.streampack.sample D/CCodecBufferChannel: [c2.qti.avc.encoder#766] Query input allocators returned 0 params => BAD_INDEX (6)
2021-06-03 14:43:43.291 21219-21444/com.github.thibaultbee.streampack.sample D/CCodecBufferChannel: [c2.qti.avc.encoder#766] Using basic input block pool with poolID 1 => got 1 - OK (0)
2021-06-03 14:43:43.291 21219-21444/com.github.thibaultbee.streampack.sample D/CCodecBufferChannel: [c2.qti.avc.encoder#766] Query output allocators returned 0 params => BAD_INDEX (6)
2021-06-03 14:43:43.292 21219-21437/com.github.thibaultbee.streampack.sample D/CCodecBuffers: [c2.android.aac.encoder#865:Output[N]] popFromStashAndRegister: output format changed to AMessage(what = 0x00000000) = {
int32_t aac-sbr-mode = 3
int32_t bitrate = 128000
int32_t channel-count = 2
int32_t level = 0
int32_t max-bitrate = 128000
string mime = "audio/mp4a-latm"
int32_t profile = 2
int32_t sample-rate = 48000
Buffer csd-0 = {
00000000: 11 90 ..
}
}
2021-06-03 14:43:43.295 21219-21444/com.github.thibaultbee.streampack.sample I/CCodecBufferChannel: [c2.qti.avc.encoder#766] Created output block pool with allocatorID 16 => poolID 287 - OK
2021-06-03 14:43:43.296 21219-21444/com.github.thibaultbee.streampack.sample D/CCodecBufferChannel: [c2.qti.avc.encoder#766] Configured output block pool ids 287 => OK
2021-06-03 14:43:43.492 21219-21600/com.github.thibaultbee.streampack.sample D/GraphicBufferSource: got buffer with new dataSpace #104
2021-06-03 14:43:43.492 21219-21600/com.github.thibaultbee.streampack.sample D/C2OMXNode: dataspace changed to 0x104 pixel format: 0x1
2021-06-03 14:43:43.601 21219-21271/com.github.thibaultbee.streampack.sample E/FMQ: grantorIdx must be less than 3
2021-06-03 14:43:48.316 21219-21437/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 243/248 (recycle/alloc) - 6/487 (fetch/transfer)
2021-06-03 14:43:53.376 21219-21437/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 480/485 (recycle/alloc) - 8/961 (fetch/transfer)
2021-06-03 14:43:58.417 21219-21437/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 717/722 (recycle/alloc) - 10/1435 (fetch/transfer)
2021-06-03 14:44:03.480 21219-21271/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 954/959 (recycle/alloc) - 10/1910 (fetch/transfer)
2021-06-03 14:44:08.519 21219-21271/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 1190/1195 (recycle/alloc) - 12/2382 (fetch/transfer)
2021-06-03 14:44:13.581 21219-21271/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 1427/1432 (recycle/alloc) - 13/2856 (fetch/transfer)
2021-06-03 14:44:18.656 21219-21437/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 1665/1670 (recycle/alloc) - 13/3331 (fetch/transfer)
2021-06-03 14:44:23.697 21219-21437/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 1902/1907 (recycle/alloc) - 13/3805 (fetch/transfer)
2021-06-03 14:44:28.761 21219-21271/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 2139/2144 (recycle/alloc) - 13/4280 (fetch/transfer)
2021-06-03 14:44:33.803 21219-21271/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 2375/2380 (recycle/alloc) - 14/4752 (fetch/transfer)
2021-06-03 14:44:38.879 21219-21271/com.github.thibaultbee.streampack.sample D/BufferPoolAccessor2.0: bufferpool2 0xb40000730bb06798 : 5(20480 size) total buffers - 4(16384 size) used buffers - 2613/2618 (recycle/alloc) - 15/5228 (fetch/transfer)

from streampack.

There is nothing concerning SRT messages in here.
Sorry, I don't have much lead right now.
Can you try to use the FileWriter instead of the SrtProducer?
In the settings, set the Endpoint Type to Write to a file. It will generate a TS file in your DCIM directory. Then play the TS file with ffplay.

from streampack.

I tried with an emulated Pixel 4A (Android 11). I had no issue with the stream.

from streampack.

I changed the endpoint to write to a file, but that didn't work either, the file doesn't get created.

Are you using SRT v1.4.3 on your server?

from streampack.

Yes, I also use SRT v1.4.3 on my ffmpeg server.
I just checked the TS file is located in /storage/emulated/0/Android/data/com.github.thibaultbee.streampack.sample/files/DCIM/StreamPackTest.ts

from streampack.

Okay so the file is generated! It looks okay. But for some reason it is not being delivered properly via SRT.

from streampack.

It seems to be an SRT configuration issue (not sure if it is on TX or on RX). Could be that RX recv buffer is too small... You can try to reduce quality (resolution + video bitrate).

I have only tested on Wi-Fi. Are you on Wi-Fi aswell?

from streampack.

I think it's a TX issue, because I have tested with a different library on iOS, and the Larix Broadcaster app on Android, and it works fine.

I tried reducing the quality, and I'm on a very good Wi-Fi connection, still the same issue.

from streampack.

Do you have access to on these library configuration?

from streampack.

I used the default configuration on the Larix Broadcaster app. All I setup was the server address.

from streampack.

Hi,

Can you test branch https://github.com/ThibaultBee/StreamPack/tree/test/clock_source? And join logcat?
I have the feeling that your camera does have the same timestamp source. I deactivated srcTime on SRT so that SRT receiver consumes frames ASAP (ie without to wait srcTime + latency). The consequence: they will be an issue with audio/video synchronization. Anyway, this test branch is just to be sure that the issue comes from the timestamp source. It is not the final solution.

from streampack.

Yup that works much better! There is an issue with the audio sync, as you mentioned.

from streampack.

Hi,
It is not fixed yet it is not that easy. At least we know it comes from the timestamps.

Your camera timestamp source is from SENSOR_INFO_TIMESTAMP_SOURCE_REALTIME source whereas mine is from SENSOR_INFO_TIMESTAMP_SOURCE_UNKNOWN.
SENSOR_INFO_TIMESTAMP_SOURCE_REALTIME will have to be compare SystemClock.elapsedRealtimeNanos() whereas SENSOR_INFO_TIMESTAMP_SOURCE_UNKNOWN need to be compare with SystemClock.uptimeMillis() (which is also SRT internal timestamp source).

from streampack.

You are right!

The timestamp issue has been moved to #12.
The display issue is fixed for now.

from streampack.