Anamorphic video reports incorrect onVideoSizeChanged about exoplayer HOT 25 CLOSED

I've had a little dig through the DASH code, specifically the FragmentedMp4Extractor. It reads the Width and Height from the STSD, but ignores the (optional) PASP (Pixel Aspect Ratio Box). This could contain relevant information which could be used to adjust the Width and Height used when creating the MediaFormat. As to whether the MediaFormat should be Pixel Aspect Ratio aware, i'm not sure :) The underlying video player could also read this information from the H.264 Bitstream, but it's not.

I guess a simple hack to test would be to just hard code in the PAR adjusted width/height before the MediaFormat was created. Then I could see if the "real" width / height are reported via onVideoSizeChanged and whether it had any side effects to playback.

from exoplayer.

Another option would be to get the information from the H264 stream, we have been having the same kind of issues with hls: martinbonnin#18. Not sure which one is best though.

from exoplayer.

Does the normal media player handle this type of content correctly?

from exoplayer.

Unfortunately not. MediaPlayer always reports Width/Height, not taking into account non-square pixels.

For some videos, i'm able to know the Pixel Aspect Ratio up front, which I manually use when determining the actual aspect ratio of the video. However, if we end up playing the video using DASH, I have no way to extract this information. Anamorphic content isn't rare either, which is why I was keen to explore solutions to this.

from exoplayer.

@IanDBird do you have any temporary solution for now for HLS that i can do right away in my app to fix this.

from exoplayer.

@ojw28 for hls, yes media player handles this perfectly fine with square pixels at least.

from exoplayer.

@hiteshgupta33 - Just to clarify you're point, the MediaPlayer does handle anamorphic content, but it does report it's Width and Height ignoring the size of the individual pixels, right? This means that if you're scaling the video and using those reported values, it's very difficult it compute the correct size maintaining the original aspect ratio of the video.

I'm sorry, our workaround isn't suitable for others. We perform some analysis of the video sources first (on a server) and then report this information to the client. This analysis includes the Pixel Aspect Ratio. I'm essentially getting this information out-of-band and using it on the Android client to compute the "true" width and height of the video and therefore can scale the video appropriately. This works when i'm "Direct Playing" the video (via the FrameworkSampleSource) but is not possible when streaming via DASH. This is because converting the video for DASH "could" alter the size of the pixels, for which I don't have the same out-of-band reporting mechanism.

from exoplayer.

@IanDBird yeah MediaPlayer does ignore the size of the individual pixel but it does render properly but yes not in the absolute correct aspect ratio.

But at least does not distort like this:

This is using exoplayer (hls)

from exoplayer.

Yeah, although that screenshot / distortion suggests a bug in the HLS implementation, since the underlying Media API can handle it (as shown by the native MediaPlayer working). I can try and play anamorphic DASH content here and see if that results in any rendering artefacts, if that's of any use?

from exoplayer.

@IanDBird yeah eventually we have to shift to DASH. It will be of great use

thanks

from exoplayer.

I just tried playing some anamorphic content using DASH. There are no rendering issues, the video looks fine, it's just the wrong aspect ratio. As I mentioned in my original post, i'm really just wanting the VideoSurface to maintain it's original aspect ratio. But since it doesn't consider non-square pixels, i'm left with something like the following:

from exoplayer.

• For DASH I think you're correct in stating that FragmentedMp4Parser should parse out the PASP box (assuming that's the correct box; I didn't actually check :)).
• We shouldn't adjust the MediaFormat width and height values. These should be in pixels, and so are already correct. My initial thought would be to add an additional pixelWidthHeightRatio field to MediaFormat (default value 1), and just have it be propagated all the way through to onVideoSizeChanged. I think width and height should still be reported as pixels, but the ratio could be passed as an additional argument.

from exoplayer.

That sounds like a great solution, but would I be correct in assuming that when using the FrameworkSampleSource it's unlikely we'll get this additional information?

As for the PASP, I took that from ISO BMFF (ISO/IEC 14496-12)

The pixel aspect ratio and clean aperture of the video may be specified using the ‘pasp’ and ‘clap’ sample entry boxes, respectively. These are both optional; if present, they over-ride the declarations (if any) in structures specific to the video codec, which structures should be examined if these boxes are absent. For maximum compatibility, these boxes should follow, not precede, any boxes defined in or required by derived specifications.

In the PixelAspectRatioBox, hSpacing and vSpacing have the same units, but those units are unspecified: only the ratio matters. hSpacing and vSpacing may or may not be in reduced terms, and they may reduce to 1/1. Both of them must be positive.

from exoplayer.

p.s. If you can provide some sample DASH content then I can take a look. Otherwise feel free to propose a change. Note for the latter, you'd need to sign our CLA: https://github.com/google/ExoPlayer/blob/master/CONTRIBUTING.md

from exoplayer.

Re FrameworkSampleSource - FrameworkSampleSource hooks into the underlying media support provided by the platform, so if there's no support there [it sounds like you've concluded that there isn't]. then it wont be possible to provide the additional information.

from exoplayer.

Fair enough. For my use, I can continue to use the out-of-band mechanism to determine the PAR, but unfortunately not helpful for others.

I'll try to generate some representative DASH content, and also confirm that it does actually contain the useable atoms.

from exoplayer.

Yeah, the underlying MediaPlayer doesn't report the width/height for anamorphic content, so with out knowing ahead of time, it will always be in the wrong aspect ratio (typically 4:3 instead of 16:9). This has been a long standing issue with the built in MediaPlayer.

from exoplayer.

Yeah, I guess I was hoping there was some mechanism of using the MediaExtractor to read the STSD separately and parse out the additional information manually.

from exoplayer.

@ojw28 - Here you go, https://drive.google.com/file/d/0B5F60IW5qC-vMkV3c2VMRWFWX0k/edit?usp=sharing. You'll need to modify the MPD depending on how you host it.

I was able to use mp4file --dump to confirm that the PASP is contained within the initial.mp4:

        "initial.mp4": type pasp (moov.trak.mdia.minf.stbl.stsd.avc1.pasp)
         "initial.mp4": hSpacing = 64 (0x00000040)
         "initial.mp4": vSpacing = 45 (0x0000002d)

from exoplayer.

Do you need to add the initialization segments to the zip? Or did I miss them :).

from exoplayer.

So sorry, i've updated the ZIP (https://drive.google.com/file/d/0B5F60IW5qC-vMkV3c2VMRWFWX0k/edit?usp=sharing). I was trying not to include the entire movie to spare your download :)

from exoplayer.

Thanks. I have a fix for anything that uses the fragmented mp4 extractor (i.e. DASH / SmoothStreaming).

from exoplayer.

That's awesome, thanks! For our application we can detect anamorphic content and force it via our (on the fly) transcoder to DASH. If the content was originally direct playable, it would simply remux the video/audio and therefore be pretty quick as an end result for the user.

Let me know when there is code available, and I can give it a go.

from exoplayer.

Fixed in the dev branch in:
ec90eac

from exoplayer.

Thanks @ojw28, I've just tested and can confirm that the fix works.

from exoplayer.