Git Product home page Git Product logo

mediapy's Introduction

Read/write/show images and videos in an IPython/Jupyter notebook.

Unittests PyPI version

[GitHub source]   [API docs]   [Colab example]

Examples:

See the notebook   mediapy_examples.ipynb     Open In Colab   Open in Binder

Images:

    !pip install -q mediapy
    import mediapy as media
    import numpy as np

    image = media.read_image('https://github.com/hhoppe/data/raw/main/image.png')
    print(image.shape, image.dtype)  # It is a numpy array.
    media.show_image(image)

    checkerboard = np.kron([[0, 1] * 16, [1, 0] * 16] * 16, np.ones((4, 4)))
    media.show_image(checkerboard)

    media.show_image(media.color_ramp((128, 128)), height=48, title='ramp')

    images = {
        'original': image,
        'brightened': media.to_float01(image) * 1.5,
    }
    media.show_images(images)

    media.write_image('/tmp/checkerboard.png', checkerboard)

Videos:

    url = 'https://github.com/hhoppe/data/raw/main/video.mp4'
    video = media.read_video(url)
    print(video.shape, video.dtype)  # It is a numpy array.
    print(video.metadata.fps)  # The 'metadata' attribute includes framerate.
    media.show_video(video)  # Play the video using the retrieved framerate.

    media.show_images(video, height=80, columns=4)  # Show frames side-by-side.

    video = media.moving_circle((128, 128), num_images=10)
    media.show_video(video, fps=10)

    media.write_video('/tmp/video.mp4', video, fps=60)

    # Darken a video frame-by-frame:
    filename_in = '/tmp/video.mp4'
    filename_out = '/tmp/out.mp4'
    with media.VideoReader(filename_in) as r:
      print(f'shape={r.shape} fps={r.fps} bps={r.bps}')
      darken_image = lambda image: media.to_float01(image) * 0.5
      with media.VideoWriter(
          filename_out, shape=r.shape, fps=r.fps, bps=r.bps) as w:
        for image in r:
          w.add_image(darken_image(image))
    media.show_video(media.read_video(filename_out), fps=60)

Setup:

Video I/O relies on the external program ffmpeg, which must be present in the system PATH. On Unix, it can be installed using:

    apt install ffmpeg

or within a notebook using:

    !command -v ffmpeg >/dev/null || (apt update && apt install -y ffmpeg)

mediapy's People

Contributors

conchylicultor avatar hawkinsp avatar hedpeter avatar hhoppe avatar kmaninis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mediapy's Issues

Will support transform BGR to RGB for 3-channel images?

Hi mediapy team!

Thanks for sharing this awesome tool. It seems that the color map only support grey to RGB currently. Because opencv uses BGR by default and it's also widely used. I want to know if there are some plans to support this channel mode, and if this mode is already supported, could you give me some hints where this interface is?

mediapy load, visualize or write a video with audio

Hi, would like to ask if mediapy can load, visualize or write a video with audio in a colab notebook.

I checked the functions in mediapy, but cannot find any audio related code.

I wonder if adding audio into the function is possible or not?

Thank you.

AttributeError: module 'PIL.Image' has no attribute 'Resampling'

I encounter the following error when reading an image on Google Colab.

%pip install mediapy
import mediapy as media

img = media.read_image('/content/images/yorkshire_terrier_8.jpg')
---------------------------------------------------------------------------

AttributeError                            Traceback (most recent call last)

[<ipython-input-16-5d047d0e728a>](https://localhost:8080/#) in <cell line: 1>()
----> 1 import mediapy as media
      2 
      3 img = media.read_image('/content/images/yorkshire_terrier_8.jpg')

2 frames

[/usr/local/lib/python3.9/dist-packages/PIL/Image.py](https://localhost:8080/#) in __getattr__(name)
     75             name, 10, f"{old_resampling[name]} or Resampling.{old_resampling[name]}"
     76         )
---> 77         return Resampling[old_resampling[name]]
     78     msg = f"module '{__name__}' has no attribute '{name}'"
     79     raise AttributeError(msg)

AttributeError: module 'PIL.Image' has no attribute 'Resampling'

According to this StackOverflow answer, this is due to a recent version of Pillow. The version on Google Colab is 9.5.0.

%pip list
[...]
mediapy                       1.1.6
[...]
Pillow                        9.5.0
[...]

By downgrading Pillow, as suggested by the StackOverflow answer, the issue disappears.

%pip install --ignore-installed Pillow==9.0.0

mediapy `display_images()` downsamples images seemingly arbitrarily....

Issue tested with version: #41

a) If I call media.show_images() once passing it a dict of 32 images, each with shape (512, 512, 3), it stubbornly displays a row matrix of images that are (256, 256, 3) each.

b) If I call media.show_images() 32 times passing it a dict with a single image each time, each with shape (512, 512, 3), it displays the images at the correct resolution (however the images are displayed one below each other which is not what I want).

c) In case a), if I pass the argument height=512 then the images are displayed pixelated 2x: it looks like mediapy makes a 256x256 internal image buffer and displays it pixelated at 512x512.

d) Passing downsample=False doesn't fix the issue either (too easy-no points).

e) I also have a repro case where it dowsamples from 480 to 240 so this seems like a 2x downsample bug.

f) Aha!!! It looks like displaying a smaller number of images (6 vs. 32) works as intended.

Looks like it is some kind of cap on total memory it can consume?

question about write_video

Hi, thanks for open-sourcing the great library!

I am just curious in your demo notebook, when saving the video:

# Write a video to a file:
media.write_video('/tmp/video1.mp4', video1, fps=10, qp=10)

what does 'qp' here mean? I searched online but didn't find anything.

Thanks in advance!

ffmpeg with copy codecs is not guaranteed to contain number of frames in output

Noticed this when using ffmpeg installed from apt, when vcodec and acodec are copy, the frame= XXX line does not appear in the output, causing api like read_video to fail. The actual output of the ffmpeg -nostdin -i videofile.m4v -acodec copy -vcodec copy -f null - call is

ffmpeg version 6.1-3 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13 (Debian 13.2.0-5)
  configuration: --prefix=/usr --extra-version=3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libsvtav1 --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'videofile.m4v':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2023-04-26T22:39:08.000000Z
  Duration: 00:00:16.50, start: 0.000000, bitrate: 5329 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, progressive), 640x480, 5114 kb/s, 30.10 fps, 29.67 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2023-04-26T22:39:08.000000Z
      handler_name    : ISO Media file produced by Corp Inc. Created on: 04/26/2023.
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of 90.00 degrees
  Stream #0:1[0x2](und): Data: none (mett / 0x7474656D), 20 kb/s (default)
    Metadata:
      creation_time   : 2023-04-26T22:39:08.000000Z
      handler_name    : ISO Media file produced by Corp Inc. Created on: 04/26/2023.
  Stream #0:2[0x3](und): Data: none (mett / 0x7474656D), 64 kb/s (default)
    Metadata:
      creation_time   : 2023-04-26T22:39:08.000000Z
      handler_name    : ISO Media file produced by Corp Inc. Created on: 04/26/2023.
  Stream #0:3[0x4](und): Data: none (mett / 0x7474656D), 64 kb/s (default)
    Metadata:
      creation_time   : 2023-04-26T22:39:08.000000Z
      handler_name    : ISO Media file produced by Corp Inc. Created on: 04/26/2023.
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    encoder         : Lavf60.16.100
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, progressive), 640x480, q=2-31, 5114 kb/s, 30.10 fps, 29.67 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2023-04-26T22:39:08.000000Z
      handler_name    : ISO Media file produced by Corp Inc. Created on: 04/26/2023.
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of 90.00 degrees
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
[out#0/null @ 0x55bbe6083100] video:10287kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
size=N/A time=00:00:16.47 bitrate=N/A speed=2.13e+03x

Unable to parse video framerate in line

city-cond-0

I want to use mediapy.read_video to read it but there is something wrong because this gif only has 2 frames and has not fps metadata. how to read it by using mediapy?

Unrecognized option 'qp'

I run the offifial example code but go wrong.

from mediapy import *
video = moving_circle((100, 100), num_images=10)
show_video(video, fps=10)

I am not sure it is relative to torchvision or ffmpeg

when i dont conda install torchvision, the code can run well. but after i conda install torchvision (and ffmpeg also installed automatedly), mediapy can not run well, so how to solve the problem?

torchvision and ffmpeg version are as follows:
image

BrokenPipeError                           Traceback (most recent call last)
File [~/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py:1663](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a224e4b552d43562d4c41422d5562756e74753132227d.vscode-resource.vscode-cdn.net//~/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py:1663), in write_video(path, images, **kwargs)
   [1662](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1661) for image in images:
-> [1663](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1662)   writer.add_image(image)

File [~/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py:1572](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a224e4b552d43562d4c41422d5562756e74753132227d.vscode-resource.vscode-cdn.net//~/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py:1572), in VideoWriter.add_image(self, image)
   [1571](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1570) assert self._proc.stdin
-> [1572](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1571) if self._proc.stdin.write(data) != len(data):
   [1573](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1572)   self._proc.wait()

BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
Cell In[13], line 2
      1 video = moving_circle((100, 100), num_images=10)
----> 2 show_video(video, fps=10)

File [~/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py:1772](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a224e4b552d43562d4c41422d5562756e74753132227d.vscode-resource.vscode-cdn.net//~/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py:1772), in show_video(images, title, **kwargs)
   [1749](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1748) def show_video(
   [1750](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1749)     images: Iterable[_NDArray], *, title: str | None = None, **kwargs: Any
   [1751](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1750) ) -> str | None:
   [1752](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1751)   """Displays a video in the IPython notebook and optionally saves it to a file.
...
   [1587](file:///home/ubuntu/anaconda3/envs/MSPred/lib/python3.9/site-packages/mediapy/__init__.py?line=1586) self._popen = None

RuntimeError: Error writing '/tmp/tmpugt6gx5m/file.mp4': Unrecognized option 'qp'.
Error splitting the argument list: Option not found

and i tried

ffmpeg pkgs/main/linux-64::ffmpeg-4.2.2-h20bf706_0 runs well

but ffmpeg pytorch/linux-64::ffmpeg-4.3-hf484d3e_0 cannot run well

Intermittent test failure

I got a test failure in https://github.com/hhoppe/mediapy/actions/runs/5408425104/jobs/9827481857 on HEAD:

=============================== warnings summary ===============================
mediapy_test.py::MediapyTest::test_to_type_extreme_value
  /home/runner/work/mediapy/mediapy/mediapy/__init__.py:421: RuntimeWarning: invalid value encountered in cast
    dst = dst.astype(dtype)

Possibly this is due to differences in precision across machines when processing uint64 arrays.
I'm working on a PR to omit dst_dtype=uint64 from the test, and to refactor using @parameterized.parameters so we can identify the precise test parameters that cause a failure.

video show no content

simple code, but output video shows no content.

image

checked that variant video has value
and have installed ffmpeg and set it in the system path.

show_image() shows jaggies when the desktop or browser is magnified

(This affects show_image(), show_images(), show_video(), and show_videos().)

By design, when any image or video content is magnified, mediapy shows it pixelated (via a CSS style) -- each image pixel appears as a block of screen pixel values (rather than some bilinear or bicubic filtered reconstruction).
Showing this pixelated view is useful to reveal the original resolution of the image, for example in closeups.
(If one does desire a smoother interpolated version of the image, it can be achieved by first calling media.resize_image().)

With default settings (no width or height attribute), a show_*() call displays the image or video at native resolution, and in that case it should not matter.
However, this is not actually the case.
When there is either (1) desktop DPI scaling (e.g., common in Windows) or (2) browser scaling (e.g., in Chrome using control-+),
the image is displayed over more or fewer pixels than its native resolution.
And in this case, the CSS pixelated setting introduces unexpected ugly jaggies, particularly for vector graphics. See the image below.

v1

The solution is to introduce the CSS pixelated setting only when the show_*() call has explicit height/width parameters that are larger than the image content, i.e., intended to introduce magnification. For flexibility, it's also useful to have a separate pixelated parameter.

I am readying a PR to address this.

`codec="gif"` gives not RGB color

Hi mediapy,
This is an AWESOME tool! One quick question

When I use the codec="gif", the color is not right;
image

But if I remove that, the color is back:
image

Do I miss some parameters in order to have the rgb color?

Again, thanks for open-sourcing this wonderful tool!

Large videos

Hii, I would like to ask how can I open large videos (.avi format) using mediapy.
The problem is when I try to open large video the system freezes for a while than it is closed with no errors.
Hence I would like to know if mediapy doesn't support large files.
Thanks in advance

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.