Git Product home page Git Product logo

vidio's Introduction

Vidio

A simple Video I/O library written in Go. This library relies on FFmpeg, and FFProbe which must be downloaded before usage and added to the system path.

All frames are encoded and decoded in 8-bit RGBA format.

For Audio I/O using FFmpeg, see the aio project.

Installation

go get github.com/AlexEidt/Vidio

Video

The Video struct stores data about a video file you give it. The code below shows an example of sequentially reading the frames of the given video.

Calling the Read() function will fill in the Video struct framebuffer with the next frame data as 8-bit RGBA data, stored in a flattened byte array in row-major order where each pixel is represented by four consecutive bytes representing the R, G, B and A components of that pixel. Note that the A (alpha) component will always be 255. When iteration over the entire video file is not required, we can lookup a specific frame by calling ReadFrame(n int). By calling ReadFrames(n ...int), we can immediately access multiple frames as a slice of RGBA images and skip the framebuffer.

vidio.NewVideo(filename string) (*vidio.Video, error)
vidio.NewVideoStreams(filename string) ([]*vidio.Video, error)

FileName() string
Width() int
Height() int
Depth() int
Bitrate() int
Frames() int
Stream() int
Duration() float64
FPS() float64
Codec() string
HasStreams() bool
FrameBuffer() []byte
MetaData() map[string]string
SetFrameBuffer(buffer []byte) error

Read() bool
ReadFrame(n int) error
ReadFrames(n ...int) ([]*image.RGBA, error)
Close()

If all frames have been read, video will be closed automatically. If not all frames are read, call video.Close() to close the video.

Camera

The Camera can read from any cameras on the device running Vidio. It takes in the stream index. On most machines the webcam device has index 0.

vidio.NewCamera(stream int) (*vidio.Camera, error)

Name() string
Width() int
Height() int
Depth() int
FPS() float64
Codec() string
FrameBuffer() []byte
SetFrameBuffer(buffer []byte) error

Read() bool
Close()

VideoWriter

The VideoWriter is used to write frames to a video file. The only required parameters are the output file name, the width and height of the frames being written, and an Options struct. This contains all the desired properties of the new video you want to create.

vidio.NewVideoWriter(filename string, width, height int, options *vidio.Options) (*vidio.VideoWriter, error)

FileName() string
StreamFile() string
Width() int
Height() int
Bitrate() int
Loop() int
Delay() int
Macro() int
FPS() float64
Quality() float64
Codec() string

Write(frame []byte) error
Close()
type Options struct {
	Bitrate    int     // Bitrate.
	Loop       int     // For GIFs only. -1=no loop, 0=infinite loop, >0=number of loops.
	Delay      int     // Delay for final frame of GIFs in centiseconds.
	Macro      int     // Macroblock size for determining how to resize frames for codecs.
	FPS        float64 // Frames per second for output video.
	Quality    float64 // If bitrate not given, use quality instead. Must be between 0 and 1. 0:best, 1:worst.
	Codec      string  // Codec for video.
	StreamFile string  // File path for extra stream data.
}

The Options.StreamFile parameter is intended for users who wish to process a video stream and keep the audio (or other streams). Instead of having to process the video and store in a file and then combine with the original audio later, the user can simply pass in the original file path via the Options.StreamFile parameter. This will combine the video with all other streams in the given file (Audio, Subtitle, Data, and Attachments Streams) and will cut all streams to be the same length. Note that Vidio is not a audio/video editing library.

This means that adding extra stream data from a file will only work if the filename being written to is a container format.

Images

Vidio provides some convenience functions for reading and writing to images using an array of bytes. Currently, only png and jpeg formats are supported. When reading images, an optional buffer can be passed in to avoid array reallocation.

Read(filename string, buffer ...[]byte) (int, int, []byte, error)
Write(filename string, width, height int, buffer []byte) error

Examples

Copy input.mp4 to output.mp4. Copy the audio from input.mp4 to output.mp4 as well.

video, _ := vidio.NewVideo("input.mp4")
options := vidio.Options{
	FPS: video.FPS(),
	Bitrate: video.Bitrate(),
}
if video.HasStreams() {
	options.StreamFile = video.FileName()
}

writer, _ := vidio.NewVideoWriter("output.mp4", video.Width(), video.Height(), &options)

defer writer.Close()

for video.Read() {
    writer.Write(video.FrameBuffer())
}

Read 1000 frames of a webcam stream and store in output.mp4.

webcam, _ := vidio.NewCamera(0)
defer webcam.Close()

options := vidio.Options{FPS: webcam.FPS()}
writer, _ := vidio.NewVideoWriter("output.mp4", webcam.Width(), webcam.Height(), &options)
defer writer.Close()

count := 0
for webcam.Read() && count < 1000 {
	writer.Write(webcam.FrameBuffer())
	count++
}

Create a gif from a series of png files enumerated from 1 to 10 that loops continuously with a final frame delay of 1000 centiseconds.

w, h, _, _ := vidio.Read("1.png") // Get frame dimensions from first image

options := vidio.Options{FPS: 1, Loop: 0, Delay: 1000}
gif, _ := vidio.NewVideoWriter("output.gif", w, h, &options)
defer gif.Close()

for i := 1; i <= 10; i++ {
	w, h, img, _ := vidio.Read(fmt.Sprintf("%d.png", i))
	gif.Write(img)
}

Write all frames of video.mp4 as jpg images.

video, _ := vidio.NewVideo("video.mp4")

img := image.NewRGBA(image.Rect(0, 0, video.Width(), video.Height()))
video.SetFrameBuffer(img.Pix)

frame := 0
for video.Read() {
	f, _ := os.Create(fmt.Sprintf("%d.jpg", frame))
	jpeg.Encode(f, img, nil)
	f.Close()
	frame++
}

Write the last frame of video.mp4 as jpg image (without iterating over all video frames).

video, _ := video.NewVideo("video.mp4")

img := image.NewRGBA(image.Rect(0, 0, video.Width(), video.Height()))
video.SetFrameBuffer(img.Pix)

video.ReadFrame(video.Frames() - 1)

f, _ := os.Create(fmt.Sprintf("%d.jpg", video.Frames() - 1))
jpeg.Encode(f, img, nil)
f.Close()

Write the first and last frames of video.mp4 as jpg images (without iterating over all video frames).

video, _ := vidio.NewVideo("video.mp4")

frames, _ := video.ReadFrames(0, video.Frames() - 1)

for index, frame := range frames {
	f, _ := os.Create(fmt.Sprintf("%d.jpg", index))
	jpeg.Encode(f, frame, nil)
	f.Close()
}

Acknowledgements

vidio's People

Contributors

alexeidt avatar jille avatar krzysztofz01 avatar lgaribaldi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vidio's Issues

Opening video starting from a specific frame. A proposal on introducing a new API

I come again with a proposal for a new API, which, I could implement if it receives positive feedback. What I mean is the possibility to open a video starting from a specific frame. The ability to open such a file with a " leap" would be very helpful when working with video using multiple threads/goroutines: each goroutine would create its own process in which it would work with its video chunk.

I also wonder how such a solution would compare performance-wise with a solution where there is one process with access to the file, and the work with the frame would be done in a pool of goroutines.

I am curious to hear the opinions of those interested.
Best regards!

Is Screen sharing possible?

Is it possible that we can combine this with electron js ( golang to node.js module

Can we recording of enter screen in electron.js based app using webrtc screen sharing api & then stream then recorded video to any rtmp server in real-time.

Support for audio?

This package works great for me and is much faster for my use case than PyAV! Sadly, this package is missing a feature that my Python program had: the ability to copy over audio from the video I'm modifying. Since you're using the FFmpeg CLI directly, the best way to do this would likely be to open up another subprocess to handle audio, and then use named pipes in order to mux video and audio together at once.

Access to specific frames, performance. A proposal on introducing a new API

Yesterday i forked the Vidio repository in order to experiment with the Read() function.
One of the projects on which I'm working is using Vidio in a few separate steps:

  • First we open a video file to perform an analysis step on all frames and later the program is processing the data and is returning a collection of frame numbers.
  • Then I needed to repeat the iteration over the whole video in order to access and export the specified frames.

The iteration over the whole video takes a lot of time due to the frameBuffer writing operation. Here is a solution that I tried to implement:

I created a ReadOffset(n int) function, that skips n frames and writes the n+1 frame to the frameBuffer. The io.ReadFull involves the creation of a buffer and the main goal was to prevent allocations and unnecessary writing. The usage of io.SkipN with the source set to video.pipe and destination io.Discard performed the same on the benchmark and profiling that i performed. Later, I thought about using the Seek() operation, but the pipe is ReadCloser and not ReadSeekCloser, and a cast would not help, because we are not waiting for the command to end, so the length is changing and some unexpected bahabiour may occur.

All these problems lead to the idea of introducing a separate API. I think that accessing a specific frame is a common operation and iteration over the whole video file is overkill, especially when we know the specific frame numbers. Also, these kinds of changes:

  • DO NOT expose the underlying FFMpeg implementation details, so the API would still seem FFMpeg agnostic
  • DO NOT change the existing API so no breaking changes will be made
  • allow to increase efficiency of applications using Vidio

I though about functions like

GetFrame(filename string, index int) *image.RGBA

GetFrames(filename string, index ...int) []*image.RGBA

The underlying FFMpeg API can use the select flag. Example flag value to export frames 2, 5 and 10:

... select='eq(n\,1)+eq(n\,4)+eq(n\,9)' ...

I would appreciate your opinion on this case and would be happy to implement these changes if my proposal is accepted.

Thank you for creating and maintaining this library.
Best regards!

Support for additional FFmpeg flags

I thought about opening a pull request with a feature implementation I wanted to propose, but I think, first I'm gonna consult it via an issue.
I thought about the possibility of passing additional FFmpeg flags and parameters to VideoWriter. This could be achieved, for example, by adding a map[string]string or a []string to the Options struct. This change wouldn't introduce any breaking-changes but may open new possibilities, more use-cases and would make the library generally more versatile.

Identify an error in rm format video

	var vdo *vidio.Video
	vdo, err = vidio.NewVideo(filepath)
	if err != nil {
		return
	}

err:
no video data found in /tmp/upload/acd9bc2b5543f08633.rm

and the file exists.

Issue running on Docker

Issue

Running video.Read() on Docker seems to run into the issue where the following loop ends after the first iteration:

for frameCount := 0; video.Read(); frameCount++  {

  // ... do some logic on frames

}

Not sure where I am going wrong, running the binary on macOS seems to give me no issues, but running on both alpine:latest and ubuntu:latest gives me this issue.

Things I tried

  • Using different base images
  • Copying video file directly into the container

What my docker image looks like

# Use the official Golang image as the base image
FROM golang:1.18 AS builder

# Set the working directory inside the container
WORKDIR /app

# Copy the Go module files
COPY go.mod go.sum ./

# Download and cache the Go module dependencies
RUN go mod download

# Copy the source code into the container
COPY . .

# Build the Go application
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o vidstreamsplit .


# Use a minimal Alpine image as the base image for the final image
FROM ubuntu:latest

# Install FFmpeg and AWS CLI
RUN apt update && apt install -y ffmpeg 

# Set the working directory inside the container
WORKDIR /app

# Copy the built binary from the previous stage
COPY --from=builder /app/vidstreamsplit .

# Make the binary executable
RUN chmod +x vidstreamsplit

# Copy the scripts directory into the container
COPY scripts ./scripts

# Make all the scripts in the scripts package executable
RUN chmod +x scripts/*.sh

Please let me know if you've run into this issue before. Happy to take a look and try to submit a change if you could point me in the write direction.

Get video extension

I have a route which receives a video through an http post, basically what I want is to validate the extension of this video to only work with mp4 videos. Can I do this?

Getting scrambled frames for portrait videos

By using the example code to read a video and write the output directly and save the frames, I am getting all frames and the output video scrambled for certain input videos. This seems to only happen in portrait videos as far as I can tell.
This is an example frame, as you can see it looks like the stride is off:
1

I'll submit a PR with the fix that I am using

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.