sdroege / gst-plugin-rs Goto Github PK

There are various other features (specifically virtual methods and their arguments) that GstBaseSink has and we don't wrap yet. This should probably come with adding support for libgstbase in gstreamer-rs: Might make sense to do this first: sdroege/gstreamer-rs#7

Most of these features can be wrapped basically 1-1:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-libs/html/GstBaseSink.html

Hi,

We loved the idea of the fallbacksrc and very needed. But our pipeline working with rtspsrc but not wrking with fallbacksrc.

function create_rtsp_sources() { echo "Rtsp sources creating" for ((n = 0; n < $num_of_src; n++)); do src_name="SRC_${n}" src_name="${!src_name}" rtsp_sources+="rtspsrc async-handling=true location=$src_name name=source_$n message-forward=true ! \ decodebin3 ! \ queue name=custom_preprocess_q_$n leaky=no max-size-buffers=32 max-size-bytes=0 max-size-time=0 ! \ $decode_scale_elements ! videoconvert n-threads=8 ! \ video/x-raw,pixel-aspect-ratio=1/1 ! \ fun.sink_$n sid.src_$n ! \ queue name=comp_q_$n leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! \ comp.sink_$n " #echo "$n -- $rtsp_sources \n\n" done }

isnt the fallbacksrc is the same output of the rtspsrc ?
How we can replace the rtspsrc with fallbacksrc for above pipeline ?

Currently we have lots of handcrafted bindings for various GStreamer types or call FFI functions directly. This should be moved to using the proper bindings from gstreamer-rs.

Best approach here would probably be to first do (one after another) source.rs, sink.rs, demuxer.rs and then get rid of everything in the other files.

Types like FlowReturn would be wrapped in our custom FlowError (omitting the Ok case for Result<_,_>). Whatever makes sense should become a wrapper around the gstreamer-rs types with additional functionality.

In the context of this, also see sdroege/gstreamer-rs#3 which can be done before or after (in which case we would have a custom newtype for that here for the time being).

See https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-libs/html/GstBaseTransform.html
Might make sense to do this first: sdroege/gstreamer-rs#7

First step would be to come up with an API design (mostly the main trait) in the style of Source, Sink and Demuxer. That is:

• all functions return a value and don't call into other elements. So return value would be some kind of enum
• the general idea of the C base class would be kept (i.e. you have some kind of transform function, transform_caps, etc)
• implementors of the trait do not have to worry about threading

As part of this should also be a simple transform element. Maybe just a minimal "volume" element, or mono to stereo converter (the latter would at least make use of caps transformations).

There are various other features (specifically virtual methods and their arguments) that GstBaseSrc has and we don't wrap yet. This should probably come with adding support for libgstbase in gstreamer-rs: Might make sense to do this first: sdroege/gstreamer-rs#7

Most of these features can be wrapped basically 1-1:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-libs/html/GstBaseSrc.html

See https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/GstAudioEncoder.html

As part of this should also be a simple encoder element. Maybe just a minimal element that takes e.g. a ALAW/MULAW encoder.

See https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-GstVideoEncoder.html

It probably makes sense to first wrap more parts libgstvideo in gstreamer-rs so that things like GstVideoCodecFrame and GstVideoInfo/Frame can possibly be directly used without writing manual bindings. sdroege/gstreamer-rs#5

As part of this should also be a simple decoder element. Maybe just a minimal element that takes e.g. a BMP image and encodes it (or rather adds the header).

There was no 1.18 branch for gst-plugin-rs, is there any gst-plugin-rs version compatible with gstreamer 1.18?

See rust-lang/rust#33553

Reduces shared library size a lot

The proposed repo is offline: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs

Also causing build issue:
Updating git repository https://gitlab.freedesktop.org/gstreamer/gstreamer-rs warning: spurious network error (3 tries remaining): unexpected http status code: 504; class=Http (34) warning: spurious network error (2 tries remaining): unexpected http status code: 504; class=Http (34) warning: spurious network error (1 tries remaining): unexpected http status code: 504; class=Http (34) error: failed to get gstreameras a dependency of packagegst-plugin-tutorial v0.9.11 (/home/bwilma/BuildAgent/work/100b0d6fce2ffa2f/playground/gs_prestudy/webrtc/builddir/gst-plugins-rs-cross/tutorial)`

So that the plain wrappers can be released on crates.io without worrying about the API of the simple traits

Around https://docs.rs/vobsub/0.2.3/vobsub/, probably best as GstVideoDecoder subclass once we have it... or GstElement.

Needs some changes to expose the &[u8] based API that is currently internally, and accepts elementary stream packets.

The repository also has some SRT code, might be worth adding something around that too.

CC @emk who brought this to my attention on reddit

0. building on Mac M1Max
1. Newbie (<3 days old) Rust user
2. Trying to build the gst-plugin-rs project
3. Installed gstreamer "official" packages - devel + non-level
4. gstreamer works fine (e.g. gst-launch-1.0 videotestsrc ! autovideosink)
5. "cargo build" fails with this errors. I am probably missing a step or two - any pointers would be helpful 🙏

error occurred: Command "cc" "-O1" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-arch" "arm64" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/gstreamer-1.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/glib-2.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/lib/glib-2.0/include" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/gstreamer-1.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include" "-I" "/Users/matt/Projects/cerbero-1.20/build/dist/darwin_universal/include/orc-0.4" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/gstreamer-1.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/glib-2.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/lib/glib-2.0/include" "-Wall" "-Wextra" "-DRTPJitterBuffer=TsRTPJitterBuffer" "-DRTPJitterBufferClass=TsRTPJitterBufferClass" "-DRTPJitterBufferPrivate=TsRTPJitterBufferClass" "-o" "/Users/cgadgil/dev/src/scalers-ai/common/tmp/gst-plugin-rs/target/debug/build/gst-plugin-threadshare-6bbad1f4e13769c6/out/src/jitterbuffer/rtpjitterbuffer.o" "-c" "src/jitterbuffer/rtpjitterbuffer.c" with args "cc" did not execute successfully (status code exit status: 1).

warning: build failed, waiting for other jobs to finish...

Not sure what is missing at this point apart from: seeking and lots of minor details/features and cleanup at least. And #15

Should be compared with the the C demuxer: https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/gst/flv/gstflvdemux.c

There's currently some C code for the demuxer. This should be replaced by corresponding, unsafe Rust code. Just like for Source and Sink.

It might make sense to refactor/redesign the demuxer trait at the same time. See #16

See https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-libs/html/GstBaseParse.html

As part of this should also be a simple parser element. Maybe just a minimal element that takes e.g. a PNG or JPEG parser

See https://github.com/sdroege/gst-plugin-rs/tree/async-http

This should ideally become feature-equivalent with souphttpsrc to become a proper replacement.

I want to use rsaudioecho filter in C code (is it even possible?), but it panicks at gst_init() if GST_PLUGIN_PATH is specified

(macOS Sierra 10.12.6, Xcode 8.3.3 (8E3004b), clang-802.0.42, rustc 1.23.0-nightly, cargo 0.24.0-nightly, GStreamer 1.12.3)

int main(int argc, char *argv[]) {
    setenv("RUST_BACKTRACE", "1", 1);
    setenv("GST_PLUGIN_PATH","/Users/username/sources/gstreamer/gst-plugin-rs",1);

    /* Initialize GStreamer */
    gst_init (&argc, &argv);

    return 0;
}

output:

thread '<unnamed>' panicked at 'assertion failed: ::types::instance_of::<Self>(ptr as *const _)', /Users/username/.cargo/git/checkouts/gstreamer-rs-8719441e7bb90fbc/2f7ee30/gstreamer/src/auto/plugin.rs:14:0
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:69
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:381
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:397
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:577
   5: std::panicking::begin_panic
             at /Users/travis/build/rust-lang/rust/src/libstd/panicking.rs:538
   6: <gstreamer::auto::plugin::Plugin as glib::translate::FromGlibPtrBorrow<*mut gstreamer_sys::GstPlugin>>::from_glib_borrow
             at /Users/username/sources/gstreamer/gst-plugin-rs/gst-plugin-audiofx/<panic macros>:3
   7: glib::translate::from_glib_borrow
             at /Users/username/.cargo/git/checkouts/glib-928cf7b282977403/a51dc67/src/translate.rs:825
   8: gstrsaudiofx::plugin_desc::plugin_init_trampoline
             at /Users/username/sources/gstreamer/gst-plugin-rs/gst-plugin-audiofx/<plugin_define macros>:24
   9: gst_plugin_register_func
  10: _priv_gst_plugin_load_file_for_registry
  11: exchange_packets
  12: _gst_plugin_loader_client_run
  13: main
fatal runtime error: failed to initiate panic, error 5
Program ended with exit code: 9

BTW, rsaudioecho works well with gst-launch-1.0 and gst-inspect-1.0

I tried to build this cargo.toml file

[package]
...

[dependencies]
glib = "0.4"
gstreamer = "0.10"
gstreamer-base = "0.10"
gst-plugin = "0.1"
gstreamer-video = "0.10"

[lib]
name = "gstrstutorial"
crate-type = ["cdylib"]
path = "src/lib.rs"

I'm getting this error when I try cargo build

Compiling gstreamer-video-sys v0.4.0
error: failed to run custom build command for `gstreamer-video-sys v0.4.0`
process didn't exit successfully: `/home/.../Desktop/GStreamer/plugin tests/target/debug/build/gstreamer-video-sys-0a43cf028baefe7d/build-script-build` (exit code: 1)
--- stderr
`"pkg-config" "--libs" "--cflags" "gstreamer-video-1.0 >= 1.8"` did not exit successfully: exit code: 1
--- stderr
Package gstreamer-video-1.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `gstreamer-video-1.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gstreamer-video-1.0' found

It would be nice to have this more generic so that it could e.g. also be used to build custom GTK widgets or the like.

A start exists in https://github.com/sdroege/gst-plugin-rs/tree/video_filter

See https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/GstAudioDecoder.html

As part of this should also be a simple decoder element. Maybe just a minimal element that takes e.g. a ALAW/MULAW decoder.

LGPL isn't the best for static linking. This is a proposal to switch to a more appropriate permissive license.

Which one do people recommend? I suggest MIT but IANAL. An other alternative is to explicitly add an exception about static linking, like GNU Guile does.
https://en.wikipedia.org/wiki/GPL_linking_exception

Hi,

I am using udpsrc, I tried to replace it with ts-udpsrc, but "retrieve-sender-address" property is not there yet. I am using the property to retrieve address by adding a probe in udpsrc src pad.

struct _GstNetAddressMeta *meta = gst_buffer_get_net_address_meta(buffer);                                                                                                                    
assert(meta);                                                                                                                                                                                 
                                                                                                                                                                                              
GInetSocketAddress *sockAddr = (GInetSocketAddress *) meta->addr;                                                                                                                             
assert(sockAddr);

Is there any other way to achieve this in ts-udpsrc ?

See https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-GstVideoDecoder.html

It probably makes sense to first wrap more parts libgstvideo in gstreamer-rs so that things like GstVideoCodecFrame and GstVideoInfo/Frame can possibly be directly used without writing manual bindings. sdroege/gstreamer-rs#5

As part of this should also be a simple decoder element. Maybe just a minimal element that takes e.g. a BMP image and decodes it (or rather parses the header and throws away the header).

Hello Sebastian,

thank you very much for your work!

I haven't wrote any Gstreamer plugins yet but I have one in a plan :)
I believe your projects will help me a lot.

I have noticed that repository like on crates.io is broken

https://github.com/sdroege/gst-plugin-rs/gst-plugin

If you meant it to point to gst-plugin workspace shouldn't it be?

https://github.com/sdroege/gst-plugin-rs/tree/master/gst-plugin

I have tried with both the library provided signaller as well as a custom "echo server" and running webrtcsink with gst-launch will simply never connect to the websocket. It doesn't change anything whether I provide a signaller::uri or not. Needless to say because of this, the usage instructions from the documentation do not work either, as it will not connect to the rust signaller (can be confirmed by no debugging message showing up whatsoever), so the webpage will not show the test stream. Am I doing something wrong or is this a known bug ?

Thanks!

Would it be possible to implement streaming to Youtube over HLS ? Which is needed if you'd like to stream in H265 (HEVC)

See documentation here

A start of this can be found here: https://github.com/sdroege/gst-plugin-rs/tree/rust-av

Needs more work

The default implementation would just abort(), element and others could catch, report an error message and poison the object.

This probably would have to be on the ObjectType trait

Hi,

I’m trying to use “whipsink”, I followed the instruction mentioned here Fernando Jiménez Moreno / gst-plugins-rs · GitLab to build. When running this command cargo cbuild --manifest-path video/cdg/Cargo.tom, it produces the error

Error: CliError { error: Some(failed to select a version for `gstreamer`.
    ... required by package `gst-plugin-claxon v0.6.0 (/home/a/Desktop/x/gst-plugins-rs/audio/claxon)`
versions that meet the requirements `*` are: 0.23.0

the package `gst-plugin-claxon` depends on `gstreamer`, with features: `v1_14` but `gstreamer` does not have these features.

I have no idea how can I fix this error, I’ve searched on Google but there is no post relate to the feature of v1_14. Any help would be appreciated.

Thank you very much!