webrtc-sdk / webrtc Goto Github PK

We're having an issue where the audio processing audio unit gets stopped when all the tracks are muted. This is a change to libwebrtc done here - 272127d#diff-96494536ec12249ed599120315bbc5df4ca4b7612c4098e5fdce65e72065328dR101

Normally this would be a great addition, stop accessing the mic, stop showing the mic access bar/dot when users think the mic shouldn't be "listening". In our case, it's not great because once you stop an audio unit in the background, you can't restart it unless you're in the foreground or your app is using callkit (as far as I'm aware anyway)

I believe this functionality should have an opt out - libwebrtc allows for the use of feature flags to enable and disable a lot of functionality so I think this should definitely be enabled by default but allow users who need to, the ability to opt out of this behaviour.

cc @davidzhao

Hello, I would like to know if webrtc supports H.265 ?

If not, will you support it recently ?

Thanks a lot!

To reproduce on iOS:

• have a video track with simulcast with 3 layers with H264 hardware encoding
• disable one layer by setting encoding.isActive to false in peer connection
• start screencast with H264 hardware encoding
• the H264 encoder will fail to init with Failed to create compression session: -12915 error, this is caused by ios encoders limit exceeded (twilio/twilio-video-ios#17 ) however, we should use just 3 encoders (2 simulcast layers + 1 screencast)
• this is probably caused by the encoder not being reset. There is a check here:

  webrtc/video/video_stream_encoder.cc

  Line 131 in a828846

  // No need to reset when stream is inactive.

  that doesn't reset if layer is only disabled and nothing else changed

Hello.

Could someone explain how to compile this sdk for Android?
I tried to replace this repo files with google's files but build_aar.py fails

Thank you in advance

Below are the sample candidates got added while connecting using WiFi which never connects.

1. RTCIceCandidate_YqwKRfA2 - local-candidate - 1686915646988416.0 - {candidateType: host, protocol: udp, address: 127.0.0.1, transportId: RTCTransport_0_1, port: 37661, vpn: false, isRemote: false, ip: 127.0.0.1, networkAdapterType: loopback, networkType: unknown, priority: 2121932543}.
2. RTCIceCandidate_NLGxfctm - local-candidate - 1686915646988416.0 - {candidateType: host, protocol: tcp, address: 127.0.0.1, transportId: RTCTransport_0_1, port: 33007, vpn: false, isRemote: false, ip: 127.0.0.1, networkAdapterType: loopback, networkType: unknown, priority: 1517952767}.
3. RTCIceCandidate_Ab+ycNxZ - local-candidate - 1686915646988416.0 - {candidateType: host, protocol: tcp, address: 0.0.0.0, transportId: RTCTransport_0_1, port: 60505, vpn: false, isRemote: false, ip: 0.0.0.0, networkAdapterType: cellular, networkType: cellular, priority: 1518083839}.
4. RTCIceCandidate_Fl3Aldbc - local-candidate - 1686915646988416.0 - {candidateType: host, protocol: tcp, address: 0.0.0.0, transportId: RTCTransport_0_1, port: 50047, vpn: false, isRemote: false, ip: 0.0.0.0, networkAdapterType: wifi, networkType: wifi, priority: 1518214911}.

Clone or fork the repo and switch to master

git clone https://github.com/webrtc-sdk/webrtc.git -b master webrtc-src
cd webrtc-src

Add upstream

git remote add upstream https://chromium.googlesource.com/external/webrtc.git
git fetch upstream

Edit the upstream .git/config to download branch-heads/* from googl webrtc.

• add fetch = +refs/branch-heads/*:refs/remotes/upstream/branch-heads/* to .git/config

[remote "upstream"]
        url = https://chromium.googlesource.com/external/webrtc
        fetch = +refs/heads/*:refs/remotes/upstream/*
        fetch = +refs/branch-heads/*:refs/remotes/upstream/branch-heads/*

• Pull all branches from upstream.

git fetch upstream
git branch -a

• You can see that all branches look like this.

...
remotes/upstream/branch-heads/4006
remotes/upstream/branch-heads/4007
...

You can see the all branches in https://chromiumdash.appspot.com/branches.

for example:
Sync the m92 version to our repo.

M92 = refs/branch-heads/4515

git checkout -b branch-m92 remotes/upstream/branch-heads/4515
git push --set-upstream origin branch-m92

Now that you have an m version branch, you can add your own patches and compile them.

Cheers.

Currently, webrtc compiles a mac library with objective-c interface by default. I want to compile a c++ interface that can run on the mac, without objective-c code specific to the mac platform. How to do this?

I am looking for a way to build WebRTC-SDK xcframework (for ios) from my forked webrtc git after editing some code.
I tried to follow the process from the git below.
https://github.com/alexpiezo/WebRTC

But it appear that the generated framework file doesn't contain some header files present in the original code.
Does anyone know what I might have done wrong?

thank you

I have use gclient to fetch the webrtc code, compiled the aar file for android platform, and get a libwebrtc.aar,
But when I compare the aar file with 'com.github.webrtc-sdk:android:92.4515.03', I get the diffrent directory.

In src directory: I compile the code by command:
./tools_webrtc/android/build_aar.py --verbose --build-dir out/aar-release --arch arm64-v8a

I'm trying to compile the project to generate iOS and Android binaries but I can't found that configuration file.

Description

I want to support soft codec in ios. so I use rtc_use_h264=true, but I can't compile it.

Trace

ld64.lld: error: library not found for -lrt
ld64.lld: error: undefined symbol: _av_buffer_create
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _av_image_check_size
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _avcodec_align_dimensions
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _av_frame_alloc
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _avcodec_open2
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _avcodec_find_decoder
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _avcodec_alloc_context3
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _av_frame_unref
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _av_packet_free
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _av_buffer_get_opaque
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _avcodec_receive_frame
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _avcodec_send_packet
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _av_packet_alloc
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

ld64.lld: error: undefined symbol: _avcodec_free_context
>>> referenced by obj/modules/video_coding/libwebrtc_h264.a(h264_decoder_impl.o)

Last

If there's any suggestion, thank you very much

error does not occur in m104_release, but error occurs in m114_release

SyntaxError: Inner class (MediaStreamTrack.MediaType) can not be used directly by JNI. Please import the outer class, probably:
import org.webrtc.MediaStreamTrack;

.gclient
solutions = [
{
"name": "src",
"url": "https://github.com/webrtc-sdk/webrtc.git@m114_release",
"deps_file": "DEPS",
"managed": False,
"custom_deps": {},
},
]
target_os = ["android", "unix"]

gn gen out/Debug --args='target_os="android" target_cpu="arm64"'
ninja -C out/Debug

Hello!

I have compiled the sdk for android and generated the libwebrtc.aar with your sources.
Now, my problem is this:

e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (182, 9): 'setRates' overrides nothing
e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (182, 58): Unresolved reference: RateControlParameters
e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (183, 35): Not enough information to infer type variable T
e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (183, 42): Not enough information to infer type variable V
e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (183, 77): Unresolved reference: setRates
e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (192, 9): 'getEncoderInfo' overrides nothing
e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (192, 53): Unresolved reference: EncoderInfo
e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (193, 35): Not enough information to infer type variable T
e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (193, 42): Not enough information to infer type variable V
e: /Library/Flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.8.7/android/src/main/java/com/cloudwebrtc/webrtc/SimulcastVideoEncoderFactoryWrapper.kt: (193, 77): Unresolved reference: encoderInfo

any ideas?

Thank you in advance

I'm not sure whether to report this in Flutter WebRTC or here in the WebRTC-sdk. @cloudwebrtc You'll probably want to be generally aware of this too.

Describe the bug

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:mergeAppCenterReleaseAssets'.
> Could not resolve all files for configuration ':app:appCenterReleaseRuntimeClasspath'.
   > Could not resolve io.flutter:armeabi_v7a_release:1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.
     Required by:
         project :app
      > Could not resolve io.flutter:armeabi_v7a_release:1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.
         > Could not get resource 'https://plugins.gradle.org/m2/io/flutter/armeabi_v7a_release/1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6/armeabi_v7a_release-1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.pom'.
            > Could not HEAD 'https://jcenter.bintray.com/io/flutter/armeabi_v7a_release/1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6/armeabi_v7a_release-1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.pom'. Received status code 502 from server: Bad Gateway
   > Could not resolve io.flutter:arm64_v8a_release:1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.
     Required by:
         project :app
      > Could not resolve io.flutter:arm64_v8a_release:1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.
         > Could not get resource 'https://plugins.gradle.org/m2/io/flutter/arm64_v8a_release/1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6/arm64_v8a_release-1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.pom'.
            > Could not HEAD 'https://jcenter.bintray.com/io/flutter/arm64_v8a_release/1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6/arm64_v8a_release-1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.pom'. Received status code 502 from server: Bad Gateway
   > Could not resolve io.flutter:x86_64_release:1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.
     Required by:
         project :app
      > Could not resolve io.flutter:x86_64_release:1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.
         > Could not get resource 'https://plugins.gradle.org/m2/io/flutter/x86_64_release/1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6/x86_64_release-1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.pom'.
            > Could not HEAD 'https://jcenter.bintray.com/io/flutter/x86_64_release/1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6/x86_64_release-1.0.0-890a5fca2e34db413be624fc83aeea8e61d42ce6.pom'. Received status code 502 from server: Bad Gateway
   > Could not resolve com.github.webrtc-sdk:android:93.4577.01.
     Required by:
         project :app > project :flutter_webrtc
      > Skipped due to earlier error
      > Could not resolve com.github.webrtc-sdk:android:93.4577.01.
         > Could not get resource 'https://jcenter.bintray.com/com/github/webrtc-sdk/android/93.4577.01/android-93.4577.01.pom'.
            > Could not GET 'https://jcenter.bintray.com/com/github/webrtc-sdk/android/93.4577.01/android-93.4577.01.pom'. Received status code 502 from server: Bad Gateway

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1m 45s
Running Gradle task 'bundleRelease'...                            106.7s
[!] Gradle threw an error while downloading artifacts from the network. Retrying to download...
FLUTTER_ROOT: C:\flutter

FAILURE: Build failed with an exception.

            > Could not GET 'https://jcenter.bintray.com/com/github/webrtc-sdk/android/93.4577.01/android-93.4577.01.pom'. Received status code 502 from server: Bad Gateway

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org
BUILD FAILED in 21s
Running Gradle task 'bundleRelease'...                             21.9s[!] Gradle threw an error while downloading artifacts from the network. Retrying to download...
Gradle task bundleRelease failed with exit code 1

To Reproduce
Discovered while attempting to run flutter build appbundle with Flutter WebRTC as a dependency. Or, visit the WebRTC pom in Bintray. Or just the root domain itself https://jcenter.bintray.com/. You'll notice a 502 Gateway Error. The required dependencies fail to download.

Expected behavior
The required dependencies succeed downloading.

Platform information
N/A - Happens everywhere.

Notes
This issue is most likely temporary and the outage will be restored soon. However, it looks like jcenter/bintray are read-only repositories right now. While the company plans to run the repository indefinitely this way, its clear interruptions like this can still continue to occur. If possible, consider migrating dependencies away from this service.

flutter_webrtc 80 issues
x Undefined symbol:OBJC_CLASS$_RTCAudioTrack
x Undefined symbol:OBJC_CLASS$RTCCVPixelBuffer
x Undefined symbol:
OBJC_CLASS$RTCCameraVideoCapturer
x Undefined symbol:OBJC_CLASS$RTCConfigurationx Undefined symbol:OBJC_CLASS$RTCCryptoOptionsx Undefined symbol:OBJC_CLASS$RTCDataBuffer
x Undefined symbol: OBJC_CLASS$RTCDataChannel
x Undefined symbol:
OBJC_CLASS$RTCDataChannelConfiguration
Undefined symbol:
OBJC_CLASS$RTCDefaultVideoDecoderFactory
Undefined symbol:
x
OBJC_CLASS$RTCDefaultVideoEncoderFactory
x Undefined symbol:
OBJC_CLASS$ RTCDesktopMediaList
x Undefined symbol:OBJC_CLASS$RTCFrameCryptor
x Undefined symbol:
OBJC_CLASS$RTCFrameCryptorKeyProvider
Undefined symbol:
x
OBJC CLASS$ RTCH264ProfileLevelld
xUndefined symbol:OBJC_CLASS$_RTCI420Buffer
x Undefined symbol:OBJC_CLASS$_RTCIceCandidate
x Undefined symbol:OBJC_CLASS$RTCIceServer
x Undefined symbol:
OBJC CLASS $ RTCMediaConstraints
Undefined symbol:
OBJC CLASS $RTCMediaStreamTrackx Undefined symbol:OBJC_CLASS$RTCPeerConnection
Undefined symbol:OBJC CLASS$ RTCPeerConnectionFactory
Undefined symbol:
X
OBJC_CLASS$_RTCPeerConnectionFactoryOptionsUndefined symbol:

Hi All,

I am trying to build the libraries in Mac with MacOS13.3.sdk.

I am getting below error

../../modules/desktop_capture/mac/screen_capturer_mac.mm:453:11: error: 'CGDisplayStreamUpdateGetRects' is only available on macOS 13.0 or newer [-Werror,-Wunguarded-availability-new] CGDisplayStreamUpdateGetRects(updateRef, kCGDisplayStreamUpdateDirtyRects, &count);

Then I tried changing minimum supported version to -mmacosx-version-min=13.0. Then I got below error

../../sdk/objc/components/capturer/RTCCameraVideoCapturer.m:129:27: error: 'devicesWithMediaType:' is deprecated: first deprecated in macOS 10.15 - Use AVCaptureDeviceDiscoverySession instead. [-Werror,-Wdeprecated-declarations]

Would someone like to give any inputs ?

BRs
Kiran

I can see lot of commits to fix / overcome issues are available on branch m104_release but not on m114_release.
I would like to know about how we are planning to include them.

We recently integrated WebRTC on both iOS and Android. Everything seems to be working fine, except that my backend isn't receiving the REMB packets on iOS, while Android works fine. Can anyone help or provide some pointers on how we can enable this on iOS? It doesn't seem like the RTCPeerConnection interface provides any configuration options for that.

Any help is appreciated.

int32_t ObjCDesktopMediaList::UpdateSourceList(bool force_reload, bool get_thumbnail) {
  if (force_reload) {
    for (auto source : sources_) {
      [objcMediaList_ mediaSourceRemoved:source.get()];
    }
    sources_.clear();
  }

  webrtc::DesktopCapturer::SourceList new_sources;

  thread_->BlockingCall([this, &new_sources] {
    capturer_->GetSourceList(&new_sources);
  });

  typedef std::set<DesktopCapturer::SourceId> SourceSet;
  SourceSet new_source_set;
  for (size_t i = 0; i < new_sources.size(); ++i) {
    if (type_ == kScreen && new_sources[i].title.length() == 0) {
      new_sources[i].title = std::string("Screen " + std::to_string(i + 1));
    }
    new_source_set.insert(new_sources[i].id);
  }
  // Iterate through the old sources to find the removed sources.
  for (size_t i = 0; i < sources_.size(); ++i) {
    if (new_source_set.find(sources_[i]->id()) == new_source_set.end()) {
      [objcMediaList_ mediaSourceRemoved:(*(sources_.begin() + i)).get()];
      sources_.erase(sources_.begin() + i);
      --i;
    }
  }
  // Iterate through the new sources to find the added sources.
  if (new_sources.size() > sources_.size()) {
    SourceSet old_source_set;
    for (size_t i = 0; i < sources_.size(); ++i) {
      old_source_set.insert(sources_[i]->id());
    }
    for (size_t i = 0; i < new_sources.size(); ++i) {
      if (old_source_set.find(new_sources[i].id) == old_source_set.end()) {
        MediaSource *source = new MediaSource(this, new_sources[i], type_);
        sources_.insert(sources_.begin() + i, std::shared_ptr<MediaSource>(source));
        [objcMediaList_ mediaSourceAdded:source];
        GetThumbnail(source, true);
      }
    }
  }

  RTC_DCHECK_EQ(new_sources.size(), sources_.size());

  // Find the moved/changed sources.
  size_t pos = 0;
  while (pos < sources_.size()) {
    if (!(sources_[pos]->id() == new_sources[pos].id)) {
      // Find the source that should be moved to |pos|, starting from |pos + 1|
      // of |sources_|, because entries before |pos| should have been sorted.
      size_t old_pos = pos + 1;
      for (; old_pos < sources_.size(); ++old_pos) {
        if (sources_[old_pos]->id() == new_sources[pos].id) break;
      }
      RTC_DCHECK(sources_[old_pos]->id() == new_sources[pos].id);

      // Move the source from |old_pos| to |pos|.
      auto temp = sources_[old_pos];
      sources_.erase(sources_.begin() + old_pos);
      sources_.insert(sources_.begin() + pos, temp);
      //[objcMediaList_ mediaSourceMoved:old_pos newIndex:pos];
    }

    if (sources_[pos]->source.title != new_sources[pos].title) {
      sources_[pos]->source.title = new_sources[pos].title;
      [objcMediaList_ mediaSourceNameChanged:sources_[pos].get()];
    }
    ++pos;
  }

  if (get_thumbnail) {
    for (auto source : sources_) {
      GetThumbnail(source.get(), true);
    }
  }
  return sources_.size();
}

This method crashes when called simultaneously by multiple threads with force_reload = true. I suggest modifying the implementation to use a mutex lock to protect shared resources and ensure safe access in a multi-threaded environment. Here is the suggested modification:

int32_t ObjCDesktopMediaList::UpdateSourceList(bool force_reload, bool get_thumbnail) {
  std::lock_guard<std::mutex> lock(mutex_);

  if (force_reload) {
    for (auto source : sources_) {
      [objcMediaList_ mediaSourceRemoved:source.get()];
    }
    sources_.clear();
  }

  // CODE  ..

  return sources_.size();
}

According to Google library of webrtc and documentation this should be the behavior when disabling a stream:
https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack/enabled

It should keep sending audio frames with 0 value and in video should send black frames.

The current behavior was changed in the webrtc library at some point and the mute is changed to true when disabling the track.
This causes unexpected behavior and is different than the web google behavior.

Can you please fix this?