Microphone support? about showmewebcam HOT 10 OPEN

For a good setup, you'd want a dedicated microphone. A shabby microphone such as the one that is on a board on the Pi and far away from your face would require a lot of work to get rid of the noise while a clip-on microphone such as one in this list will serve you very well.

Not that I'm against the idea of supporting microphones eventually, but I think it will take some time to sort out (we need a professional person in sound DSP) while having a good picture quality is always going to be the top priority. For a good webcam, we don't have a lot of good and cheap solution but for a good microphone, we already had good and cheap solutions :)

from showmewebcam.

@htruong there is your i2s mic repo https://github.com/opencardev/snd-i2s_rpi

Maybe https://invensense.tdk.com/wp-content/uploads/2015/02/Microphone-Array-Beamforming.pdf might interest you.
You can create an endfire beamformer by just delaying by 2 samples and adding the inverse of the rear to the front mic.
The frequency dependent response can be fixed with alsaeq https://www.hifiberry.com/docs/software/guide-adding-equalization-using-alsaeq/ and you should have quite a decent quality directional mic sort of shotgun style?

@ 48khz the min mic spacing of 7mmm and overall spacing that creates the null notch needs some play but thinking it would be an excellent addition if it works? PS the SPH0645LM4H might be only 18bit but they will do 64Khz.

Its not going to be a cutting edge high speed dsp beamformer but provide a degree of noise reduction and far field directionality that might be fit for purpose.

Probably rather than a driver maybe a utility with params endfire -i -o -c -g (-i input, -o output, -c compressor level, -g agc level)
Added -c, -g as if you doing processing those could be tacked on fairly easy to the stream)
Use a snd_loopback sink side for -o and its then avail as a normal alsa source on the other side of the loopback and the endfire just acts as a middleman util so the same driver can either be stereo or beamformer depending if you run endfire or not.

You might not want to add the AGC and like alsaeq just use the speex alsa-plugin raspbian has an outdated speex/speexdsp version that if you compile the release and then recompile alsa-plugins its no longer missing and you can use the AGC of speex which works quite well.

Its prob likely to be better on a Pi3a+ as that 10$ extra over a Zero for me is just so worth it, but dunno if the otg problems still exist.

from showmewebcam.

+1 It seems like this would be a great addition to the project. This would allow us to sync the video track to the sound track more easily I'd think?

from showmewebcam.

Its sort of important to record sound as part of the process than as a separate file to be latency adjusted and lip synced after.

If the Pi4 ever comes to fruition then there are USB that can use just plain old USB soundcards.

The IQaudio Codec Zero hat is also another interesting product as it has 2x I2S mics and a mono electret input.
There are also extremely cheap PCM1802 modules that as far as I know just need a slight modification to the https://github.com/opencardev/snd-i2s_rpi code.

As a product review I suggest giving the boya-MM1 a go as it can be found for £15 on ebay and from me and a rake of reviews on youtube its excellent for the $

The downside to the omnidirectional I2S mics is without algs they are poor and capture much which is not front of cam.

from showmewebcam.

Something like this could work:

https://ceworkbench.wordpress.com/2019/07/21/setting-up-a-composite-ethernet-audio-usb-gadget/

I'm currently probably not going to have tons of time looking at it but this sounds plausible with some minimal effort. I do have a i2s mic and will try to work on it whenever it's convenient. Anyone else that knows alsa is also welcome to do so.

from showmewebcam.

Never thought of it as a composite driver as 1.1.x and 1.3.x provide support for MJPEG and H264 video formats as well so you gain compression as well.
I am a noob to the project and is raw YUY2 purely due to load on a Pi Zero?

https://hmbd.wordpress.com/2016/08/01/raspberry-pi-video-and-audio-recording-and-streaming-guide/
https://gist.github.com/moritzmhmk/48e5ed9c4baa5557422f16983900ca95
https://www.reddit.com/r/raspberry_pi/comments/69o721/finally_got_my_raspberry_to_capture_picam_video/
https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=177990&p=1157514#p1137469

Also with the tests how come the compute module isn't there as isn't that the only one with both 4 lane CSI & usb gadget mode?

from showmewebcam.

Just created a draft pull request that somewhat works.

I've tested it with an Infineon IM69D130 MEMS microphone + Analog Devices ADAU7002 PDM-to-i2s converter (eval board) but any i2s microphone should suffice.

It successfully produces audio from the microphone on the host device, however I'm seeing some odd behavior:

• Using the camera + mic combo on Zoom works fine, but using any WebRTC service introduces artifacts in the video
• On my Mac, I'm unable to simultaneously record audio and video using QuickTime Player for more than 1-3 seconds, however recording only audio or only video works fine
• alsaloop CPU utilization is unusually high at ~30-40% when idle, but drops to 10-18% when the mic is in use
• When I replace alsaloop with speaker-test -D plughw:1,0 -r48000 -c2 I'm able to record audio and video in QuickTime with no problems

It seems like there's something going on between the uac2 Linux driver and alsaloop but I'm not sure how to proceed next. I tried setting CONFIG_USB_DWC2_DEBUG=y but it produced way too much output. Suggestions welcome!

from showmewebcam.

What about a dual mic Hat for the Pi Zero like this?

https://shop.pimoroni.com/products/respeaker-2-mics-phat

although might make the webcam a bit back-heavy with both a zero and this stacked on the back...

from showmewebcam.

What about a dual mic Hat for the Pi Zero like this?

https://shop.pimoroni.com/products/respeaker-2-mics-phat

although might make the webcam a bit back-heavy with both a zero and this stacked on the back...

I've got one on the back of my Pi Zero HQ cam combo and it is sitll front heavy (on the camera lens side). I'm probably going to try and make a cover for the back so that it focuses the audio. I might get some stand-off screws so that I can turn the boards around and have the mics more at the front and the cable out of the way.

from showmewebcam.

Its far from ideal as any extended ports are going to be essentially a bass tube.

What you can do is mount 2x I2s mics direct to gpio as adafruit https://learn.adafruit.com/adafruit-i2s-mems-microphone-breakout/raspberry-pi-wiring-test

The aliexpress I2s mics are even cheaper https://www.aliexpress.com/item/32970296252.html 14mm dia is fairly small, but there are a range of modules to choose from.

If you install pulseaudio and just the beamforming part of WebRTC AEC then at least you have a bit of beamforming and essentially the stereo pair becomes a single channel shotgun mic as also with the 2 mic hats having 2x onmi-directional channels with a 58mm spacing is a whole load of pointless. Its just load on a Pi Zero and have never checked as I have used them for voiceAI on a Pi3

That way the ports can be on the same axis as lens and the modules are fairly easy to provide some rear isolation.

from showmewebcam.