Comments (2)
I have a rough version of AudioStreamTrack, a variation of your implementation and aiortc's implementation
class AudioStreamTrack(MediaStreamTrack):
"""
A dummy audio track which reads silence.
"""
def __init__(self, callback: AudioSourceCallback) -> None:
super().__init__()
self.kind = "audio"
self._callback = callback
self._started_at: Optional[float] = None
self._pts: Optional[int] = None
# _start: float
# _timestamp: int
async def recv(self) -> av.frame.Frame:
print('sending some stuff')
if self.readyState != "live":
raise MediaStreamError
sample_rate = 8000
samples = int(AUDIO_PTIME * sample_rate)
if hasattr(self, "_timestamp"):
self._timestamp += samples
wait = self._start + (self._timestamp / sample_rate) - time.time()
print(wait)
await asyncio.sleep(wait)
else:
self._start = time.time()
self._timestamp = 0
frame = AudioFrame(format="s16", layout="mono", samples=samples)
for p in frame.planes:
p.update(bytes(p.buffer_size))
frame.pts = self._timestamp
frame.sample_rate = sample_rate
frame.time_base = fractions.Fraction(1, sample_rate)
return frame
def _call_callback(self) -> av.AudioFrame:
try:
frame = self._callback()
except Exception as exc:
logger.error(
"AudioSourceCallbackTrack: Audio frame callback raised an exception: %s", # noqa: E501
exc,
)
raise
return frame
Seems to be working, hard to tell since it's suppose to be silent.
However, I'd like to send an audio for speech to text, then receive an audio track. Basically fully duplex audio, each track with their respective handlers. Here is an example webrtc object.
webrtc_streamer(
key="player",
mode=WebRtcMode.SENDRECV,
audio_frame_callback=process_audio,
source_audio_track=AudioStreamTrack(audio_source_callback),
media_stream_constraints={"video": False, "audio": True},
on_change=on_change,
async_processing=True
)
Only the source audio track is played. I am not able to process the received audio frames process_audio
.
What's the distinction between source_audio_track
and audio_frame_callback
.
from streamlit-webrtc.
hello, thanks for the great work.
I am interested in the audio source. My use case is that I would like to stream audio to the client without the player component. Do you have any pointers on how that can be done?
from streamlit-webrtc.
Related Issues (20)
- how to mix audio track HOT 2
- Error when running Streamlit app: FileNotFoundError
- Failed to get any callback frames to render HOT 1
- Receiving an existing WebRTC stream as an input track
- Incompatibility between the av==10.0.0 library and streamlit-webrtc==0.47.1 HOT 1
- how to let the ctx player running background with hidden ?
- Encoding too slow - option for hardware acceleration? HOT 3
- How do i set a trigget function for webrtc_streamer
- multi users audio process
- Is there a way to send local video frame by socket to this tool ? HOT 1
- How can I stream both audio and video using streamlit_webrtc and only save the audio to a .wav file?
- audio implementation with lags and discontinuities
- CPU-heavy task causes frame delay HOT 2
- No module named 'sample_utils' HOT 1
- Low quality video streaming HOT 1
- While I import streamlit webrtc in my project I got an error on executing the code - ImportError: DLL load failed while importing _rust: The specified procedure could not be found.
- How to run on Runpod?
- Non of the samples and demos work! The video/audio stuck loading forever. HOT 3
- Why error : AttributeError: 'NoneType' object has no attribute 'call_exception_handler' HOT 11
- webrtc_streamer doesn't have attribute tobytes() to convert the captured image to img_bytes and make an API call HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from streamlit-webrtc.