Comments (22)
Hi and thank you for your interest in our video call application, Kvazzup!
Currently Kvazzup uses Qt for its camera input. If the external camera is supported by Qt, then it should work in Kvazzup. You may have to select it in the settings.
Which operating system are you using? I'm pretty sure there is a bug in Qt Gstreamer module on Linux in that it doesn't support changing the camera. You may have more luck with using other Qt media interfaces, but I'm afraid I cannot help with this at this time.
If you have a custom camera you want to integrate that is not supported by Qt, the easiest way would be to inherit the Filter module and replace the CameraFilter in FilterGraph with the new class. This makes it so that the video feed in the application uses your camera and ignores the settings. The settings will need additional modifications if you want them working with the new camera.
Hope this helps.
from kvazzup.
@polusto thanks for your answer!
I am using Ubuntu 20.04.
Maybe I need your help to extent current framework for working with CustomerCamera(), and configuration possibility for switching between built-in and customer’s camera!
Can help me by extending a template CustomerCamera?
Thanks
from kvazzup.
What kind of camera are you using? It looks like it may be the Qt bug that is causing issues.
from kvazzup.
I am using a prototype (non-standard) camera, so that I can get images from a special image capturing.
For testing CustomerCamera, I am thinking about using e.g. openCV VideoCapture.
What do you think?
from kvazzup.
It is possible. If you want the selection to also work, that is more effort, but it should be very easy to just get the camera working.
There is no template for filters. You just have to create a new class that inherits the Filter class in media/processing/filter.h. Then all you have to do is call sendOutput with the frame as a parameter (and replace the cameraFilter in FilterGraph). Take a look at the existing cameraFilter process function on how to do that.
There are also additional tricks that can be done with threads, but it is hard to comment on that since I'm not very familiar with openCV. Current implementation does the copying in cameraFilter thread by calling wakeup every time a frame arrives. This is needed so that Qt main thread is not slowed down, but it may be unnecessary with OpenCV.
Custom comes from word customized, not customer and it was just an example :)
from kvazzup.
I found an example and modified it as CustomerCamera, which uses openCV VideoCapture as images source.
class CustomerCamera : public QThread
{
Q_OBJECT
private:
bool stop;
QMutex mutex;
QWaitCondition condition;
Mat frame;
int frameRate;
VideoCapture *capture;
Mat RGBframe;
QImage img;
signals:
//Signal to output frame to be displayed
void processedImage(const QImage &image);
protected:
void run();
void msleep(int ms);
public:
//Constructor
CustomerCamera(QObject *parent = 0);
//Destructor
~CustomerCamera();
//Load a video from memory
bool loadVideo(std::string filename);
//Play the video
void Play();
//Stop the video
void Stop();
//check if the player has been stopped
bool isStopped() const;
void setCurrentFrame( int frameNumber);
//
double getFrameRate();
double getCurrentFrame();
double getNumberOfFrames();
};
CustomerCamera::CustomerCamera(QObject *parent)
: QThread(parent)
{
stop = true;
}
bool CustomerCamera::loadVideo(string filename) {
capture = new cv::VideoCapture(filename);
if (capture->isOpened())
{
frameRate = (int) capture->get(CV_CAP_PROP_FPS);
return true;
}
else
return false;
}
void CustomerCamera::Play()
{
if (!isRunning()) {
if (isStopped()){
stop = false;
}
start(LowPriority);
}
}
void CustomerCamera::run()
{
int delay = (1000/frameRate);
while(!stop){
if (!capture->read(frame))
{
stop = true;
}
if (frame.channels()== 3){
cv::cvtColor(frame, RGBframe, 4); //CV_BGR2RGB
img = QImage((const unsigned char*)(RGBframe.data),
RGBframe.cols,RGBframe.rows,QImage::Format_RGB888);
}
else
{
img = QImage((const unsigned char*)(frame.data),
frame.cols,frame.rows,QImage::Format_Indexed8);
}
emit processedImage(img);
this->msleep(delay);
}
}
CustomerCamera::~CustomerCamera()
{
mutex.lock();
stop = true;
capture->release();
delete capture;
condition.wakeOne();
mutex.unlock();
wait();
}
void CustomerCamera::Stop()
{
stop = true;
}
void CustomerCamera::msleep(int ms){
struct timespec ts = { ms / 1000, (ms % 1000) * 1000 * 1000 };
nanosleep(&ts, NULL);
}
bool CustomerCamera::isStopped() const{
return this->stop;
}
double CustomerCamera::getCurrentFrame(){
return capture->get(CV_CAP_PROP_POS_FRAMES);
}
double CustomerCamera::getNumberOfFrames(){
return capture->get(CV_CAP_PROP_FRAME_COUNT);
}
double CustomerCamera::getFrameRate(){
return frameRate;
}
void CustomerCamera::setCurrentFrame( int frameNumber )
{
capture->set(CV_CAP_PROP_POS_FRAMES, frameNumber);
}
from kvazzup.
Now, I can use my external simple camera (loading image from files using Qt) after modifying this CustomCamera, + CustomCameraFilter .
It works well now. Thanks
from kvazzup.
Great to hear!
Want to share what is the purpose of this Kvazzup integration? I'm very curious what people are using Kvazzup for.
from kvazzup.
In my current version, I try to load images from a folder and send them continuously.
void CustomCamera::run()
{
int delay = (1000/frameRate);
while(!stop)
{
// simple simulation of camera capturing function
const QString url = R"(faceImage.bmp)";
QImage img(url);
emit processedImage(img);
}
}
BUT I am thinking about how handle with a custom video stream:
I have a custom camera, which is used for the main application, I call application A.
Video Conference is application B, already startet and in standby mode.
If application A found a defined interesting situation, e.g. "unknown people in my garden at night", it forwards the current video stream of camera to Application B, it wakes up Application B and make a video conference to predefined person (e.g. superwise people).
There are some ways for doing the communication tasks:
- Using share memory for data exchange between app A and app B
- using socket communication
BUT I do not know
- what is better, or is there any alternative?
- how to wake up Video conference by incomming video stream?
Do you have any idea?
Best thanks
from kvazzup.
I don't exactly understand what you are trying to do. I would do it so that the Controller would try to call the application A using SIP (this can be done very easyli by just calling sip_.startCall() at the right point) and set an option in settings called auto-accept on so user would not have to click accept with it. If the application A does not exist, the call would fail (and hopefully not cause problems).
I cannot comment on other kinds of ways for doing this. They may or may not be easier. There is also some risk of encountering bugs that I do not have time to fix if you modify Kvazzup very heavily.
from kvazzup.
Thanks.
Here is again what I am trying to do with functional blocks:
There are 2 PCs:
- PC1 has ApplicationA + Host-VideoConference
a) ApplicationA captures images from a live-camera and does image processing somehows
b) when necessary, ApplicationA sends a video stream X to the Host-VideoConference
c) Host-VideoConference uses Video stream X as its own "virtual camera".
d) Host-VideoConference starts a video conference by calling PC2
e) PC2 will see Video stream X from PC1 - PC2 has VideoConference in waiting mode. It waits for any call from PC1
I understand that the current Controller has a SIP, which is responsible for video conference. I do not know if it can have the 2nd task (connection to ApplicationA ans waiting for any video stream from there)?
Do you mean I need the second SIP for communicating to ApplicationA?
Thanks
from kvazzup.
Ok, now I understand more. In that case I would not involve SIP since you are trying to change your own camera feed and not have a call with the camera application.
This is quite far from what Kvazzup is currently able to do. One option would be to integrate everything into the camerafilter using some custom protocol.
Another option would be to somehow manage which camera is used inside the filter graph by somehow first connecting the RTP stream (created in delivery) to the outgoing stream. The filter graph is pretty flexible, but the logic would have to be rewritten for stream construction and graph construction. This also needs some additional way of starting the stream in application A.
Combining application A + conference host would get rid of a need to communicate the start of the video feed.
from kvazzup.
I think ApplicationA can connect to my CustomCamera via RTP but I do not know how to implement RTP
- server in my CustomCamera and
- RTP client in ApplicationA
Do you have any example for RTP client/server?
Thanks
from kvazzup.
Kvazzup uses https://github.com/ultravideo/uvgRTP/ There are good examples there on how to use it.
Another option would be just to change the order of filters in filter graph since I don't think there is any rule stating that you cannot replace your camera with RTP receiver, which would then forward the stream. This would require however having it as an alternative to current implementation which would require some planning.
from kvazzup.
Here is my code CustomCamera with uvgRTP for your review. Thanks
#include <QMutex>
#include <QThread>
#include <QImage>
#include <QWaitCondition>
#include <QPixmap>
#include <string>
using namespace std;
class CustomCamera : public QThread
{
Q_OBJECT
private:
bool stop;
QMutex mutex;
QWaitCondition condition;
uint16_t frameRate;
signals:
//Signal to output frame to be displayed
void processedImage(const QImage &image);
protected:
void run();
void msleep(int ms);
public:
//Constructor
CustomCamera(QObject *parent = 0);
//Destructor
~CustomCamera();
void setFrameRate(uint16_t fps){frameRate = fps;};
bool isStopped() const;
void Start();
void Stop();
};
#include <QTextStream>
CustomCamera::CustomCamera(QObject *parent)
: QThread(parent)
{
stop = true;
// initialize RTP-receiver
}
void CustomCamera::Start()
{
if (!isRunning())
{
if (isStopped()) stop = false;
start(QThread::Priority::LowPriority);
}
}
void CustomCamera::run()
{
int delay = (1000/frameRate);
while(!stop)
{
// RTP-receiver is waiting for new image from video streaming
// 1. receive new image and copy it into QImage img;
const QString url = R"(faceImage.bmp)";
QImage img(url);
emit processedImage(img);
this->msleep(delay);
}
}
CustomCamera::~CustomCamera()
{
mutex.lock();
if (!stop)
{
exit(); // stop thread
stop = true;
}
condition.wakeOne();
mutex.unlock();
wait();
}
void CustomCamera::Stop()
{
if (!stop)
{
exit(); // stop thread
stop = true;
}
}
void CustomCamera::msleep(int ms){
struct timespec ts = { ms / 1000, (ms % 1000) * 1000 * 1000 };
nanosleep(&ts, NULL);
}
bool CustomCamera::isStopped() const{
return this->stop;
}
from kvazzup.
I understand that RTP receiver looks like. What do you think?
uvgrtp::media_stream myRTP("localhost", 12345, 12345);
while (1)
{
size_t timeout = 1;
uvgrtp::frame::rtp_frame* frame = myRTP.pull_frame(timeout);
if (frame_is_valid)
convert RTP frame into QImgage
sleep(10);
}
from kvazzup.
These codes has nothing to do with Kvazzup. I don't understand why you have posted it.
I'm here mostly to fix bugs in existing Kvazzup. I can also help you better understand how kvazzup works or where you should make changes if it does not take too much of my time, but I cannot help you with new code, especially if it does not involve Kvazzup at all.
from kvazzup.
I think it relates to Kvazzup, where I want to extend it for handling the input video streaming instead of data from camera.
I have another alternative solution:
- create a new class rtpCameraStreamingFilter, as a copy of UvgRTPReceiver() and
- modify it so that it connects to video stream source via "localhost" and special port 1234567 AND
- its signal is connected to SLOT(handleFrame(QImage)) of Filter()
In principle, rtpCameraStreamingFilter has all functions like CameraFramegrabber() + CameraFilter() for built-in camera.
If you think OK, I will write it and test...
from kvazzup.
But I do not know how to create QFuture<uvg_rtp::media_stream *> stream;
which is needed to initialize UvgRTPReceiver
UvgRTPReceiver(uint32_t sessionID, QString id, StatisticsInterface *stats,
DataType type, QString media, QFuture<uvg_rtp::media_stream *> stream);
I am not sure if it is correct:
std::string addr = "localhost";
int src_port = 1234567;
int dst_port = 1234567;
rtp_format_t fmt = value????
int flags = value???
QFuture<uvg_rtp::media_stream *> stream(addr, src_port, dst_port, fmt, flags);
Can you help me? thanks
from kvazzup.
@polusto I use sender and receiver example of uvgRTP, but it is NOT allowed to run sender and receiver in the same PC using localhost. I got always error by running sender after receiver:
Address already in use
[RTPLIB][ERROR][::bind] Biding to port 8888 failed!
[RTPLIB][ERROR][::log_platform_error] Failed to initialize the underlying socket: Address already in use 98
[RTPLIB][ERROR][::create_stream] Failed to initialize media stream 127.0.0.1:8888/8889
Segmentation fault (core dumped)
Is my argument correct? Thanks
from kvazzup.
@polusto Now, I can implement 2 additional modules for handling with local cameras:
- module A using the existing UvgRTPReceiver() in file "uvgrtpreceiver.cpp" , which directly binds in FilterGraph. This module waits for any video stream from local PC, e.g. video stream from Application 1 as mentioned before.
- module B using my CustomCamera, which can be realized using any communication protocol, e.g. a very simple uvg_rtp::media_stream.
I prefer to the implementation A due to clear SW architecture.
from kvazzup.
I'm closing this issue for the following reasons:
-
Issues are primarily meant for solving bugs and other problems in Kvazzup master branch. I don't currently see any direct way this discussion can be thought to help with this.
-
You are asking questions related to uvgRTP. Those do not belong in Kvazzup repository. Use uvgRTP repository or Stack Overflow.
-
I do not have time to answer basic programming questions. I can give high level answers to Kvazzup related issues, but not specific and especially not ones that have nothing to do with Kvazzup. Use Stack Overflow.
-
I nor this repository benefits from these questions being answered. Sure, I do have time to answer one persons questions, but since I try to treat everyone equally, I do not have time to answer if more people start asking these kinds of questions. Use Stack Overflow or something similar.
Any one of these would merit the closing of this issue. If you are looking to do a pull request or something similar, that is a separate discussion, but currently I don't see how this discussion belongs in the issues section of Kvazzup.
from kvazzup.
Related Issues (20)
- STUN adress? HOT 35
- Screen sharing is not working HOT 2
- "Illegal instruction" crash when starting a call on Linux in release mode HOT 10
- bad audio quality in Ubuntu 20.04 HOT 58
- crashed after stop conference HOT 5
- Compile problem with VS 2017 HOT 4
- Issue template
- Question: How Kvazzup filter graph handles sending video HOT 4
- Qt is unable to get Camera parameters on Windows Surface Pro 7 HOT 8
- Pad video frames not divisible by 8 HOT 1
- Reducing audio latency HOT 1
- Fine tuning AEC HOT 2
- Kvazzup crashes when connection established HOT 8
- Authentication via SIP to a SIP proxy HOT 1
- Linking problem at latest Ubuntu HOT 6
- Migrating to Qt 6 HOT 2
- Screen sharing without a camera HOT 1
- Make sure the order of received audio frames is correct
- Installation on Ubuntu 20.04 - unable to find QT6 dependencies HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kvazzup.