Git Product home page Git Product logo

Comments (22)

jrsnen avatar jrsnen commented on August 21, 2024

Hi and thank you for your interest in our video call application, Kvazzup!

Currently Kvazzup uses Qt for its camera input. If the external camera is supported by Qt, then it should work in Kvazzup. You may have to select it in the settings.

Which operating system are you using? I'm pretty sure there is a bug in Qt Gstreamer module on Linux in that it doesn't support changing the camera. You may have more luck with using other Qt media interfaces, but I'm afraid I cannot help with this at this time.

If you have a custom camera you want to integrate that is not supported by Qt, the easiest way would be to inherit the Filter module and replace the CameraFilter in FilterGraph with the new class. This makes it so that the video feed in the application uses your camera and ignores the settings. The settings will need additional modifications if you want them working with the new camera.

Hope this helps.

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

@polusto thanks for your answer!
I am using Ubuntu 20.04.
Maybe I need your help to extent current framework for working with CustomerCamera(), and configuration possibility for switching between built-in and customer’s camera!
Can help me by extending a template CustomerCamera?
Thanks

from kvazzup.

jrsnen avatar jrsnen commented on August 21, 2024

What kind of camera are you using? It looks like it may be the Qt bug that is causing issues.

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

I am using a prototype (non-standard) camera, so that I can get images from a special image capturing.
For testing CustomerCamera, I am thinking about using e.g. openCV VideoCapture.
What do you think?

from kvazzup.

jrsnen avatar jrsnen commented on August 21, 2024

It is possible. If you want the selection to also work, that is more effort, but it should be very easy to just get the camera working.

There is no template for filters. You just have to create a new class that inherits the Filter class in media/processing/filter.h. Then all you have to do is call sendOutput with the frame as a parameter (and replace the cameraFilter in FilterGraph). Take a look at the existing cameraFilter process function on how to do that.

There are also additional tricks that can be done with threads, but it is hard to comment on that since I'm not very familiar with openCV. Current implementation does the copying in cameraFilter thread by calling wakeup every time a frame arrives. This is needed so that Qt main thread is not slowed down, but it may be unnecessary with OpenCV.

Custom comes from word customized, not customer and it was just an example :)

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

I found an example and modified it as CustomerCamera, which uses openCV VideoCapture as images source.

class CustomerCamera : public QThread
{    
  Q_OBJECT
 private:
    bool stop;
    QMutex mutex;
    QWaitCondition condition;
    Mat frame;
    int frameRate;
    VideoCapture *capture;
    Mat RGBframe;
    QImage img;
 signals:
 //Signal to output frame to be displayed
      void processedImage(const QImage &image);
 protected:
     void run();
     void msleep(int ms);
 public:
    //Constructor
    CustomerCamera(QObject *parent = 0);
    //Destructor
    ~CustomerCamera();
    //Load a video from memory
    bool loadVideo(std::string filename);
    //Play the video
    void Play();
    //Stop the video
    void Stop();
    //check if the player has been stopped
    bool isStopped() const;
    void setCurrentFrame( int frameNumber);
    //
    double getFrameRate();
    double getCurrentFrame();
    double getNumberOfFrames();
};

CustomerCamera::CustomerCamera(QObject *parent)
 : QThread(parent)
{
    stop = true;
}

bool CustomerCamera::loadVideo(string filename) {

    capture  =  new cv::VideoCapture(filename);

    if (capture->isOpened())
    {
        frameRate = (int) capture->get(CV_CAP_PROP_FPS);
        return true;
    }
    else
        return false;
}

void CustomerCamera::Play()
{
    if (!isRunning()) {
        if (isStopped()){
            stop = false;
        }
        start(LowPriority);
    }
}

void CustomerCamera::run()
{
    int delay = (1000/frameRate);
    while(!stop){
        if (!capture->read(frame))
        {
            stop = true;
        }
        if (frame.channels()== 3){
            cv::cvtColor(frame, RGBframe, 4);   //CV_BGR2RGB
            img = QImage((const unsigned char*)(RGBframe.data),
                              RGBframe.cols,RGBframe.rows,QImage::Format_RGB888);
        }
        else
        {
            img = QImage((const unsigned char*)(frame.data),
                                 frame.cols,frame.rows,QImage::Format_Indexed8);
        }
        emit processedImage(img);
        this->msleep(delay);
    }
}

CustomerCamera::~CustomerCamera()
{
    mutex.lock();
    stop = true;
    capture->release();
    delete capture;
    condition.wakeOne();
    mutex.unlock();
    wait();
}
void CustomerCamera::Stop()
{
    stop = true;
}
void CustomerCamera::msleep(int ms){
    struct timespec ts = { ms / 1000, (ms % 1000) * 1000 * 1000 };
    nanosleep(&ts, NULL);
}
bool CustomerCamera::isStopped() const{
    return this->stop;
}

double CustomerCamera::getCurrentFrame(){

    return capture->get(CV_CAP_PROP_POS_FRAMES);
}

double CustomerCamera::getNumberOfFrames(){

    return capture->get(CV_CAP_PROP_FRAME_COUNT);
}

double CustomerCamera::getFrameRate(){
    return frameRate;
}

void CustomerCamera::setCurrentFrame( int frameNumber )
{
    capture->set(CV_CAP_PROP_POS_FRAMES, frameNumber);
}

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

Now, I can use my external simple camera (loading image from files using Qt) after modifying this CustomCamera, + CustomCameraFilter .
It works well now. Thanks

from kvazzup.

jrsnen avatar jrsnen commented on August 21, 2024

Great to hear!

Want to share what is the purpose of this Kvazzup integration? I'm very curious what people are using Kvazzup for.

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

In my current version, I try to load images from a folder and send them continuously.

void CustomCamera::run()
{
  int delay = (1000/frameRate);
  while(!stop)
  {
    // simple simulation of camera capturing function
    const QString url = R"(faceImage.bmp)";
    QImage img(url);
    emit processedImage(img);
  }
}

BUT I am thinking about how handle with a custom video stream:
I have a custom camera, which is used for the main application, I call application A.
Video Conference is application B, already startet and in standby mode.
If application A found a defined interesting situation, e.g. "unknown people in my garden at night", it forwards the current video stream of camera to Application B, it wakes up Application B and make a video conference to predefined person (e.g. superwise people).

There are some ways for doing the communication tasks:

  1. Using share memory for data exchange between app A and app B
  2. using socket communication

BUT I do not know

  1. what is better, or is there any alternative?
  2. how to wake up Video conference by incomming video stream?

Do you have any idea?
Best thanks

from kvazzup.

jrsnen avatar jrsnen commented on August 21, 2024

I don't exactly understand what you are trying to do. I would do it so that the Controller would try to call the application A using SIP (this can be done very easyli by just calling sip_.startCall() at the right point) and set an option in settings called auto-accept on so user would not have to click accept with it. If the application A does not exist, the call would fail (and hopefully not cause problems).

I cannot comment on other kinds of ways for doing this. They may or may not be easier. There is also some risk of encountering bugs that I do not have time to fix if you modify Kvazzup very heavily.

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

Thanks.
Here is again what I am trying to do with functional blocks:
There are 2 PCs:

  • PC1 has ApplicationA + Host-VideoConference
    a) ApplicationA captures images from a live-camera and does image processing somehows
    b) when necessary, ApplicationA sends a video stream X to the Host-VideoConference
    c) Host-VideoConference uses Video stream X as its own "virtual camera".
    d) Host-VideoConference starts a video conference by calling PC2
    e) PC2 will see Video stream X from PC1
  • PC2 has VideoConference in waiting mode. It waits for any call from PC1

I understand that the current Controller has a SIP, which is responsible for video conference. I do not know if it can have the 2nd task (connection to ApplicationA ans waiting for any video stream from there)?
Do you mean I need the second SIP for communicating to ApplicationA?
Thanks

from kvazzup.

jrsnen avatar jrsnen commented on August 21, 2024

Ok, now I understand more. In that case I would not involve SIP since you are trying to change your own camera feed and not have a call with the camera application.

This is quite far from what Kvazzup is currently able to do. One option would be to integrate everything into the camerafilter using some custom protocol.

Another option would be to somehow manage which camera is used inside the filter graph by somehow first connecting the RTP stream (created in delivery) to the outgoing stream. The filter graph is pretty flexible, but the logic would have to be rewritten for stream construction and graph construction. This also needs some additional way of starting the stream in application A.

Combining application A + conference host would get rid of a need to communicate the start of the video feed.

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

I think ApplicationA can connect to my CustomCamera via RTP but I do not know how to implement RTP

  1. server in my CustomCamera and
  2. RTP client in ApplicationA

Do you have any example for RTP client/server?
Thanks

from kvazzup.

jrsnen avatar jrsnen commented on August 21, 2024

Kvazzup uses https://github.com/ultravideo/uvgRTP/ There are good examples there on how to use it.

Another option would be just to change the order of filters in filter graph since I don't think there is any rule stating that you cannot replace your camera with RTP receiver, which would then forward the stream. This would require however having it as an alternative to current implementation which would require some planning.

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

Here is my code CustomCamera with uvgRTP for your review. Thanks

#include <QMutex>
#include <QThread>
#include <QImage>
#include <QWaitCondition>
#include <QPixmap>

#include <string>

using namespace std;

class CustomCamera : public QThread
{    
  Q_OBJECT
private:
  bool stop;
  QMutex mutex;
  QWaitCondition condition;

  uint16_t frameRate;
    
signals:
 //Signal to output frame to be displayed
  void processedImage(const QImage &image);

protected:
  void run();
  void msleep(int ms);
public:
  //Constructor
  CustomCamera(QObject *parent = 0);
  //Destructor
  ~CustomCamera();
  void setFrameRate(uint16_t fps){frameRate = fps;};
  bool isStopped() const;
  void Start();
  void Stop();
};

#include <QTextStream>

CustomCamera::CustomCamera(QObject *parent)
 : QThread(parent)
{
  stop = true;
  // initialize RTP-receiver
}

void CustomCamera::Start()
{
  if (!isRunning())
  {
    if (isStopped())  stop = false;
    start(QThread::Priority::LowPriority);
  }
}

void CustomCamera::run()
{
  int delay = (1000/frameRate);
  while(!stop)
  {
    // RTP-receiver is waiting for new image from video streaming
    // 1. receive new image and copy it into QImage img;
    const QString url = R"(faceImage.bmp)";
    QImage img(url);
    emit processedImage(img);
    this->msleep(delay);
  }
}

CustomCamera::~CustomCamera()
{
  mutex.lock();
  if (!stop)
  {
    exit();  // stop thread
    stop = true;
  }
  condition.wakeOne();
  mutex.unlock();
  wait();
}

void CustomCamera::Stop()
{
  if (!stop)
  {
    exit();  // stop thread
    stop = true;
  }
}

void CustomCamera::msleep(int ms){
  struct timespec ts = { ms / 1000, (ms % 1000) * 1000 * 1000 };
  nanosleep(&ts, NULL);
}

bool CustomCamera::isStopped() const{
  return this->stop;
}

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

I understand that RTP receiver looks like. What do you think?

uvgrtp::media_stream myRTP("localhost", 12345, 12345);

while (1)
{
  size_t timeout = 1;
  uvgrtp::frame::rtp_frame* frame = myRTP.pull_frame(timeout);
  if (frame_is_valid)
    convert RTP frame into QImgage
  sleep(10);
}

from kvazzup.

jrsnen avatar jrsnen commented on August 21, 2024

These codes has nothing to do with Kvazzup. I don't understand why you have posted it.

I'm here mostly to fix bugs in existing Kvazzup. I can also help you better understand how kvazzup works or where you should make changes if it does not take too much of my time, but I cannot help you with new code, especially if it does not involve Kvazzup at all.

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

I think it relates to Kvazzup, where I want to extend it for handling the input video streaming instead of data from camera.
I have another alternative solution:

  1. create a new class rtpCameraStreamingFilter, as a copy of UvgRTPReceiver() and
  2. modify it so that it connects to video stream source via "localhost" and special port 1234567 AND
  3. its signal is connected to SLOT(handleFrame(QImage)) of Filter()

In principle, rtpCameraStreamingFilter has all functions like CameraFramegrabber() + CameraFilter() for built-in camera.
If you think OK, I will write it and test...

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

But I do not know how to create QFuture<uvg_rtp::media_stream *> stream;
which is needed to initialize UvgRTPReceiver
UvgRTPReceiver(uint32_t sessionID, QString id, StatisticsInterface *stats,
DataType type, QString media, QFuture<uvg_rtp::media_stream *> stream);

I am not sure if it is correct:
std::string addr = "localhost";
int src_port = 1234567;
int dst_port = 1234567;
rtp_format_t fmt = value????
int flags = value???
QFuture<uvg_rtp::media_stream *> stream(addr, src_port, dst_port, fmt, flags);

Can you help me? thanks

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

@polusto I use sender and receiver example of uvgRTP, but it is NOT allowed to run sender and receiver in the same PC using localhost. I got always error by running sender after receiver:

Address already in use
[RTPLIB][ERROR][::bind] Biding to port 8888 failed!
[RTPLIB][ERROR][::log_platform_error] Failed to initialize the underlying socket: Address already in use 98

[RTPLIB][ERROR][::create_stream] Failed to initialize media stream 127.0.0.1:8888/8889
Segmentation fault (core dumped)

Is my argument correct? Thanks

from kvazzup.

Thanh-Binh avatar Thanh-Binh commented on August 21, 2024

@polusto Now, I can implement 2 additional modules for handling with local cameras:

  1. module A using the existing UvgRTPReceiver() in file "uvgrtpreceiver.cpp" , which directly binds in FilterGraph. This module waits for any video stream from local PC, e.g. video stream from Application 1 as mentioned before.
  2. module B using my CustomCamera, which can be realized using any communication protocol, e.g. a very simple uvg_rtp::media_stream.

I prefer to the implementation A due to clear SW architecture.

from kvazzup.

jrsnen avatar jrsnen commented on August 21, 2024

I'm closing this issue for the following reasons:

  1. Issues are primarily meant for solving bugs and other problems in Kvazzup master branch. I don't currently see any direct way this discussion can be thought to help with this.

  2. You are asking questions related to uvgRTP. Those do not belong in Kvazzup repository. Use uvgRTP repository or Stack Overflow.

  3. I do not have time to answer basic programming questions. I can give high level answers to Kvazzup related issues, but not specific and especially not ones that have nothing to do with Kvazzup. Use Stack Overflow.

  4. I nor this repository benefits from these questions being answered. Sure, I do have time to answer one persons questions, but since I try to treat everyone equally, I do not have time to answer if more people start asking these kinds of questions. Use Stack Overflow or something similar.

Any one of these would merit the closing of this issue. If you are looking to do a pull request or something similar, that is a separate discussion, but currently I don't see how this discussion belongs in the issues section of Kvazzup.

from kvazzup.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.