Home » Code » Converting Between cv::Mat and QImage or QPixmap
      

Converting Between cv::Mat and QImage or QPixmap

posted by Andy Maloney

In a previous article, I outlined how to compile OpenCV for Mac OS X and to build and run a small example. In that post I used OpenCV‘s High-Level GUI (highgui) module to display a simple interface using the Qt framework. This is great if you’re writing a one-off program or are just experimenting with OpenCV, but how do you interface with your own Qt-based project if you don’t want to (or can’t) use the highgui module?

OpenCV Library

OpenCV Library

+

Qt Library

Qt Library

The first hurdle is converting between data formats so you can work with images in both OpenCV and Qt. So how do you move an image’s data between a cv::Mat and a QImage or QPixmap so you can display it in a widget?

In this article, I present some functions to handle these conversions and explain how to use them.

I chose to put all the functions in their own header file and in their own namespace. In the first section there are functions to convert from cv::Mat to QImage and from cv::Mat to QPixmap.

The first thing to note is that the cv::Mat to QPixmap conversion (cvMatToQPixmap) simply relies on cvMatToQImage() and uses Qt’s QPixmap::fromImage() to return a QPixmap, so that’s pretty straightforward.

The second thing to notice is the switch statement in the cv::Mat to QImage conversion (cvMatToQImage). Because both OpenCV and Qt provide multiple data formats for images, we need to do different conversions depending on their internal layout. In this case I am interested in three OpenCV image formats: CV_8UC4 (8-bit unsigned, 4 channels), CV_8UC3 (8-bit unsigned, 3 channels), and CV_8UC1 (8-bit unsigned, 1 channel – grayscale).

The CV_8UC4 conversion to QImage::Format_RGB32 is straightforward – simply construct a QImage and return it. Since QImage uses implicit data sharing, we don’t need to worry about any memory management issues.

The CV_8UC3 conversion to QImage::Format_RGB888 is almost as straightforward. The only difference is that we need to swap the red and blue components of all pixels since OpenCV uses the opposite to Qt.

The final case – CV_8UC1 – adds a small twist. The grayscale QImage needs a color lookup table, so the first time this code is executed, it will create and fill in the static table. No point in doing it every time. We then set the color table on the image and return it.

If we happen to pass a cv::Mat with a format that is not supported here, we give a warning and return a NULL QImage. If you find you need other formats, you can simply extend this switch statement and the one below in QImageToCvMat().

In the second section of code, we handle conversion the other way – from QImage to cv::Mat and from QPixmap to cv::Mat.

As with the conversion the other way, the conversion from QPixmap simply calls our QImage version and uses the built-in QPixmap::toImage() to handle to conversion to QImage.

Aside from the issue of memory management and cloning the data discussed below, the function to convert from QImage to cv::Mat (QImageToCvMat) is pretty straightforward. It supports three QImage formats that are analogous to the OpenCV ones discussed above: QImage::Format_RGB32 (8-bit unsigned, 4 channels), QImage::Format_RGB888 (8-bit unsigned, 3 channels), and QImage::Format_Indexed8 (8-bit unsigned, 1 channel – grayscale).

Unlike conversions from cv::Mat, however, with conversions to cv::Mat we now have to be concerned about memory management. Keep in mind that QImage uses implicit data sharing, so when assigning one QImage to another, it keeps track of how many QImages are using the same data and releases it when nobody is using it. Because our conversions here rely on us pointing at that internal data, and because we have no way of telling the QImage we’re pointing at it, we either have to copy it, or ensure the the lifetime of the new cv::Mat does not exceed that of the original QImage.

If you know (programmatically) that the lifetime of the cv::Mat is shorter than the QImage, then you can be more efficient by passing  false to QPixmapToCvMat or QImageToCvMat for the  inCloneImageData argument. This will share the QImage data. For example you might have a function like this:

In this case you know the QPixmap exists longer than the cv::Mat, so there’s no reason to clone the data.

One exception to this is the QImage::Format_RGB888 format.  In this case, we need to create a temporary QImage to swap the red and blue.  Because it is a local variable, it will go away when the function returns.  This means we can’t just point at its data like we do with the others, so we are forced to clone it.  If this function is called with an image in this format and inCloneImageData is false, we produce a warning and return a clone anyways.

If you are unsure, it is safest to let these functions clone the data.

I hope that’s useful to somebody.  Please feel free to point out any errors or offer a better solution!

18 Comments

  1. Comment by alem:

    This is the best walk-through online! Most of the stuff on the internet regarding QT and OpenCV is messy, thanks! It may be out of the scope of what you’re doing, but could you discuss the image formats OpenCV uses? Im new to image processing and im also learning both QT and OpenCV at the same time. One thing I was wondering is, what does it mean to have 1 or 2 or 3 or 4 channels?? The data representation used by OpenCV is pretty clear, CV_{U|S|F}C(number of channels)…but the last bit of information, the channels, and what they mean to me, the guy whose doing the processing on these images, isnt too clear.

    • Comment by Andy Maloney:

      alem:

      You can think of the channels as the individual components that are combined to create an image. In the cases we’re dealing with here, they are combined with the bit size to define the format of the image – the way the bits are laid out in memory.

      The simplest is 1 channel. This is simply a greyscale (or black & white) image.

      As far as I know, 2 channel does not exist… If it does, it’s not useful for our discussion :-)

      The most common 3 channel format is RGB, so it will have one channel for red, one for blue, and one for green. Combining them will create your colour image.

      Another common 3 channel format is HSV (hue, saturation, value) which stores the colour info differently.

      RGB can be extended to RGBA – where A is alpha – to create a 4 channel image. CMYK (cyan, magenta, yellow, and black) is another 4 channel image format.

      What my switch statements in the code above are doing are mapping the QImage formats to the OpenCV formats and vice-versa.

      With OpenCV you almost certainly need to know the specifics of the format you’re dealing with if you’ll be doing any pixel manipulation. So when you are processing an image, you will need to know whether you’re working with greyscale, RGB, RGBA, HSV, or CMYK.

      On the Qt side of the fence, they don’t talk in channels, they just define the format including its bit layout explicitly, like QImage::Format_RGB888. Almost all the QImage functions are high-level enough that you probably don’t care what the internal format is.

      Some more info can be found on the Wikipedia page (but I find it’s not as clear as it could be).

      Hope that helps and doesn’t further confuse the issue.

      Thank you for taking the time to comment!

  2. Comment by Librehat:

    Thank you very much. This code helps me a lot.

  3. Comment by Przemek:

    Thanks a lot – this was a realy great tutorial!

  4. Comment by Cristi:

    Awesome, thanks!

  5. Comment by Bitman:

    Thanks for your article :) It helped me a lot :)
    To which QFormat would you convert a CV_32FC1 image ? Format_Indexed8?
    And how would you do it, would you first convert CV_32FC1 to CV_8UC1 using opencv function convertTo(), or do something else ?
    Thanks again :)

    • Comment by Andy Maloney:

      You’re welcome! Glad you found it useful.

      Keeping in mind that I’m not an expert on this, I think your proposal seems like a good solution.

      First convert to CV_8UC1 using OpenCV’s convertTo(), then convert to QImage::Format_Indexed8 as I do in ASM::cvMatToQImage() above.

  6. Comment by Jassie:

    Thanks for your article. But I have a problem when I convert Qimage to mat. When I get a image which format is Format_ARGB32, and I want to keep alpha channel.But it always crashes.Is there any problem? Here is my code:

    cv::Mat mat( image.height(), image.width(), CV_8UC4, const_cast(image.bits()), image.bytesPerLine() );

    Could you please help me ?

    • Comment by Andy Maloney:

      I haven’t been working in this area in a very long time, but I would start by looking at two things:

      1) Make sure the lifetime of the cv::Mat is managed properly so you aren’t working on a temporary (see comments towards the end of the post).

      2) Check that a QImage::Format_ARGB32 can actually be converted to CV_8UC4. I suspect this is incorrect since what you have here is the same code as for QImage::Format_RGB32. If this is wrong it shouldn’t crash though – it should give you an incorrect image.

  7. Comment by Bastian:

    Thanks for the code. But I’d like to know why you are converting a CV_8UC4 image (which has four 8-bit channels) to a Format_RGB32 image (which should have three 8bit channels) and the other way around.
    The QImage documentation states that RGB32 images are saved as 0xffRRGGBB, which I read as one byte for blue, one for green and one for red and the first one is always 255.

    If I open for example a png saved with transparency (should have 4 channels) as QImage it has got the format ARGB32, which is not considered in your code. If I open one without transparency it is opened as RGB32 (so only 3 channels, or the alpha channel is white).

    Can you please explain this to me?

    • Comment by Andy Maloney:

      As I mentioned above it’s been a long time since I looked at any of this and I’m not using it in any active projects anymore, but it seems that I’m missing two things:

      1) CV_8UC4 should be converted to QImage::Format_ARGB32 in cvMatToQImage

      2)QImageToCvMat should add another case for QImage::Format_ARGB32 which does the same thing as QImage::Format_RGB32

      This is probably what Jassie was referring to in the comment above (though I don’t know about his crashing problem).

      I’ll have to experiment when I get the chance.

      Thanks for the input!

      • Comment by Bastian:

        Ok, I’ve been trying around a little bit and changing the line to

        QImage(mat.data, mat.cols, mat.rows, mat.step, QImage::Format_ARGB32);

        works fine (alpha channel is correctly converted).

        I added some more cases to the conversion function from QImage to cv::Mat.

        If the QImage is Format_ARGB32 or Format_ARGB32_Premultiplied it can be converted by:

        cv::Mat(image.height(), image.width(), CV_8UC4, const_cast(image.bits()), image.bytesPerLine()).clone();

        If it is Format_RGB32 then the problem is, that the image has got an alpha channel which is completely white. So you can either convert it to a CV_8UC4 image, or the other way would be doing the following:

        QImage converted = image.convertToFormat(QImage::Format_RGB888).rgbSwapped();

        mat = cv::Mat(converted.height(), converted.width(), CV_8UC3, const_cast(converted.bits()), converted.bytesPerLine()).clone();

        Then the resulting image has only got the three colour channels. Unfortunately it has to be converted which might be expensive, I don’t know.

        If you like you can have a look at the complete code here:

        http://goo.gl/UEAQrb

        Just have a look at the two last functions in the file. Note: I’m converting QPixmap to cv::Mat. If that would be possible without first converting to a QImage, that would also be very nice.

        Best regards
        Bastian

  8. Comment by Théo Friberg:

    There seems to be an error on the third #include line, it reads:
    #include “opencv2/imgproc.hpp”
    to my understanding this should be:
    #include “opencv2/imgproc/imgproc.hpp”

Leave a Reply

Your email address will not be published. Required fields are marked *