Home » Code » Converting Between cv::Mat and QImage or QPixmap
      

Converting Between cv::Mat and QImage or QPixmap

posted by Andy Maloney

In a previous article, I outlined how to compile OpenCV for Mac OS X and to build and run a small example. In that post I used OpenCV‘s High-Level GUI (highgui) module to display a simple interface using the Qt framework. This is great if you’re writing a one-off program or are just experimenting with OpenCV, but how do you interface with your own Qt-based project if you don’t want to (or can’t) use the highgui module?

OpenCV Library

OpenCV Library

+

Qt Library

Qt Library

The first hurdle is converting between data formats so you can work with images in both OpenCV and Qt. So how do you move an image’s data between a cv::Mat and a QImage or QPixmap so you can display it in a widget?

In this article, I present some functions to handle these conversions and explain how to use them.

I chose to put all the functions in their own header file and in their own namespace. In the first section there are functions to convert from cv::Mat to QImage and from cv::Mat to QPixmap.

The first thing to note is that the cv::Mat to QPixmap conversion (cvMatToQPixmap) simply relies on cvMatToQImage() and uses Qt’s QPixmap::fromImage() to return a QPixmap, so that’s pretty straightforward.

The second thing to notice is the switch statement in the cv::Mat to QImage conversion (cvMatToQImage). Because both OpenCV and Qt provide multiple data formats for images, we need to do different conversions depending on their internal layout. In this case I am interested in three OpenCV image formats: CV_8UC4 (8-bit unsigned, 4 channels), CV_8UC3 (8-bit unsigned, 3 channels), and CV_8UC1 (8-bit unsigned, 1 channel – grayscale).

The CV_8UC4 conversion to QImage::Format_RGB32 is straightforward – simply construct a QImage and return it. Since QImage uses implicit data sharing, we don’t need to worry about any memory management issues.

The CV_8UC3 conversion to QImage::Format_RGB888 is almost as straightforward. The only difference is that we need to swap the red and blue components of all pixels since OpenCV uses the opposite to Qt.

The final case – CV_8UC1 – adds a small twist. The grayscale QImage needs a color lookup table, so the first time this code is executed, it will create and fill in the static table. No point in doing it every time. We then set the color table on the image and return it.

If we happen to pass a cv::Mat with a format that is not supported here, we give a warning and return a NULL QImage. If you find you need other formats, you can simply extend this switch statement and the one below in QImageToCvMat().

In the second section of code, we handle conversion the other way – from QImage to cv::Mat and from QPixmap to cv::Mat.

As with the conversion the other way, the conversion from QPixmap simply calls our QImage version and uses the built-in QPixmap::toImage() to handle to conversion to QImage.

Aside from the issue of memory management and cloning the data discussed below, the function to convert from QImage to cv::Mat (QImageToCvMat) is pretty straightforward. It supports three QImage formats that are analogous to the OpenCV ones discussed above: QImage::Format_RGB32 (8-bit unsigned, 4 channels), QImage::Format_RGB888 (8-bit unsigned, 3 channels), and QImage::Format_Indexed8 (8-bit unsigned, 1 channel – grayscale).

Unlike conversions from cv::Mat, however, with conversions to cv::Mat we now have to be concerned about memory management. Keep in mind that QImage uses implicit data sharing, so when assigning one QImage to another, it keeps track of how many QImages are using the same data and releases it when nobody is using it. Because our conversions here rely on us pointing at that internal data, and because we have no way of telling the QImage we’re pointing at it, we either have to copy it, or ensure the the lifetime of the new cv::Mat does not exceed that of the original QImage.

If you know (programmatically) that the lifetime of the cv::Mat is shorter than the QImage, then you can be more efficient by passing  false to QPixmapToCvMat or QImageToCvMat for the  inCloneImageData argument. This will share the QImage data. For example you might have a function like this:

In this case you know the QPixmap exists longer than the cv::Mat, so there’s no reason to clone the data.

One exception to this is the QImage::Format_RGB888 format.  In this case, we need to create a temporary QImage to swap the red and blue.  Because it is a local variable, it will go away when the function returns.  This means we can’t just point at its data like we do with the others, so we are forced to clone it.  If this function is called with an image in this format and inCloneImageData is false, we produce a warning and return a clone anyways.

If you are unsure, it is safest to let these functions clone the data.

I hope that’s useful to somebody.  Please feel free to point out any errors or offer a better solution!

8 Comments

  1. Comment by alem:

    This is the best walk-through online! Most of the stuff on the internet regarding QT and OpenCV is messy, thanks! It may be out of the scope of what you’re doing, but could you discuss the image formats OpenCV uses? Im new to image processing and im also learning both QT and OpenCV at the same time. One thing I was wondering is, what does it mean to have 1 or 2 or 3 or 4 channels?? The data representation used by OpenCV is pretty clear, CV_{U|S|F}C(number of channels)…but the last bit of information, the channels, and what they mean to me, the guy whose doing the processing on these images, isnt too clear.

    • Comment by Andy Maloney:

      alem:

      You can think of the channels as the individual components that are combined to create an image. In the cases we’re dealing with here, they are combined with the bit size to define the format of the image – the way the bits are laid out in memory.

      The simplest is 1 channel. This is simply a greyscale (or black & white) image.

      As far as I know, 2 channel does not exist… If it does, it’s not useful for our discussion :-)

      The most common 3 channel format is RGB, so it will have one channel for red, one for blue, and one for green. Combining them will create your colour image.

      Another common 3 channel format is HSV (hue, saturation, value) which stores the colour info differently.

      RGB can be extended to RGBA – where A is alpha – to create a 4 channel image. CMYK (cyan, magenta, yellow, and black) is another 4 channel image format.

      What my switch statements in the code above are doing are mapping the QImage formats to the OpenCV formats and vice-versa.

      With OpenCV you almost certainly need to know the specifics of the format you’re dealing with if you’ll be doing any pixel manipulation. So when you are processing an image, you will need to know whether you’re working with greyscale, RGB, RGBA, HSV, or CMYK.

      On the Qt side of the fence, they don’t talk in channels, they just define the format including its bit layout explicitly, like QImage::Format_RGB888. Almost all the QImage functions are high-level enough that you probably don’t care what the internal format is.

      Some more info can be found on the Wikipedia page (but I find it’s not as clear as it could be).

      Hope that helps and doesn’t further confuse the issue.

      Thank you for taking the time to comment!

  2. Comment by Librehat:

    Thank you very much. This code helps me a lot.

  3. Comment by Przemek:

    Thanks a lot – this was a realy great tutorial!

  4. Comment by Cristi:

    Awesome, thanks!

Leave a Reply

Your email address will not be published. Required fields are marked *