将CMSampleBufferRef转换为cv::Mat

Convert CMSampleBufferRef to cv::Mat

本文关键字：Mat cv CMSampleBufferRef 转换      更新时间：2023-10-16

我正试图将CMSampleBufferRef（作为iOS中AVCaptureVideoDataOutputSampleBufferDelegate的一部分）转换为OpenCV Mat，以尝试半实时稳定输出。

我现在正在运行一个测试应用程序，但在创建和使用Mat.时一直会遇到问题

Swift控制器

let wrapper : OpenCVWrapper = OpenCVWrapper()
...
func captureOutput(captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, fromConnection connection: AVCaptureConnection!) {
    self.wrapper.processBuffer(sampleBuffer, self.previewMat)
}

OpenCVWrapper

- (void)processBuffer:(CMSampleBufferRef)buffer :(UIImageView*)previewMat {
    // Convert current buffer to Mat
    CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(buffer);
    CVPixelBufferLockBaseAddress( pixelBuffer, 0);
    CGFloat bufferWidth = CVPixelBufferGetWidth(pixelBuffer);
    CGFloat bufferHeight = CVPixelBufferGetHeight(pixelBuffer);
    unsigned char *pixel = (unsigned char *)CVPixelBufferGetBaseAddress(pixelBuffer);
    Mat tmp(bufferWidth, bufferHeight, CV_8UC4, pixel);
    Mat cur = tmp.clone();
    dispatch_async(dispatch_get_main_queue(), ^{
        [previewMat setImage:[UIImage imageWithCVMat:cur]];
    });
    CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
}

在Mat cur = tmp.clone()中，我得到了一个EXC_BAD_ACCESS

你觉得我在这里做错了什么吗？

我已经尝试过bufferWidth、CGFloat和int，并在Mat的构造函数中切换它们，同样的问题。

改进的解决方案，修复了"仅前30%"的问题：

- (cv::Mat)matFromBuffer:(CMSampleBufferRef)buffer {
    CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(buffer);
    CVPixelBufferLockBaseAddress( pixelBuffer, 0 );
    //Processing here
    int bufferWidth = (int)CVPixelBufferGetWidth(pixelBuffer);
    int bufferHeight = (int)CVPixelBufferGetHeight(pixelBuffer);
    unsigned char *pixel = (unsigned char *)CVPixelBufferGetBaseAddress(pixelBuffer);
    //put buffer in open cv, no memory copied
    cv::Mat mat = cv::Mat(bufferHeight,bufferWidth,CV_8UC4,pixel,CVPixelBufferGetBytesPerRow(pixelBuffer));
    //End processing
    CVPixelBufferUnlockBaseAddress( pixelBuffer, 0 );
    cv::Mat matGray;
    cvtColor(mat, matGray, CV_BGR2GRAY);
    return matGray;
}

也许这会起作用：

- (void)processBuffer:(CMSampleBufferRef)buffer :(UIImageView*)previewMat {
{
    CVImageBufferRef imgBuf = CMSampleBufferGetImageBuffer(buffer);
    // lock the buffer
    CVPixelBufferLockBaseAddress(imgBuf, 0);
    // get the address to the image data
    void *imgBufAddr = CVPixelBufferGetBaseAddressOfPlane(imgBuf, 0);
    // get image properties
    int w = (int)CVPixelBufferGetWidth(imgBuf);
    int h = (int)CVPixelBufferGetHeight(imgBuf);
    // create the cv mat
    cv::Mat image;
    image.create(h, w, CV_8UC4);
    // memcpy(image.data, imgBufAddr, w * h); copies only 25% of the image
    memcpy(image.data, imgBufAddr, w * h* 4); // copies all pixels
    // unlock again
    CVPixelBufferUnlockBaseAddress(imgBuf, 0);
    dispatch_async(dispatch_get_main_queue(), ^{
        [previewMat setImage:[UIImage imageWithCVMat:image]];
    });
}

图像类型不相等，您需要尝试进行某种

cvtColor(image, image_copy, CV_BGRA2BGR);

尝试使用其他类型的CV_BGRA2BGR

希望能有所帮助。

相关文章：