How to get CVPixelBuffer handle from UnsafeMutablePointer<UInt8> in Swift? - swift

Question

I got a decoded AVFrame whose format shows 160/Videotoolbox_vld. After googled some articles(here) and viewed the FFmpeg source code(here, and here), the CVBuffer handle should be at AVFrame.data[3]. But the CVBuffer I got seems invalid, any CVPixelBufferGetXXX() function returns 0 or nil.
If I used the av_hwframe_transfer_data() like the ffmpeg's example/hw_decode.c did, the sample can be downloaded from HW to SW buffer. Its AVFrame.format will be nv12. After converted via sws_scale to bgra, the sample can be showed on view with correct content.
I think the VideoToolbox decoded frame is OK. The way I convert AVFrame.data[3] to CVBuffer may be wrong. Just learned accessing c pointer in swift but I am not sure how to read a resource handle(CVBuffer) in a pointer correctly.
The following is how I try to extract CVBuffer from AVFrame
var pFrameOpt: UnsafeMutablePointer<AVFrame>? = av_frame_alloc()
avcodec_receive_frame(..., pFrameOpt)
let data3: UnsafeMutablePointer<UInt8>? = pFrameOpt?.pointee.data.3
data3?.withMemoryRebound(to: CVBuffer.self, capacity: 1) { pCvBuf in
let fW = pFrameOpt!.pointee.width // print 3840
let fH = pFrameOpt!.pointee.height // print 2160
let fFmt = pFrameOpt!.pointee.format // print 160
let cvBuf: CVBuffer = pCvBuf.pointee
let a1 = CVPixelBufferGetDataSize(cvBuf) // print 0
let a2 = CVPixelBufferGetPixelFormatType(cvBuf) // print 0
let a3 = CVPixelBufferGetWidth(cvBuf) // print 0
let a4 = CVPixelBufferGetHeight(cvBuf) // print 0
let a5 = CVPixelBufferGetBytesPerRow(cvBuf) // print 0
let a6 = CVPixelBufferGetBytesPerRowOfPlane(cvBuf, 0) // print 0
let a7 = CVPixelBufferGetWidthOfPlane(cvBuf, 0) // print 0
let a8 = CVPixelBufferGetHeightOfPlane(cvBuf, 0) // print 0
let a9 = CVPixelBufferGetPlaneCount(cvBuf) // print 0
let a10 = CVPixelBufferIsPlanar(cvBuf) // print false
let a11 = CVPixelBufferGetIOSurface(cvBuf) // print nil
let a12 = CVPixelBufferGetBaseAddress(cvBuf) // print nil
let a13 = CVPixelBufferGetBaseAddressOfPlane(cvBuf, 0) // print nil
let b1 = CVImageBufferGetCleanRect(cvBuf) // print 0, 0, 0, 0
let b2 = CVImageBufferGetColorSpace(cvBuf) // print nil
let b3 = CVImageBufferGetDisplaySize(cvBuf) // print 0, 0, 0, 0
let b4 = CVImageBufferGetEncodedSize(cvBuf) // print 0, 0, 0, 0
let b5 = CVImageBufferIsFlipped(cvBuf) // print false
// bad exec
var cvTextureOut: CVMetalTexture?
CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, ..., cvBuf, nil, .bgra8Unorm, 3840, 2160, 0, ...)
}

Answer 1

CVBuffer is not a fixed size, so rebinding the memory won't work in this way. You need to do this:
Unmanaged<CVBuffer>.fromOpaque(data!).takeRetainedValue()
However, the bottom line is FFmpeg's VideoToolbox backend is not creating a CVPixelBuffer with kCVPixelBufferMetalCompatibilityKey set to true. You won't be able to call CVMetalTextureCacheCreateTextureFromImage(...) successfully in any case.
You could consider using a CVPixelBufferPool with appropriate settings (including kCVPixelBufferMetalCompatibilityKey set to true) and then using VTPixelTransferSession to quickly copy FFmpeg's pixel buffer to your own.

Answer 2

It seems like I wrongly cast void* to CVPixelBuffer* instead of casting void* directly to CVPixelBuffer. I cannot find a swift way to do such c style casting from pointer to numeric value. (Using as! CVPixelBuffer causes crash).
So I create a function for void* to CVPixelBufferRef in C code to do such casting job.
// util.h
#include <CoreVideo/CVPixelBuffer.h>
CVPixelBufferRef CastToCVPixelBuffer(void* p);
// util.c
CVPixelBufferRef CastToCVPixelBuffer(void* p)
{
return (CVPixelBufferRef)p;
}
// BridgeHeader.h
#include "util.h"
Then pass the UnsafeMutablePointer<UInt8> in, get CVPixelBuffer handle out.
let pFrameOpt: UnsafeMutablePointer<AVFrame>? = ...
let data3: UnsafeMutablePointer<UInt8>? = pFrameOpt?.pointee.data.3
let cvBuf: CVBuffer = CastToCVPixelBuffer(data3).takeUnretainedValue()
let width = CVPixelBufferGetWidth(cvBuf) // print 3840
let height = CVPixelBufferGetHeight(cvBuf) // print 2160

Answer 3

Try this
let cvBuf: CVBuffer = Array(UnsafeMutableBufferPointer(start: data3, count: 3))
.withUnsafeBufferPointer {
$0.baseAddress!.withMemoryRebound(to: CVBuffer.self, capacity: 1) { $0 }
}.pointee
or maybe even
let cvBuf: CVBuffer = unsafeBitcast(UnsafeMutableBufferPointer(start: data3, count: 3), to: CVBuffer.self)

Answer 4

/**
#function CVPixelBufferGetBaseAddressOfPlane
#abstract Returns the base address of the plane at planeIndex in the PixelBuffer.
#discussion Retrieving the base address for a PixelBuffer requires that the buffer base address be locked
via a successful call to CVPixelBufferLockBaseAddress. On OSX 10.10 and earlier, or iOS 8 and
earlier, calling this function with a non-planar buffer will have undefined behavior.
#param pixelBuffer Target PixelBuffer.
#param planeIndex Identifying the plane.
#result Base address of the plane, or NULL for non-planar CVPixelBufferRefs.
*/
#available(iOS 4.0, *)
public func CVPixelBufferGetBaseAddressOfPlane(_ pixelBuffer: CVPixelBuffer, _ planeIndex: Int) -> UnsafeMutableRawPointer?
maybe you can try use CVPixelBufferLockBaseAddress before use CVPixelBufferGetBaseAddressOfPlane