Tutorial
Get frame data
1
2
3
| guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
/// Handle an error. We failed to get image buffer.
} // CVImageBufferRef
|
Convert frame to a Metal texture
This part is slightly more tricky, since we need to know which pixel format we are using. Remember last time I’ve mentioned you have two options: either use RGB or YCbCr? Well, now our steps would be slightly different for each of those two formats, since in case of RGB iOS will do a bit more work and provide you with a joint single RGB texture in the sample buffer, whereas for YCbCr, being a hardware native format, it’s not doing any extra effort and provides you the camera data as is, which is actually two textures: one would hold Y pixel component, and the other one the CbCr component.
1
2
3
4
5
6
7
8
9
| /// Texture cache we will use for converting frame images to textures
var textureCache: CVMetalTextureCache?
/// `MTLDevice` we need to initialize texture cache
var metalDevice = MTLCreateSystemDefaultDevice()
guard let device = metalDevice, CVMetalTextureCacheCreate(nil, nil, device, nil, &textureCache) == kCVReturnSuccess else {
return
}
|
Get frame’s dimensions
1
2
| let width = CVPixelBufferGetWidth(imageBuffer)
let height = CVPixelBufferGetHeight(imageBuffer)
|
Get an unmanaged reference to a CVMetalTexture
1
2
3
| var imageTexture: CVMetalTexture?
let result = CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, textureCache.takeUnretainedValue(), imageBuffer, nil, pixelFormat, width, height, planeIndex, &imageTexture)
|
Grab the actual texture from the CVMetalTexture
1
2
3
4
5
6
7
| guard
let unwrappedImageTexture = imageTexture,
let texture = CVMetalTextureGetTexture(unwrappedImageTexture),
result == kCVReturnSuccess
else {
throw MetalCameraSessionError.failedToCreateTextureFromImage
}
|
