Using the CVPixelBuffer, one can read depth map information and convert it to a 32 bit float32 pixel data set. This gizmo is a bit of a fad as many video game consoles no longer support depth maps, but with a little bit of tweaking, you can get your pixels looking pretty slick.
CVPixelBuffer is not all that expensive to maintain, and you may only need a few per frame of video. In a pinch, you can use a byte buffer to access the 8 bit component of a 32 bit pixel data set, although this will eat into the memory allocator’s bandwidth. As with any other memory allocation, if you want to store more than one image, you may have to invest in multiple pixel buffers. A well crafted video player will ensure you get the most out of each one.
The best way to get a decent picture is to use a video player that supports the Core Video API. This is particularly important if you are using a 3D game console, as it will allow you to display a virtual image with a depth map. Similarly, you can use a video player that supports OpenGL if you need to show an image that has no depth map. You will also want to ensure that your video player is configured to retrieve the most relevant video data for display. If you need to know what the CVPixelBuffer is all about, you can refer to the Core Video Programming Guide.
The Core Video API is best utilized by high-end 3D game consoles and Vision framework-powered applications such as Core ML and Vuforia, but you can also use it to play around with the OpenGL ES API, and even to render your own custom 3D video. If you want to make use of this technology, you should read the Core Video Programming Guide and the Core Video SDK User’s Guide. The latter is a quick and dirty guide to all things Core Video, and it’s an excellent starting point. With this documentation in hand, you should be well on your way to producing impressive 3D visuals. With all of that said, don’t be afraid to ask for help if you have any questions. The community at large is a great resource for information and advice. If you have any specific questions, feel free to leave a comment in the forum section below. We’ll be happy to answer them.
The pixel based CVPixelBuffer is probably the most popular, but you may want to consider using a byte buffer if you want to access the 8 bit component of a pixel data set. This will ensure that you are accessing the most relevant video data for your application, and will save you the drudgery of reading pixel data from memory.