ios - migrating opengl to metal

In iOS 8, Apple released its own API for GPU-accelerated 3D graphics: Metal.

Metal is similar to OpenGL ES in that it’s a low-level API for interacting with 3D graphics hardware.

The difference is that Metal is not cross-platform. Instead, it’s designed to be extremely efficient with Apple hardware, offering improved speed and low overhead compared to using OpenGL ES.

Note: Metal apps do not run on the iOS simulator; they require a device with an Apple A7 chip or later. To complete this tutorial, you’ll need an [A7 device](http://en.wikipedia.org/wiki/Apple_A7#Products_that_include_the_Apple_A7) or newer.

Metal vs. SpriteKit, SceneKit or Unity

Metal is a low-level 3D graphics API, similar to OpenGL ES, but with lower overhead meaning better performance. It’s a very thin layer above the GPU, which means that, in doing just about anything, such as rendering a sprite or a 3D model to the screen, it requires you to write all of the code to do this. The trade-off is that you have full power and control.

Conversely, higher-level game frameworks like SpriteKit, SceneKit and Unity are built on top of a lower-level 3D graphics APIs like Metal or OpenGL ES. They provide much of the boilerplate code you normally need to write in a game, such as rendering a sprite or 3D model to the screen.

Metal vs. OpenGL ES

OpenGL ES is designed to be cross platform. That means you can write C++ OpenGL ES code, and, most of the time, with some small modifications, you can run it on other platforms, such as Android.

Apple realized that, although the cross-platform support of OpenGL ES was nice, it was missing something fundamental to how Apple designs its products: the famous Apple integration of the operating system, hardware and software as a complete package.

So Apple took a clean-room approach to see what it would look like if it were to design a graphics API specifically for Apple hardware with the goal of being extremely low overhead and performant, while supporting the latest and greatest features.

The result is Metal, which can provide up to 10✕ the number of draw calls for your app compared to OpenGL ES. This can result in some amazing effects — you may remember from the Zen Garden example in the WWDC 2014 keynote, as an example.

There are seven steps required to set up Metal so that you can begin rendering. You need to create a:

MTLDevice
CAMetalLayer
Vertex Buffer
Vertex Shader
Fragment Shader
Render Pipeline
Command Queue

var device: MTLDevice!
device = MTLCreateSystemDefaultDevice()

var metalLayer: CAMetalLayer!
metalLayer = CAMetalLayer()          // 1
metalLayer.device = device           // 2
metalLayer.pixelFormat = .bgra8Unorm // 3
metalLayer.framebufferOnly = true    // 4
metalLayer.frame = view.layer.frame  // 5
view.layer.addSublayer(metalLayer)   // 6

Create a new CAMetalLayer.
You must specify the MTLDevice the layer should use. You simply set this to the device you obtained earlier.
Set the pixel format to bgra8Unorm, which is a fancy way of saying “8 bytes for Blue, Green, Red and Alpha, in that order — with normalized values between 0 and 1.” This is one of only two possible formats to use for a CAMetalLayer, so normally you’d just leave this as-is.
Apple encourages you to set framebufferOnly to true for performance reasons unless you need to sample from the textures generated for this layer, or if you need to enable compute kernels on the layer drawable texture. Most of the time, you don’t need to do this.
You set the frame of the layer to match the frame of the view.
Finally, you add the layer as a sublayer of the view’s main layer.

Creating a Vertex Buffer

Everything in Metal is a triangle. In this app, you’re just going to draw one triangle, but even complex 3D shapes can be decomposed into a series of triangles.

In Metal, the default coordinate system is the normalized coordinate system, which means that by default you’re looking at a 2x2x1 cube centered at (0, 0, 0.5).

let vertexData: [Float] = [
   0.0,  1.0, 0.0,
  -1.0, -1.0, 0.0,
   1.0, -1.0, 0.0
]

var vertexBuffer: MTLBuffer!

let dataSize = vertexData.count * MemoryLayout.size(ofValue: vertexData[0]) // 1
vertexBuffer = device.makeBuffer(bytes: vertexData, length: dataSize, options: []) // 2

You need to get the size of the vertex data in bytes. You do this by multiplying the size of the first element by the count of elements in the array.
You call makeBuffer(bytes:length:options:) on the MTLDevice to create a new buffer on the GPU, passing in the data from the CPU. You pass an empty array for default configuration.

Creating a Vertex Shader

The vertices that you created in the previous section will become the input to a little program that you’ll write called a vertex shader.

A vertex shader is simply a tiny program that runs on the GPU, written in a C++-like language called the Metal Shading Language.

vertex float4 basic_vertex(                           // 1
  const device packed_float3* vertex_array [[ buffer(0) ]], // 2
  unsigned int vid [[ vertex_id ]]) {                 // 3
  return float4(vertex_array[vid], 1.0);              // 4
}

All vertex shaders must begin with the keyword vertex. The function must return (at least) the final position of the vertex. You do this here by indicating float4 (a vector of four floats). You then give the name of the vertex shader; you’ll look up the shader later using this name.
The first parameter is a pointer to an array of packed_float3 (a packed vector of three floats) – i.e., the position of each vertex. Use the [[ ... ]] syntax to declare attributes, which you can use to specify additional information such as resource locations, shader inputs and built-in variables. Here, you mark this parameter with [[ buffer(0) ]] to indicate that the first buffer of data that you send to your vertex shader from your Metal code will populate this parameter.
The vertex shader also takes a special parameter with the vertex_id attribute, which means that the Metal will fill it in with the index of this particular vertex inside the vertex array.
Here, you look up the position inside the vertex array based on the vertex id and return that. You also convert the vector to a float4, where the final value is 1.0 — long story short, this is required for 3D math.

Creating a Fragment Shader

After the vertex shader completes, Metal calls another shader for each fragment (think pixel) on the screen: the fragment shader.

The fragment shader gets its input values by interpolating the output values from the vertex shader. For example, consider the fragment between the bottom two vertices of the triangle:

The input value for this fragment will be a 50/50 blend of the output value of the bottom two vertices.

The job of a fragment shader is to return the final color for each fragment.

fragment half4 basic_fragment() { // 1
  return half4(1.0);              // 2
}

All fragment shaders must begin with the keyword fragment. The function must return (at least) the final color of the fragment. You do so here by indicating half4 (a four-component color value RGBA). Note that half4 is more memory efficient than float4 because you’re writing to less GPU memory.
Here, you return (1, 1, 1, 1) for the color, which is white.

Creating a Render Pipeline

Now that you’ve created a vertex and fragment shader, you need to combine them — along with some other configuration data — into a special object called the render pipeline.

One of the cool things about Metal is that the shaders are precompiled, and the render pipeline configuration is compiled after you first set it up. This makes everything extremely efficient.

var pipelineState: MTLRenderPipelineState!

// 1
let defaultLibrary = device.makeDefaultLibrary()!
let fragmentProgram = defaultLibrary.makeFunction(name: "basic_fragment")
let vertexProgram = defaultLibrary.makeFunction(name: "basic_vertex")
    
// 2
let pipelineStateDescriptor = MTLRenderPipelineDescriptor()
pipelineStateDescriptor.vertexFunction = vertexProgram
pipelineStateDescriptor.fragmentFunction = fragmentProgram
pipelineStateDescriptor.colorAttachments[0].pixelFormat = .bgra8Unorm
    
// 3
pipelineState = try! device.makeRenderPipelineState(descriptor: pipelineStateDescriptor)

You can access any of the precompiled shaders included in your project through the MTLLibrary object you get by calling device.makeDefaultLibrary()!. Then, you can look up each shader by name.
You set up your render pipeline configuration here. It contains the shaders that you want to use, as well as the pixel format for the color attachment — i.e., the output buffer that you’re rendering to, which is the CAMetalLayer itself.
Finally, you compile the pipeline configuration into a pipeline state that is efficient to use here on out.

Creating a Command Queue

The final one-time-setup step that you need to do is to create an MTLCommandQueue.

Think of this as an ordered list of commands that you tell the GPU to execute, one at a time.

var commandQueue: MTLCommandQueue!

commandQueue = device.makeCommandQueue()

Rendering the Triangle

This is done in five steps:

Create a Display Link
Create a Render Pass Descriptor
Create a Command Buffer
Create a Render Command Encoder
Commit your Command Buffer

Note: In theory, this app doesn’t actually need to render things once per frame, because the triangle doesn’t move after it’s drawn. However, most apps do have moving pieces, so you’ll do things this way to learn the process. This also gives a nice starting point for future tutorials.

Creating a Display Link

You need a way to redraw the screen every time the device screen refreshes.

CADisplayLink is a timer synchronized to the displays refresh rate. The perfect tool for the job! To use it, add a new property to the class:

var timer: CADisplayLink!
timer = CADisplayLink(target: self, selector: #selector(gameloop))
timer.add(to: RunLoop.main, forMode: .default)

Creating a Render Pass Descriptor

The next step is to create an MTLRenderPassDescriptor, which is an object that configures which texture is being rendered to, what the clear color is and a bit of other configuration.

guard let drawable = metalLayer?.nextDrawable() else { return }
let renderPassDescriptor = MTLRenderPassDescriptor()
renderPassDescriptor.colorAttachments[0].texture = drawable.texture
renderPassDescriptor.colorAttachments[0].loadAction = .clear
renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColor(
  red: 0.0, 
  green: 104.0/255.0, 
  blue: 55.0/255.0, 
  alpha: 1.0)

First, you call nextDrawable() on the Metal layer you created earlier, which returns the texture in which you need to draw in order for something to appear on the screen.

Next, you configure the render pass descriptor to use that texture. You set the load action to Clear, which means “set the texture to the clear color before doing any drawing,” and you set the clear color to the green color used on the site.

Creating a Command Buffer

The next step is to create a command buffer. Think of this as the list of render commands that you wish to execute for this frame. The cool thing is that nothing actually happens until you commit the command buffer, giving you fine-grained control over when things occur.

let commandBuffer = commandQueue.makeCommandBuffer()!

Creating a Render Command Encoder

To create a render command, you use a helper object called a render command encoder.

let renderEncoder = commandBuffer
  .makeRenderCommandEncoder(descriptor: renderPassDescriptor)!
renderEncoder.setRenderPipelineState(pipelineState)
renderEncoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)
renderEncoder
  .drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3, instanceCount: 1)
renderEncoder.endEncoding()

Here, you create a command encoder and specify the pipeline and vertex buffer that you created earlier.

The most important part is the call to drawPrimitives(type:vertexStart:vertexCount:instanceCount:). Here, you’re telling the GPU to draw a set of triangles, based on the vertex buffer. To keep things simple, you are only drawing one. The method arguments tell Metal that each triangle consists of three vertices, starting at index 0 inside the vertex buffer, and there is one triangle total.

When you’re done, you simply call endEncoding().

Committing Your Command Buffer

commandBuffer.present(drawable)
commandBuffer.commit()

The first line is needed to make sure that the GPU presents the new texture as soon as the drawing completes. Then you commit the transaction to send the task to the GPU.