The Future of Graphics Programming on MacOS

With Apple releasing the new Macbook Air, Macbook Pro 13 and Mac Mini with their custom M1 chips, I want to discuss the future of Graphics Programming and the API’s you can use in the Mac ecosystem.

Apple have just released their first ‘Apple Silicon’ macs which contain their ARM based M1 chips. From the initial benchmarks that have been released, they apparently seem to be little power machines.

What I am interested in as well, is the GPU capability, how developers will view these machines and see if they are viable machines for computer graphics developers.

Ever since macOS Mojave, OpenGL (and it’s compute counterpart) has been deprecated. Apple have stated that they will still work, but that users should not expect support or updates. This is a shame, as OpenGL has, for years, been the API that many programmers use when they learning graphics programming for the first time. It’s cross platform, and there are a ton of resources, books and tutorials available to learn from. Also, the OpenGL support on macOS wasn’t that great anyway, with it only supporting up to core profile 4.1. This means no compute shaders or advanced rendering features such as multidraw indirect.

That is not to say that OpenGL is still getting updates itself. The entity that is responsible for its standardization/specification, the Khronos Group, has also effectively put it into legacy mode and focused it’s efforts on it’s ‘replacement’, Vulkan. As you may know, Vulkan is a very low level API and is designed to allow developers to get the best performance from the hardware, without a lot of the driver overhead. OpenGL, by design and due to its long evolution, has built up a lot of cruft over the years and things have been tacked on, extension by extension, to add addition capability, but at the expense of a muddy API and drastically different performance characteristics across different hardware vendors. This means that the same OpenGL program that runs fine on AMD cards, might run poorly on intel or Nvidia cards, and you would have to be in the know, to write different code paths that use different opengl features to get the best performance depending on what card the program is being run on.

So why is it sad that OpenGL is being left on the sidelines? Is there a future for it?

Apple have even gone their own route by designing their own API, Metal, that is macOS/iOS/iPadOS only. Many were hoping that they would adopt Vulkan as the one true API that we would all use and then we would only have to learn one thing.

Enter the translation layer

Something very interesting happened in the last few years. Various so called ‘translatation layers’ have started to pop up which take code from one API and translate it into code that targets another API. A couple of examples of this are ANGLE, which started life as a library that allowed opengl to run on top of directX. Then MoltenVK and MoltenGL we created which allowed Vulkan and OpenGL ES to run on Metal. What’s amazing, is that The Khronons Group has got behind these efforts and has helped guide these projects under what it calls the ‘portability initiative’. They are actively encouraging the sheparding these projects to effectively allow anyone to be able to write code in their API of choice and have it runs almost anywhere else on any platform. Not all of these are complete and fully working yet, but they a lot of them show promise.

Of particular mention is MoltenVK which started it’s life as a commercial project. Thanks to what I believe is funding from the Khronos group, this has now become open source and is released as part of the vulkan lugarG SDK. This bring macOS and it’s iOS counterparts back into the fold of cross platform API’s. It is running Metal under the hood, and it is not 100% of the performance and capability of Vulkan (I believe at the time I was writing this, Vulkan 1.1 was fully supported), but it is an amazing achievement.

Back to OpenGL

So where does this leave OpenGL? Well in the case of macOS, there is the ANGLE project which has been extended from its original goal of just opengl on directx to having backends from multiple APIs on multiple platforms. This means that a Metal backend is being worked on as well as vulkan backends. This means that OpenGL ES 3.2 might be able to be run on top of MoltenVK which runs of top of vulkan. This is OpenGL ES though, not desktop OpenGL.

I Zink, therefore I am….

Even more recently, there has been some very exciting development over on the Mesa Project side of things. If you don’t know, Mesa is a project that is comprised of multiple components, but mainly, it is an OpenGL interface layer or state tracker and secondly, it has many real open source hardware drivers for running on actual cards which are known as ‘Gallium drivers’. There is even a software renderer (LLVMPipe) which acts like a hardware driver to allow rendering on the CPU. This means that applications can write standard opengl code, then link their application against the mesa opengl library as opposed to the system or hardware vendor supplied one, and their code will run using Mesa.

Mesa has mainly been focused on the Linux side of things for the last few years, but it is technically possible to run the state tracker and software renderer on windows and macOS.

The new development has been the creation of a new ‘driver’ Zink, which is actually a translation layer that uses vulkan as a back end. A developer Mike Blumenkrantz has been working furiously in his spare time to get features and performance of Zink up to OpenGL 4.6 and running at 95% of some native drivers that were tested. This is very encouraging as it seems like there is more to go in terms of performance, even though there is still some work on the compatibility side still to do.

And hot off the press, some patches have recently started to come in which allows the use of MoltekVK as a back-back-end. This will allow OpenGL->Vulkan->MoltenVK->Metal as a valid and workable solution to using OpenGL on macOS. In fact being able to use OpenGL on anywhere there is a Vulkan driver is amazing, and might means that in future, new cards only ship with a Vulkan Driver, and thanks to Zink will be able to run OpenGL applications out of the box without all of the complicated driver issues that has plagued OpenGL for many years.

What will this bring us?

What I am hoping this will provide, is a valid route, for people who want to learn graphics programming can take, to get started and build upon the years of work and educational material that has been created over the last 25 years. It will let developers still be able to write a simple OpenGL 1.1 application with glBegin() and glEnd(). It will then allow someone to experiment with basic pixel and vertex shaders without having to worry about separate spirv compilers or libraries.

I know there are many arguments against OpenGL as an introduction to Graphics Programming, and that Modern Game Engines like Unity and Unreal Engine should be were people start. I do agree with that. I am a huge fan of UE4 myself and do a lot of my creative hobby endeavours within UE4, but going from UE4 straight to vulkan is a massive leap. There needs to be middle ground. I believe OpenGL is that middle ground. Specifically, Modern OpenGL, which encourages the ADZO techniques that have been popular in recent years. This means Direct State Access, MultiDraw indirect, efficient texture usage and manual synchronization for buffer updates.

For more information on these techniques, please see my tutorial series and blog posts on Modern OpenGL.

Who know what the new macs will do for graphics programming. I, for one, am excited about the power efficiency and performance they bring, and am hopeful that the Operating system remains open enough for us to be able to tinker and hack away at fun little programs, all in the name of learning for learning’s sake.

I’m happy to see some healthy discussion on the topic. Thanks for reading.

Dan Dokipen Elliott Jones