Replace OpenGL rendering with Vulkan or other modern 3D graphics library

[Tom Goddard]

Might consider switching from OpenGL to Vulkan graphics.

On macOS we get reports of graphics driver crashes multiple times per week. I just experienced such a crash using ChimeraX only 3 minutes on small data while changing a map threshold. #4669.

Vulkan is much lower level making the application responsible for all memory allocation on CPU and GPU, memory transfers, command synchronization. It would be much much harder to implement our graphics. But it would probably be more reliable since it would use Apple's proprietary Metal graphics through the MoltenVK Vulkan shim layer.

Apple deprecated OpenGL in 2018.

I think Vulkan is way too low level for ChimeraX. It would take possibly years of programming effort to make it work well. Vulkan was intended for game engines like Unity, not higher level end-user applications like ChimeraX. So it seems wrong to use Vulkan. One argument in favor of Vulkan is that scientific data can be very large and ChimeraX must limit graphics memory use to work reliably and OpenGL gives no API to even know how much video memory is available. Also the OpenGL "out of memory" error almost never happens, instead a crash results, same even on Windows where the driver is more stable.

Maybe if an Apple OpenGL layer implemented on top of Metal became available that would improve reliability of ChimeraX on macOS.

[Tristan Croll]

Might be worth a look: ​https://github.com/bkaradzic/bgfx. Abstraction layer over quite a range of lower-level rendering engines... seems to be in quite wide use.

[Tom Goddard]

Our CZI Essential Software grant proposal was funded that included attempting replacing OpenGL, so we have resources to work on this.

[Tom Goddard]

Possibly the Anari rendering library could replace our OpenGL. Check out ​https://www.khronos.org/api/index_2017/anari. It's "not a scene graph" C API that can work with many backends. If it works for us, we would be able to remove our OpenGL code in ChimeraX (unless we wanted to write our own backend). There are VMD example images in the Phoronix article at ​https://www.phoronix.com/scan.php?page=news_item&px=Khronos-ANARI-1.0-Provisional.

[Tom Goddard]

Well, I like the "high level" part of ANARI. What concerns me is the "Unidirectional. Data flows from the app to ANARI". We need picking to work, which means we need information back from the renderer.

--Eric

Our picking in ChimeraX does not use the GPU. It knows the shapes of the objects and computes the intersection of the ray under the mouse using the CPU only with no graphics rendering. In Chimera we used the GPU.

Tom

[Tom Goddard]

ANARI seems worth checking out. I'd be most concerned about the weird shader stuff we do, like selection outlines, highly tuned ambient occlusion shadows, direction dependent transparency, blending multi-channel image data (light microscopy) on the GPU. I bet ANARI backends do many fancy things that we don't do like raytracing that we could take advantage of. But we may lose the opportunity to do our own unusual graphics, including things that have long been standard capabilities in ChimeraX and Chimera. It would be a big tradeoff.

[Tom Goddard]

One of the primary motivations for replacing OpenGL is because of the frequently reported OpenGL crashes in the drivers on macOS (reports ever week). But in the past year no graphics driver crashes have been reported for the Apple M1 CPU/GPU and Apple is replacing all its computer line Intel and discrete GPUs with Apple M1. There have been 43 bug reports on the Apple M1. So maybe the main reason for trying to replace OpenGL is no longer relevant. Another possibility is that it is crashing but the Intel ChimeraX crash reports on Apple M1 often have C++ stack traces that just give "?" for the function names, so it may have crashed in the driver and we would not know. Should survey our bug reports to see how often the C++ stack traces are garbled on Apple M1.

Apple has deprecated OpenGL about 3 or 4 years ago and it is possible they could drop support for it entirely, but it seems unlikely.