Mirror VR on desktop display

[Tom Goddard]

Would like to mirror the view from a virtual reality headset in the ChimeraX graphics window. There is an option for this in the vr command ("mirror true"), but it can cause stuttering. Currently I use the SteamVR mirror display window. But it would be simpler for the user if the main graphics window updated.

I would expect the "mirror true" option to always cause stuttering because OpenGL syncing to the 60 Hz desktop display causes the 90 Hz headset updates to drop frames. But tests show it does not appear to drop frames for simple scenes (p97 binding site) but does drop frames for complex scenes (ribosome antibiotic). I think if I could set the OpenGL swap interval (e.g. with wglSwapIntervalExt(0)) then OpenGL would not sync to the desktop display and mirroring should minimally slowdown the VR headset. Unfortunately the needed extension does not work in PyOpenGL 3.1.1 due to Python3 bytes versus string issues. This is a bug in PyOpenGL. It does not look like PyOpenGL is well maintained (last change 6 months ago, only one developer). Qt5 also can set the swap interval using QSurfaceFormat.setSwapInterval(). But it does not appear the swap interval can be changed after creating the OpenGL context and the graphics window from the Qt5 interface.

[Tom Goddard]

Tried to fix PyOpenGL 3.1.1. First needed to add as_8_bit() to prefix and version_prefix in OpenGL/raw/WGL/_types.py. This avoids the str/bytes startswith error, but find extensions.hasExtension('WGL_EXT_swap_control') returns false and the methods from

from OpenGL.WGL.EXT import swap_control
swap_control.wglGetSwapIntervalEXT()
swap_control.wglSwapIntervalEXT(0)

both calls report that the function is not available. Using

GL.glGetString(GL.GL_EXTENSIONS)

on vive.cgl.ucsf.edu Windows 10 with Nvidia GTX 1080 does report WGL_EXT_swap_control. So it seems likely that some other PyOpenGL problem exists. Will need more work to see why it does not find the extension.

[Tom Goddard]

Tried calling Qt5 method to set swap interval.

fmt.setSwapInterval(0)

before creating context. Had no effect on Windows 10 / GTX 1080 system (also used command "graphics maxFrameRate 500").

But using the Nvidia control panel to set the swap interval to 0 (overriding application) does give faster than 60 frames per second, get 250 frames per second full screen on desktop display (no VR).

But the ribosome antibiotic scene with mirroring on and swap interval 0 still stutters several times a second. Better than with swap interval 1 which seems to drop ten frames each second but still unacceptable stuttering. Setting the Qt redraw interval timer to 0 instead of 1 msec resolves that stuttering. Should probably make that change. Even without setting the interval timer to 0, the stutter disappears if I undisplay the hand controllers and mirror. So it seems the rendering is fast enough but the redraw timing is causing some stutter.

In testing the antibiotic ribosome even with no mirror it tends to stutter for the first 20 seconds of use -- maybe not all the vertex buffers are over on the graphics card until then. So in testing it is necessary to try for 20-30 seconds to see if stuttering goes away.

[Tom Goddard]

Fixed.

vr command now turns of synchronization to vertical refresh and mirrors headset view on desktop.