VR glitches on Nvidia 4070 Ti graphics

[phil.cruz@…]

Phil Cruz reports that the BCBB BioViz lab has several new computers with Nvidia 4070 Ti graphics that exhibit glitches in ChimeraX VR.

Begin forwarded message:

From: "Cruz, Phil (NIH/NIAID) [C]"
Subject: Re: [EXTERNAL] ChimeraX VR and Quest Air Link
Date: August 11, 2023 at 2:52:34 PM PDT
To: Tom Goddard, "McCarthy, Meghan (NIH/NIAID) [C]"

Hi Tom,

Thanks for the tip! We have totally switched over to using link cables for now, but this will be useful when we go back to working with Air Link again after our event next week.

We have encountered another issue, however, but only with our newest set of five computers with 4070 video cards. When running ChimeraX VR there is a “flicker”. The best way to describe it is that it looks like the outline of a city skyline flickering across the model (and/or controllers) every couple of seconds. Like the visual manifestation of some sort of step function. Swapping in a 3000 series card from one of the computers we got late last year fixes the problem, so it is likely some setting that makes the 4070 cards incompatible with ChimeraX VR. We have spent a lot of time troubleshooting, and were wondering if you have some time on Monday to discuss troubleshooting ideas with Victor.

Regarding the self-driving cars, that sounds totally creepy. Be afraid, very afraid?

Thanks,

Phil

[Tom Goddard]

Since ChimeraX VR works fine on a 30x0 series card on the same computer, this sounds like a problem with the Oculus Windows PC app that drives the headset not working well with the 4070 Ti driver. Phil said in a meeting yesterday that posts online indicate some other VR applications have also had problems with the 40x0 series cards.

Here are some ideas to help debug this problems.

1) It sounds like you observed this with Quest headsets (since you mention AirLink). Have you tried the 4070 Ti with a SteamVR headset such as Vive Pro? If that does not glitch then it points to the Oculus software being the problem.

2) You might try turning off mirroring the VR to the ChimeraX desktop graphics window. You use the mirror option, "vr on mirror false" to do this. Unfortunately the mirror false option was not working and you will need a ChimeraX daily build dated August 25 or newer to fix it. Rendering to the desktop display and headset may be a trigger for the bug.

3) Try ChimeraX using OpenXR. I just put my initial OpenXR support into ChimeraX today so it will be in the August 25, 2023 daily build. It uses the "xr" command with all the same options as the "vr" command. The "vr" command still uses SteamVR. I have done only basic testing of OpenXR and it is not ready for regular use, but maybe it will fix the 4070 Ti rendering. You need to tell Windows whether the Oculus app or SteamVR is accepting OpenXR connections. For the Quest headset you want to enable the Oculus app to handle OpenXR connections. You do this in the Oculus app under Settings / General / OpenXR Runtime / Set Oculus as Active. If you want to make SteamVR handle the OpenXR connections (for testing a Vive Pro headset with OpenXR) you set that in the SteamVR app under Settings / OpenXR.

Hopefully the OpenXR API fixes the 4070 glitches you are seeing.

[Tom Goddard]

This post claims that Windows "game mode" caused flicker every second on 4070 Ti and that game mode is turned on for individual apps and can be turned off using Windows game bar.

​https://www.reddit.com/r/OculusQuest2/comments/12wh34q/after_reinstalling_windows_10_with_new_gpu_rtx/jhfsmcv/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

I had similar problems but learned that Windows have a special gaming mode than can interfere with VR. Disabled that, and then everything worked again.

Edit: https://www.reddit.com/r/EliteDangerous/comments/7n4biv/stutteringfreezing_in_vr_on_oculus_fixed_check/

[Tom Goddard]

My initial OpenXR was working with Vive Pro but was broken with Quest due to a desktop mirroring bug #9651 that I just fixed.

I also observed other bugs with the OpenXR rendering on Quest. It flickers badly if mirroring is on with Quest (but not with Vive Pro), so I turned mirroring off by default. And if you take the headset off and put it on again the rendering is broken. These bugs will get fixed over the next few weeks of testing. Tonight's daily build or yesterday's with "xr on mirror off" should be adequate to test if the Nvidia 4070 Ti glitches are resolved by OpenXR.

[victor.starrkramer@…]

Hi Tom,

Thank you for all the suggested tips and troubleshooting steps. Meghan and I spend some time yesterday walking through the suggested and we found that when we turned off the quest pro mirroring to the desktop via \u201cvr on mirror false\u201d the image tearing and jitter immediately stopped. We also did the initial setup with steam using open XR versus Oculus. This seem to solve the problem we were having with image tearing and jittering. It seems you may have come to this conclusion with the quest pro as well.

We will continue to do some more testing, but we are excited to share that your suggested solved our problem!

Thank you,

Victor

[Tom Goddard]

Great! It is strange that mirroring to the desktop causes glitches on 4070 Ti but not on other graphics cards. With the new OpenXR code I saw severe flicker when mirroring on a Quadro P6000 graphics card. The desktop mirror is rendering a 3rd image after the two VR eye images. One problem this can cause is that the desktop display may only render at 60 frames per second and that can slow down the VR rendering. But ChimeraX checks for that and if the desktop display would slow down rendering it does not do the mirroring and issues a warning. I don't think that is the problem. Instead the problem is something more subtle, likely that the Oculus app handling the two eye images and ChimeraX desktop mirror rendering are happening at the same time and are competing for the GPU resources.

I hope to be able to fix this so that mirroring does not interfere by rendering the desktop window earlier before the Oculus app is given the two eye images.

The mirroring is useful when giving demos so other people can see what the person in VR is seeing, although there are other ways to do the mirroring (Oculus casting, or SteamVR display mirror) that won't cause flicker because they are done by the VR library that also handles the 2 eye images.

[Tom Goddard]

From discussions at NIAID meetings in December 2023 and January 2024 we hear from Phil and Meghan that the VR skyline artifact with 4070 Ti on desktop computers and 4070 on laptop computers still is present even when mirroring is turned off. Maybe the earlier email that turning of mirroring fixed it really meant that turning of mirroring reduced its occurrence.

We have an Nvidia 4070 desktop GPU that I put in a Windows 10 machine to try to reproduce this bug. I am using ChimeraX 1.7 and Nvidia 536.23 driver and Vive Pro and SteamVR 1.27.5. I have not seen any visual artifacts using the VR for 10 minutes looking at a very large structure PDB 7rro (1.3 million atoms) together with a small structure 1aio. The 4070 renders these smoothly with low sphere quality at 10 triangles per sphere. Bumping up the quality up (graphics quality atom 50) to 50 triangles per sphere produces the usual flicker from dropping frames but no "skyline" artifacts. At 20 triangles/sphere it drops a small number of frames. I also tried with selection outlines, and soft lighting. In all tests I only saw the usual frame drop when it can't render fast enough.

Next steps are to try Quest Pro since that is what you reported it for. Not sure if you were using link cable or Air Link.

Other likely differences between your 4070 Ti desktop systems and my setup are: 1) I have a 4070, not 4070 Ti, 2) you probably have Windows 11, I have Windows 10, 3) I have SteamVR 1.27.5, you probably have newer SteamVR 2, 4) I have Nvidia driver 536.23, yours is probably a different version, my CPU is an old i7-6700K 4 GHz, yours is probably something much newer.

[Tom Goddard]

I updated SteamVR to the current version 2.2.3 and did the same testing as in the previous comment with 7rro and also EMDB map 12873 and saw no artifacts and smooth rendering with Vive Pro.

Next I'm going to try with Quest Pro.

It could be that Windows 11 is needed to see your artifact. This machine cannot be updated to Windows 11 because of its old CPU. Our other Windows 11 machine has only a 500 Watt power supply which is inadequate for the 4070 which needs 650W minimum.

[Tom Goddard]

What is the wattage rating of the power supply on the desktop PCs where you see this artifact?

I was suprised to see that our Windows 11 PC had only 500W supply when the minimum recommended for the Nvidia 3070 we have in that machine is 650W. This is asking for trouble under load spikes.

[Tom Goddard]

Aha! I was able to see the skyline artifact you describe using Quest 3 headset.

I tried with the Quest 3 connected by Quest Link (USBC cable) and Windows PC Oculus App version 60.0.0.162.352.

I only saw it under extreme conditions where ambient lighting (soft or full) is turned on and I am trying to move one model relative to another with the drag model mouse mode. This causes a massive recomputation of shadows from all directions as long as the motion is in progress. So if I click on a model with the drag model mouse mode and hold the button down, so it is constantly making little motions and recomputing ambient lighting then I see the artifact. I tested with EMDB 12873 and PDB 6eee.

The visual appearance is easiest to describe near the yellow and blue hand cones there is a stair step yellow or blue line about a foot above the cones extending a foot to the left and right and rapidly flickering and changing shape almost like a stair step line graph.

While I hold the button to keep the ambient shadows constantly recomputing the SteamVR frame rate monitor turns from green to orange or red. Orange or red means the VR runtime is doing "reprojection", ie using old frames because it isn't getting enough new frames from ChimeraX because the frame rate has dropped. I suspect this artifact is just some bad behavior of that reprojection when starved for new rendered frames.

I'll test to see if Vive Pro does this.

I don't know when you see it. It is ill-advised to turn on soft or full lighting in VR -- it is automatically turned off when you enter VR. So I suspect you were not doing that. But the same kinds of artifacts are likely to occur whenever the rendering becomes way to slow and the Oculus runtime reprojection tries to reuse the old frames.

[Tom Goddard]

Vive Pro with SteamVR does not exhibit the artifact with PDB 6eee and EMDB 12873 and full lighting and dragging one model relative to the other with drag model mouse mode. The SteamVR frame rate monitor does not indicate reprojection is occuring, although the framerate graph changes from green to a greenish yellow. If I also open the massive PDB 7rro then drag one model the SteamVR frame rate indicator goes to solid red and still no skyline artifacts -- only normal flicker from low framerate. If I hide 7rro and increase EMDB 12873 from step 2 to step 1 then drag I can get yellow or orange or red frame rate SteamVR graphs but I never see the skyline artifact under any of these differing levels of frame drop.

[Tom Goddard]

There are two possible sources of the skyline artifact. The most probable I think is that that the video compression done for the Quest 3 over the USB cable or AirLink introduces these skyline artifacts when the framerate drops. With the Vive Pro there is no video compression since it uses a DisplayPort cable to transmit video. This is the most complained about issue online for PC VR with Quest headsets -- they compress the video stream because they are using USB or Wifi instead of a video transmission format like display port or hdmi. This is just a fundamental problem with Meta Quest PC VR. Meta could have added a display port or hdmi port on the headset but they have no interest in PC VR, so we only have USB and Wifi that are not adequate for very high frame rate very high resolution video without using video compression methods.

The other possibility is that the Quest reprojection method of using previous rendered frames when the VR application rendering slows down is buggy, while the SteamVR reprojection method works. I don't think this is the problem. Reprojection is a basic feature even in standalone Quest VR and I think it is very unlikely that it alone introduces the skyline artifacts.

So why don't you see the skyline artifacts with a 30x0 series graphics card? (I have not tested that but Phil, Meghan and Victor say they are not seen.) The video compression is done on the graphics card. My guess is that the 40x0 series cards offer some different video compression capability than the 30x0 card and this is just an unfortunate problem with the 40x0 GPU video compression, or in how the Quest runtime is using the 40x0 series video compression capabilities. I would guess this is probably a bug in Meta's use of 40x0 graphics hardware video compression, possibly when the graphics is simultaneously being used to do reprojection. The chance of Meta caring about this is near 0 since 1) PC VR is not their market, and 2) if it only happens under degraded rendering conditions (unfortunately common in unconstrained scientific data visualization) Meta is especially uninterested because the main market of video games tightly controls their scene depictions to never get into slow framerate situations.

[Tom Goddard]

The next step is for Phil, Meghan and Victor to tell me if they can produce the artifact when scenes are not so complex that dropped frames are constantly happening. My guess it is always associated with exceeding the rendering capacity of the graphics. If that is right I can at least advise on specific scenes you tell me about, why those scenes are exceeding the graphics capabilities.

[Tom Goddard]

I tried 6eee and 12873 on an Nvidia 3070, Windows 11, ChimeraX 1.7, SteamVR 2.2.3, soft lighting and dragging one model continuously. Saw no skyline artifacts with Vive Pro. Saw no skyline artifacts with Quest 3 and link cable. This agrees with what Phil reported that the skyline artifacts only appear on Nvidia 40x0 graphics, not on Nvidia 30x0 graphics.

[Tom Goddard]

I tried SteamLink v2.0.4.677 with Quest 3 on the Nvidia 3070 and Nvidia 4070 systems using 6eee and 12873, soft lighting, and drag single model. Neither showed skyline artifacts. Both performed very well, with little flicker and map step 2, and tolerable at map step 1.

I tried AirLink with the Nvidia 4070 system. And the skyline artifacts were frequent especially with map at step 1 and especially when releasing and pressing the drag model button, less so when dragging.

So AirLink and QuestLink which both use the Windows PC Oculus App (version 60.0.0.162.352) exhibit the problem, but SteamLink which does not use the Oculus App and handles the wifi connection using Steam does not have the problem.

It appears to me this is a bug in Oculus Windows App support for QuestLink (ie usb cable connection) and AirLink (wifi connection) possibly related to video compression on Nvidia 40x0 series cards.

[Tom Goddard]

Besides the skyline-like line, yellow near the yellow cone, blue near the blue cone, and gray near the gray emdb 12873 map, I also see finger of background color that flicker blocking the views of the models. It looks like the colored lines are on the border of these background color lines. I saw it was the background color by trying white, gray or black backgrounds. It also was clear that the skyline lines and finger were not aligned with the VR displays. Originally I thought they were vertical stair steps as if the tiled video compression failed in some parts. But that is not what it looks like. The fingers can project in any direction. I also was able to see the artifacts with simple lighting.

All of this makes me think the bug is more likely related to the reprojection algorithm going haywire on the Nvidia 40x0 cards when the rendering gets slow.

It is unlikely you will see this in other VR apps because almost every VR app (game) controls the full scene complexity so it never ends up dropping lots of frames and using reprojection heavily. But viewing scientific data of arbitrary size with expensive lighting modes (soft, full) will often cause these bad slow rendering scenarios.

The Oculus Debug Tool allows changing the Quest Link bit rate. I tried 50 Mbit/sec or 5 Mbit/sec, presumably slower than the default which is dynamic and typically around 100 Mbit/sec according to online sources. It was unclear if the change even worked. The Oculus software is has horrible usability so I could not tell if any change I made actually was being used.

Aha! I searched online for how to disable reprojection and learned that the Oculus Debug Tool has a setting "(PC) Asynchronous Spacewarp". The default setting was "Auto". I changed it to "Disabled" and the skyline artifacts vanished. Turning it back to "Auto" brings back the artifacts. All I have to do is start Oculus Debug Tool (C:/Program Files/Oculus/Support/oculus-diagnostics/OculusDebugTool.exe) while running on Quest Link with ChimeraX in VR, then change the dropdown menu setting and it takes effect immediately.

[Tom Goddard]

Here is a summary of the problem and a few solutions.

The artifacts are caused by Oculus PC Asynchronous Spacewarp. This tries to reuse old rendered frames when drawing new VR frames is too slow. For unknown reasons it produces these skyline artifacts on Nvidia 4070 graphics, but not on Nvidia 3070 graphics. Because this spacewarp is a feature of Oculus PC software it does not effect Vive Pro with SteamVR, and also it does not appear to effect the Quest headsets running with SteamLink since that does not use the Oculus App.

Here are some ways to avoid the artifacts:

1) Use SteamLink with Quest headsets (not Air Link, and not Quest Link usb cable). SteamLink worked very well in my tests, more reliably connecting and staying connected than AirLink or QuestLink.

2) If you want to use Quest Link or Air Link, then use Oculus Debug Tool to disable PC Asynchronous Spacewarp. I'm not sure if Oculus Debug Tool can change defaults, it may only change for the current VR session, so very painful. But this post suggests that Oculus Tray Tool can change the default. I have not tried it. I attached an image of the Oculus Debug Tool version 60.0.0.162.352 so you can see what the option looks like.

3) Try to avoid complex scenes and time consuming actions in ChimeraX that cause the framerate to drop. This is kind of a joke. With scientific data and the ChimeraX application that was not designed to do everything in under a few milliseconds there are going to be dropped frames in VR. But almost all commercial VR games control everything to minimize frame drop as it is critical to good VR experience. In ChimeraX that means knowing what operations cause severe frame drop.

4) Just accept the artifacts. At least in all my tests they were only slightly more annoying than the flicker induced by dropped frames. The VR should not be used on data so big that there is constant dropped frames -- that will make a user sick. So the artifacts should only occur in isolated bursts when doing a time consuming action. These artifacts significantly reduce usability and increase discomfort, only very simple uses of ChimeraX can avoid that.

[Tom Goddard]

Oculus Debug Tool showing PC Asynchronous Spacewarp option disabled that fixes skyline artifacts.

[Tom Goddard]

We tried disabling Async spacewarp at the NIH BioViz lab during our visit in February and it indeed fixed the problem. Perhaps using SteamLink is a better fix.

Possibly Oculus will fix this in the future. For now these methods of working around the problem are the best we can do since it is a general problem that occurs whenever rendering is not fast enough to keep up with the 90Hz headset refresh rate. We have no way to keep ChimeraX consistently rendering that fast when so many of its operations (like computing surfaces) take fractions of a second and block rendering.