Stuck VR hand controller during 3 person meeting

[goddard@…]

The following bug report has been submitted:
Platform:        Darwin-19.6.0-x86_64-i386-64bit
ChimeraX Version: 1.2.dev202009020044 (2020-09-02 00:44:18 UTC)
Description
In a 3 person VR meeting with Phil Cruz and Victor Kramer, Phil's hand controller was stuck moving the models even though he had no buttons held down.  Taking the battery out of the controller did not help since is turned off controller still fought with our working hand controllers when we tried to move the models.  Apparently the ChimeraX VR software decided the button was held down and was not going to give up on that idea.

Log:
UCSF ChimeraX version: 1.2.dev202009020044 (2020-09-02)  
© 2016-2020 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open 2jv7 format mmcif fromDatabase pdb

2jv7 title:  
NMR Solution Structure of Histoplasma capsulatum CBP Homodimer [more info...]  
  
Chain information for 2jv7  
---  
Chain | Description  
1.1/A 1.2/A 1.3/A 1.4/A 1.5/A 1.6/A 1.7/A 1.8/A 1.9/A 1.10/A 1.11/A 1.12/A
1.13/A 1.14/A 1.15/A 1.16/A 1.17/A 1.18/A 1.19/A 1.20/A 1.1/B 1.2/B 1.3/B
1.4/B 1.5/B 1.6/B 1.7/B 1.8/B 1.9/B 1.10/B 1.11/B 1.12/B 1.13/B 1.14/B 1.15/B
1.16/B 1.17/B 1.18/B 1.19/B 1.20/B | Calcium-binding protein  
  

> select #1.7/B:17

24 atoms, 23 bonds, 1 residue, 1 model selected  

> ui mousemode right "translate selected models"

> ui mousemode right "tape measure"

> marker segment #2 position 2.37,1.404,5.824 toPosition 6.46,-8.087,1.563
> color yellow radius 0.1 label 11.18 labelHeight 1.118 labelColor yellow

> marker segment #2 position -1.211,7.912,6.795 toPosition 3.956,8.564,5.777
> color yellow radius 0.1 label 5.307 labelHeight 0.5307 labelColor yellow




OpenGL version: 4.1 ATI-3.10.16
OpenGL renderer: AMD Radeon Pro Vega 20 OpenGL Engine
OpenGL vendor: ATI Technologies Inc.Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro15,3
      Processor Name: 8-Core Intel Core i9
      Processor Speed: 2.4 GHz
      Number of Processors: 1
      Total Number of Cores: 8
      L2 Cache (per Core): 256 KB
      L3 Cache: 16 MB
      Hyper-Threading Technology: Enabled
      Memory: 32 GB
      Boot ROM Version: 1037.147.4.0.0 (iBridge: 17.16.16610.0.0,0)

Software:

    System Software Overview:

      System Version: macOS 10.15.6 (19G2021)
      Kernel Version: Darwin 19.6.0
      Time since boot: 21 days 21:11

Graphics/Displays:

    Intel UHD Graphics 630:

      Chipset Model: Intel UHD Graphics 630
      Type: GPU
      Bus: Built-In
      VRAM (Dynamic, Max): 1536 MB
      Vendor: Intel
      Device ID: 0x3e9b
      Revision ID: 0x0002
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1

    Radeon Pro Vega 20:

      Chipset Model: Radeon Pro Vega 20
      Type: GPU
      Bus: PCIe
      PCIe Lane Width: x8
      VRAM (Total): 4 GB
      Vendor: AMD (0x1002)
      Device ID: 0x69af
      Revision ID: 0x00c0
      ROM Revision: 113-D2060I-087
      VBIOS Version: 113-D20601MA0T-016
      Option ROM Version: 113-D20601MA0T-016
      EFI Driver Version: 01.01.087
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1
      Displays:
        Color LCD:
          Display Type: Built-In Retina LCD
          Resolution: 2880 x 1800 Retina
          Framebuffer Depth: 24-Bit Color (ARGB8888)
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: No
          Connection Type: Internal

PyQt version: 5.12.3
Compiled Qt version: 5.12.4
Runtime Qt version: 5.12.9
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    appnope: 0.1.0
    Babel: 2.8.0
    backcall: 0.2.0
    blockdiag: 2.0.1
    certifi: 2020.6.20
    chardet: 3.0.4
    ChimeraX-AddH: 2.1
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.1
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.8.2
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AxesPlanes: 2.0
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 1.0.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.0
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.0
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.0
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-CommandLine: 1.1.2
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.2.dev202009020044
    ChimeraX-CoreFormats: 1.0
    ChimeraX-Coulombic: 1.0.1
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-DataFormats: 1.0
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1
    ChimeraX-DistUI: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.0
    ChimeraX-Hbonds: 2.0
    ChimeraX-Help: 1.0
    ChimeraX-HKCage: 1.0
    ChimeraX-IHM: 1.0
    ChimeraX-ImageFormats: 1.0
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0
    ChimeraX-Label: 1.0
    ChimeraX-ListInfo: 1.0
    ChimeraX-Log: 1.1.1
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Map: 1.0.1
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.0
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 1.1
    ChimeraX-MDcrds: 2.0
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.0
    ChimeraX-mmCIF: 2.2
    ChimeraX-MMTF: 2.0
    ChimeraX-Modeller: 1.0
    ChimeraX-ModelPanel: 1.0
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.0
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0
    ChimeraX-OpenCommand: 1.2.1
    ChimeraX-PDB: 2.1
    ChimeraX-PDBBio: 1.0
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0
    ChimeraX-PubChem: 2.0
    ChimeraX-Read-Pbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.0
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.2
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SeqView: 2.2
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.0
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.0
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.0.4
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.0
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.0
    ChimeraX-ToolshedUtils: 1.0
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.2.3
    ChimeraX-uniprot: 2.0
    ChimeraX-ViewDockX: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.3
    comtypes: 1.1.7
    cxservices: 1.0
    cycler: 0.10.0
    Cython: 0.29.21
    decorator: 4.4.2
    distlib: 0.3.1
    docutils: 0.16
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    h5py: 2.10.0
    html2text: 2020.1.16
    idna: 2.10
    ihm: 0.17
    imagecodecs: 2020.5.30
    imagecodecs-lite: 2020.1.31
    imagesize: 1.2.0
    ipykernel: 5.3.4
    ipython: 7.18.1
    ipython-genutils: 0.2.0
    jedi: 0.17.2
    Jinja2: 2.11.2
    jupyter-client: 6.1.7
    jupyter-core: 4.6.3
    kiwisolver: 1.2.0
    line-profiler: 2.1.2
    lxml: 4.5.2
    MarkupSafe: 1.1.1
    matplotlib: 3.3.2
    msgpack: 1.0.0
    netifaces: 0.10.9
    networkx: 2.5
    numexpr: 2.7.1
    numpy: 1.19.2
    numpydoc: 1.1.0
    openvr: 1.12.501
    packaging: 20.4
    parso: 0.7.1
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 7.2.0
    pip: 20.2.3
    pkginfo: 1.5.0.1
    prompt-toolkit: 3.0.6
    psutil: 5.7.2
    ptyprocess: 0.6.0
    pycollada: 0.7.1
    pydicom: 2.0.0
    Pygments: 2.7.1
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PyQt5-commercial: 5.12.3
    PyQt5-sip: 4.19.19
    PyQtWebEngine-commercial: 5.12.1
    python-dateutil: 2.8.1
    pytz: 2020.1
    pyzmq: 19.0.2
    qtconsole: 4.7.7
    QtPy: 1.9.0
    RandomWords: 0.3.0
    requests: 2.24.0
    scipy: 1.5.2
    setuptools: 50.3.0
    sfftk-rw: 0.6.6.dev0
    six: 1.15.0
    snowballstemmer: 2.0.0
    sortedcontainers: 2.2.2
    Sphinx: 3.2.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 1.0.3
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.4
    suds-jurko: 0.6
    tables: 3.6.1
    tifffile: 2020.9.3
    tinyarray: 1.2.3
    tornado: 6.0.4
    traitlets: 4.3.3
    urllib3: 1.25.10
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.35.1

[Tom Goddard]

Phil's stuck controller was an oculus rift s controller.

I see from the code that if a button up event is lost there is safety code that tests if the button is no longer up each frame and releases it if it is found not up. I was able to test this using a quirk of the code. If while a button is held down its mode assignment is changed, then when the button is released the mode will not be removed from the active drag modes list. Then the safety code is required to unstick the button. I held the grip button in move mode, switched it to zoom mode using the vr button command as I held the button down, then I could still move the model, and when I released the button the move mode ended.

So it seems likely that the button state flags during this bug continued to say the button was down. Other possibilities exist. The code looks not so robust. The safety code relies on the mode._button_down being set correctly. Although an incorrect setting could only preserve the drag mode if it was for a button that was down, and Phil said no buttons were being held down. The safety code relies on getControllerState(device_id) returning success. If the controller were off (battery died) it would probably fail, but then the controller would not move and Phil was moving the model in the stuck state.

I tried turning the Oculus Rift S controller off in my tests and Phil did too. Holding the system button for a long time does not do it, and the Oculus control panel in VR presents no option to turn off a controller, and the Oculus runtime app has no option, and searches online suggest there is no way to turn of a Rift S controller short of taking out the battery or waiting for it to sleep on its own. I took out the battery and the controller simply froze in its last position. If I had the move button held down when I took the battery out then ChimeraX continued to think the button was down and doing move with the live controller caused it to zoom. ChimeraX detects when a controller deactivated event occurs and no such event seems to have happened when the battery was removed. When Phil removed the battery we also still saw his controller (in vr meeting), and it still acted like the move models button was held down.

[Tom Goddard]

If the controller changed its device index and ChimeraX did not know then the button up safety code might fail because the device index is invalid. Also we might get no button up. if the device index changed while the button was held down. But motion was working -- could we get motion if the device index were wrong? It uses device index to get the position. The device index could have switched with that used by the other controller. But then button release safety should work.

If all buttons are up and the safety code is working then all drag modes should be cleared. So the only explanation that makes sense so far is that SteamVR really was reporting the button was always down. There is currently no way to recover from that.

Maybe I should make some mechanism to clear the drag modes. A natural try is press the button again to release it. That may not work because if SteamVR thinks the button was never released then it may not generate a button pressed event. Could have pressing any button clear any other drag modes. Right now it allows two drag modes at the same time. Maybe that is not reasonable. Could easily have starting any drag mode first clear previous ones.

Initially grip and trigger both move scene. So it seems like even if one button were stuck the other would unstick it by taking over the move drag mode. But in fact the code uses two separate move scene modes for the two buttons.

[Tom Goddard]

Although I think it would be better not to allow two drag modes to operate at the same time using one hand controller, clearing all but the most recent drag mode has a problem. Clearing should probably act like button release. But then there would later be a real button release, so the code would see two release calls. Don't want that. Maybe more troubling, suppose a drag is swapping an amino acid and you accidentally press lighting on the grip button to move models. Then the amino acid is chosen but that grip press. Not good. It seems that allowing two simultaneous drag modes is not great but is probably better than last one wins.

[Tom Goddard]

I see no good way to recover from stuck down VR buttons. I could make a recovery command "vr unstick". I guess that would sometimes be better than just quitting although no one would know to use it. Phil quit and rejoined and the problem disappeared. So who knows if we will see this again. Maybe the best course is to wait and see if it happens again.

[Tom Goddard]

If this happens again in a meeting might try a few things. Have person use vr off then vr on and see if button becomes unstuck. If still stuck, try vr off, start a new ChimeraX copy (can leave the old one running), open a model and try vr on and see if the button is stuck in that copy of ChimeraX. If it is quit that ChimeraX and start another SteamVR app and see if button works, if so, go back and try vr on in ChimeraX again.

The point of these tests will be to see if SteamVR is telling all apps the button is down, or whether just the original ChimeraX where it got stuck thinks it is down.

If I observe a stuck button I should directly use the Python shell to test if SteamVR reports the button down.