VR laser pointer for picking

[Tom Goddard]

Wilian notes in the message below that VR bond rotation is difficult for new users. I also have observed this.

Certainly some visual feedback would help. It is probably too computationally demanding to constantly see what a laser pointer hits, and also visually it is not wanted much of the time, and we have no way of knowing when the user wants it.

Maybe a laser pointer that appears when the press the bond rotation controller button and the laser either strikes the bond indicating success or goes off into the distance indicating a miss and giving the user a clue about how they missed. The laser pointer should probably disappear in 0.5 seconds instead of flying around as they hold the button and rotate after a successful pick. I could see the user after a miss holding the button and moving to make the laser hit the bond, but then being surprised that the laser passes right through the bond. Not sure what to do about that misleading visual feedback.

Ideally this laser would apply to all picking actions, not just bond rotation, for example swapaa, or pressing a button on the user interface.

Another bit of visual feedback might be to select the bond when it is picked so the user knows it is hit and does not release the button before rotating.

I could also make two distinct sounds, one for a successful pick and one for a miss.

Eric suggests possibly using a different color laser beam for a hit versus a miss.

Begin forwarded message:

From: "Cortopassi, Wilian"
Subject: Feedback and some questions
Date: February 12, 2020 at 3:46:39 PM PST
To: Tom Goddard

Hi Tom,

Initial demo sessions here in Emeryville have been very nice – we have been working with both Vive and Oculus in the same session without major issues. People seemed to really enjoy it and find it useful for their projects. Being able to see the density files together with the 3D models is very impactful. We have also been able to do cross-site communications without firewall restrictions, which was interesting. Thanks for your full support, it is highly appreciated. It has been a very nice experience using VR with ChimeraX.

As for questions/feedback, do you believe it would be better to send them through another channel or may I keep asking questions directly to you? Either works for me.

...

(vii) The bond rotation function is usually quite difficult to execute;

...

These were the notes I took during the sessions, and I thought it would be useful to share them with you. I have more sessions next week, so I may have more notes to share as more people give it a try. Overall, very positive experience!

Have a nice week,

Wilian

[Tom Goddard]

Discussed with Phil Cruz and Meghan McCarthy having a VR laser pointer at NIAID biweekly meeting. Main trouble is being able to detect what the laser hits so it does not pass right through an atom. That has to be done fast enough (e.g. 1 millisecond) that it does not cause the VR rendering to stutter. Current code for picking is likely too slow for this (should test that). Putting it in a separate thread has the problem that Python can't run multiple threads due to global interpreter lock. Would take significant work to make the pick all done in C++ in a separate thread, although that may be feasible. Could pass C++ numpy arrays (triangles or spheres or cylinders) and be sure they won't go away as long as reference count incremented.

[pett]

Maybe a button press to initiate a pick instead of "always on" would be better. That way if there's some stutter the user has some ability to anticipate it and may not be so badly affected.

[goddard@…]

Definitely pressing a button fixes most of the problem, but also loses most of the benefit.  The advantage of the laser pointer is that you can see what your are pointing at before you press a button that selects, or tugs, or mutates a residue, or places a marker....

The trouble is a laser beam should end on what it hits to clearly show what a button press would select.  But constantly looking for what is hit needs a separate thread which runs into many difficulties when data can be closed at any time.  But maybe if the laser beam just passes through ignoring what it hits it would still be useful.  That would not have performance problems and might be worth a try.