Add multitouch trackpad rotation and translation

[olibclarke@…]

Begin forwarded message:

From: Oliver Clarke
Subject: Re: show/hide panels
Date: September 10, 2016 at 7:53:02 AM PDT
To: Eric Pettersen
Cc: Tom Goddard
...
Also, will full trackpad support be added to ChimeraX at some point (or maybe it is possible to set this up now via the command line)? I really like the way I have it set up in Chimera 1 (two-finger swipe to rotate, thee finger swipe to translate, pinch to zoom), would be great to see in ChimeraX.

Cheers
Oli

[Tom Goddard]

Begin forwarded message:

From: Edoardo Gianni
Subject: [chimerax-users] mutitouch mac
Date: September 15, 2017 at 3:39:36 AM PDT
To: chimerax-users@…

Hello,

I was wondering if multitouch was available on chimerax for mac, or if you had plans to implement it in the future.

Thanks and best wishes,
Edoardo

[Tom Goddard]

A few years ago I used the Qt window toolkit to handle multitouch events. Qt 5.2(?) had multitouch disabled on Mac OS because it supposedly interfered with scrolling. I don't think I observed that scrolling problem. But I had to write some C++ code that called a native Mac OS method to get touch events, a pretty ugly hack to get it to work.

The current state in Qt 5.9 seems that multitouch events work with Qt widgets. But we need them to work on the graphics area which is a QWindow (for drawing OpenGL 3D graphics). The following Qt bug report (53874) from June 2016 indicates that getting multitouch events on Mac in a QWindow does not work despite the documentation saying it is supposed to work.

​https://bugreports.qt.io/browse/QTBUG-53874

A comment from the Qt developer Morten Sørvig assigned to this ticket indicates it still will require a C++ hack to get multitouch events. "Touch events need to be enabled on a per-QWindow basis on OS X. Widgets and Quick currently do this via the private function QCocoaNativeInterface::registerTouchWindow(). This means there is currently no way to get touch evens via QWindow::touchEvent() on OS X using public API only."

I think the same hack I used in the past can get by this problem. But I don't thick trackpad handling on Mac is important enough relative to other priorities to work on it right now.

[Tom Goddard]

Added 2-finger and 3-finger Mac trackpad drags to rotate, zoom and translate models. This should behave exactly as in Chimera 1. It is enabled by default and there is a preference setting to disable it.

The implementation uses a hack that may break in future releases of Qt because Qt 5.9 still cannot provide touch events on Mac as described in 2016 bug report

​https://bugreports.qt.io/browse/QTBUG-53874

To work around this I added some Objective-C code to ChimeraX that directly enables touch events on the graphics window native Mac NSView.

[olibclarke@…]

Fantastic!!! Thank you thank you :)

Have a great weekend!

Cheers
Oli

[olibclarke@…]

Hi,

Just tried this out, great to have multitouch support!

Couple of minor issues:

1. Trackpad rotation sensitivity needs to be increased - currently, one swipe across the trackpad gives ~90 degree rotation - I would suggest doubling or tripling the degrees per unit swipe, it currently looks like it is ~1/2 as sensitive as click-and-drag rotation. The same is true for three-finger-swipe translation and pinch-to-zoom, all should be more sensitive.

2. Trackpad pinch-to-zoom is not currently compatible with orthographic projection - it changes clipping instead if camera ortho is set (relatedly, it would be great to have a ~/.chimerax file to set startup preferences, e.g. to allow for automatically setting orthographic projection, aliases etc…).

Thanks again for implementing this!

Cheers
Oli

[goddard@…]

Thanks for trying out the trackpad support.  I think I made the sensitivity identical to Chimera 1 (just copied the code).  I will try doubling the sensitivity.  And I will add an option so you can set and save the sensitivity in preferences.  Also will fix ortho pinch zoom -- pinch is just moving you closer which does not change scale in ortho.  Today is Veteran's day holiday -- will look at it tomorrow.

We do need a start-up file of ChimeraX commands.  Currently we read and import all Python code from ~/chimerax_start so that is a way to do it if you are desperate.

	Tom

[goddard@…]

I see Chimera 1 trackpad sensitivity is twice as high -- I must have missed something when I brought it over to ChimeraX.  Will be easy to fix.

[Tom Goddard]

Ok, I tuned the trackpad sensitivity.

Also I added four-finger drag to zoom (down is zoom in, up is zoom out).

Here are the nitty gritty rules for sensitivity it is using now.

1) 4 centimeter two-finger drag does a 360 rotation.
2) 4 cm three-finger drag moves model across full width of window
3) pinch zoom or 4-finger zoom translates in z 3-times faster than xy motion
4) twist rotation rotates 6-times more than finger rotation

An overall scale factor modulates these sensitivity values and is set in the preferences dialog under trackpad.

[olibclarke@…]

Hi,

These alterations sound great but don’t seem to be in the latest build (the sensitivity appears unaltered)

Probably they just didn’t make it in time for commit and will be in the next daily?

Cheers
Oli

[goddard@…]

When I try the current daily build (UCSF ChimeraX version: 0.8 (2018-11-14)) it has the new trackpad features including the sensitivity setting in Preferences / Trackpad.  You must be running an older build.  Last nights daily went up at 3 AM Pacific time.

	Tom

[olibclarke@…]

No, I was running the new build, and the trackpad preferences are present.

However, I was testing it by loading a previous structure (from the quick launch panel of recently opened files at the start). This structure was first opened using the previous build.

When I do this, the old sensitivity settings persist. When I load a fresh pdb at launch (e.g. by opening directly from the PDB), the new trackpad settings are used. Weird.

Cheers
Oli

[olibclarke@…]

I’d also note that pinch-to-zoom is still not compatible with orthographic projection (`camera ortho`) - would be great if it worked with both perspective and orthographic as it does in original Chimera. Thanks for this, it is a big usability improvement!

Cheers
Oli

[goddard@…]

The trackpad sensitivity has nothing to do with what structure you load.  Even if you mean you opened a session saved with a previous ChimeraX that did not have the trackpad sensitivity, it should have no effect on your current sensitivity setting.  So the problem you describe makes no sense to me.  Could you give a more detailed explanation?  Exactly what types of files are you opening (PDB or session), how do you know the sensitivity is not the expected one (look in Preferences / Trackpad or just by testing 2-finger rotations), have you saved a trackpad sensitivity different form the default 1.0?

[olibclarke@…]

Hi Tom,

It is indeed weird.

I am referring to the autosaved recent structures that are present in the quick-launch panel that loads directly upon opening ChimeraX (see screenshot).

If I load one of those that was initially created in a previous version of ChimeraX (e.g. here 2bbv, 4cmc..build), the trackpad sensitivity is the same as it was originally (very sluggish, <90deg rotation for a full trackpad swipe). This is very evident from a two-finger swipe - it is why I initially thought that the changes you described had not yet been committed.

If I open a new pdb (or an auto-save that was created in the latest daily), the sensitivity is as you describe - a full swipe on the trackpad gives >360deg rotation. This behavior (old structures having sluggish trackpad sensitivity) persists across multiple restarts of ChimeraX.

No, I did not alter the trackpad preferences, and the sensitivity there is listed as 1.0 in both cases.

Cheers
Oli

[olibclarke@…]

Added by email2trac

[Tom Goddard]

I fixed zooming with trackpad in orthographic camera.

[Tom Goddard]

I don't understand the sluggish trackpad sensitivity you observe.

The thumbnail images when ChimeraX starts are not "auto-save" -- it is just a history of your recently opened files. So clicking the thumbnail for 2bbv just runs the command "open 2bbv". And it makes no difference whether you originally opened 2bbv a month ago or in today's build -- it is just saving the command "open 2bbv" in the history, nothing else. This set of thumbnails is called either "file history", or "rapid access".

So could you try opening a structure by clicking a thumbnail from a few days ago that you say gives low trackpad sensitivity. Then restart ChimeraX and try using the "open ..." command to open the same structure -- that command should in fact be sitting in the command line when you restart since it was the last command issued. I expect both to give the same trackpad sensitivity.

[olibclarke@…]

e.g. for a large model and map, compare trackpad rotation with PDB-5TAQ & EMD-8382, alone and together in the same session, and compare with PDB-1BL8

Cheers
Oli

[olibclarke@…]

Ah! You’re right Tom, it isn’t related to the history entries per se, it is related to the specific pdb (still weird).

Try trackpad gestures on a large pdb (e.g. 5taq, 4v6x) and compare with a small pdb (e.g. 1bl8).

The large pdb rotates only ~90deg with a full trackpad swipe, while 1bl8 rotates ~450deg with a swipe of the same length.

Interestingly, if I open a map of a corresponding size, that rotates more or less normally. If I then open a large pdb in the same session (e.g. 5taq), sluggish rotation appears again.

I’m not sure that clears it up at all, but hopefully you can now at least reproduce...?

Cheers
Oli

[Tom Goddard]

Ok, I see. It is actually slow rendering that causes trackpad sensitivity to decrease because trackpad events are thrown away. I've fixed that so now sensitivity will stay the same for fast and slow rendering models.

I also changed the default sensitivity behavior a bit since even with fast rendering small models this change made the trackpad more sensitive. So I changed the 4 centimeters for one full rotation to 6 centimeters for one full rotation, likewise 4 -> 6 for a full width translation.

Oliver, thanks for catching this sensitivity problem.

[olibclarke@…]

Ah that makes sense, thanks for the fix Tom!

Cheers
Oli

[Tom Goddard]

Begin forwarded message:

From: Oliver Clarke
Subject: Re: [ChimeraX] #1454: showPivot performance problems
Date: November 16, 2018 at 11:46:55 AM PST
To: ChimeraX-bugs@…
Cc: goddard

Thanks Tom!! Also one more wrinkle as regards multitouch, before I forget - model rotation still needs a tweak to fix the following behavior:

• If I click on the main frame (where the model is), two finger drag works as expected, rotating the model.

• If I then click on another frame, e.g. the reply log, the next two finger drag operation zooms, rather than rotating.

• It seems like rotation with two-finger-drag only works properly if the main window is “active” somehow, and otherwise it defaults to the old behaviour (two finger drag = scroll wheel = zoom).

Have a great weekend!

Cheers
Oli

[Tom Goddard]

What I observe on macOS 10.13.6 is that 2-finger rotation requires the mouse pointer is over the graphics window. If it is over the Log panel then it does not do rotation (it will scroll the log instead). Clicking on the log panel, and then moving the pointer back to the graphics window and doing a 2-finger drag still does rotation for me.

But there is some weirdness where sometimes the 2-finger drag does zooming. If you can identify a reproducible case of that (give exact steps to reproduce it) then I can look at it.

[olibclarke@…]

Hi Tom,

I think this is somehow a system-dependent thing… I have two macs (an iMac with magic trackpad and a MacBook Pro).

Both are on OS X 10.14.1. The MacBook pro reproducibly shows the behavior I describe, every time. The iMac shows a slightly different behavior - when I click to the reply log and then hover over the main graphics window and perform a two finger drag, nothing happens - the reply log does not scroll, and the model neither rotates nor zooms. Subsequent two finger drag operations rotate the model correctly, once the main graphics window has focus.

Thanks for adding the startup section of preferences by the way, that is very helpful!

Cheers
Oli

[Tom Goddard]

I can now reproduce the bug where two-finger drag does not do rotation after clicking the Log panel and moving the pointer back to the graphics pane. I get 3 different outcomes, sometimes it zooms, sometimes it does nothing, and sometimes it correctly rotates. After lifting my two fingers and trying the two-finger drag again it always correctly does the rotation. When I do multiple tests clicking the Log pane, I usually get the same result each time, but if I restart ChimeraX I sometimes get a different result.

Debugging shows that ChimeraX gets no touch events in the zooming case. In the no motion case it gets a single finger touch event even though 2 fingers are on the trackpad. It does not appear anything ChimeraX is doing causes the touch events to not be received -- it is simply a Qt bug. Since Qt has had Mac multitouch support disabled for many years (see QtBug 53874 for status) it is amazing that it works at all. I had to do a very ugly hack to enable Mac multitouch in Qt. I think the missing multitouch events can only be solved by Qt developers. I don't think they will fix it. Multitouch will likely remain too exotic to handle in a cross-platform toolkit -- only apps written in the native mac window toolkit can reliably use it.

I have made a small improvement that it now no longer will incorrectly zoom. Instead the first two-finger drag after clicking reply log will do nothing or will rotate, but never zoom. I'm afraid this is the best that can be done with current Qt.