Add support for 3d labels

[Tom Goddard]

I surveyed 3d label support in Chimera and here are some observations.

1) Chimera has the label, rlabel, and labelopt command to control label display on atoms and residues and what text is shown. Label preferences controls font, and font size and style (bold, italic, ...) and those setting apply to all 3d labels. The color command sets the label colors. Bonds and pseudobonds have labels but have no command specifically for showing them, instead selection inspector or setattr can be used.

2) ChimeraX should have a simpler interface, probably just a single label command like 2dlabel that sets text, font, size, color.

3) Labels are part of the Atom, Residue, Bond, Pseudobond C++ classes in Chimera with specific text, color, and offset attributes.

4) ChimeraX could instead make all label properties belong to a label model that is a child model of the Structure the labels belong to. The Label model would keep track of the atoms, residues and bonds labeled, their fonts and colors. The labels could easily be hidden by undisplaying that model, and redisplayed without losing any settings.

5) Chimera has a "labels on top" preference default true, which means labels are never obscured by any other graphics -- they essentially behave like 2d labels as an overlay on top of the molecular graphics. Maybe only the "on top" true setting is needed -- not sure whether having molecules occlude labels is ever useful (ask Elaine).

6) Labels have an x,y,z offset that is in screen coordinates. For an atom or bond label the default offset depends on the radius (0.2 + atom radius along x). When the atom / bond display style is changed the label position adjusts.

7) The labelopt command allows putting attributes into the label text like "labelopt resinfo %(name)s %(number)s" shows subsequent residue labels like “TYR 151”. And numeric parameters can be handle too "labelopt info B = %(bfactor).2f" shows “B = 43.70”.

8) For residue labels the label can be positioned at the residue centroid, backbone centroid, or backbone principle atom (CA, C5').

9) 3D labels are of course saved in sessions.

10) 3D labels maintain fixed size in pixels as you zoom in and out.

11) Sometimes Chimera users have wanted labels not attached to an atom, just on a point in space. This would be easy to support in ChimeraX, although some id might be needed to have a way to delete such a label. This was not easy in Chimera because every rotation moved the models in the global coordinate system so tying a label to global coordinates would not move it with the models.

12) Chimera has a mouse mode to move 3d label positions, but this capability is more important for 2d labels.

Here are some implementation considerations.

13) The code could treat the 3d labels as an overlay so they are like 2d labels only the move around to stay positioned relative to their atoms as the camera moves. Or it could treat 3d labels as rectangular billboards that are real models at some depth in the scene. If the above idea (4) to make a labels child model of the Structure is used then it can't be an overlay -- overlays are not models and are not part of the hierarchy where model position is inherited. As a 3D model the drawing routine will need to resize the billboard for the text based on distance from the camera and orient the billboard perpendicular to the camera. Another advantage of using a real model is it could be shown in a VR scene, while overlays do not display correctly in a VR scene.

14) If each label is rendered as a separate Drawing under a Label model this could bog down if thousands of labels are shown. Probably real use cases only have tens to a few hundred labels. Need to avoid hang if user accidentally labels every atom in a ribosome. Faster rendering could pack all label text into a single texture and use one Drawing with geometry consisting of all the label billboards. That is considerably more complex and could be done later to optimize performance.

15) Label model would turn of depth test to implement "always on top" mode. If we allowed labels to be occluded, allowing turning off on top mode, then labels might not show through a transparent surface because of one transparent layer, and the billboard showing the label is transparent. Possibly the label could be drawn in the opaque pass, but would need alpha blending enabled.

16) Labels on global pseudobond groups could be a child model of that group. Not sure if this introduces any complexities.

17) Because the label billboards have to constantly change geometry this can lead to recomputing bounding boxes which can be slow -- may want to suppress that.

18) Currently there is no support for models to be exempt from ambient occlusion shadows and direct shadows. Labels need to not cast ambient shadows since their geometry changes every frame and this will kill interactive performance.

[Tom Goddard]

A usable implementation of some of the 3d label features will take about 3 full days of effort. To get all the features in that Chimera supports might take another 5 days or more.

We milestoned this for alpha2 before we ran into funding difficulties. It may make more sense now to work on novel features since those are more likely to contribute to new funding. I'm getting very little time in on novel features because of all the porting of basic features so I think a reevaluation of this as an alpha2 goal is needed.

[Tom Goddard]

I added basic 3d label support using a new "label" command that is similar to the 2dlabel command. Labels appear in a child model of the structure or pseudobond group of the items labeled. Session saving is supported (although there is a bug with session restore that effects pseudobond labels). Also added 2d label session saving.

Would like to add a "label fonts" command to list available font names. To do this I want to switch to rendering the text using Qt QPainter instead of the current use of the Python Image Library. This is the subject of another ticket.

[Tom Goddard]

Small labels look very poor quality with fuzzy edges and dim. Probably I need to use mipmap textures (multiple level of detail) to get the best appearance. Otherwise small mismatches of the texture grid and screen grid create ugly edges.

[Elaine Meng]

Command structure/usability issues:

(1) probably should take blank atomspec, e.g. "label residues" and "label delete"

(2) command structure causes some problems, mainly that if you are displaying specific labels, it is inconvenient to change just their color, size, etc. without accidentally displaying ALL labels. Also, one might want to change size (etc.) of all labels but keep the hidden ones hidden and the shown ones shown. Need some separation of show/hide from other settings. Some ideas:

(A) top-level "label" just does show/hide/delete (maybe offset) and
omnibus subcommand "label style" controls other settings:
color,size,font,typeface,onTop (maybe offset).
Analogous to "cartoon" and "cartoon style" or for atoms, "show" and "style".
-OR-
(B) leave all the settings in top-level "label" but make subcommands
"label show/hide/delete" to control display. Somewhat analogous to "surface".
-OR-
(C) command "label current" that only affects the currently shown labels.
(meh… doesn’t seem like a very good solution)

(3) tied in with the previous point, consider whether certain settings should apply to subsequently created labels. Say I want all my labels to be size 40; it's tedious to specify that in every subsequent label-creation command, or if you created them already, to specify only the existing labels so as not to display more labels.

For 3-D labels, the concept of their not existing until you create them is unfamiliar. Instead one is likely to think of the set of atom (residue, pseudobond) labels as partly shown and partly hidden and would often want to act on all of them collectively, which circles back to the previous need to separate parameters from display. This mental picture is reinforced by the label automatically being hidden when its object (atom, etc.) is hidden, and reshown when the object is reshown.

Of course you want to allow labels of different sizes to coexist. The question is whether “label” (or “label style”) sets the subsequent default, or whether that will only be done separately with Preferences.

(4) unlike color/size, "onTop" always applies to all 3D labels (is global). I'm OK with that, just noting it as something that has to be explained in documentation if the command structure does not differentiate.

[Tom Goddard]

I've improved the label command behavior. The atom spec can now be omitted and then the command acts on existing labels. For example

label size 30
label res color blue
label delete

changes existing labels to have size 30, colors all residue labels blue, and deletes all labels. "label residues" does nothing (actually gives a warning that an atom spec is needed to create labels). I think that is ok and "label res #1" is not too much to type. Also we probably don't want "labels" to try to label every atom since this will probably hang ChimeraX for minutes with a ribosome structure.

I think the above change addresses point 2, allowing changing existing labels without re-specifying the atom spec for existing labels.

For point 3) that you might want to change the default size for all future labels, this is related to preferences. Probably we want the capability to save the new default label size as a preference. But we don't currently have preferences support. So I'm leaving this for later when preferences are available.

For point 4) I agree it is a bit weird that onTop is per-label-model while all the other settings (size, color, ...) are per-label. Allowing onTop to be per label will either slow down the rendering in all cases, or will require complex optimization code. The case of allowing some labels to be on top and others not is just too obscure to justify this.

[Elaine Meng]

Re (1) allowing blank atomspec:
I disagree about requiring an atomspec for “label” and “label residues”  to show (create) labels … It’s a pain for such a common operation to be forced to remember which model number you have shown and whether the “residues” (or “atoms”) part is before or after the atomspec. Of course it’s bad to show too many labels, but there could be a mechanism like we have now in Chimera1 to detect a request for >N labels and show a warning message with an abort button.  As in Chimera1, it's already smart enough to show only the labels for the atoms or residues that  are currently displayed.

On that note, however, it seems like all the undisplayed labels also slow things down considerably, which seems like a bug (e.g. open 2gbp, show only ligand, label #1). I’ll make a separate ticket for that.

Re (4) I’m not suggesting that onTop should be per-label.  I don’t think it should.  It is just something that will have to be described in documentation since the command structure gives no indication it’s different from the other parameters.

Acting on all current labels with blank atomspec helps somewhat, but I still prefer my suggestion in (2) to separate show/hide/delete into a top-level “label” command from parameter-setting in a “label style” subcommand. It's analogous to atom/bond display (show/hide is “show”, parameter-setting is “style”) and I think easier to understand and work with.  However, I’ll try to work with the current command and I hope the others will too, so that we have more than two testers before everything is “locked in.”  (CCing Conrad on this mail in case he has time to try it out)

Elaine

[Tom Goddard]

Added session support for 3d labels.

I think the current capabilities and command syntax are good enough for the first pass. Since the command syntax is different from Chimera 1 I think we should consider it experimental until we get more experience with it.