Custom atom/residue properties?

[Tristan Croll]

There are various situations where one might want to associate custom per-atom and/or per-residue values with a model. Conservation analysis, for example - or in my case, per-atom scaling factors to allow fine-grained tuning of the coupling between atoms and maps during an ISOLDE simulation. Superficially straightforward, but gets a little more complicated when one considers structure editing (i.e. addition/removal of atoms/residues). I wonder if it would be possible to implement something along the lines of the following at the Structure/AtomicStructure level?

def add_custom_atom_property(self, name, data_type, default_value):
    '''
    Adds a new array of custom properties to the atoms in this model.
    Args:
        name:
            a key for later retrieval of this property
        data_type:
            the data type of the property values (e.g. float, int, object)
        default_value:
            the default value associated with each atom
    '''

def get_custom_property_values(self, name, atoms):
    '''
    Returns a numpy array of values for the given custom property 
    associated with the given atoms. Raises an error if any atoms are not
    in this model.
    '''

def set_custom_property_values(self, name, atoms, values):
    '''
    Sets the values of the named custom property for the given atoms.
    '''

def _update_custom_properties_on_model_changes(self):
    '''
    Updates all existing custom properties (removing obsolete values for
    deleted atoms, adding default values for new atoms, and re-indexing)
    when self.new_atoms() is run.
    '''     

[pett]

Hi Tristan,

The plan for custom attributes has always been to have them live in the corresponding Python objects, rather than via some indirection through Structure or a Collection. This is already possible of course, but such attributes are ephemeral in that they won't live through session save/restore cycles, and won't exist in newly created objects of the given class.
So the plan for custom attributes is that you would register them with the class object (not implemented yet). You would specify the attribute name and optionally it's default value and type. The registration would create a property and ensure the attribute and its registration are preserved in sessions. That's basically it.
On top of that basic mechanism we could layer convenience mechanisms for fetching custom attributes from Collections if the attribute has a registered type (which in turn means that it would be pretty easy to get custom attrs from a Structure).
The whole thing isn't terribly efficient since it's in Python. If efficiency is a major concern, we would have to work up some mechanism where custom attributes could be created in the C++ layer. Doing that kind of thing in C++ is never easy, so we will try to use the Python layer approach first and see how it goes.

--Eric

[tic20@…]

Hi Eric,

The reason I suggested implementing it at the Structure level is that it 
would avoid a lot of the complexity you would otherwise encounter with 
mixed populations: what would you do with an Atoms collection where 
different atoms have different custom properties, and some have none? It 
seems to me that would get messy quickly. Implementation at the 
Structure level would in contrast be relatively easy:

   - define a default value (or constructor function for an array of 
objects) v, name and dtype
   - self._custom_values[name] = numpy.array([v]*len(self.atoms), dtype)

... then define a retrieval function, i.e.:

Structure.get_custom_value(name, atoms)

... which internally just runs self.indices(atoms) and returns the 
corresponding values from the array, an equivalent setter, and some 
housekeeping functions to update the internal array if atoms are added 
to/removed from the structure. That could all be done entirely in 
Python, but should still be fast since there would be no need to loop 
through individual Atom objects - and I think it would be very 
flexible/powerful. It would also make it relatively easy to save them in 
a session state to continue on later. I can think of a number of useful 
applications in my own context:

   - handling rotamer restraints. At present I have Rotamer objects 
defined for each protein residue, stored in a residue:Rotamer dict. It 
works well enough for now, but it means an extra loop through all mobile 
residues every time I start a simulation, and I'll have to define 
functions to run on model changes to check if the set of residues has 
changed and act accordingly.
   - Similarly for distance and position restraints, except now we're 
talking individual atoms. Due to the way OpenMM works, any restraint you 
want to be able to adjust (or simply switch on) interactively needs to 
be defined before the simulation starts to avoid costly 
re-initialisations. That means one position restraint for every heavy 
atom, and sets of distance restraints for key strategic atom pairs (e.g. 
On-Nn+4 for alpha helix H-bonds).
   - Defining TLS groups. These are used extensively in crystallographic 
refinement to define portions of the structure that are treated as rigid 
bodies for the purpose of calculating anisotropic B-factors, but at the 
moment there are few tools that make it really easy to define them. 
Having the ability to simply make a selection in the ChimeraX window and 
then assign an integer (or name) to its 'TLS' custom property would be 
very useful in itself.

All of these things can of course be achieved in other ways (and I 
already have working implementations of everything but the TLS 
assignment), but I think they'd be made much easier by this sort of 
functionality.

Cheers,

Tristan

On 2017-05-30 22:18, ChimeraX wrote:

[pett]

Well, as I mentioned, the registration process would create a _property_ in the _class_ and therefore all instances of the class would instantly have the attribute at it's default value, including instances created later. This seems simple and clean and looks much more Pythonic from the outside (e.g. you get an atom's charge as a.charge). What it may not be is efficient enough in some usage cases. I prefer to first implement the simple clean solution, and then if necessary implement additional more complex solutions as needed.

--Eric

[Greg Couch]

Would like support for additional properties in residues and atomic structures for the nucleotides extension.

[tic20@…]

I'd also like to bump this. I've done some further work on my C++ 
Dihedral implementation, with a higher-level Dihedral_Mgr class. My 
current idea is to create a Dihedral_Mgr for each AtomicStructure to 
hold a vector of all defined proper dihedrals (and later another for 
improper dihedrals defining chiral centres), with fast lookup by residue 
and name (e.g. (<Residue obj>, 'chi1')). The necessary functions are all 
in place, and it all cleans itself up using the existing 
DestructionUser/DestructionObserver framework. For the Python interface, 
though, it would be great to be able to add methods directly to the 
Residue/Residues classes to access named dihedral angles, Ramachandran 
scores etc.

On 2018-01-17 22:52, ChimeraX wrote:

[pett]

This ticket is only for additional _attributes_ in atomic classes, not properties or methods. I do intend to add phi/psi/chi angle support to residues (ticket #493), but that is down the road -- when I have time to get to rotamer support.

For now, your extension can simply add methods and properties into the Python Residue class itself. Chimera1 does that for phi/psi/chi angle support for instance.

--Eric

[pett]

Custom attribute registration is implemented for these classes: Atom, AtomicStructure, Bond, PseudobondGorup, PseudobondManager, Residue, and Structure

and not yet in these classes: Chain, CoordSet, Pseuodbond, Sequence, StructureSeq

The protocol for registering an attribute 'charge' for Atom with type float and default value 0.0 would be:

from chimerax.core.atomic import Atom
Atom.register_attr(session, "charge", "My Tool", default_value=0.0, attr_type=float)

"My Tool" is some string that identifies the registerer for use in error messages. The keywords are optional. If no default_value is supplied, then accessing the attribute when it hasn't been explicitly set will raise AttributeError. The attr_type is currently only used to check that multiple registrations of the same attribute from different registerers are in fact compatible, though more extensive use in the future could occur.

[pett]

Sequence has no way to gather its per-session instances and therefore Sequence/StructureSeq/Chain will not be included in this mechanism. Instead, it will continue to use its already implemented 'attrs' dict method of preserving miscellaneous attributes. Extensions can insert properties into Sequence to fetch/set values if desired.

[pett]

CoordSet now supported

[pett]

Pseudobond now supported, which finishes the classes I intend to support. 'setattr' now registers the attributes it creates. I don't think I'm going to do the "Collection convenience fetch" since (1) it has to look at the existing instances which you could easily do yourself, and (2) I can't think of any reasonable way to distinguish between completely missing attributes and attributes whose value is None in whatever the convenience function returns. Therefore, I an closing this ticket as finished.

[pett]

The current implementation has a session-restore race condition and needs a major revamp under the hood, though the API will remain the same.

[pett]

The under-the-hood revamp is complete and it seems to work with ViewDockX's custom attribute, so I'm closing this ticket now.