Opened 4 years ago

Closed 4 years ago

#4894 closed defect (fixed)

ResiduesArg slow with outermost ~

Reported by: Tristan Croll Owned by: Eric Pettersen
Priority: normal Milestone:
Component: Performance Version:
Keywords: Cc:
Blocked By: Blocking:
Notify when closed: Platform: all
Project: ChimeraX

Description 

The following bug report has been submitted:
Platform:        Linux-3.10.0-1160.25.1.el7.x86_64-x86_64-with-glibc2.14
ChimeraX Version: 1.3.dev202106210544 (2021-06-21 05:44:33 UTC)
Description
Parsing a `ResiduesArg` gets *very* slow in some circumstances with multiple models involved. Here I have three models open (#1: large multi-chain complex, #2-3: each a single small ~25-residue chain).
With a single residue from #1 selected:

from chimerax.core.commands import run
%prun -s cumulative run(session, 'isolde ignore ~sel')

         377768 function calls (363559 primitive calls) in 24.040 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   24.040   24.040 {built-in method builtins.exec}
        1    0.000    0.000   24.040   24.040 <string>:1(<module>)
        1    0.000    0.000   24.040   24.040 run.py:14(run)
        1    0.000    0.000   24.040   24.040 cli.py:2770(run)
        1    0.000    0.000   24.024   24.024 cli.py:2560(_process_positional_arguments)
        1    0.000    0.000   24.024   24.024 cli.py:2441(_parse_arg)
        1    0.085    0.085   24.024   24.024 args.py:67(parse)
     2388    0.002    0.000   18.944    0.008 molarray.py:196(__and__)
     2391    0.016    0.000   18.942    0.008 molarray.py:208(intersect)
     2395    0.003    0.000   16.955    0.007 molarray.py:239(mask)
     2395    0.022    0.000   16.948    0.007 molarray.py:318(includes_each)
     2395   14.858    0.006   14.870    0.006 molarray.py:300(_cpp_table_pointer)

%prun -s cumulative run(session, 'isolde ignore #1&~sel)

         33222 function calls (30675 primitive calls) in 0.465 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.465    0.465 {built-in method builtins.exec}
        1    0.000    0.000    0.465    0.465 <string>:1(<module>)
        1    0.002    0.002    0.465    0.465 run.py:14(run)
        1    0.000    0.000    0.463    0.463 cli.py:2770(run)
        1    0.000    0.000    0.447    0.447 cli.py:2560(_process_positional_arguments)
        1    0.000    0.000    0.447    0.447 cli.py:2441(_parse_arg)
        1    0.000    0.000    0.447    0.447 args.py:67(parse)
      2/1    0.000    0.000    0.415    0.415 atomspec.py:1041(evaluate)
        5    0.000    0.000    0.366    0.073 atomspec.py:1109(add_implied_bonds)
        5    0.253    0.051    0.254    0.051 molarray.py:588(intra_bonds)
        5    0.000    0.000    0.100    0.020 pbgroup.py:366(interatom_pseudobonds)
       18    0.020    0.001    0.092    0.005 molc.py:189(get_prop)

  

OpenGL version: 3.3.0 NVIDIA 465.19.01
OpenGL renderer: NVIDIA TITAN Xp/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation
Manufacturer: Dell Inc.
Model: Precision T5600
OS: CentOS Linux 7 Core
Architecture: 64bit ELF
Virutal Machine: none
CPU: 32 Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.10GHz
Cache Size: 20480 KB
Memory:
	              total        used        free      shared  buff/cache   available
	Mem:            62G         11G         29G        217M         21G         50G
	Swap:          4.9G          0B        4.9G

Graphics:
	03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP102 [TITAN Xp] [10de:1b02] (rev a1)	
	Subsystem: NVIDIA Corporation Device [10de:11df]	
	Kernel driver in use: nvidia
Locale: ('en_GB', 'UTF-8')
PyQt5 5.15.2, Qt 5.15.2
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    Babel: 2.9.1
    backcall: 0.2.0
    blockdiag: 2.0.1
    certifi: 2021.5.30
    cftime: 1.5.0
    chardet: 4.0.0
    ChimeraX-AddCharge: 1.1.4
    ChimeraX-AddH: 2.1.7
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.1
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.23
    ChimeraX-AtomicLibrary: 3.3
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AtomSearchLibrary: 1.0
    ChimeraX-AxesPlanes: 2.0
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 1.1.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.5.2
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.1
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.1
    ChimeraX-Clipper: 0.17.0
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.3.2
    ChimeraX-CommandLine: 1.1.4
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.3.dev202106210544
    ChimeraX-CoreFormats: 1.0
    ChimeraX-coulombic: 1.3
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0
    ChimeraX-DataFormats: 1.2
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1.4
    ChimeraX-DistUI: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.1
    ChimeraX-Hbonds: 2.1
    ChimeraX-Help: 1.1
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.1
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ISOLDE: 1.3.dev20
    ChimeraX-ItemsInspection: 1.0
    ChimeraX-Label: 1.1
    ChimeraX-LinuxSupport: 1.0
    ChimeraX-ListInfo: 1.1.1
    ChimeraX-Log: 1.1.4
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.1
    ChimeraX-Map: 1.1
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 1.2.1
    ChimeraX-MDcrds: 2.3
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.1
    ChimeraX-mmCIF: 2.3
    ChimeraX-MMTF: 2.1
    ChimeraX-Modeller: 1.0.2
    ChimeraX-ModelPanel: 1.1
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.1
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0.2
    ChimeraX-OpenCommand: 1.6
    ChimeraX-PDB: 2.4.2
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0.1
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0.1
    ChimeraX-PubChem: 2.1
    ChimeraX-ReadPbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-Sample: 0.1
    ChimeraX-SaveCommand: 1.4
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.4.1
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.1
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.6
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.1
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.1
    ChimeraX-ToolshedUtils: 1.2
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.10
    ChimeraX-uniprot: 2.1
    ChimeraX-UnitCell: 1.0
    ChimeraX-ViewDockX: 1.0.1
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-Voyager: 0.1
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.4
    comtypes: 1.1.10
    cxservices: 1.0
    cycler: 0.10.0
    Cython: 0.29.23
    decorator: 4.4.2
    distlib: 0.3.1
    distro: 1.5.0
    docutils: 0.17.1
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    h5py: 3.2.1
    html2text: 2020.1.16
    idna: 2.10
    ihm: 0.20
    imagecodecs: 2021.4.28
    imagesize: 1.2.0
    ipykernel: 5.5.5
    ipython: 7.23.1
    ipython-genutils: 0.2.0
    jedi: 0.18.0
    Jinja2: 2.11.3
    jupyter-client: 6.1.12
    jupyter-core: 4.7.1
    kiwisolver: 1.3.1
    line-profiler: 3.3.0
    lxml: 4.6.3
    lz4: 3.1.3
    MarkupSafe: 1.1.1
    matplotlib: 3.4.2
    matplotlib-inline: 0.1.2
    msgpack: 1.0.2
    netCDF4: 1.5.6
    networkx: 2.5.1
    numexpr: 2.7.3
    numpy: 1.20.3
    numpydoc: 1.1.0
    openvr: 1.16.801
    packaging: 20.9
    ParmEd: 3.2.0
    parso: 0.8.2
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 8.2.0
    pip: 21.1.1
    pkginfo: 1.7.0
    prompt-toolkit: 3.0.19
    psutil: 5.8.0
    ptyprocess: 0.7.0
    pycollada: 0.7.1
    pydicom: 2.1.2
    Pygments: 2.9.0
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PyQt5-commercial: 5.15.2
    PyQt5-sip: 12.8.1
    PyQtWebEngine-commercial: 5.15.2
    python-dateutil: 2.8.1
    pytz: 2021.1
    pyzmq: 22.1.0
    qtconsole: 5.1.0
    QtPy: 1.9.0
    RandomWords: 0.3.0
    requests: 2.25.1
    scipy: 1.6.3
    setuptools: 57.0.0
    sfftk-rw: 0.7.0.post1
    six: 1.16.0
    snowballstemmer: 2.1.0
    sortedcontainers: 2.4.0
    Sphinx: 4.0.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 2.0.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.5
    suds-jurko: 0.6
    tables: 3.6.1
    tifffile: 2021.4.8
    tinyarray: 1.2.3
    tornado: 6.1
    traitlets: 5.0.5
    urllib3: 1.26.5
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.36.2
    wheel-filename: 1.3.0

Change History (9)

comment:1 by Eric Pettersen, 4 years ago

Component: Unassigned → Command Line
Owner: set to Eric Pettersen
Platform: → all
Project: → ChimeraX
Status: new → accepted
Summary: ChimeraX bug report submission → ResiduesArg slow with outermost ~

comment:2 by Eric Pettersen, 4 years ago

Component: Command Line → Performance

I'm pretty sure it's because of the special handling that '~' gets when it's the outermost operator of atom spec in the context of residues. Without the special handling, if one atom is selected in a structure and you ask for the residues in "~sel" you would get all residues. It may be quite difficult to speed up. I will certainly investigate.

in reply to:  3 ; comment:3 by Tristan Croll, 4 years ago 

Hoist by my own petard! (I think I was the one that suggested that feature)
________________________________
From: ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
Sent: 13 July 2021 18:48
Cc: pett@cgl.ucsf.edu <pett@cgl.ucsf.edu>; Tristan Croll <tic20@cam.ac.uk>
Subject: Re: [ChimeraX] #4894: ResiduesArg slow with outermost ~

#4894: ResiduesArg slow with outermost ~
------------------------------------+----------------------------
          Reporter:  Tristan Croll  |      Owner:  Eric Pettersen
              Type:  defect         |     Status:  accepted
          Priority:  normal         |  Milestone:
         Component:  Performance    |    Version:
        Resolution:                 |   Keywords:
        Blocked By:                 |   Blocking:
Notify when closed:                 |   Platform:  all
           Project:  ChimeraX       |
------------------------------------+----------------------------
Changes (by Eric Pettersen):

 * component:  Command Line => Performance


Comment:

 I'm pretty sure it's because of the special handling that '~' gets when
 it's the outermost operator of atom spec in the context of residues.
 Without the special handling, if one atom is selected in a structure and
 you ask for the residues in "~sel" you would get all residues.  It may be
 quite difficult to speed up.  I will certainly investigate.

--
Ticket URL: <https://www.rbvi.ucsf.edu/trac/ChimeraX/ticket/4894#comment:2>
ChimeraX <https://www.rbvi.ucsf.edu/chimerax/>
ChimeraX Issue Tracker

in reply to:  4 ; comment:4 by Tristan Croll, 4 years ago 

A more general question related to this: is there a clear choice behind why selecting a bond doesn't automatically select its endpoint atoms? It seems to me that would be quite intuitive to the general user (perhaps more so than the current situation), and would dramatically simplify the logic for things like this.
________________________________
From: ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
Sent: 13 July 2021 19:11
To: pett@cgl.ucsf.edu <pett@cgl.ucsf.edu>; Tristan Croll <tic20@cam.ac.uk>
Subject: Re: [ChimeraX] #4894: ResiduesArg slow with outermost ~

#4894: ResiduesArg slow with outermost ~
------------------------------------+----------------------------
          Reporter:  Tristan Croll  |      Owner:  Eric Pettersen
              Type:  defect         |     Status:  accepted
          Priority:  normal         |  Milestone:
         Component:  Performance    |    Version:
        Resolution:                 |   Keywords:
        Blocked By:                 |   Blocking:
Notify when closed:                 |   Platform:  all
           Project:  ChimeraX       |
------------------------------------+----------------------------

Comment (by Tristan Croll):

 {{{
 Hoist by my own petard! (I think I was the one that suggested that
 feature)
 ________________________________
 From: ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
 Sent: 13 July 2021 18:48
 Cc: pett@cgl.ucsf.edu <pett@cgl.ucsf.edu>; Tristan Croll <tic20@cam.ac.uk>
 Subject: Re: [ChimeraX] #4894: ResiduesArg slow with outermost ~

 #4894: ResiduesArg slow with outermost ~
 ------------------------------------+----------------------------
           Reporter:  Tristan Croll  |      Owner:  Eric Pettersen
               Type:  defect         |     Status:  accepted
           Priority:  normal         |  Milestone:
          Component:  Performance    |    Version:
         Resolution:                 |   Keywords:
         Blocked By:                 |   Blocking:
 Notify when closed:                 |   Platform:  all
            Project:  ChimeraX       |
 ------------------------------------+----------------------------
 Changes (by Eric Pettersen):

  * component:  Command Line => Performance


 Comment:

  I'm pretty sure it's because of the special handling that '~' gets when
  it's the outermost operator of atom spec in the context of residues.
  Without the special handling, if one atom is selected in a structure and
  you ask for the residues in "~sel" you would get all residues.  It may be
  quite difficult to speed up.  I will certainly investigate.

 --
 Ticket URL:
 <https://www.rbvi.ucsf.edu/trac/ChimeraX/ticket/4894#comment:2>
 ChimeraX <https://www.rbvi.ucsf.edu/chimerax/>
 ChimeraX Issue Tracker
 }}}

--
Ticket URL: <https://www.rbvi.ucsf.edu/trac/ChimeraX/ticket/4894#comment:3>
ChimeraX <https://www.rbvi.ucsf.edu/chimerax/>
ChimeraX Issue Tracker

comment:5 by Eric Pettersen, 4 years ago

Does unselecting the bond unselect its endpoint atoms?

in reply to:  6 ; comment:6 by Tristan Croll, 4 years ago 

Realised I never responded to this. To me, the most intuitive answer would be “yes”.

comment:7 by Eric Pettersen, 4 years ago

Do other selected bonds connected to the endpoint atoms also get deselected? What about atoms connected to those?

This also means that you can't delete a bond by selecting it and using Actions→Atoms/Bonds→Delete, since that also deletes the endpoint atoms. My feeling is that the cure is worse than the disease in this case.

in reply to:  8 ; comment:8 by Tristan Croll, 4 years ago 

Counterpoint: having a single menu entry that deletes both atoms and bonds can be a bit scary, since when a bond is selected it's often difficult to tell visually whether or not the atoms are also selected. But I take your point about the deselection challenge.
________________________________
From: ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
Sent: 27 July 2021 18:05
Cc: pett@cgl.ucsf.edu <pett@cgl.ucsf.edu>; Tristan Croll <tic20@cam.ac.uk>
Subject: Re: [ChimeraX] #4894: ResiduesArg slow with outermost ~

#4894: ResiduesArg slow with outermost ~
------------------------------------+----------------------------
          Reporter:  Tristan Croll  |      Owner:  Eric Pettersen
              Type:  defect         |     Status:  accepted
          Priority:  normal         |  Milestone:
         Component:  Performance    |    Version:
        Resolution:                 |   Keywords:
        Blocked By:                 |   Blocking:
Notify when closed:                 |   Platform:  all
           Project:  ChimeraX       |
------------------------------------+----------------------------

Comment (by Eric Pettersen):

 Do other selected bonds connected to the endpoint atoms also get
 deselected?  What about atoms connected to those?

 This also means that you can't delete a bond by selecting it and using
 Actions→Atoms/Bonds→Delete, since that also deletes the endpoint atoms.
 My feeling is that the cure is worse than the disease in this case.

--
Ticket URL: <https://www.rbvi.ucsf.edu/trac/ChimeraX/ticket/4894#comment:7>
ChimeraX <https://www.rbvi.ucsf.edu/chimerax/>
ChimeraX Issue Tracker

comment:9 by Eric Pettersen, 4 years ago

Resolution: → fixed
Status: accepted → closed

Sped up by two orders of magnitude for the common case where it's reasonably easy to discern the specific '~' character responsible for the "outermost inversion" (and therefore easy to create the corresponding uninverted spec needed to quickly remove partially selected residues that don't belong in the inversion).

Note: See TracTickets for help on using tickets.