Atomic/session memory leak

[Tristan Croll]

The following bug report has been submitted:
Platform:        Linux-5.15.0-60-generic-x86_64-with-glibc2.35
ChimeraX Version: 1.6.dev202301180300 (2023-01-18 03:00:03 UTC)
Description
Just had a long-running ISOLDE session (working on a particularly big model - 8em7) die unexpectedly, killed by the system out-of-memory killer. Investigation reveals a few possible culprits - it looks like the OpenMM `Topology` objects aren't being cleaned up (possibly a circular reference issue?), but It also looks like there's a big memory leak in session saving. Toy example below, but in my real case it eats about 55 MB per save (which I do fairly regularly in case of disaster).

open 7lto
show

#in the shell

m = session.models[0]

import tracemalloc

tracemalloc.start()

before = tracemalloc.take_snapshot()

pwd
Out[5]: '/home/tcroll/my-gits/isolde/isolde'

cd ~
/home/tcroll

from chimerax.core.commands import run

for i in range(10):
    run(session, 'save test.cxs')
    

after = tracemalloc.take_snapshot()

top_stats = after.compare_to(before, 'lineno')

for stat in top_stats[:10]:
    print(stat)
/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/chimerax/atomic/molobject.py:1954: size=17.1 MiB (+17.1 MiB), count=197119 (+197119), average=91 B
<frozen importlib._bootstrap_external>:647: size=448 KiB (+448 KiB), count=4140 (+4140), average=111 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/qtconsole/console_widget.py:1713: size=64.0 KiB (+64.0 KiB), count=1 (+1), average=64.0 KiB
<frozen importlib._bootstrap>:228: size=30.1 KiB (+30.1 KiB), count=253 (+253), average=122 B
/usr/bin/../../usr/lib/ucsf-chimerax-daily/lib/python3.9/abc.py:123: size=28.5 KiB (+28.5 KiB), count=350 (+350), average=83 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/chimerax/core/serialize.py:65: size=27.5 KiB (+27.5 KiB), count=151 (+151), average=187 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/ctypes/__init__.py:510: size=25.5 KiB (+23.7 KiB), count=383 (+326), average=68 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/pygments/formatters/html.py:507: size=20.1 KiB (+20.1 KiB), count=242 (+242), average=85 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/numpy/core/_internal.py:255: size=20.1 KiB (+13.1 KiB), count=161 (+105), average=128 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/chimerax/core_formats/__init__.py:81: size=11.3 KiB (+11.3 KiB), count=51 (+51), average=226 B

for i in range(10):
    run(session, 'save test.cxs')

after = tracemalloc.take_snapshot()

top_stats = after.compare_to(before, 'lineno')

for stat in top_stats[:10]:
    print(stat)
/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/chimerax/atomic/molobject.py:1954: size=34.1 MiB (+34.1 MiB), count=394269 (+394269), average=91 B
<frozen importlib._bootstrap_external>:647: size=447 KiB (+447 KiB), count=4137 (+4137), average=111 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/tracemalloc.py:115: size=95.9 KiB (+95.9 KiB), count=1227 (+1227), average=80 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/tracemalloc.py:129: size=78.1 KiB (+78.1 KiB), count=1111 (+1111), average=72 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/chimerax/mouse_modes/mousemodes.py:514: size=64.0 KiB (+64.0 KiB), count=1 (+1), average=64.0 KiB
/usr/lib/ucsf-chimerax-daily/lib/python3.9/tracemalloc.py:193: size=62.3 KiB (+62.3 KiB), count=1330 (+1330), average=48 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/tracemalloc.py:498: size=58.2 KiB (+58.2 KiB), count=1234 (+1234), average=48 B
/usr/lib/ucsf-chimerax-daily/lib/python3.9/ctypes/__init__.py:510: size=52.8 KiB (+51.0 KiB), count=686 (+629), average=79 B
<frozen importlib._bootstrap>:228: size=30.1 KiB (+30.1 KiB), count=253 (+253), average=122 B
/usr/bin/../../usr/lib/ucsf-chimerax-daily/lib/python3.9/abc.py:123: size=28.5 KiB (+28.5 KiB), count=350 (+350), average=83 B

Log:
> isolde shorthand
    
    
    Initialising ISOLDE-specific command aliases:
    Alias	Equivalent full command
    -------------------------------------------------
    st	isolde step {arguments}
    aw	isolde add water {arguments}
    awsf	isolde add water {arguments} sim false
    al	isolde add ligand {arguments}
    aa	isolde add aa $1 sel {arguments}
    ht	isolde mod his sel {arguments}
    so	setattr sel atoms occupancy {arguments}
    ab	isolde adjust bfactors {arguments}
    ss	isolde sim start sel
    rt	isolde release torsions sel {arguments}
    rd	isolde release distances sel {arguments}
    ra	rd; rt
    pf	isolde pepflip sel
    cf	isolde cisflip sel
    cbb	color bfactor {arguments}
    cbo	color byattr occupancy {arguments}
    cbc	color {arguments} bychain; color {arguments} byhet
    cs	clipper set contourSensitivity {arguments}
    

  
UCSF ChimeraX version: 1.6.dev202301180300 (2023-01-18)  
© 2016-2023 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open 7lto format mmcif fromDatabase pdb

7lto title:  
Nse5-6 complex [more info...]  
  
Chain information for 7lto #1  
---  
Chain | Description | UniProt  
A | Non-structural maintenance of chromosome element 5 | NSE5_YEAST 1-556  
B | Ubiquitin-like protein SMT3,DNA repair protein KRE29 chimera | SMT3_YEAST
-98--1, KRE29_YEAST 1-464  
  

> ui tool show Shell

> show

> save test.cxs

[Repeated 19 time(s)]




OpenGL version: 3.3.0 NVIDIA 515.86.01
OpenGL renderer: NVIDIA GeForce RTX 3070/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation

Python: 3.9.11
Locale: en_GB.UTF-8
Qt version: PyQt6 6.4.0, Qt 6.4.0
Qt runtime version: 6.4.2
Qt platform: xcb

XDG_SESSION_TYPE=x11
DESKTOP_SESSION=ubuntu
XDG_SESSION_DESKTOP=ubuntu
XDG_CURRENT_DESKTOP=ubuntu:GNOME
DISPLAY=:1
Manufacturer: Dell Inc.
Model: XPS 8950
OS: Ubuntu 22.04 Jammy Jellyfish
Architecture: 64bit ELF
Virtual Machine: none
CPU: 20 12th Gen Intel(R) Core(TM) i7-12700
Cache Size: 25600 KB
Memory:
	               total        used        free      shared  buff/cache   available
	Mem:            31Gi        17Gi       8.3Gi       710Mi       5.5Gi        12Gi
	Swap:          2.0Gi       956Mi       1.1Gi

Graphics:
	0000:01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 3070 Lite Hash Rate] [10de:2488] (rev a1)	
	Subsystem: Dell GA104 [GeForce RTX 3070 Lite Hash Rate] [1028:c903]	
	Kernel driver in use: nvidia

Installed Packages:
    alabaster: 0.7.13
    appdirs: 1.4.4
    asttokens: 2.2.1
    Babel: 2.11.0
    backcall: 0.2.0
    blockdiag: 3.0.0
    build: 0.8.0
    certifi: 2022.12.7
    cftime: 1.6.2
    charset-normalizer: 2.1.1
    ChimeraX-AddCharge: 1.5.8
    ChimeraX-AddH: 2.2.3
    ChimeraX-AlignmentAlgorithms: 2.0.1
    ChimeraX-AlignmentHdrs: 3.3.1
    ChimeraX-AlignmentMatrices: 2.1
    ChimeraX-Alignments: 2.8
    ChimeraX-AlphaFold: 1.0
    ChimeraX-AltlocExplorer: 1.0.3
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.1
    ChimeraX-Atomic: 1.43.5
    ChimeraX-AtomicLibrary: 10.0.1
    ChimeraX-AtomSearch: 2.0.1
    ChimeraX-AxesPlanes: 2.3.2
    ChimeraX-BasicActions: 1.1.2
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 2.1.2
    ChimeraX-BondRot: 2.0.1
    ChimeraX-BugReporter: 1.0.1
    ChimeraX-BuildStructure: 2.7.2
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.2
    ChimeraX-ButtonPanel: 1.0.1
    ChimeraX-CageBuilder: 1.0.1
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.3.2
    ChimeraX-ChangeChains: 1.0.2
    ChimeraX-CheckWaters: 1.3.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.2.4
    ChimeraX-Clipper: 0.20.0
    ChimeraX-ColorActions: 1.0.3
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.5.3
    ChimeraX-CommandLine: 1.2.5
    ChimeraX-ConnectStructure: 2.0.1
    ChimeraX-Contacts: 1.0.1
    ChimeraX-Core: 1.6.dev202301180300
    ChimeraX-CoreFormats: 1.1
    ChimeraX-coulombic: 1.4.2
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0.1
    ChimeraX-DataFormats: 1.2.3
    ChimeraX-Dicom: 1.1
    ChimeraX-DistMonitor: 1.3.1
    ChimeraX-DockPrep: 1.1
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ESMFold: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0.1
    ChimeraX-FunctionKey: 1.0.1
    ChimeraX-Geometry: 1.2.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.1.1
    ChimeraX-Hbonds: 2.4
    ChimeraX-Help: 1.2.1
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.2
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ISOLDE: 1.6.dev0
    ChimeraX-ItemsInspection: 1.0.1
    ChimeraX-Label: 1.1.7
    ChimeraX-LinuxSupport: 1.0.1
    ChimeraX-ListInfo: 1.1.1
    ChimeraX-Log: 1.1.5
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.2
    ChimeraX-Map: 1.1.4
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0.1
    ChimeraX-MapFilter: 2.0.1
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1.1
    ChimeraX-Markers: 1.0.1
    ChimeraX-Mask: 1.0.2
    ChimeraX-MatchMaker: 2.0.11
    ChimeraX-MDcrds: 2.6
    ChimeraX-MedicalToolbar: 1.0.2
    ChimeraX-Meeting: 1.0.1
    ChimeraX-MLP: 1.1.1
    ChimeraX-mmCIF: 2.11
    ChimeraX-MMTF: 2.2
    ChimeraX-Modeller: 1.5.8
    ChimeraX-ModelPanel: 1.3.6
    ChimeraX-ModelSeries: 1.0.1
    ChimeraX-Mol2: 2.0
    ChimeraX-Mole: 1.0
    ChimeraX-Morph: 1.0.2
    ChimeraX-MouseModes: 1.2
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0.3
    ChimeraX-OpenCommand: 1.10
    ChimeraX-PDB: 2.6.12
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0.2
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0.1
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.1
    ChimeraX-PubChem: 2.1
    ChimeraX-ReadPbonds: 1.0.1
    ChimeraX-Registration: 1.1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-RenderByAttr: 1.0
    ChimeraX-RenumberResidues: 1.1
    ChimeraX-ResidueFit: 1.0.1
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 3.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.5.1
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0.1
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0.1
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.8.1
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0.1
    ChimeraX-Shortcuts: 1.1.1
    ChimeraX-ShowSequences: 1.0.1
    ChimeraX-SideView: 1.0.1
    ChimeraX-Smiles: 2.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.10.1
    ChimeraX-STL: 1.0.1
    ChimeraX-Storm: 1.0
    ChimeraX-StructMeasure: 1.1.1
    ChimeraX-Struts: 1.0.1
    ChimeraX-Surface: 1.0.1
    ChimeraX-SwapAA: 2.0.1
    ChimeraX-SwapRes: 2.2
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.1.2
    ChimeraX-ToolshedUtils: 1.2.1
    ChimeraX-Topography: 1.0
    ChimeraX-Tug: 1.0.1
    ChimeraX-UI: 1.26
    ChimeraX-uniprot: 2.2.2
    ChimeraX-UnitCell: 1.0.1
    ChimeraX-ViewDockX: 1.1.6
    ChimeraX-VIPERdb: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0.1
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0.2
    ChimeraX-WebServices: 1.1.1
    ChimeraX-Zone: 1.0.1
    colorama: 0.4.5
    comm: 0.1.2
    contourpy: 1.0.7
    cxservices: 1.2
    cycler: 0.11.0
    Cython: 0.29.32
    debugpy: 1.6.5
    decorator: 5.1.1
    distro: 1.7.0
    docutils: 0.19
    entrypoints: 0.4
    executing: 1.2.0
    filelock: 3.7.1
    fonttools: 4.38.0
    funcparserlib: 1.0.1
    grako: 3.16.5
    h5py: 3.7.0
    html2text: 2020.1.16
    idna: 3.4
    ihm: 0.35
    imagecodecs: 2022.9.26
    imagesize: 1.4.1
    importlib-metadata: 6.0.0
    ipykernel: 6.19.2
    ipython: 8.7.0
    ipython-genutils: 0.2.0
    jedi: 0.18.2
    Jinja2: 3.1.2
    jupyter-client: 7.4.8
    jupyter-core: 5.1.3
    kiwisolver: 1.4.4
    line-profiler: 3.5.1
    lxml: 4.9.1
    lz4: 4.0.2
    MarkupSafe: 2.1.2
    matplotlib: 3.6.2
    matplotlib-inline: 0.1.6
    msgpack: 1.0.4
    nest-asyncio: 1.5.6
    netCDF4: 1.6.0
    networkx: 2.8.8
    numexpr: 2.8.4
    numpy: 1.23.5
    openvr: 1.23.701
    packaging: 23.0
    ParmEd: 3.4.3
    parso: 0.8.3
    pep517: 0.13.0
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 9.3.0
    pip: 22.2.2
    pkginfo: 1.8.3
    platformdirs: 2.6.2
    prompt-toolkit: 3.0.36
    psutil: 5.9.4
    ptyprocess: 0.7.0
    pure-eval: 0.2.2
    pycollada: 0.7.2
    pydicom: 2.3.0
    Pygments: 2.12.0
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 3.0.9
    PyQt6-commercial: 6.4.0
    PyQt6-Qt6: 6.4.2
    PyQt6-sip: 13.4.0
    PyQt6-WebEngine-commercial: 6.4.0
    PyQt6-WebEngine-Qt6: 6.4.2
    python-dateutil: 2.8.2
    pytz: 2022.7.1
    pyzmq: 25.0.0
    qtconsole: 5.4.0
    QtPy: 2.3.0
    RandomWords: 0.4.0
    rdkit-pypi: 2022.9.2
    requests: 2.28.1
    scipy: 1.9.3
    setuptools: 65.1.1
    sfftk-rw: 0.7.2
    six: 1.16.0
    snowballstemmer: 2.2.0
    sortedcontainers: 2.4.0
    Sphinx: 5.1.1
    sphinx-autodoc-typehints: 1.19.1
    sphinxcontrib-applehelp: 1.0.3
    sphinxcontrib-blockdiag: 3.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 2.0.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.5
    stack-data: 0.6.2
    tables: 3.7.0
    tifffile: 2022.10.10
    tinyarray: 1.2.4
    tomli: 2.0.1
    tornado: 6.2
    traitlets: 5.8.0
    urllib3: 1.26.14
    wcwidth: 0.2.6
    webcolors: 1.12
    wheel: 0.37.1
    wheel-filename: 1.4.1
    zipp: 3.11.0

[Tristan Croll]

Continuing from the saved session file on my Windows Laptop (as opposed to
Linux yesterday) shows no sign of the accumulating OpenMM topology cruft I
saw yesterday - memory usage started at 3 GB and was 3.2 GB after about 6
hours of fairly continuous work involving innumerable simulations (I also
opened a new 1600-residue AlphaFold model and its PAE matrix in the
meantime). So either that's something Linux-specific or the garbage
collection was just being a bit sluggish. Saving the session ten times over
in a loop permanently increased memory usage by about 1GB.

On Tue, Feb 21, 2023 at 7:01 PM ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
wrote:

[pett]

Was under the misapprehension that appending to a list works the same as list insertion (which steals a reference). It doesn't -- it adds a reference. Added a bunch of Py_DEREFs to the code.