Opened 3 years ago

Last modified 2 years ago

#7101 closed defect

Memory leak opening and closing atomic models — at Version 3

Reported by: goddard@… Owned by: Eric Pettersen
Priority: normal Milestone:
Component: Performance Version:
Keywords: Cc: Tom Goddard, Greg Couch
Blocked By: Blocking:
Notify when closed: Platform: all
Project: ChimeraX

Description (last modified by Eric Pettersen) 

The following bug report has been submitted:
Platform:        macOS-12.4-arm64-arm-64bit
ChimeraX Version: 1.5.dev202206081745 (2022-06-08 17:45:34 UTC)
Description
ChimeraX leaks a lot of memory opening and closing atomic models.  This makes it painful to run nogui processing scripts on large collections of atomic models because they need to be divided into batches.

I've attached a Python script that demonstrates the memory leak.  Here are some test results opening and closing various models 100 times in nogui mode.

# Starting chimerax in nogui on macOS 12.4 gives 68 Mbytes memory use reported by Mac Activity Monitor.
# For small structure 17gs (0.38 MB cif file) memory use goes to 145 Mbytes.
# For larger structure 7azs (42 MB cif file) memory use goes to 526 Mbytes.
# For a PDB format file 1s72 (9 MB pdb file) memory use goes to 1450 Mbytes.
# For EMDB map 1080 (4 MB ccp4 file) memory use goes to 71 Mbyte.
# For 7azs opening without adding to open models memory use goes to 280 Mbytes.




Log:
UCSF ChimeraX version: 1.5.dev202206081745 (2022-06-08)  
© 2016-2022 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  




OpenGL version: 4.1 Metal - 76.3
OpenGL renderer: Apple M1 Max
OpenGL vendor: Apple

Python: 3.9.11
Locale: UTF-8
Qt version: PyQt6 6.3.0, Qt 6.3.0
Qt runtime version: 6.3.0
Qt platform: cocoa
Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro18,2
      Chip: Apple M1 Max
      Total Number of Cores: 10 (8 performance and 2 efficiency)
      Memory: 32 GB
      System Firmware Version: 7459.121.3
      OS Loader Version: 7459.121.3

Software:

    System Software Overview:

      System Version: macOS 12.4 (21F79)
      Kernel Version: Darwin 21.5.0
      Time since boot: 20 days 11:47

Graphics/Displays:

    Apple M1 Max:

      Chipset Model: Apple M1 Max
      Type: GPU
      Bus: Built-In
      Total Number of Cores: 32
      Vendor: Apple (0x106b)
      Metal Family: Supported, Metal GPUFamily Apple 7
      Displays:
        Color LCD:
          Display Type: Built-in Liquid Retina XDR Display
          Resolution: 3456 x 2234 Retina
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: No
          Connection Type: Internal


Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    appnope: 0.1.3
    Babel: 2.10.1
    backcall: 0.2.0
    blockdiag: 3.0.0
    certifi: 2021.10.8
    cftime: 1.6.0
    charset-normalizer: 2.0.12
    ChimeraX-AddCharge: 1.2.3
    ChimeraX-AddH: 2.1.11
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2.1
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.4.2
    ChimeraX-AlphaFold: 1.0
    ChimeraX-AltlocExplorer: 1.0.2
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.39.2
    ChimeraX-AtomicLibrary: 7.0
    ChimeraX-AtomSearch: 2.0.1
    ChimeraX-AxesPlanes: 2.1
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 2.1.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.7
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.1
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.2
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.2.3
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.5.1
    ChimeraX-CommandLine: 1.2.3
    ChimeraX-ConnectStructure: 2.0.1
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.5.dev202206081745
    ChimeraX-CoreFormats: 1.1
    ChimeraX-coulombic: 1.3.2
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0
    ChimeraX-DataFormats: 1.2.2
    ChimeraX-Dicom: 1.1
    ChimeraX-DistMonitor: 1.1.5
    ChimeraX-Dssp: 2.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.2
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.1
    ChimeraX-Hbonds: 2.1.2
    ChimeraX-Help: 1.2
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.2
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ItemsInspection: 1.0
    ChimeraX-Label: 1.1
    ChimeraX-ListInfo: 1.1.1
    ChimeraX-Log: 1.1.5
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.1
    ChimeraX-Map: 1.1
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 2.0.6
    ChimeraX-MDcrds: 2.6
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.1
    ChimeraX-mmCIF: 2.7
    ChimeraX-MMTF: 2.1
    ChimeraX-Modeller: 1.5.5
    ChimeraX-ModelPanel: 1.3.2
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.1
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0.2
    ChimeraX-OpenCommand: 1.9
    ChimeraX-PDB: 2.6.6
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0.2
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.1
    ChimeraX-PubChem: 2.1
    ChimeraX-ReadPbonds: 1.0.1
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0.1
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.5.1
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.6
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.1
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.8
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-StructMeasure: 1.0.1
    ChimeraX-Struts: 1.0.1
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.1.1
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.1
    ChimeraX-ToolshedUtils: 1.2.1
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.20.2
    ChimeraX-uniprot: 2.2
    ChimeraX-UnitCell: 1.0
    ChimeraX-ViewDockX: 1.1.2
    ChimeraX-VIPERdb: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.4
    cxservices: 1.2
    cycler: 0.11.0
    Cython: 0.29.26
    debugpy: 1.6.0
    decorator: 5.1.1
    docutils: 0.17.1
    entrypoints: 0.4
    filelock: 3.4.2
    fonttools: 4.33.2
    funcparserlib: 1.0.0a1
    grako: 3.16.5
    html2text: 2020.1.16
    idna: 3.3
    ihm: 0.27
    imagecodecs: 2022.2.22
    imagesize: 1.3.0
    ipykernel: 6.6.1
    ipython: 7.31.1
    ipython-genutils: 0.2.0
    jedi: 0.18.1
    Jinja2: 3.0.3
    jupyter-client: 7.1.0
    jupyter-core: 4.10.0
    kiwisolver: 1.4.2
    line-profiler: 3.4.0
    lxml: 4.7.1
    lz4: 3.1.10
    MarkupSafe: 2.1.1
    matplotlib: 3.5.1
    matplotlib-inline: 0.1.3
    msgpack: 1.0.3
    nest-asyncio: 1.5.5
    netCDF4: 1.5.8
    networkx: 2.6.3
    numexpr: 2.8.1
    numpy: 1.22.1
    openvr: 1.16.802
    packaging: 21.0
    ParmEd: 3.4.3
    parso: 0.8.3
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 9.0.1
    pip: 21.3.1
    pkginfo: 1.8.2
    prompt-toolkit: 3.0.29
    psutil: 5.9.0
    ptyprocess: 0.7.0
    pycollada: 0.7.2
    pydicom: 2.2.2
    Pygments: 2.11.2
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 3.0.8
    PyQt6: 6.3.0
    PyQt6-Qt6: 6.3.0
    PyQt6-sip: 13.3.1
    PyQt6-WebEngine: 6.3.0
    PyQt6-WebEngine-Qt6: 6.3.0
    python-dateutil: 2.8.2
    pytz: 2022.1
    pyzmq: 22.3.0
    qtconsole: 5.3.0
    QtPy: 2.0.1
    requests: 2.27.1
    scipy: 1.7.3
    setuptools: 59.8.0
    six: 1.16.0
    snowballstemmer: 2.2.0
    sortedcontainers: 2.4.0
    Sphinx: 4.3.2
    sphinx-autodoc-typehints: 1.15.2
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 3.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 2.0.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.5
    suds-community: 1.0.0
    tables: 3.7.0
    tifffile: 2021.11.2
    tinyarray: 1.2.4
    tornado: 6.1
    traitlets: 5.1.1
    urllib3: 1.26.9
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.37.1
    wheel-filename: 1.3.0
File attachment: test_memleak.py

test_memleak.py​

Change History (4)

by goddard@…, 3 years ago

Attachment: test_memleak.py​ added

Added by email2trac

comment:1 by Tom Goddard, 3 years ago

Component: Unassigned → Performance
Owner: set to Eric Pettersen
Platform: → all
Project: → ChimeraX
Status: new → assigned
Summary: ChimeraX bug report submission → Memory leak opening and closing atomic models

comment:2 by Eric Pettersen, 3 years ago

Status: assigned → accepted

comment:3 by Eric Pettersen, 2 years ago

Description: modified (diff)

According to tracemalloc, the top 10 differences after opening and closing 7azs once are:

Top 10 differences:
/Users/chimera/Applications/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/chimerax/atomic/molc.py:80: size=3504 KiB (+3504 KiB), count=53655 (+53655), average=67 B
/Users/chimera/Applications/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/chimerax/atomic/molarray.py:159: size=2525 KiB (+2525 KiB), count=11 (+11), average=230 KiB
<frozen importlib._bootstrap>:228: size=549 KiB (+549 KiB), count=6520 (+6520), average=86 B
/Users/chimera/Applications/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/chimerax/atomic/molarray.py:86: size=261 KiB (+261 KiB), count=21 (+21), average=12.5 KiB
/Users/chimera/Applications/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/chimerax/atomic/molarray.py:1393: size=40.4 KiB (+40.4 KiB), count=171 (+171), average=242 B
/Users/chimera/Applications/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py:510: size=24.4 KiB (+24.4 KiB), count=393 (+393), average=63 B
/Users/chimera/Applications/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/chimerax/core/orderedset.py:36: size=20.4 KiB (+20.4 KiB), count=134 (+134), average=156 B
/Users/chimera/Applications/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/_internal.py:255: size=19.8 KiB (+19.8 KiB), count=169 (+169), average=120 B
/Users/chimera/Applications/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/chimerax/graphics/drawing.py:255: size=14.2 KiB (+14.2 KiB), count=26 (+26), average=558 B
/Users/chimera/Applications/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/chimerax/atomic/molobject.py:1461: size=10.1 KiB (+10.1 KiB), count=9 (+9), average=1146 B

Last edited 2 years ago by Eric Pettersen (previous) (diff)
Note: See TracTickets for help on using tickets.