Saved session file huge for small atomic model

[drorn@…]

The following bug report has been submitted:
Platform:        macOS-10.15.7-x86_64-i386-64bit
ChimeraX Version: 1.2.5 (2021-05-24 04:13:57 UTC)
Description
Session files are unresonably large even for structures without molecular surfaces. In the following link, a session containing only the pdb structure 5hpz imported directly from the pdb and saved in a session without further manipulation. The file size is 124.5 MB
https://www.dropbox.com/sh/f908f5qexdjz4yw/AABsrloWwWTlXKuKc6knz_jKa?dl=0

OpenGL version: 4.1 ATI-3.10.22
OpenGL renderer: AMD Radeon Pro 560X OpenGL Engine
OpenGL vendor: ATI Technologies Inc.Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro15,1
      Processor Name: 6-Core Intel Core i7
      Processor Speed: 2.2 GHz
      Number of Processors: 1
      Total Number of Cores: 6
      L2 Cache (per Core): 256 KB
      L3 Cache: 9 MB
      Hyper-Threading Technology: Enabled
      Memory: 16 GB
      Boot ROM Version: 1554.140.20.0.0 (iBridge: 18.16.14759.0.1,0)

Software:

    System Software Overview:

      System Version: macOS 10.15.7 (19H1323)
      Kernel Version: Darwin 19.6.0
      Time since boot: 9 days 11:46

Graphics/Displays:

    Intel UHD Graphics 630:

      Chipset Model: Intel UHD Graphics 630
      Type: GPU
      Bus: Built-In
      VRAM (Dynamic, Max): 1536 MB
      Vendor: Intel
      Device ID: 0x3e9b
      Revision ID: 0x0000
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1

    Radeon Pro 560X:

      Chipset Model: Radeon Pro 560X
      Type: GPU
      Bus: PCIe
      PCIe Lane Width: x8
      VRAM (Total): 4 GB
      Vendor: AMD (0x1002)
      Device ID: 0x67ef
      Revision ID: 0x00c2
      ROM Revision: 113-C980AL-075
      VBIOS Version: 113-C97501U-005
      EFI Driver Version: 01.A1.075
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1
      Displays:
        Color LCD:
          Display Type: Built-In Retina LCD
          Resolution: 2880 x 1800 Retina
          Framebuffer Depth: 24-Bit Color (ARGB8888)
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: No
          Connection Type: Internal

Locale: (None, 'UTF-8')
PyQt5 5.15.2, Qt 5.15.2
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    appnope: 0.1.2
    Babel: 2.9.1
    backcall: 0.2.0
    blockdiag: 2.0.1
    certifi: 2020.12.5
    cftime: 1.5.0
    chardet: 3.0.4
    ChimeraX-AddCharge: 1.0.1
    ChimeraX-AddH: 2.1.6
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.1
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.13.2
    ChimeraX-AtomicLibrary: 3.1.3
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AtomSearchLibrary: 1.0
    ChimeraX-AxesPlanes: 2.0
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 1.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.5.2
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.1
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.1
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.2.1
    ChimeraX-CommandLine: 1.1.4
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.2.5
    ChimeraX-CoreFormats: 1.0
    ChimeraX-coulombic: 1.1.1
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0
    ChimeraX-DataFormats: 1.1
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1.3
    ChimeraX-DistUI: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.0
    ChimeraX-Hbonds: 2.1
    ChimeraX-Help: 1.1
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.0
    ChimeraX-ImageFormats: 1.1
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-Label: 1.0
    ChimeraX-ListInfo: 1.1.1
    ChimeraX-Log: 1.1.2
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.1
    ChimeraX-Map: 1.0.2
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.0
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 1.2.1
    ChimeraX-MDcrds: 2.2
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.1
    ChimeraX-mmCIF: 2.3
    ChimeraX-MMTF: 2.1
    ChimeraX-Modeller: 1.0.1
    ChimeraX-ModelPanel: 1.0.1
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.1
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0.1
    ChimeraX-OpenCommand: 1.5
    ChimeraX-PDB: 2.4.1
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0.1
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0.1
    ChimeraX-PubChem: 2.0.1
    ChimeraX-ReadPbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.4
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SeqView: 2.3
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.0
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.0.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.3.1
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.1
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.0.1
    ChimeraX-ToolshedUtils: 1.2
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.7.6
    ChimeraX-uniprot: 2.1
    ChimeraX-UnitCell: 1.0
    ChimeraX-ViewDockX: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.3
    comtypes: 1.1.7
    cxservices: 1.0
    cycler: 0.10.0
    Cython: 0.29.21
    decorator: 5.0.9
    distlib: 0.3.1
    docutils: 0.16
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    h5py: 2.10.0
    html2text: 2020.1.16
    idna: 2.10
    ihm: 0.17
    imagecodecs: 2020.5.30
    imagesize: 1.2.0
    ipykernel: 5.3.4
    ipython: 7.18.1
    ipython-genutils: 0.2.0
    jedi: 0.17.2
    Jinja2: 2.11.2
    jupyter-client: 6.1.7
    jupyter-core: 4.7.1
    kiwisolver: 1.3.1
    line-profiler: 2.1.2
    lxml: 4.6.2
    lz4: 3.1.0
    MarkupSafe: 2.0.1
    matplotlib: 3.3.2
    matplotlib-inline: 0.1.2
    msgpack: 1.0.0
    netCDF4: 1.5.4
    networkx: 2.5
    numexpr: 2.7.3
    numpy: 1.19.2
    numpydoc: 1.1.0
    openvr: 1.14.1501
    packaging: 20.9
    ParmEd: 3.2.0
    parso: 0.7.1
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 7.2.0
    pip: 21.0.1
    pkginfo: 1.5.0.1
    prompt-toolkit: 3.0.18
    psutil: 5.7.2
    ptyprocess: 0.7.0
    pycollada: 0.7.1
    pydicom: 2.0.0
    Pygments: 2.7.1
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PyQt5-commercial: 5.15.2
    PyQt5-sip: 12.8.1
    PyQtWebEngine-commercial: 5.15.2
    python-dateutil: 2.8.1
    pytz: 2021.1
    pyzmq: 22.0.3
    qtconsole: 4.7.7
    QtPy: 1.9.0
    RandomWords: 0.3.0
    requests: 2.24.0
    scipy: 1.5.2
    setuptools: 50.3.2
    sfftk-rw: 0.6.7.dev1
    six: 1.15.0
    snowballstemmer: 2.1.0
    sortedcontainers: 2.2.2
    Sphinx: 3.2.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 2.0.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.5
    suds-jurko: 0.6
    tables: 3.6.1
    tifffile: 2020.9.3
    tinyarray: 1.2.3
    tornado: 6.1
    traitlets: 5.0.5
    urllib3: 1.25.11
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.36.0
    wheel-filename: 1.3.0

[Tom Goddard]

I looked at your session file. It contains logs of previous sessions so I do not think it was made by a newly started ChimeraX followed by open "5hpz" and save test.cxs. You had previously opened and closed some sessions. Can you test with a newly started ChimeraX? I guess then the file size will be small and the problem is caused by the previously closed sessions.

I see that the test.cxs session contains 998889 atomic charge values which cannot possibly be right for 5hpz. So maybe somehow assigned charges from a previous session that you closed are making it into this file.

[Eric Pettersen]

That session restores 362 structures, of which one is 5hpz and 361 are an unknown structure with 6408 atoms. We need to figure out what is referencing those other structures and storing them in this session.

[Eric Pettersen]

Those 361 structures all have the same name: "copy of 6m32 2 1". This implies they are the temporary copies that the coulombic command makes to add hydrogens to in order to assign charges back to the original structure. All those copies get explicitly deleted. Not sure what would produce a model name of "6m32 2 1" other than a file by that name.

[Eric Pettersen]

Adding to the mystery is that "6m32" appears nowhere in the log, nor do any coulombic commands.

[Tom Goddard]

Reassigning to Eric since the bloated session seems to be caused by undisplayed molecule copies created by Coulombic for adding hydrogens.

[Eric Pettersen]

My best guess is that:

1) This session file originally predates the saving of the log in sessions, and is therefore missing references to opening 6m32 and running coulombic commands.
2) That coulombic was run many times and failed many times on that structure for some reason, and therefore .delete() was never called and the structure got saved into sessions because of its custom attributes.

I don't have great confidence in this scenario due to needing hundreds of coulombic failures, but nonetheless I added additional bulletproofing to the charge-addition code (there already was some) to try to absolutely ensure that the copied structure gets deleted no matter what.

[drorn@…]

Hi all,
My problem now is that this is carried over to every session that I’m saving and these are growing larger and larger…I’m already getting 0.5GB session files. Could you recommend a way to reset chimeraX so that I will not get these huge files?
Thanks
Dror

[Eric Pettersen]

Okay, now I understand what's happening and why there are so many copies of that structure in the session file.  As you run ChimeraX and open and close session files, any sessions that have these "zombie" structures will add them to the session, but they won't be removed when the session file is closed, so they accumulate and more and more of them get written to later sessions.

I will be committing a fix for this later today which will be available In tomorrow's daily build.  In the interim I have attached a Python script which cleans out these zombie structure which you can run (simply by opening the file in ChimeraX) to remove these zombie structures just before you write a new session file.

--Eric

clean.py​

[Eric Pettersen]

Added by email2trac

[Eric Pettersen]

Fix applied. Will be in next daily build.