Saving mmCIF files too slow to work with large structures

[goddard@…]

The following bug report has been submitted:
Platform:        macOS-10.15.7-x86_64-i386-64bit
ChimeraX Version: 1.2.dev202010140223 (2020-10-14 02:23:41 UTC)
Description
Saving  1.2 million atom 5y6p as an mmCIF took about 10 minutes.  This is unusably slow.  Since ChimeraX is trying to serve cryoEM researchers with very large structures this needs to be faster.

Log:
UCSF ChimeraX version: 1.2.dev202010140223 (2020-10-14)  
© 2016-2020 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> time open 5y6p

> open 5y6p

5y6p title:  
Structure of the phycobilisome from the red alga Griffithsia pacifica [more
info...]  
  
Chain information for 5y6p #1  
---  
Chain | Description  
14 R8 W5 W9 aW bW | LRC1  
24 S8 X5 X9 aX bX | LR1  
34 A2 U8 Y5 Z9 a3 aY b8 b9 bY dw dx ey ez | LR_gamma4  
44 C2 D2 T8 Y9 Z5 aZ bZ c3 c8 c9 d3 d8 d9 e8 e9 | LR_gamma5  
A1 B1 | LRC6  
A3 A4 A5 A6 A7 A9 B8 C3 C4 C5 C6 C7 C9 D8 E2 E3 E4 E5 E6 E7 E9 F8 G2 G3 G4 G5
G6 G7 G9 H8 I2 I3 I4 I5 I6 I7 I9 J8 K2 K3 K4 K5 K6 K7 K9 L8 M2 M3 M4 M5 M6 M7
M9 N8 O2 O3 O4 O5 O6 O7 O9 P8 Q2 Q3 Q4 Q5 Q6 Q7 Q9 S2 S3 S4 S5 S6 S7 S9 U2 U3
U4 U5 U6 U7 U9 W2 W3 W4 W6 W7 Y2 Y3 Y4 Y6 Y7 aA aC aE aG aI aK aM aO aQ aS aU
am ao aq as au aw ay bA bC bE bG bI bK bM bO bQ bS bU bm bt bu bv bw bx by cA
cC cD cG cI cK cS cT cU cV cW cX cZ cm ct cu cv cw cx cy dA dC dE dG dI dJ dM
dO dQ dS dU dW dZ da db dc de dg dk dm do dq ds du e2 e3 eA eC eE eG eI eK eM
eO eQ eS eU eW ea ed ef eh ej ek em eo eq es eu f6 f7 fb fe fg fh fk fl fn fp
fr ft fv fx g2 g3 g6 g7 gb gc gf gh gj gl gm go gq gs gu gw h6 h7 hA hC hE hF
hI hK hM hO hQ hS hU hW i2 i3 i6 i7 i8 iA iB j6 j7 j8 k2 k3 k6 k7 k8 l8 la lc
le lg li lk lm lo lq ls lu lw m2 m3 m4 m5 m6 m7 m8 m9 o2 o3 o4 o5 o6 o7 o9 q2
q3 q4 q5 q6 q7 q8 q9 r8 s2 s3 s4 s5 s6 s7 s8 s9 t8 u2 u3 u4 u5 u6 u7 u8 u9 v8
w2 w3 w4 w5 w6 w7 w9 x8 y2 y3 y4 y5 y9 z8 | R-phycoerythrin α chain  
A8 B3 B4 B5 B6 B7 B9 C8 D3 D4 D5 D6 D7 D9 E8 F2 F3 F4 F5 F6 F7 F9 G8 H2 H3 H4
H5 H6 H7 H9 I8 J2 J3 J4 J5 J6 J7 J9 K8 L2 L3 L4 L5 L6 L7 L9 M8 N2 N3 N4 N5 N6
N7 N9 O8 P2 P3 P4 P5 P6 P7 P9 Q8 R2 R3 R4 R5 R6 R7 R9 T2 T3 T4 T5 T6 T7 T9 V2
V3 V4 V5 V6 V7 V9 X2 X3 X4 X6 X7 Z2 Z3 Z4 Z6 Z7 a6 a7 aB aD aF aH aJ aL aN aP
aR aT aV an ap ar at av ax az b6 b7 bB bD bF bH bJ bL bN bP bR bT bV bn bo bp
bq br bs bz c6 c7 cB cE cF cH cJ cL cM cN cO cP cQ cR cY cn co cp cq cr cs cz
d6 d7 dB dD dF dH dK dL dN dP dR dT dV dX dY dd df dh di dj dl dn dp dr dt dv
e6 e7 eB eD eF eH eJ eL eN eP eR eT eV eX eb ec ee eg ei el en ep er et ev f2
f3 f8 fa fc ff fi fj fm fo fq fs fu fw fy g8 ga gd ge gg gi gk gn gp gr gt gv
gx h2 h3 h8 hB hD hG hH hJ hL hN hP hR hT hV hX iC iD iE iF iG iH j2 j3 jI jJ
jK jL jM jN jO jP kQ kR kS kT kU kV kW kX l2 l3 l6 l7 lb ld lf lh lj ll ln lp
lr lt lv lx n2 n3 n4 n5 n6 n7 n8 n9 o8 p2 p3 p4 p5 p6 p7 p8 p9 r2 r3 r4 r5 r6
r7 r9 t2 t3 t4 t5 t6 t7 t9 v2 v3 v4 v5 v6 v7 v9 w8 x2 x3 x4 x5 x6 x7 x9 y6 y7
y8 z2 z3 z4 z5 z6 z7 z9 | R-phycoerythrin β chain  
B2 b3 | LRC2  
C1 D1 | LRC4  
E1 F1 | LRC5  
G1 I1 K1 N1 P1 R1 T1 X1 Z1 b1 d1 f1 i1 k1 o1 q1 s1 u1 w1 y1 | alpha_APC  
H1 J1 L1 M1 O1 S1 U1 W1 Y1 a1 c1 e1 h1 j1 l1 n1 p1 r1 t1 v1 x1 z1 | beta_APC  
Q1 g1 | ApcF  
V1 m1 | ApcD  
V8 X8 Z8 a4 a5 aa ac ae ag ai ak b4 b5 ba bc be bg bi bk ca cc ce cg ci ck d4
d5 f4 f5 g9 i4 i5 i9 k4 k5 k9 | alpha_PC  
W8 Y8 ab ad af ah aj al bb bd bf bh bj bl c4 c5 cb cd cf ch cj cl e4 e5 f9 g4
g5 h4 h5 h9 j4 j5 j9 l4 l5 l9 | beta_PC  
a2 b2 | LCM  
a8 a9 | LRC3  
c2 d2 | LC  
dy dz | LR3  
eY eZ | LR_gamma7  
ew ex fd fz | LR2  
gy gz | LR_gamma6  
hY hZ ly lz | LR_gamma8  
ka kb | LR9  
  
Non-standard residues in 5y6p #1  
---  
CYC — phycocyanobilin  
PEB — phycoerythrobilin  
PUB — phycourobilin  
PVN — phycoviolobilin (Phycoviolobilin, bound form)  
  
command time 28.53 seconds  
draw time 0.1007 seconds  

> select ligand

88064 atoms, 94225 bonds, 2048 residues, 1 model selected  

> select all

1234795 atoms, 1257007 bonds, 4 pseudobonds, 155291 residues, 2 models
selected  

> select clear

> delete ligand

> save test.cif




OpenGL version: 4.1 ATI-3.10.18
OpenGL renderer: AMD Radeon Pro Vega 20 OpenGL Engine
OpenGL vendor: ATI Technologies Inc.Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro15,3
      Processor Name: 8-Core Intel Core i9
      Processor Speed: 2.4 GHz
      Number of Processors: 1
      Total Number of Cores: 8
      L2 Cache (per Core): 256 KB
      L3 Cache: 16 MB
      Hyper-Threading Technology: Enabled
      Memory: 32 GB
      Boot ROM Version: 1037.147.4.0.0 (iBridge: 17.16.16610.0.0,0)

Software:

    System Software Overview:

      System Version: macOS 10.15.7 (19H2)
      Kernel Version: Darwin 19.6.0
      Time since boot: 33 days 20:03

Graphics/Displays:

    Intel UHD Graphics 630:

      Chipset Model: Intel UHD Graphics 630
      Type: GPU
      Bus: Built-In
      VRAM (Dynamic, Max): 1536 MB
      Vendor: Intel
      Device ID: 0x3e9b
      Revision ID: 0x0002
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1

    Radeon Pro Vega 20:

      Chipset Model: Radeon Pro Vega 20
      Type: GPU
      Bus: PCIe
      PCIe Lane Width: x8
      VRAM (Total): 4 GB
      Vendor: AMD (0x1002)
      Device ID: 0x69af
      Revision ID: 0x00c0
      ROM Revision: 113-D2060I-087
      VBIOS Version: 113-D20601MA0T-016
      Option ROM Version: 113-D20601MA0T-016
      EFI Driver Version: 01.01.087
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1
      Displays:
        Color LCD:
          Display Type: Built-In Retina LCD
          Resolution: 2880 x 1800 Retina
          Framebuffer Depth: 24-Bit Color (ARGB8888)
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: No
          Connection Type: Internal

PyQt version: 5.15.1
Compiled Qt version: 5.15.1
Runtime Qt version: 5.15.1
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    appnope: 0.1.0
    Babel: 2.8.0
    backcall: 0.2.0
    blockdiag: 2.0.1
    certifi: 2020.6.20
    cftime: 1.2.1
    chardet: 3.0.4
    ChimeraX-AddH: 2.1.1
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.1
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.9.3
    ChimeraX-AtomicLibrary: 1.0
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AtomSearchLibrary: 1.0
    ChimeraX-AxesPlanes: 2.0
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 1.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.2
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.0
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.1
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-CommandLine: 1.1.3
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.2.dev202010140223
    ChimeraX-CoreFormats: 1.0
    ChimeraX-coulombic: 1.0.1
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-DataFormats: 1.0
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1
    ChimeraX-DistUI: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.0
    ChimeraX-Hbonds: 2.1
    ChimeraX-Help: 1.0
    ChimeraX-HKCage: 1.2
    ChimeraX-IHM: 1.0
    ChimeraX-ImageFormats: 1.0
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0
    ChimeraX-Label: 1.0
    ChimeraX-ListInfo: 1.0
    ChimeraX-Log: 1.1.1
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Map: 1.0.1
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.0
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 1.1
    ChimeraX-MDcrds: 2.1
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.0
    ChimeraX-mmCIF: 2.2
    ChimeraX-MMTF: 2.0
    ChimeraX-Modeller: 1.0
    ChimeraX-ModelPanel: 1.0
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.0
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0
    ChimeraX-OpenCommand: 1.2.1
    ChimeraX-PDB: 2.1.1
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0
    ChimeraX-PubChem: 2.0.1
    ChimeraX-Read-Pbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.0
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.2
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SeqView: 2.2.1
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.0
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.0.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.1
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.0
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.0
    ChimeraX-ToolshedUtils: 1.0
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.3.1
    ChimeraX-uniprot: 2.0
    ChimeraX-ViewDockX: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.3
    comtypes: 1.1.7
    cxservices: 1.0
    cycler: 0.10.0
    Cython: 0.29.21
    decorator: 4.4.2
    distlib: 0.3.1
    docutils: 0.16
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    h5py: 2.10.0
    html2text: 2020.1.16
    idna: 2.10
    ihm: 0.17
    imagecodecs: 2020.5.30
    imagecodecs-lite: 2020.1.31
    imagesize: 1.2.0
    ipykernel: 5.3.4
    ipython: 7.18.1
    ipython-genutils: 0.2.0
    jedi: 0.17.2
    Jinja2: 2.11.2
    jupyter-client: 6.1.7
    jupyter-core: 4.6.3
    kiwisolver: 1.2.0
    line-profiler: 2.1.2
    lxml: 4.5.2
    MarkupSafe: 1.1.1
    matplotlib: 3.3.2
    msgpack: 1.0.0
    netCDF4: 1.5.4
    netifaces: 0.10.9
    networkx: 2.5
    numexpr: 2.7.1
    numpy: 1.19.2
    numpydoc: 1.1.0
    openvr: 1.12.501
    packaging: 20.4
    parso: 0.7.1
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 7.2.0
    pip: 20.2.3
    pkginfo: 1.5.0.1
    prompt-toolkit: 3.0.7
    psutil: 5.7.2
    ptyprocess: 0.6.0
    pycollada: 0.7.1
    pydicom: 2.0.0
    Pygments: 2.7.1
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PyQt5-commercial: 5.15.1
    PyQt5-sip: 12.8.1
    PyQtWebEngine-commercial: 5.15.1
    python-dateutil: 2.8.1
    pytz: 2020.1
    pyzmq: 19.0.2
    qtconsole: 4.7.7
    QtPy: 1.9.0
    RandomWords: 0.3.0
    requests: 2.24.0
    scipy: 1.5.2
    Send2Trash: 1.5.0
    SEQCROW: 0.18
    setuptools: 50.3.0
    sfftk-rw: 0.6.6.dev0
    six: 1.15.0
    snowballstemmer: 2.0.0
    sortedcontainers: 2.2.2
    Sphinx: 3.2.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 1.0.3
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.4
    suds-jurko: 0.6
    tables: 3.6.1
    tifffile: 2020.9.3
    tinyarray: 1.2.3
    tornado: 6.0.4
    traitlets: 5.0.4
    urllib3: 1.25.10
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.34.2

[Greg Couch]

If you're willing to give up fast reading of the mmCIF file, you can give the "fixedWidth false" keyword argument when saving a mmCIF file. But that only improves it a little bit, < 10%. I got a 6% speedup for 5xnl, which has 98986 atoms and 56 chains, for 29 seconds instead of 31. This request is for making it an order of magnitude or more faster. Will probably need to rewrite parts in C++.

[Greg Couch]

Speed up mmCIF code by using Cython. Mostly with "cdef class CIFTable" and by implementing string quoting in C++. The mmCIF writing code is compiled with Cython, but has no Cython optimizations