Read CHAI-1 PAE .npy files

[Tristan Croll]

The following bug report has been submitted:
Platform:        Windows-10-10.0.22631
ChimeraX Version: 1.8 (2024-06-10 23:15:52 UTC)
Description
Yet another new structure-prediction tool on the block (CHAI-1 - https://www.chaidiscovery.com/blog/introducing-chai-1), with yet another choice of format for the PAE files. In this case a straightforward dump of the PAE matrix to .npy. The below snippet was sufficient to convert them to JSON files I could load using the existing ChimeraX 1.8 plugin, but thought you might want to consider adding built-in support.

def pae_to_json():
    import numpy,json,glob
    for file in glob.glob('*.npy'):
        with open(file, 'rb') as infile:
            pae = numpy.load(infile)
            with open(file.replace('npy','json'), 'wt') as outfile:
                json.dump({'pae':pae.tolist()}, outfile)

OpenGL version: 3.3.0 NVIDIA 556.12
OpenGL renderer: NVIDIA GeForce RTX 3070 Laptop GPU/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation

Python: 3.11.4
Locale: en_GB.cp1252
Qt version: PyQt6 6.6.1, Qt 6.6.1
Qt runtime version: 6.6.3
Qt platform: windows

Manufacturer: HP
Model: HP ZBook Studio 15.6 inch G8 Mobile Workstation PC
OS: Microsoft Windows 11 Pro (Build 22631)
Memory: 34,007,068,672
MaxProcessMemory: 137,438,953,344
CPU: 16 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
OSLanguage: en-GB

Installed Packages:
    alabaster: 0.7.16
    appdirs: 1.4.4
    asteval: 1.0.5
    asttokens: 2.4.1
    autograd: 1.7.0
    Babel: 2.15.0
    beautifulsoup4: 4.12.3
    blockdiag: 3.0.0
    blosc2: 2.0.0
    build: 1.2.1
    certifi: 2024.6.2
    cftime: 1.6.4
    charset-normalizer: 3.3.2
    ChimeraX-AddCharge: 1.5.17
    ChimeraX-AddH: 2.2.6
    ChimeraX-AlignmentAlgorithms: 2.0.2
    ChimeraX-AlignmentHdrs: 3.5
    ChimeraX-AlignmentMatrices: 2.1
    ChimeraX-Alignments: 2.12.7
    ChimeraX-AlphaFold: 1.0
    ChimeraX-AltlocExplorer: 1.1.1
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.1
    ChimeraX-Atomic: 1.57.1
    ChimeraX-AtomicLibrary: 14.0.6
    ChimeraX-AtomSearch: 2.0.1
    ChimeraX-AxesPlanes: 2.4
    ChimeraX-BasicActions: 1.1.2
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 2.4.6
    ChimeraX-BondRot: 2.0.4
    ChimeraX-BugReporter: 1.0.1
    ChimeraX-BuildStructure: 2.12.1
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.2.7
    ChimeraX-ButtonPanel: 1.0.1
    ChimeraX-CageBuilder: 1.0.1
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.4
    ChimeraX-ChangeChains: 1.1
    ChimeraX-CheckWaters: 1.4
    ChimeraX-ChemGroup: 2.0.1
    ChimeraX-Clashes: 2.2.4
    ChimeraX-Clipper: 0.23.1
    ChimeraX-ColorActions: 1.0.5
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.5.6
    ChimeraX-CommandLine: 1.2.5
    ChimeraX-ConnectStructure: 2.0.1
    ChimeraX-Contacts: 1.0.1
    ChimeraX-Core: 1.8
    ChimeraX-CoreFormats: 1.2
    ChimeraX-coulombic: 1.4.3
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0.1
    ChimeraX-DataFormats: 1.2.3
    ChimeraX-Dicom: 1.2.4
    ChimeraX-DiffPlot: 1.0
    ChimeraX-DistMonitor: 1.4.2
    ChimeraX-DockPrep: 1.1.3
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ESMFold: 1.0
    ChimeraX-FileHistory: 1.0.1
    ChimeraX-FunctionKey: 1.0.1
    ChimeraX-Geometry: 1.3
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.1.1
    ChimeraX-Hbonds: 2.4
    ChimeraX-Help: 1.2.2
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.2
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ISOLDE: 1.8
    ChimeraX-ItemsInspection: 1.0.1
    ChimeraX-IUPAC: 1.0
    ChimeraX-Label: 1.1.10
    ChimeraX-ListInfo: 1.2.2
    ChimeraX-Log: 1.1.6
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.9.1
    ChimeraX-Map: 1.2
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0.1
    ChimeraX-MapFilter: 2.0.1
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1.1
    ChimeraX-Markers: 1.0.1
    ChimeraX-Mask: 1.0.2
    ChimeraX-MatchMaker: 2.1.3
    ChimeraX-MCopy: 1.0
    ChimeraX-MDcrds: 2.7.1
    ChimeraX-MedicalToolbar: 1.0.3
    ChimeraX-Meeting: 1.0.1
    ChimeraX-MLP: 1.1.1
    ChimeraX-mmCIF: 2.14.1
    ChimeraX-MMTF: 2.2
    ChimeraX-Modeller: 1.5.17
    ChimeraX-ModelPanel: 1.5
    ChimeraX-ModelSeries: 1.0.1
    ChimeraX-Mol2: 2.0.3
    ChimeraX-Mole: 1.0
    ChimeraX-Morph: 1.0.2
    ChimeraX-MouseModes: 1.2
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nifti: 1.2
    ChimeraX-NMRSTAR: 1.0.2
    ChimeraX-NRRD: 1.2
    ChimeraX-Nucleotides: 2.0.3
    ChimeraX-OpenCommand: 1.13.5
    ChimeraX-PDB: 2.7.5
    ChimeraX-PDBBio: 1.0.1
    ChimeraX-PDBLibrary: 1.0.4
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0.1
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.1.1
    ChimeraX-PubChem: 2.2
    ChimeraX-QScore: 1.1
    ChimeraX-ReadPbonds: 1.0.1
    ChimeraX-Registration: 1.1.2
    ChimeraX-RemoteControl: 1.0
    ChimeraX-RenderByAttr: 1.4.1
    ChimeraX-RenumberResidues: 1.1
    ChimeraX-ResidueFit: 1.0.1
    ChimeraX-RestServer: 1.2
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 4.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.5.1
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0.2
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0.1
    ChimeraX-Segmentations: 3.0.15
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.11.2
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0.1
    ChimeraX-Shortcuts: 1.1.1
    ChimeraX-ShowSequences: 1.0.3
    ChimeraX-SideView: 1.0.1
    ChimeraX-Smiles: 2.1.2
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.16.5
    ChimeraX-STL: 1.0.1
    ChimeraX-Storm: 1.0
    ChimeraX-StructMeasure: 1.2.1
    ChimeraX-Struts: 1.0.1
    ChimeraX-Surface: 1.0.1
    ChimeraX-SwapAA: 2.0.1
    ChimeraX-SwapRes: 2.5
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-TaskManager: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.1.2
    ChimeraX-ToolshedUtils: 1.2.4
    ChimeraX-Topography: 1.0
    ChimeraX-ToQuest: 1.0
    ChimeraX-Tug: 1.0.1
    ChimeraX-UI: 1.39.1
    ChimeraX-uniprot: 2.3
    ChimeraX-UnitCell: 1.0.1
    ChimeraX-ViewDockX: 1.4.3
    ChimeraX-VIPERdb: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0.1
    ChimeraX-vrml: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0.2
    ChimeraX-WebServices: 1.1.4
    ChimeraX-Zone: 1.0.1
    colorama: 0.4.6
    comm: 0.2.2
    comtypes: 1.4.1
    contourpy: 1.2.1
    cxservices: 1.2.2
    cycler: 0.12.1
    Cython: 3.0.10
    debugpy: 1.8.1
    decorator: 5.1.1
    dill: 0.3.9
    docutils: 0.20.1
    executing: 2.0.1
    filelock: 3.13.4
    fonttools: 4.53.0
    funcparserlib: 2.0.0a0
    glfw: 2.7.0
    grako: 3.16.5
    h5py: 3.11.0
    html2text: 2024.2.26
    idna: 3.7
    ihm: 1.0
    imagecodecs: 2024.1.1
    imagesize: 1.4.1
    ipykernel: 6.29.2
    ipython: 8.21.0
    ipywidgets: 8.1.3
    jedi: 0.19.1
    jinja2: 3.1.4
    jupyter-client: 8.6.0
    jupyter-core: 5.7.2
    jupyterlab-widgets: 3.0.11
    kiwisolver: 1.4.5
    line-profiler: 4.1.2
    lmfit: 1.3.2
    lxml: 5.2.1
    lz4: 4.3.3
    MarkupSafe: 2.1.5
    matplotlib: 3.8.4
    matplotlib-inline: 0.1.7
    mpmath: 1.3.0
    msgpack: 1.0.8
    nest-asyncio: 1.6.0
    netCDF4: 1.6.5
    networkx: 3.3
    nibabel: 5.2.0
    nptyping: 2.5.0
    numexpr: 2.10.0
    numpy: 1.26.4
    openvr: 1.26.701
    packaging: 24.1
    ParmEd: 4.2.2
    parso: 0.8.4
    pep517: 0.13.1
    pillow: 10.3.0
    pip: 24.0
    pkginfo: 1.10.0
    platformdirs: 4.2.2
    prompt-toolkit: 3.0.47
    psutil: 5.9.8
    pure-eval: 0.2.2
    py-cpuinfo: 9.0.0
    pybindingcurve: 1.2.3
    pycollada: 0.8
    pydicom: 2.4.4
    pygments: 2.17.2
    pynmrstar: 3.3.4
    pynrrd: 1.0.0
    PyOpenGL: 3.1.7
    PyOpenGL-accelerate: 3.1.7
    pyopenxr: 1.0.3401
    pyparsing: 3.1.2
    pyproject-hooks: 1.1.0
    PyQt6-commercial: 6.6.1
    PyQt6-Qt6: 6.6.3
    PyQt6-sip: 13.6.0
    PyQt6-WebEngine-commercial: 6.6.0
    PyQt6-WebEngine-Qt6: 6.6.3
    python-dateutil: 2.9.0.post0
    pytz: 2024.1
    pywin32: 306
    pyzmq: 26.0.3
    qtconsole: 5.5.1
    QtPy: 2.4.1
    RandomWords: 0.4.0
    requests: 2.31.0
    scipy: 1.13.0
    Send2Trash: 1.8.3
    SEQCROW: 1.8.8
    setuptools: 69.5.1
    sfftk-rw: 0.8.1
    six: 1.16.0
    snowballstemmer: 2.2.0
    sortedcontainers: 2.4.0
    soupsieve: 2.5
    sphinx: 7.2.6
    sphinx-autodoc-typehints: 2.0.1
    sphinxcontrib-applehelp: 1.0.8
    sphinxcontrib-blockdiag: 3.0.0
    sphinxcontrib-devhelp: 1.0.6
    sphinxcontrib-htmlhelp: 2.0.5
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.7
    sphinxcontrib-serializinghtml: 1.1.10
    stack-data: 0.6.3
    superqt: 0.6.3
    tables: 3.8.0
    tcia-utils: 1.5.1
    tifffile: 2024.1.30
    tinyarray: 1.2.4
    tornado: 6.4.1
    traitlets: 5.14.2
    typing-extensions: 4.12.2
    tzdata: 2024.1
    uncertainties: 3.2.2
    urllib3: 2.2.1
    wcwidth: 0.2.13
    webcolors: 1.13
    wheel: 0.43.0
    wheel-filename: 1.4.1
    widgetsnbextension: 4.0.11
    WMI: 1.5.1
File attachment: Wnt3a-wntless_with_ptm_as_ligand.zip

Wnt3a-wntless_with_ptm_as_ligand.zip​

[Tristan Croll]

Added by email2trac

[Tom Goddard]

Thanks. The PAE .json files are very large and slow to load because they are big matrices in text, so I can understand chaii-1 going with .npy binary files. It looks easy to handle. The AlphaFold 3 PAE json files I think have useful info about which residues or atoms are used for each row. So far I'm not using that and just relying on the order following the mmCIF file. Looks like the .npy file omits that info which is a drawback.

[Tom Goddard]

Done in ChimeraX 1.9 and 1.10.

I made the AlphaFold Error Plot tool and the "alphafold pae" command read .npy PAE files produced by the Chai-1 AlphaFold 3 clone.