Opened 8 months ago

Closed 8 months ago

#17113 closed defect (fixed)

Alphafold predictions broken by Google Colab update to numpy 2

Reported by: goddard@… Owned by: Tom Goddard
Priority: high Milestone:
Component: Structure Prediction Version:
Keywords: Cc:
Blocked By: Blocking:
Notify when closed: Platform: all
Project: ChimeraX

Description

The following bug report has been submitted:
Platform:        macOS-15.3.2-arm64-arm-64bit
ChimeraX Version: 1.10.dev202502060236 (2025-02-06 02:36:00 UTC)
Description
AlphaFold predictions are broken because google colab updated to numpy 2 today.  The colabfold install uninstalls numpy 2.0.2 and installs 1.26.x.  It fails once the prediction tries to import colabfold which then tries to import from a numpy submodule numpy.random.mtrand.RandomState  which failse with "ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject" apparently because both part of numpy 2 and part of numpy 1 have been imported.

Log:
Startup Messages  
---  
note | available bundle cache has not been initialized yet  
  
UCSF ChimeraX version: 1.10.dev202502060236 (2025-02-06)  
© 2016-2025 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open 1hho format mmcif fromDatabase pdb

1hho title:  
Structure of human oxyhaemoglobin At 2.1 angstroms resolution [more info...]  
  
Chain information for 1hho #1  
---  
Chain | Description | UniProt  
A | HEMOGLOBIN A (OXY) (ALPHA CHAIN) | HBA_HUMAN 1-141  
B | HEMOGLOBIN A (OXY) (BETA CHAIN) | HBB_HUMAN 1-146  
  
Non-standard residues in 1hho #1  
---  
HEM — protoporphyrin IX containing Fe (HEME)  
OXY — oxygen molecule  
PO4 — phosphate ion  
  
1hho mmCIF Assemblies  
---  
1| author_and_software_defined_assembly  
  

> ui tool show AlphaFold

> alphafold predict #1/A

Please cite ColabFold: Making protein folding accessible to all. Nature
Methods (2022) if you use these predictions.  
Running AlphaFold prediction  

Populating font family aliases took 83 ms. Replace uses of missing font family
"Monospace" with one that exists to avoid this cost.  




OpenGL version: 4.1 Metal - 89.3
OpenGL renderer: Apple M1 Max
OpenGL vendor: Apple

Python: 3.11.4
Locale: UTF-8
Qt version: PyQt6 6.7.1, Qt 6.7.1
Qt runtime version: 6.7.3
Qt platform: cocoa
Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro18,2
      Model Number: MK1H3LL/A
      Chip: Apple M1 Max
      Total Number of Cores: 10 (8 performance and 2 efficiency)
      Memory: 32 GB
      System Firmware Version: 11881.81.4
      OS Loader Version: 11881.81.4

Software:

    System Software Overview:

      System Version: macOS 15.3.2 (24D81)
      Kernel Version: Darwin 24.3.0
      Time since boot: 19 hours, 13 minutes

Graphics/Displays:

    Apple M1 Max:

      Chipset Model: Apple M1 Max
      Type: GPU
      Bus: Built-In
      Total Number of Cores: 32
      Vendor: Apple (0x106b)
      Metal Support: Metal 3
      Displays:
        Color LCD:
          Display Type: Built-in Liquid Retina XDR Display
          Resolution: 3456 x 2234 Retina
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: No
          Connection Type: Internal


Installed Packages:
    alabaster: 1.0.0
    anyio: 4.8.0
    appdirs: 1.4.4
    appnope: 0.1.4
    asttokens: 3.0.0
    auditwheel: 6.2.0
    babel: 2.17.0
    beautifulsoup4: 4.12.3
    blockdiag: 3.0.0
    blosc2: 3.0.0
    build: 1.2.1
    certifi: 2023.11.17
    cftime: 1.6.4.post1
    charset-normalizer: 3.4.1
    ChimeraX-AddCharge: 1.5.18
    ChimeraX-AddH: 2.2.6
    ChimeraX-AlignmentAlgorithms: 2.0.2
    ChimeraX-AlignmentHdrs: 3.6
    ChimeraX-AlignmentMatrices: 2.1
    ChimeraX-Alignments: 2.18.1
    ChimeraX-AlphaFold: 1.0.1
    ChimeraX-AltlocExplorer: 1.1.2
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Aniso: 1.0
    ChimeraX-Arrays: 1.1
    ChimeraX-Atomic: 1.60.5
    ChimeraX-AtomicLibrary: 14.1.13
    ChimeraX-AtomSearch: 2.0.1
    ChimeraX-AxesPlanes: 2.4
    ChimeraX-BasicActions: 1.1.3
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 3.0.0
    ChimeraX-BondRot: 2.0.4
    ChimeraX-BugReporter: 1.0.2
    ChimeraX-BuildStructure: 2.13.1
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.4.0
    ChimeraX-ButtonPanel: 1.0.1
    ChimeraX-CageBuilder: 1.0.1
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.4
    ChimeraX-ChangeChains: 1.1
    ChimeraX-CheckWaters: 1.4
    ChimeraX-ChemGroup: 2.0.1
    ChimeraX-Clashes: 2.3
    ChimeraX-ColorActions: 1.0.5
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.5.8
    ChimeraX-CommandLine: 1.2.5
    ChimeraX-ConnectStructure: 2.0.1
    ChimeraX-Contacts: 1.0.1
    ChimeraX-Core: 1.10.dev202502060236
    ChimeraX-CoreFormats: 1.2
    ChimeraX-coulombic: 1.4.4
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0.1
    ChimeraX-DataFormats: 1.2.3
    ChimeraX-Dicom: 1.2.7
    ChimeraX-DistMonitor: 1.4.2
    ChimeraX-DockPrep: 1.1.3
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ESMFold: 1.0
    ChimeraX-FileHistory: 1.0.1
    ChimeraX-FunctionKey: 1.0.1
    ChimeraX-Geometry: 1.3
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.4.1
    ChimeraX-Hbonds: 2.5
    ChimeraX-Help: 1.3
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.2
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.3
    ChimeraX-ItemsInspection: 1.0.1
    ChimeraX-IUPAC: 1.0
    ChimeraX-KVFinder: 1.5.1
    ChimeraX-Label: 1.1.14
    ChimeraX-ListInfo: 1.2.2
    ChimeraX-Log: 1.2
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.9.1
    ChimeraX-Map: 1.3
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0.1
    ChimeraX-MapFilter: 2.0.1
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1.1
    ChimeraX-Markers: 1.0.1
    ChimeraX-Mask: 1.0.2
    ChimeraX-MatchMaker: 2.1.7
    ChimeraX-MCopy: 1.0
    ChimeraX-MDcrds: 2.7.2
    ChimeraX-MedicalToolbar: 1.1
    ChimeraX-Meeting: 1.0.1
    ChimeraX-MLP: 1.1.1
    ChimeraX-mmCIF: 2.14.2
    ChimeraX-MMTF: 2.2
    ChimeraX-ModelArchive: 1.0
    ChimeraX-Modeller: 1.5.18
    ChimeraX-ModelPanel: 1.5
    ChimeraX-ModelSeries: 1.0.1
    ChimeraX-Mol2: 2.0.3
    ChimeraX-Mole: 1.0
    ChimeraX-Morph: 1.0.2
    ChimeraX-MouseModes: 1.2
    ChimeraX-Movie: 1.0
    ChimeraX-MutationScores: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nifti: 1.2
    ChimeraX-NMRSTAR: 1.0.2
    ChimeraX-NRRD: 1.2
    ChimeraX-Nucleotides: 2.0.3
    ChimeraX-OpenCommand: 1.14.1
    ChimeraX-OrthoPick: 1.0.1
    ChimeraX-PDB: 2.7.7
    ChimeraX-PDBBio: 1.0.1
    ChimeraX-PDBLibrary: 1.0.4
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0.1
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.1.2
    ChimeraX-ProfileGrids: 1.0.1
    ChimeraX-PubChem: 2.2
    ChimeraX-ReadPbonds: 1.0.1
    ChimeraX-Registration: 1.1.2
    ChimeraX-RemoteControl: 1.0
    ChimeraX-RenderByAttr: 1.6.3
    ChimeraX-RenumberResidues: 1.1
    ChimeraX-ResidueFit: 1.0.1
    ChimeraX-RestServer: 1.3.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 4.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.5.1
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0.3
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0.1
    ChimeraX-Segmentations: 3.5.7
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.16
    ChimeraX-Shape: 1.1
    ChimeraX-Shell: 1.0.1
    ChimeraX-Shortcuts: 1.2.0
    ChimeraX-ShowSequences: 1.0.3
    ChimeraX-SideView: 1.0.1
    ChimeraX-SimilarStructures: 1.0.1
    ChimeraX-Smiles: 2.1.2
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.19.1
    ChimeraX-STL: 1.0.1
    ChimeraX-Storm: 1.0
    ChimeraX-StructMeasure: 1.2.1
    ChimeraX-Struts: 1.0.1
    ChimeraX-Surface: 1.0.1
    ChimeraX-SwapAA: 2.0.1
    ChimeraX-SwapRes: 2.5.1
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-TaskManager: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.2.3
    ChimeraX-ToolshedUtils: 1.2.4
    ChimeraX-Topography: 1.0
    ChimeraX-ToQuest: 1.0
    ChimeraX-Tug: 1.0.1
    ChimeraX-UI: 1.43
    ChimeraX-Umap: 1.0
    ChimeraX-uniprot: 2.3.1
    ChimeraX-UnitCell: 1.0.1
    ChimeraX-ViewDockX: 1.4.4
    ChimeraX-VIPERdb: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0.1
    ChimeraX-vrml: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0.2
    ChimeraX-WebServices: 1.1.4
    ChimeraX-Zone: 1.0.1
    colorama: 0.4.6
    comm: 0.2.2
    contourpy: 1.3.1
    cxservices: 1.2.3
    cycler: 0.12.1
    Cython: 3.0.10
    debugpy: 1.8.12
    decorator: 5.1.1
    docutils: 0.21.2
    executing: 2.2.0
    filelock: 3.15.4
    fonttools: 4.55.8
    funcparserlib: 2.0.0a0
    glfw: 2.8.0
    grako: 3.16.5
    h11: 0.14.0
    h5py: 3.12.1
    html2text: 2024.2.26
    httpcore: 1.0.7
    httpx: 0.28.1
    idna: 3.10
    ihm: 1.3
    imagecodecs: 2024.6.1
    imagesize: 1.4.1
    ipykernel: 6.29.5
    ipython: 8.26.0
    ipywidgets: 8.1.5
    jedi: 0.19.1
    Jinja2: 3.1.5
    jupyter_client: 8.6.2
    jupyter_core: 5.7.2
    jupyterlab_widgets: 3.0.13
    kiwisolver: 1.4.8
    line_profiler: 4.1.3
    lxml: 5.2.2
    lz4: 4.3.3
    MarkupSafe: 3.0.2
    matplotlib: 3.9.2
    matplotlib-inline: 0.1.7
    msgpack: 1.0.8
    ndindex: 1.9.2
    nest-asyncio: 1.6.0
    netCDF4: 1.6.5
    networkx: 3.3
    nibabel: 5.2.0
    nptyping: 2.5.0
    numexpr: 2.10.2
    numpy: 1.26.4
    OpenMM: 8.2.0
    openvr: 1.26.701
    packaging: 23.2
    ParmEd: 4.2.2
    parso: 0.8.4
    pep517: 0.13.1
    pexpect: 4.9.0
    pickleshare: 0.7.5
    pillow: 10.4.0
    pip: 24.2
    pkginfo: 1.11.1
    platformdirs: 4.3.6
    prompt_toolkit: 3.0.50
    psutil: 6.0.0
    ptyprocess: 0.7.0
    pure_eval: 0.2.3
    py-cpuinfo: 9.0.0
    pycollada: 0.8
    pydicom: 2.4.4
    pyelftools: 0.31
    Pygments: 2.18.0
    pynmrstar: 3.3.5
    pynrrd: 1.0.0
    PyOpenGL: 3.1.7
    PyOpenGL-accelerate: 3.1.7
    pyopenxr: 1.0.3401
    pyparsing: 3.2.1
    pyproject_hooks: 1.2.0
    PyQt6: 6.7.1
    PyQt6-Qt6: 6.7.3
    PyQt6-WebEngine: 6.7.0
    PyQt6-WebEngine-Qt6: 6.7.3
    PyQt6-WebEngineSubwheel-Qt6: 6.7.3
    PyQt6_sip: 13.8.0
    python-dateutil: 2.9.0.post0
    pytz: 2025.1
    pyzmq: 26.2.1
    qtconsole: 5.5.2
    QtPy: 2.4.2
    qtshim: 1.1
    RandomWords: 0.4.0
    requests: 2.32.3
    scipy: 1.14.0
    setuptools: 72.1.0
    sfftk-rw: 0.8.1
    six: 1.16.0
    sniffio: 1.3.1
    snowballstemmer: 2.2.0
    sortedcontainers: 2.4.0
    soupsieve: 2.6
    Sphinx: 8.0.2
    sphinx-autodoc-typehints: 2.2.3
    sphinxcontrib-applehelp: 2.0.0
    sphinxcontrib-blockdiag: 3.0.0
    sphinxcontrib-devhelp: 2.0.0
    sphinxcontrib-htmlhelp: 2.1.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 2.0.0
    sphinxcontrib-serializinghtml: 2.0.0
    stack-data: 0.6.3
    superqt: 0.6.3
    tables: 3.10.2
    tcia_utils: 1.5.1
    tifffile: 2024.7.24
    tinyarray: 1.2.4
    tornado: 6.4.2
    traitlets: 5.14.3
    typing_extensions: 4.12.2
    tzdata: 2025.1
    urllib3: 2.3.0
    wcwidth: 0.2.13
    webcolors: 24.6.0
    wheel: 0.43.0
    wheel-filename: 1.4.1
    widgetsnbextension: 4.0.13

Change History (7)

comment:1 by Tom Goddard, 8 months ago

Component: UnassignedStructure Prediction
Owner: set to Tom Goddard
Platform: all
Priority: normalhigh
Project: ChimeraX
Status: newassigned
Summary: ChimeraX bug report submissionAlphafold predictions broken by Google Colab update to numpy 2

comment:2 by Tom Goddard, 8 months ago

The Colabfold notebook from the Colabfold project is also broken with the same error. I filed a bug report with the Colabfold project.

https://github.com/sokrypton/ColabFold/issues/693

comment:3 by Tom Goddard, 8 months ago

I was not able to find an announcement of the date of the Colab numpy 2 update online. But here is Colab the ticket saying it will be updated in "early March".

https://github.com/googlecolab/colabtools/issues/5115

comment:4 by Tom Goddard, 8 months ago

The Colab ticket about this update mentions a work-around where users can from the Google Colab web page Command menu choose "Use fallback runtime version" which will use the former numpy 1 runtime. I tried this after getting the error then pressed the play button on the cell and it allowed the prediction to complete successfully. The fallback setting only applies to the current Colab session.

comment:5 by Tom Goddard, 8 months ago

The Colabfold install requires numpy version < 2 and so it uninstalls the Colab default numpy 2.0.2 and installs 1.26.x. But apparently numpy is already imported so the newly installed version is not used. I tried to unload numpy from the current Python session using reload(numpy). That of course doesn't unload submodules so predictions still fail. So I also tried unloading all numpy submodules removing them from sys.modules. Prediction still fails in matplotlib. Possibly this is because matplotlib is a default Colab module that requires numpy 2 or maybe it is because matplotlib is already imported (e.g. by IPython) and holds references to the default numpy 2 even after I load numpy 1. I did not identify what this matplotlib problem was. I tried to unload matplotlib by removing it (and all its submodules) from sys.modules but I still get errors. That latest error was in numpy _umath_linalg.inv() being called with an extobj keyword argument and says inv has no such argument. numpy 1 has that argument, so apparently the _umath_linalg submodule is still numpy 2. Possibly the dyld loaded modules even after being removed from sys.modules don't end up reloading the new version since the shared is already in the process address space -- just a guess.

comment:6 by Tom Goddard, 8 months ago

I added a status page indicating that the ChimeraX Alphafold service is broken

https://www.rbvi.ucsf.edu/chimerax/data/status/alphafold23.html

This page is shown whenever the user tries to run a prediction and it prevents running it. I think so few users will know about the "Use fallback runtime version" trick it is better to give all users a clear message that the service is down.

comment:7 by Tom Goddard, 8 months ago

Resolution: fixed
Status: assignedclosed

Fixed.

Colabfold developer Milot Mirdita updated Colabfold to work with numpy 2. I tested it on a few cases (monomer, dimer with energy minimization and templates) in ChimeraX and it worked so I have updated the ChimeraX Colab script to use it and removed the prediction broken error message.

Note: See TracTickets for help on using tickets.