Read EMAN format: cannot load .dylibs/libhdf5.200.dylib

[peter.kirchweger@…]

The following bug report has been submitted:
Platform:        macOS-10.16-x86_64-i386-64bit
ChimeraX Version: 1.3.dev202110160902 (2021-10-16 09:02:36 UTC)
Description
Dear ChimeraX developers,  I ran into a problem with importing a Segmentation map produced in EMAN.  While the file opend up in ChimeraX 1.2.5,  the bug was found in version 1.3.dev202110160902 (2021-10-16.   

Log:
UCSF ChimeraX version: 1.3.dev202110160902 (2021-10-16)  
© 2016-2021 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open
> /Users/peter/Desktop/LactB_proj/decon-s0.01_nc200_lams100-PK72_2_tomo1_wbp_rec_inv_scaled.mrc
> format mrc

Opened decon-s0.01_nc200_lams100-PK72_2_tomo1_wbp_rec_inv_scaled.mrc as #1,
grid size 1024,1024,341, pixel 28.9,28.9,28.9, shown at step 1, values float32  

> open
> /Users/peter/Desktop/LactB_proj/decon-s0.01_nc200_lams100-PK72_2_tomo1_wbp_rec_inv_scaled_preproc_merged_seg.hdf

Traceback (most recent call last):  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/open_command/dialog.py", line 155, in _qt_safe  
run(session, "open " + " ".join([FileNameArg.unparse(p) for p in paths]) + (""  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/core/commands/run.py", line 36, in run  
results = command.run(text, log=log, return_json=return_json)  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/core/commands/cli.py", line 2856, in run  
result = ci.function(session, **kw_args)  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/open_command/cmd.py", line 118, in cmd_open  
models = Command(session, registry=registry).run(provider_cmd_text,
log=log)[0]  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/core/commands/cli.py", line 2856, in run  
result = ci.function(session, **kw_args)  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/open_command/cmd.py", line 168, in provider_open  
models, status = collated_open(session, None, paths, data_format, _add_models,
log_errors,  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/open_command/cmd.py", line 432, in collated_open  
return remember_data_format()  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/open_command/cmd.py", line 403, in remember_data_format  
models, status = func(*func_args, **func_kw)  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/map/__init__.py", line 151, in open  
return open_map(session, path, format=_name, **kw)  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/map/volume.py", line 3568, in open_map  
grids = open_file(path, file_type = format, log = session.logger, **kw)  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/map_data/fileformats.py", line 163, in open_file  
data.extend(open_func(p, **okw))  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/map_data/emanhdf/__init__.py", line 18, in open  
return read_eman_hdf5(path, array_name = array_name)  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/map_data/emanhdf/eman_hdf_grid.py", line 48, in
read_eman_hdf5  
d = EMAN_HDF_Data(path)  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/chimerax/map_data/emanhdf/eman_hdf_format.py", line 55, in __init__  
import warnings, tables  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/tables/__init__.py", line 99, in <module>  
from .utilsextension import (  
ImportError:
dlopen(/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/tables/utilsextension.cpython-39-darwin.so, 2): Library not loaded:
.dylibs/libhdf5.200.dylib  
Referenced from:
/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/tables/utilsextension.cpython-39-darwin.so  
Reason: image not found  
  
ImportError:
dlopen(/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/tables/utilsextension.cpython-39-darwin.so, 2): Library not loaded:
.dylibs/libhdf5.200.dylib  
Referenced from:
/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/tables/utilsextension.cpython-39-darwin.so  
Reason: image not found  
  
File
"/Applications/Peters_Programme/ChimeraX_Daily.app/Contents/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/tables/__init__.py", line 99, in  
from .utilsextension import (  
  
See log for complete Python traceback.  
  




OpenGL version: 4.1 ATI-4.6.20
OpenGL renderer: AMD Radeon Pro 580 OpenGL Engine
OpenGL vendor: ATI Technologies Inc.Hardware:

    Hardware Overview:

      Model Name: iMac
      Model Identifier: iMac18,3
      Processor Name: Quad-Core Intel Core i7
      Processor Speed: 4,2 GHz
      Number of Processors: 1
      Total Number of Cores: 4
      L2 Cache (per Core): 256 KB
      L3 Cache: 8 MB
      Hyper-Threading Technology: Enabled
      Memory: 32 GB
      System Firmware Version: 429.140.8.0.0
      SMC Version (system): 2.41f2

Software:

    System Software Overview:

      System Version: macOS 11.6 (20G165)
      Kernel Version: Darwin 20.6.0
      Time since boot: 13:36

Graphics/Displays:

    Radeon Pro 580:

      Chipset Model: Radeon Pro 580
      Type: GPU
      Bus: PCIe
      PCIe Lane Width: x16
      VRAM (Total): 8 GB
      Vendor: AMD (0x1002)
      Device ID: 0x67df
      Revision ID: 0x00c0
      ROM Revision: 113-D000AA-931
      VBIOS Version: 113-D0001A1X-025
      EFI Driver Version: 01.00.931
      Metal Family: Supported, Metal GPUFamily macOS 2
      Displays:
        iMac:
          Display Type: Built-In Retina LCD
          Resolution: Retina 5K (5120 x 2880)
          Framebuffer Depth: 30-Bit Color (ARGB2101010)
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: No
          Connection Type: Internal

Locale: (None, 'UTF-8')
PyQt5 5.15.2, Qt 5.15.2
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    appnope: 0.1.2
    Babel: 2.9.1
    backcall: 0.2.0
    blockdiag: 2.0.1
    certifi: 2021.5.30
    cftime: 1.5.1
    charset-normalizer: 2.0.7
    ChimeraX-AddCharge: 1.1.4
    ChimeraX-AddH: 2.1.10
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.2.3
    ChimeraX-AlphaFold: 1.0
    ChimeraX-AltlocExplorer: 1.0.1
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.30.2
    ChimeraX-AtomicLibrary: 4.1.4
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AtomSearchLibrary: 1.0
    ChimeraX-AxesPlanes: 2.1
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 2.0
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.6
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.1
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.2
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.1.1
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.5
    ChimeraX-CommandLine: 1.1.5
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.3.dev202110160902
    ChimeraX-CoreFormats: 1.1
    ChimeraX-coulombic: 1.3.1
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0
    ChimeraX-DataFormats: 1.2.2
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1.5
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.1
    ChimeraX-Hbonds: 2.1.2
    ChimeraX-Help: 1.2
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.2
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ItemsInspection: 1.0
    ChimeraX-Label: 1.1
    ChimeraX-ListInfo: 1.1.1
    ChimeraX-Log: 1.1.4
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.1
    ChimeraX-Map: 1.1
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 2.0.3
    ChimeraX-MDcrds: 2.6
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.1
    ChimeraX-mmCIF: 2.4
    ChimeraX-MMTF: 2.1
    ChimeraX-Modeller: 1.2.2
    ChimeraX-ModelPanel: 1.2
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.1
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0.2
    ChimeraX-OpenCommand: 1.7
    ChimeraX-PDB: 2.6.4
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0.2
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0.1
    ChimeraX-PubChem: 2.1
    ChimeraX-ReadPbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.5
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.4.5
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.1
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.6
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-StructMeasure: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.1
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.1
    ChimeraX-ToolshedUtils: 1.2
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.13.6
    ChimeraX-uniprot: 2.2
    ChimeraX-UnitCell: 1.0
    ChimeraX-ViewDockX: 1.0.1
    ChimeraX-VIPERdb: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.4
    cxservices: 1.1
    cycler: 0.10.0
    Cython: 0.29.24
    decorator: 5.1.0
    docutils: 0.17.1
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    h5py: 3.4.0
    html2text: 2020.1.16
    idna: 3.3
    ihm: 0.21
    imagecodecs: 2021.4.28
    imagesize: 1.2.0
    ipykernel: 5.5.5
    ipython: 7.23.1
    ipython-genutils: 0.2.0
    jedi: 0.18.0
    Jinja2: 3.0.1
    jupyter-client: 6.1.12
    jupyter-core: 4.8.1
    kiwisolver: 1.3.2
    lxml: 4.6.3
    lz4: 3.1.3
    MarkupSafe: 2.0.1
    matplotlib: 3.4.3
    matplotlib-inline: 0.1.3
    msgpack: 1.0.2
    netCDF4: 1.5.7
    networkx: 2.6.3
    numexpr: 2.7.3
    numpy: 1.21.2
    openvr: 1.16.801
    packaging: 21.0
    ParmEd: 3.2.0
    parso: 0.8.2
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 8.3.2
    pip: 21.2.4
    pkginfo: 1.7.1
    prompt-toolkit: 3.0.20
    psutil: 5.8.0
    ptyprocess: 0.7.0
    pycollada: 0.7.1
    pydicom: 2.1.2
    Pygments: 2.10.0
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PyQt5-commercial: 5.15.2
    PyQt5-sip: 12.8.1
    PyQtWebEngine-commercial: 5.15.2
    python-dateutil: 2.8.2
    pytz: 2021.3
    pyzmq: 22.3.0
    qtconsole: 5.1.1
    QtPy: 1.11.2
    RandomWords: 0.3.0
    requests: 2.26.0
    scipy: 1.7.1
    setuptools: 57.5.0
    sfftk-rw: 0.7.1
    six: 1.16.0
    snowballstemmer: 2.1.0
    sortedcontainers: 2.4.0
    Sphinx: 4.2.0
    sphinx-autodoc-typehints: 1.12.0
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 2.0.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.5
    suds-jurko: 0.6
    tifffile: 2021.4.8
    tinyarray: 1.2.3
    tornado: 6.1
    traitlets: 5.1.0
    urllib3: 1.26.7
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.37.0
    wheel-filename: 1.3.0

[peter.kirchweger@…]

Dear Chimera-developers,

I should add to the bug-report, the data was 32bit.
Maybe this cause the crash?

Best,
Peter
--------------------------------------------
Fass Lab
Postdoctoral Fellow
Department of Structural Biology
Weizmann Institute of Science
Herzl Street, 760001 Rehovot
ISRAEL
​--------------------------------------------

Am 18.10.2021 um 22:02 schrieb ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu<mailto:ChimeraX-bugs-admin@cgl.ucsf.edu>>:

#5427: Read EMAN format: cannot load .dylibs/libhdf5.200.dylib
-----------------------------------------+-------------------------
         Reporter:  peter.kirchweger@…  |      Owner:  Tom Goddard
             Type:  defect              |     Status:  assigned
         Priority:  normal              |  Milestone:
        Component:  Volume Data         |    Version:
       Resolution:                      |   Keywords:
       Blocked By:                      |   Blocking:
Notify when closed:                      |   Platform:  all
          Project:  ChimeraX            |
-----------------------------------------+-------------------------
Changes (by Eric Pettersen):

* status:  new => assigned
* cc: chimera-programmers (added)
* component:  Unassigned => Volume Data
* project:   => ChimeraX
* platform:   => all
* owner:  (none) => Tom Goddard


--
Ticket URL: <https://www.rbvi.ucsf.edu/trac/ChimeraX/ticket/5427#comment:1>
ChimeraX <https://www.rbvi.ucsf.edu/chimerax/>
ChimeraX Issue Tracker

[goddard@…]

Thanks.  The error is reading all HDF5 files.  It is specific to the Mac and possibly only macOS Big Sur and newer (don't have Catalina to test).  The problem seems to be not finding some shared library.  Unfortunately the PyTables python module we use to read HDF5 is no longer well supported on Mac and Windows -- they no longer provide distributions, so we need to compile it and all its required libraries specially for ChimeraX which is a lot of work.  I am looking into simpler solutions.

[Tom Goddard]

Fixed.

I recompiled the python HDF5 library PyTables on macOS and it worked in my tests on macOS 11.6 (Big Sur). Would be good to test it on macOS 10.15 Catalina but I don't have a machine with that older OS. I compiled with MACOSX_DEPLOYMENT_TARGET=10.13 so I believe it will work for older macOS. But the hdf5 C++ library and other dependencies came from homebrew so I don't know if they work with older macOS.

[peter.kirchweger@…]

Thank you!
I will try again with the new daily build!

best,
Peter

Dr. Peter Kirchweger
​-------------------------------------------------------------
Fass Lab
Postdoctoral fellow
Department of Chemical and Structural Biology
Weizmann Institute of Science
Herzl Street, 760001 Rehovot
ISRAEL
​-------------------------------------------------------------

________________________________
From: ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
Sent: Tuesday, October 19, 2021 2:12 AM
Cc: chimera-programmers@cgl.ucsf.edu <chimera-programmers@cgl.ucsf.edu>; goddard@cgl.ucsf.edu <goddard@cgl.ucsf.edu>; Peter Kirchweger <peter.kirchweger@weizmann.ac.il>
Subject: Re: [ChimeraX] #5427: Read EMAN format: cannot load .dylibs/libhdf5.200.dylib

#5427: Read EMAN format: cannot load .dylibs/libhdf5.200.dylib
-----------------------------------------+-------------------------
          Reporter:  peter.kirchweger@…  |      Owner:  Tom Goddard
              Type:  defect              |     Status:  assigned
          Priority:  normal              |  Milestone:
         Component:  Volume Data         |    Version:
        Resolution:                      |   Keywords:
        Blocked By:                      |   Blocking:
Notify when closed:                      |   Platform:  all
           Project:  ChimeraX            |
-----------------------------------------+-------------------------

Comment (by Tom Goddard):

 Fixed.

 I recompiled the python HDF5 library PyTables on macOS and it worked in my
 tests on macOS 11.6 (Big Sur).  Would be good to test it on macOS 10.15
 Catalina but I don't have a machine with that older OS.  I compiled with
 MACOSX_DEPLOYMENT_TARGET=10.13 so I believe it will work for older macOS.
 But the hdf5 C++ library and other dependencies came from homebrew so I
 don't know if they work with older macOS.

--
Ticket URL: <https://www.rbvi.ucsf.edu/trac/ChimeraX/ticket/5427#comment:4>
ChimeraX <https://www.rbvi.ucsf.edu/chimerax/>
ChimeraX Issue Tracker

[Tom Goddard]

Presumed fixed. Tested on macOS Big Sur but not tested on macOS Catalina.