Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#8105 closed defect (fixed)

ChimeraX/lib not on library path

Reported by: Tristan Croll Owned by: Zach Pearson
Priority: normal Milestone:
Component: Build System Version:
Keywords: Cc: chimerax-programmers
Blocked By: Blocking:
Notify when closed: Platform: all
Project: ChimeraX

Description 

The following bug report has been submitted:
Platform:        Linux-5.15.0-56-generic-x86_64-with-glibc2.35
ChimeraX Version: 1.6.dev202211300249 (2022-11-30 02:49:47 UTC)
Description
Building ISOLDE against the 1.6 daily builds in Linux fails because ChimeraX/lib is no longer in the library path:

c++ -pthread -pipe -fPIC -std=gnu99 -fdebug-prefix-map=/tmp/chx-develop.qAG1Ud/build=. -fstack-protector-strong -I/tmp/chx-develop.qAG1Ud/build/include -L/tmp/chx-develop.qAG1Ud/build/lib -DUSE_DYLD_GLOBAL_NAMESPACE -pthread -shared -O3 -Wall -Wextra -D_FORTIFY=2 build/temp.linux-x86_64-cpython-39/src/deps/lbfgs/src/lbfgs.o build/temp.linux-x86_64-cpython-39/src/openmm/minimize.o build/temp.linux-x86_64-cpython-39/src/openmm/openmm_interface.o -L/usr/lib -L/usr/libexec/UCSF-ChimeraX-daily/lib/python3.9/site-packages/chimerax/core/lib -L/usr/libexec/UCSF-ChimeraX-daily/lib/python3.9/site-packages/chimerax/atomic_lib/lib -L/usr/libexec/UCSF-ChimeraX-daily/lib/python3.9/site-packages/chimerax/arrays/lib -L/tmp/chx-develop.qAG1Ud/build/lib -lOpenMM -o build/lib.linux-x86_64-cpython-39/chimerax/isolde/_openmm_async.cpython-39-x86_64-linux-gnu.so -Wl,-rpath,$ORIGIN
/opt/rh/gcc-toolset-10/root/usr/bin/ld: cannot find -lOpenMM
collect2: error: ld returned 1 exit status
error: command '/opt/rh/gcc-toolset-10/root/usr/bin/c++' failed with exit code 1
make: *** [Makefile:91: wheel] Error 1

Can hack around it for now in bundle_info.xml:

      <LibraryDir platform="linux">/usr/libexec/UCSF-ChimeraX-daily/lib</LibraryDir>

The Altos software eng team have promised to lend a contractor to the OpenMM team soon to help them with their migration to PyPI, so hopefully this fiddliness should be behind us soon.


OpenGL version: 3.3.0 NVIDIA 515.86.01
OpenGL renderer: NVIDIA GeForce RTX 3070/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation

Python: 3.9.11
Locale: en_GB.UTF-8
Qt version: PyQt6 6.3.1, Qt 6.3.1
Qt runtime version: 6.3.2
Qt platform: xcb

XDG_SESSION_TYPE=x11
DESKTOP_SESSION=ubuntu
XDG_SESSION_DESKTOP=ubuntu
XDG_CURRENT_DESKTOP=ubuntu:GNOME
DISPLAY=:1
Manufacturer: Dell Inc.
Model: XPS 8950
OS: Ubuntu 22.04 Jammy Jellyfish
Architecture: 64bit ELF
Virtual Machine: none
CPU: 20 12th Gen Intel(R) Core(TM) i7-12700
Cache Size: 25600 KB
Memory:
	               total        used        free      shared  buff/cache   available
	Mem:            31Gi       5.5Gi        13Gi       535Mi        12Gi        24Gi
	Swap:          2.0Gi          0B       2.0Gi

Graphics:
	0000:01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 3070 Lite Hash Rate] [10de:2488] (rev a1)	
	Subsystem: Dell GA104 [GeForce RTX 3070 Lite Hash Rate] [1028:c903]	
	Kernel driver in use: nvidia

Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    asttokens: 2.2.0
    Babel: 2.11.0
    backcall: 0.2.0
    blockdiag: 3.0.0
    build: 0.8.0
    certifi: 2022.9.24
    cftime: 1.6.2
    charset-normalizer: 2.1.1
    ChimeraX-AddCharge: 1.5.7
    ChimeraX-AddH: 2.2.3
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.3.1
    ChimeraX-AlignmentMatrices: 2.1
    ChimeraX-Alignments: 2.8
    ChimeraX-AlphaFold: 1.0
    ChimeraX-AltlocExplorer: 1.0.3
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.43.4
    ChimeraX-AtomicLibrary: 8.0.4
    ChimeraX-AtomSearch: 2.0.1
    ChimeraX-AxesPlanes: 2.3
    ChimeraX-BasicActions: 1.1.2
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 2.1.2
    ChimeraX-BondRot: 2.0.1
    ChimeraX-BugReporter: 1.0.1
    ChimeraX-BuildStructure: 2.7.1
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.2
    ChimeraX-ButtonPanel: 1.0.1
    ChimeraX-CageBuilder: 1.0.1
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.3.1
    ChimeraX-ChangeChains: 1.0.2
    ChimeraX-CheckWaters: 1.3
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.2.4
    ChimeraX-Clipper: 0.20.0
    ChimeraX-ColorActions: 1.0.3
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.5.2
    ChimeraX-CommandLine: 1.2.5
    ChimeraX-ConnectStructure: 2.0.1
    ChimeraX-Contacts: 1.0.1
    ChimeraX-Core: 1.6.dev202211300249
    ChimeraX-CoreFormats: 1.1
    ChimeraX-coulombic: 1.4.1
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0.1
    ChimeraX-DataFormats: 1.2.2
    ChimeraX-Dicom: 1.1
    ChimeraX-DistMonitor: 1.3
    ChimeraX-DockPrep: 1.0.1
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ESMFold: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0.1
    ChimeraX-FunctionKey: 1.0.1
    ChimeraX-Geometry: 1.2
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.1
    ChimeraX-Hbonds: 2.4
    ChimeraX-Help: 1.2.1
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.2
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ItemsInspection: 1.0.1
    ChimeraX-Label: 1.1.7
    ChimeraX-LinuxSupport: 1.0.1
    ChimeraX-ListInfo: 1.1.1
    ChimeraX-Log: 1.1.5
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.2
    ChimeraX-Map: 1.1.2
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0.1
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1.1
    ChimeraX-Markers: 1.0.1
    ChimeraX-Mask: 1.0.1
    ChimeraX-MatchMaker: 2.0.9
    ChimeraX-MDcrds: 2.6
    ChimeraX-MedicalToolbar: 1.0.2
    ChimeraX-Meeting: 1.0.1
    ChimeraX-MLP: 1.1
    ChimeraX-mmCIF: 2.9
    ChimeraX-MMTF: 2.2
    ChimeraX-Modeller: 1.5.7
    ChimeraX-ModelPanel: 1.3.6
    ChimeraX-ModelSeries: 1.0.1
    ChimeraX-Mol2: 2.0
    ChimeraX-Mole: 1.0
    ChimeraX-Morph: 1.0.1
    ChimeraX-MouseModes: 1.1.1
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0.3
    ChimeraX-OpenCommand: 1.9.3
    ChimeraX-PDB: 2.6.9
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0.2
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0.1
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.1
    ChimeraX-PubChem: 2.1
    ChimeraX-ReadPbonds: 1.0.1
    ChimeraX-Registration: 1.1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-RenderByAttr: 1.0
    ChimeraX-RenumberResidues: 1.1
    ChimeraX-ResidueFit: 1.0.1
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0.1
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.5.1
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0.1
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.8
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0.1
    ChimeraX-Shortcuts: 1.1.1
    ChimeraX-ShowSequences: 1.0.1
    ChimeraX-SideView: 1.0.1
    ChimeraX-Smiles: 2.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.10.1
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-StructMeasure: 1.1
    ChimeraX-Struts: 1.0.1
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0.1
    ChimeraX-SwapRes: 2.1.3
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.1.2
    ChimeraX-ToolshedUtils: 1.2.1
    ChimeraX-Topography: 1.0
    ChimeraX-Tug: 1.0.1
    ChimeraX-UI: 1.25.2
    ChimeraX-uniprot: 2.2.1
    ChimeraX-UnitCell: 1.0.1
    ChimeraX-ViewDockX: 1.1.6
    ChimeraX-VIPERdb: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0.1
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0.1
    ChimeraX-WebServices: 1.1.0
    ChimeraX-Zone: 1.0.1
    colorama: 0.4.5
    cxservices: 1.2
    cycler: 0.11.0
    Cython: 0.29.32
    debugpy: 1.6.4
    decorator: 5.1.1
    distro: 1.7.0
    docutils: 0.19
    entrypoints: 0.4
    executing: 1.2.0
    filelock: 3.7.1
    fonttools: 4.38.0
    funcparserlib: 1.0.1
    grako: 3.16.5
    h5py: 3.7.0
    html2text: 2020.1.16
    idna: 3.4
    ihm: 0.33
    imagecodecs: 2022.7.31
    imagesize: 1.4.1
    importlib-metadata: 5.1.0
    ipykernel: 6.15.3
    ipython: 8.4.0
    ipython-genutils: 0.2.0
    jedi: 0.18.1
    Jinja2: 3.1.2
    jupyter-client: 7.3.4
    jupyter-core: 5.1.0
    kiwisolver: 1.4.4
    line-profiler: 3.5.1
    lxml: 4.9.1
    lz4: 4.0.2
    MarkupSafe: 2.1.1
    matplotlib: 3.5.2
    matplotlib-inline: 0.1.6
    msgpack: 1.0.4
    nest-asyncio: 1.5.6
    netCDF4: 1.6.0
    networkx: 2.8.5
    numexpr: 2.8.4
    numpy: 1.23.1
    openvr: 1.23.701
    packaging: 21.3
    ParmEd: 3.4.3
    parso: 0.8.3
    pep517: 0.13.0
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 9.2.0
    pip: 22.2.2
    pkginfo: 1.8.3
    platformdirs: 2.5.4
    prompt-toolkit: 3.0.33
    psutil: 5.9.1
    ptyprocess: 0.7.0
    pure-eval: 0.2.2
    pycollada: 0.7.2
    pydicom: 2.3.0
    Pygments: 2.12.0
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 3.0.9
    PyQt6-commercial: 6.3.1
    PyQt6-Qt6: 6.3.2
    PyQt6-sip: 13.4.0
    PyQt6-WebEngine-commercial: 6.3.1
    PyQt6-WebEngine-Qt6: 6.3.2
    python-dateutil: 2.8.2
    pytz: 2022.6
    pyzmq: 24.0.1
    qtconsole: 5.3.1
    QtPy: 2.3.0
    RandomWords: 0.4.0
    requests: 2.28.1
    scipy: 1.9.0
    setuptools: 65.1.1
    sfftk-rw: 0.7.2
    six: 1.16.0
    snowballstemmer: 2.2.0
    sortedcontainers: 2.4.0
    Sphinx: 5.1.1
    sphinx-autodoc-typehints: 1.19.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 3.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 2.0.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.5
    stack-data: 0.6.2
    tables: 3.7.0
    tifffile: 2022.7.31
    tinyarray: 1.2.4
    tomli: 2.0.1
    tornado: 6.2
    traitlets: 5.3.0
    urllib3: 1.26.13
    wcwidth: 0.2.5
    webcolors: 1.12
    wheel: 0.37.1
    wheel-filename: 1.4.1
    zipp: 3.11.0

Change History (8)

comment:1 by pett, 3 years ago

Cc: chimerax-programmers added
Component: Unassigned → Build System
Owner: set to Zach Pearson
Platform: → all
Project: → ChimeraX
Status: new → assigned
Summary: ChimeraX bug report submission → ChimeraX/lib not on library path

comment:2 by Zach Pearson, 3 years ago

Status: assigned → feedback

I wasn't able to reproduce this on Ubuntu 22.04 -- the bundle builder code still puts ChimeraX/lib in the library path. I built clipper using the workaround from #8266 then ISOLDE. If you decide the proposed fix in #8266 is satisfactory, try a newer daily build.

in reply to:  3 ; comment:3 by Tristan Croll, 3 years ago 

Will check tomorrow. On the off chance it makes a difference, my actual
builds happen in a CentOS 8 Singularity container.

On Wed, 18 Jan 2023 at 21:07, ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
wrote:

comment:4 by Tristan Croll, 3 years ago

Still failing today, I'm afraid. By the looks of the command, what should be the -I.../ChimeraX/include and -L.../ChimeraX/lib directories are still pointing to the temp directories that were used to build ChimeraX:

c++ -pipe -fPIC -std=gnu99 -fdebug-prefix-map=/tmp/chx-develop.1QlzKh/build=. -fstack-protector-strong -I/tmp/chx-develop.1QlzKh/build/include -L/tmp/chx-develop.1QlzKh/build/lib -DUSE_DYLD_GLOBAL_NAMESPACE -shared -O3 -Wall -Wextra -D_FORTIFY=2 build/temp.linux-x86_64-cpython-39/src/deps/lbfgs/src/lbfgs.o build/temp.linux-x86_64-cpython-39/src/openmm/minimize.o build/temp.linux-x86_64-cpython-39/src/openmm/openmm_interface.o -L/usr/lib -L/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/chimerax/core/lib -L/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/chimerax/atomic_lib/lib -L/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-packages/chimerax/arrays/lib -L/tmp/chx-develop.1QlzKh/build/lib -lOpenMM -o build/lib.linux-x86_64-cpython-39/chimerax/isolde/_openmm_async.cpython-39-x86_64-linux-gnu.so -Wl,-rpath,$ORIGIN

Here's the Singularity recipe I use to make the build environment:

Bootstrap: docker
From: rockylinux:8

%help
RHEL 8 compatible containter that can build ChimeraX

%labels
    Maintainer "Chimera Staff <chimera-staff@cgl.ucsf.edu>"
    org.label-schema.schema-version "1.0"
    org.label-scheme.description "This image has the right compiler environment to run ChimeraX"

%files
  ./chimerax-daily.rpm ./chimerax-daily.rpm

%post 
    yum --releasever=8 -y update
    yum -y install epel-release dnf-plugins-core
    yum config-manager --set-enabled powertools
    yum -y --setopt=exclude='*.i?86' --nobest group install "Development Tools"
    yum -y --setopt=exclude='*.i?86' --nobest install \
	gcc-toolset-10 \
	gcc-toolset-10-binutils-devel gcc-toolset-10-elfutils-devel gcc-toolset-10-valgrind-devel \
	tcsh python2 \
	chrpath patch openssh-clients rsync ca-certificates \
	autoconf automake libtool pkgconfig \
	flex bison yasm unzip tar gzip bzip2 xz p7zip \
	rpm-build rpmdevtools rpmlint \
	ImageMagick \
	alsa-lib-devel \
	atk-devel \
	bzip2-devel \
	cairo-devel \
	cairo-gobject-devel \
	cups-devel \
	dbus-libs \
	expat-devel \
	fftw-devel \
	fontconfig-devel \
	freetype-devel \
	gdk-pixbuf2-devel \
	glib2-devel \
	glibc-devel \
	gtk3-devel \
	hdf5-devel \
	libdrm-devel \
	libgcc \
	libstdc++ \
	libX11-devel \
	libxcb-devel \
	libXcomposite-devel \
	libXcursor-devel \
	libXdamage-devel \
	libXext-devel \
	libffi-devel \
	libXfixes-devel \
	libXi-devel \
	libXrandr-devel \
	libXrender-devel \
	libXtst-devel \
	mesa-libEGL \
	mesa-libGL-devel \
	mesa-libGLU-devel \
	mesa-libOSMesa-devel \
	mysql-libs \
	nspr-devel \
	nss-devel \
	nss-util-devel \
	openssl \
	openssl-devel \
	pango-devel \
	pulseaudio-libs-devel \
	sqlite-devel \
	xdg-utils \
	xz-devel \
	zlib-devel \
	texlive
    dnf install -y ./chimerax-daily.rpm
    alternatives --set python /usr/bin/python2

%environment
    export PATH=/usr/bin:/usr/sbin:/bin:/sbin
    export SHELL=/bin/bash
    export LANG=en_US.UTF-8
    source /opt/rh/gcc-toolset-10/enable

Built into an image (after copying the latest RedHat 8 chimerax-daily into the working directory) with:

singularity build --fakeroot rhel-8-chimerax-1_6_daily.img ./rhel-8.def

... and invoked from the bundle root directory with:

singularity exec --bind $(pwd) --pwd $(pwd) /path/to/rhel-8.img  make $@

comment:5 by Zach Pearson, 3 years ago

OK, thanks for double checking. I'll make a container from your definition and get those directories back in the path.

comment:6 by Zach Pearson, 3 years ago

Building arrays with today's daily build on bare metal Ubuntu 22.04, I see temp directories but I also see -L/usr/lib/ucsf-chimerax-daily/lib

Does this still reproduce on a recent daily build?

comment:7 by Zach Pearson, 3 years ago

Resolution: → fixed
Status: feedback → closed

I found it! On Rocky, the executable in /usr/bin is a symbolic link to the executable in /usr/libexec/UCSF-ChimeraX-daily -- we were detecting the root relative to the symlink with sys.executable as opposed to the real executable at os.path.realpath(sys.executable), and since we set our library path relative to the executable you can see how this would lead to a ficticious path making it into lib_dirs.

I've pushed a ​fix for tomorrow's daily build.

in reply to:  8 ; comment:8 by Tristan Croll, 3 years ago 

Thanks for tracking this down - sounds like it was a bit painful! :)

On Thu, Mar 16, 2023 at 10:07 PM ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
wrote:
Note: See TracTickets for help on using tickets.