Qt shim broken in Dec 6 daily builds

[Tristan Croll]

Trying to launch ChimeraX daily just gives a traceback:

NOTE: Traceback (most recent call last):

File "/usr/lib/ucsf-chimerax-daily/lib/python3.9/runpy.py", line 197, in

_run_module_as_main

return _run_code(code, main_globals, None,

File "/usr/lib/ucsf-chimerax-daily/lib/python3.9/runpy.py", line 87, in

_run_code

exec(code, run_globals)

File "/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-

packages/chimerax/core/main.py", line 1043, in <module>

exit_code = init(sys.argv)

File "/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-

packages/chimerax/core/main.py", line 579, in init

from chimerax.ui import initialize_qt

File "/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-

packages/chimerax/ui/init.py", line 21, in <module>

from .gui import MainToolWindow, initialize_qt, menu_capitalize

File "/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-

packages/chimerax/ui/gui.py", line 29, in <module>

from Qt.QtWidgets import QApplication

ModuleNotFoundError: No module named 'Qt'

BUG: ModuleNotFoundError: No module named 'Qt'

File "/usr/lib/ucsf-chimerax-daily/lib/python3.9/site-
packages/chimerax/ui/gui.py", line 29, in
from Qt.QtWidgets import QApplication

_See log for complete Python traceback._

[Tristan Croll]

Looks like PyQt6 is missing entirely... copied Qt and PyQt6* from the 1.5 release, and I can launch again.

[Tom Goddard]

Is this on Linux? Which ChimeraX installer did you use?

The Mac Dec 6 daily build works correctly.

[Tristan Croll]

Both Ubuntu 22 and CentOS 8.

On Wed, 7 Dec 2022 at 19:22, ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
wrote:

[Tom Goddard]

The build of the prerequisites fails to sync on Linux back to plato so the qtshim (site-packages/Qt) is missing.

The CentOS 8 build of all the src/bundles wheels that is done once for all Linux distributions fails. It seems that the ChimeraX libpython3.9.so.1.0 cannot load because the needed libc.so.6 cannot be found.

Here is the output from today's daily build

​http://preview.rbvi.ucsf.edu/chimerax/builds/centos-8/unified-daily.out

mkdir -p /tmp/chx-develop.QvfNXb/build/sync/
make -C src/bundles sync
make[1]: Entering directory `/tmp/chx-develop.QvfNXb/src/bundles'
/tmp/chx-develop.QvfNXb/ChimeraX.app/bin/python3.9 -I Make_dependencies.py
/tmp/chx-develop.QvfNXb/ChimeraX.app/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /tmp/chx-develop.QvfNXb/ChimeraX.app/bin/../lib/libpython3.9.so.1.0)
/tmp/chx-develop.QvfNXb/ChimeraX.app/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /tmp/chx-develop.QvfNXb/ChimeraX.app/bin/../lib/libpython3.9.so.1.0)
/tmp/chx-develop.QvfNXb/ChimeraX.app/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.26' not found (required by /tmp/chx-develop.QvfNXb/ChimeraX.app/bin/../lib/libpython3.9.so.1.0)
/tmp/chx-develop.QvfNXb/ChimeraX.app/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.27' not found (required by /tmp/chx-develop.QvfNXb/ChimeraX.app/bin/../lib/libpython3.9.so.1.0)
make[1]: *** [Makefile.dependencies] Error 1
make[1]: Leaving directory `/tmp/chx-develop.QvfNXb/src/bundles'
make: *** [sync] Error 2

[Tom Goddard]

Our nightly builds try to copy the bundle wheels to plato for inclusion in each Linux distribution. Before this happens the src/bundles/Make_dependencies.py script is run using the ChimeraX app Python. These steps are done on CentOS 7 for Linux, while the ChimeraX app is build in a singularity image with a newer Linux. And it appears that our ChimeraX app Python will no longer run in CentOS 7 due to a missing sufficiently old libc.so.6. I am not sure how this worked in the past.

I've changed our src/bundles/Makefile so it runs Make_dependencies.py using the system Python (CentOS 7). And I started a Linux daily build to see if this will fix it. It may just push the problem further along in the build if ChimeraX Python is invoked later in the build process.

[Tom Goddard]

The previous fix failed because the Make_dependencies.py script depends on PyPi package tomli that the system Python does not have. So I reverted that change.

I am now trying running the "make sync" from the build_tools/build_linux.sh script in the singularity container. Running the linux daily build again to see if that works. We really tie ourselves in knots trying to build ChimeraX on plato with CentOS 7 where ChimeraX and its Python cannot run.

[Tom Goddard]

The build seemed to work but once again failed to include the qtshim module site-packages/Qt. Closer inspection of the ubuntu 20.04 build output showed that installing PyQt 6.4.0 failed because the wheel file name did not match what was on plato (requested PyQt6_commercial-6.4.0-cp37-abi3-manylinux1_x86_64.whl but the file on plato is named PyQt6_commercial-6.4.0-cp37-abi3-manylinux_2_28_x86_64.whl). This caused the prereqs make install right before qtshim was going to be installed. Strangely the build just keeps going and makes distributions without Qt.

rsync -av plato.cgl.ucsf.edu:/usr/local/src/PyQt6/6.4.0/PyQt6_commercial-6.4.0-cp37-abi3-manylinux1_x86_64.whl .
receiving incremental file list
rsync: link_stat "/usr/local/src/PyQt6/6.4.0/PyQt6_commercial-6.4.0-cp37-abi3-manylinux1_x86_64.whl" failed: No such file or directory (2)

sent 8 bytes  received 150 bytes  105.33 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1677) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)
make[2]: *** [Makefile.wheel.qt6:72: PyQt6_commercial-6.4.0-cp37-abi3-manylinux1_x86_64.whl] Error 23
make[2]: Leaving directory '/tmp/chx-develop.p4JvIQ/prereqs/PyQt'
make[1]: *** [Makefile:49: app-install] Error 2
make[1]: Leaving directory '/tmp/chx-develop.p4JvIQ/prereqs/PyQt'
make: *** [Makefile:123: PyQt.app-install] Error 2
make: Leaving directory '/tmp/chx-develop.p4JvIQ/prereqs'

[Tom Goddard]

I've restarted the builds again with the correct PyQt wheel file name. Hopefully in a few hours the Linux daily builds on the web site will be working.

The Linux build scripts were broken in two places and yet it put the builds on the web. Definitely the builds need to detect when the fail (wheels not synced, or PyQt6 failed to install) and not put those builds on the download page.

[Tom Goddard]

With the two fixes ("make sync" done in singularity image) and PyQt6 wheel correct file name used, the Linux daily builds completed and I tested the Ubuntu 20.04 build on minsky.cgl.ucsf.edu and it worked, opening a PDB and a map.

I'll leave this ticket open so the build scripts can be fixed so they properly detect failure of make sync and PyQt install and avoid putting broken ChimeraX distributions on the download page.

[Tom Goddard]

Also the test suite seems to be broken on Linux. In the CentOS-8 daily build output it fails to load QtCore

Importing chimerax.ui
-> ERROR: libQt6Core.so.6: cannot open shared object file: No such file or directory

Maybe it is trying to run the import test on CentOS 7 instead of using the CentOS 8 environment, which would probably cause Qt libraries to fail to load. I did not investigate.

Will leave cleaning up the Linux build to Greg.

[Greg Couch]

The test suite is done in nogui/nooffscreen mode, so it should be not loading Qt. I believe that is a bug in the DICOM code.

[Tom Goddard]

It is the import test that is giving the libQt6Core.so.6 errors. It imports every Python file including GUI code which will try to import Qt. Perhaps it is being done on CentOS 7 where Qt does not work (and may have missing system library dependencies).

[Tom Goddard]

I made separate tickets for the issues where the Linux builds should fail (e.g. PyQt missing) but get put on the download page anyway, tickets #8173, #8174, #8175.

[Tom Goddard]

Fixed Linux daily builds.

There were two problems. One was the PyQt 6.4 wheel file name was wrong so fetching it from plato failed. Also the "make sync" that copies wheel files from the unified linux build to plato failed because it tried to use ChimeraX Python on CentOS 7 that was compiled on CentOS 8 and did not start.