Integrate arm64 macOS into ChimeraX's build/deployment system

[Zach Pearson]

Need to investigate whether we're going to make a universal distribution or individual distributions. Some companies (e.g. JetBrains, Adobe) release platform specific programs, but the majority of them seem to offer universal binaries so that's probably the way to go.

Prerequisites don't need run natively as arm64, but it would be nice for the speedup. '

Need to get the M1 Mac Mini building ChimeraX.

[Zach Pearson]

Attempted to compile src/apps/ChimeraX with -arch arm64 -arch x86_64 arguments, but was stopped when it attempted to link against x86_64 Python. We'll need to update the Python prereq to build both arm64 and x86_64 versions if possible.

Correct me if I'm wrong but I believe TomG does the relocatable Python for macOS. We can download a universal Python from ​https://www.python.org/downloads/macos/. Since he's on vacation I'll see what I can do.

[Zach Pearson]

Looks like simply changing the macOS version in the makefile for relocatable Python to 11 will download a universal version. My machines are both on Monterey; do we have any volunteers with an older version of macOS to test make build-from-scratch on a branch with this patch before it gets merged into develop? Don't want to break daily builds.

[Zach Pearson]

After making the relocatable python universal, I was able to make a universal src/apps/ChimeraX

[Zach Pearson]

What I found while I was making my presentation was that even if I added -arch arm64 -arch x86_64 as extra compile args in module definitions, a Python wheel will get the tag of the platform that Python has regardless of how many extra architectures are built.

There may be a way to coerce a wheel into being acknowledged as universal2 even if it's not built from a universal2 python

​https://stackoverflow.com/questions/45150304/how-to-force-a-python-wheel-to-be-platform-specific-when-building-it

But the path of least resistance for building universal2 wheels would be to use the MacOS 11+ Python on all platforms instead of just arm64 macOS in prereqs/Python. That will add -arch arm64 -arch x86_64 to module compile flags for us, and tag the wheel appropriately.

I'm going to try universal python with MACOSX_DEPLOYMENT_TARGET=10.14, but it may need to be 10.15.

[Zach Pearson]

Python does not distribute universal2 with MACOSX_DEPLOYMENT_TARGET below 11. Options are to bump MACOSX_DEPLOYMENT_TARGET, compile Python, or try the above hack to coerce dual-architecture wheels to install on arm64 and macOS.

[Zach Pearson]

We assign our own tags to Python wheels instead of letting packaging do it for us, so even using universal python which should give us the correct tag for free if we use pyproject.tomls, we're overriding that mechanism to assign x86_64 to the wheels. Which means that I have to find a new way to do this just for macOS in bundle builder.

[Zach Pearson]

Good news is that if I build ChimeraX with the changes on the m1_compat/universal_python branch, it's recognized as something my Intel mac can open. The bad news is that it can't open because it's got some platform-specific version of numpy for some reason.

Looking into it.

[Zach Pearson]

There are ways to coerce pip into only installing universal wheels, but they come with significant drawbacks that would require us to check our own dependencies. Perhaps the best way would be to, for example, vendor numpy by hosting its wheel on Plato and getting it from there at install time.

[Tom Goddard]

Hi Tom,

Can you take a look at this ticket

​https://www.rbvi.ucsf.edu/trac/ChimeraX/ticket/6778

And let me know your thoughts?

For supporting x86 and arm64, I need a universal Python, but the script requires the MACOSX_DEPLOYMENT_TARGET to be set a little higher (to 11).

Further confusing the situation, Python lists its universal2 builds as compatible with 10.9 or later.

Looking for guidance on the path forward since you’ve historically set up the macOS Python.

— Zach

[Tom Goddard]

The Python 3.9.13 download page says the universal2 build supports macOS 10.9 or higher. So what makes you think our Python 3.9 universal build which comes from Python.org does not support 10.9?

I would expect that M1 Macs only run with macOS 11 and newer. But I would also expect that the Intel build in a universal2 package does not need to have the same macOS requirements and it can be for 10.9 or higher.

[Tom Goddard]

We are using the Python 3.9 binary from Python.org with some small modifications to the dylibs so that it is relocatable. We did not compile it.

It may well be that a Mac M1 build can only be compiled with target 11 or higher and we will definitely want Intel at 10.13 or 10.14 or 10.15. And if those cannot be built at the same time on the same machine with the -arch flags then the approach would be to build M1 and Intel separately then use the lipo command to merge them.

​https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary

[Zach Pearson]

Several wheels issue variations of the following warning when they get built:

[WARNING] This wheel needs a higher macOS version than the version your Python interpreter is compiled against.  To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 12_0 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:
build/bdist.macosx-10.9-universal2/wheel/chimerax/atom_search/ast.cpython-39-darwin.so
[WARNING] This wheel needs a higher macOS version than the version your Python interpreter is compiled against. To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 12_0 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:
build/bdist.macosx-10.9-universal2/wheel/chimerax/atom_search/ast.cpython-39-darwin.so
creating build/bdist.macosx-10.9-universal2/wheel/ChimeraX_AtomSearch-2.0.1.dist-info/WHEEL
creating 'dist/ChimeraX_AtomSearch-2.0.1-cp39-cp39-macosx_12_0_universal2.whl' and adding 'build/bdist.macosx-10.9-universal2/wheel' to it

[Zach Pearson]

I'll try that but I can't predict the outcome. Python distributes an Intel build and a Universal build, not an Intel build and an arm64 build.

[Zach Pearson]

There's also the matter of needing the x86_64-arm64 Python.h from the universal2 Python.

[Tom Goddard]

The warnings you get from pip may be because pip cannot handle building two versions at once, Intel and M1 with specific target os versions. So it may be necessary for us to build M1 and Intel ChimeraX separately and lipo them together. This is what Conrad did 15 years ago for Intel and PPC. We would still get our M1 Python and Intel Python from the Python.org. If the M1 is from the Python.org universal build and it creates a problem building only an arm64 build, then it can build both for macOS 11 and when you lipo you strip out the intel version from the M1 build.

I have never done any of this before.

[Zach Pearson]

Those warnings would be from setuptools AFAIK. I'm trying a build on Chopin (Intel, 10.15) using universal Python to see if it still issues those warnings and whether that version will work on my M1 laptop.

[Zach Pearson]

It started out promising, but Chopin wants to use the 10.15 SDK to cross-compile, and the 10.15 SDK does not have arm64 code in it if this error from compiling chutil_cpp is to be believed:

note: '__uint128_t' declared here
In file included from IOFile.cpp:16:
In file included from ./IOFile.h:19:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/string:504:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/string_view:175:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/__string:57:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/algorithm:641:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/cstring:60:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/string.h:60:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/string.h:61:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:43:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_wctype_t;
        ^
note: '__uint128_t' declared here
In file included from IOFile.cpp:16:
In file included from ./IOFile.h:19:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/string:504:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/string_view:175:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/__string:57:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/algorithm:641:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/cstring:60:
In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/string.h:60:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/string.h:152:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_ssize_t.h:30:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/types.h:37:2: error: architecture not supported

[Zach Pearson]

Tom was able to hook an M1 build into the build system.