Problems encountered building bundle for Windows

[Tristan Croll]

A somewhat catch-all list of issues encountered while building the Clipper and ISOLDE plugins for Windows:

• non-gui ChimeraX command-line switches (e.g. ChimeraX -m pip ..., ChimeraX --nogui, ChimeraX --nogui --cmd ...) appear to do nothing. In all cases it simply immediately returns to the prompt. ChimeraX --cmd ... does work. I'm managing by starting the ChimeraX gui and running devel install . from the inside.

• it wasn't clear to me that toolshed install xxx could be used to install non-ChimeraX-specific wheels. I spent some time trying to figure out how to pip install the base Clipper-Python module. For what it's worth, the iPython magic !pip install xxx is really *quite* magic - it manages to install the wheel to the *system* Python rather than ChimeraX.

• #1147 is still a pain in the neck. I'm quite sure that in bundle_builder._get_bundle_dirs(), line 500 should just be

            return None, None

rather than raising an error - otherwise any bundle with a dependency that *isn't* a ChimeraX bundle fails at this point.

• In bundle_info.xml the <CLibrary> option is undocumented. Seems I've been getting by on luck up to now, using the <CModule> option for libraries that are actually just designed for ctypes and have no PyInit__ functions. The Windows build pipeline fails on linking under that scheme. I found <CLibrary> by looking through some core ChimeraX bundles. That works, once I amend bundle_builder.py to provide the ChimeraX core library directory (in _CLibrary.compile() at line 560):

            compiler.add_library_dir(os.path.join(sys.exec_prefix, '..', 'lib')

That has me as far as successfully building the Clipper plugin. Will add to this if I find any other issues going forward into building ISOLDE.

[Tristan Croll]

Can't link to OpenMM - the library files are OpenMM.lib/OpenMM.dll, but bundle_builder is looking for libOpenMM.lib.

[Tristan Croll]

Additionally, the OpenMM libraries are in $CHIMERAX/bin, which isn't on bundle_builder's library path.

[Tristan Croll]

This one's more my fault, but worth mentioning. I built Clipper-Python against my system Python, which has a newer version of numpy installed compared to ChimeraX. This triggered ChimeraX to upgrade numpy (automatically choosing the non-MKL version) on installing Clipper-Python. ISOLDE's install then failed on attempting to import scipy.linalg, and some snooping with ProcMon shows that this was because it relied on the no-longer-present MKL runtime library. Will downgrade my system numpy to something older - but it's worth noting that there's a little fragility here.

[Tristan Croll]

Libraries specified as <CLibrary> in bundle_info.xml are building and making it through unit testing, but don't make it to the final installation directory.

[Tristan Croll]

... but after copying the missing libraries over manually, I can confirm that ISOLDE runs in Windows. Hooray!

For anyone interested, the necessary files are at ​https://drive.google.com/drive/folders/1aXPck_px4MiasNo3OtJG6pRRbLwB4eVG?usp=sharing. Install Clipper_Python...whl first, then ChimeraX_Clipper, then ChimeraX_ISOLDE, then copy lib_symmetry.dll into the ChimeraX_Clipper installation directory and the other loose DLLs into ISOLDE's installation directory.

[Tristan Croll]

The problem with the missing library dir is a simple typo in bundle_builder.py. At line 552,

        if lib_dirs:
            compiler.set_library_dirs(inc_dirs)

should of course be

        if lib_dirs:
            compiler.set_library_dirs(lib_dirs)

[Tristan Croll]

I think the simplest way to deal with the <CLibrary> compiled libraries not being copied is to have _CLibrary.compile() return the library name, then in make_wheel() at line 48:

        for lib in self.c_libraries:
            self.datafiles[self.package].append(
                lib.compile(self.logger, self.dependencies, debug=debug))

[Conrad Huang]

The typo fix was added in c2acd8b back in August, and the data file copying code was added in f6f0358 in September. Can you please confirm that these actually fix the problem? Thanks.

Conrad

[Tristan Croll]

As it happens, I *just* did a build for all three operating systems, and they're all fine.

[Conrad Huang]

That's great! I can finally close this ticket.