ChimeraX in PyPi

[Tom Goddard]

Why don't we put amenable ChimeraX Python modules into PyPi so other developers can use them?

I was asked this at the 3D image hackathon when I gave a demonstration. They wanted to use just map rendering. Today Supritha Amudhu working with John Irwin on using ChimeraX with their docking-related tool was suprised that pip install of ChimeraX did not work.

Almost everything important in Python is in the PyPi ecosystem. We seem behind the times to not be putting any of our modules in PyPi.

I proposed making the PyPi ChimeraX wheel in our CZI Visual Proteomics grant proposal on VR for microscopy (Feb 2021), and also our CZI Essential Open Source Software proposal (infrastructure improvements, May 2021). The CZI grants require redistributable open source licensing and the PyPi package will address that requirement.

[Tristan Croll]

Have you considered Conda? That's where OpenMM lives, and PHENIX 2.0 (planned for late 2019, I believe) will be there as well. Conda's a lot more permissive than PyPI (where technically all Linux builds should be Centos 5 compatible), and having related packages in the same ecosystem would have obvious benefits.

[goddard@…]

Restrictions like CentOS 5 compatible are probably intended to make the packages portable which is a good thing, perhaps some trouble for the developer but good for the user.  I think it makes more sense to go with (by far?) the biggest Python ecosystem which is PyPi.  But the idea is not to try to force everything in ChimeraX into PyPi, just packages where it is not too hard and outside developers would benefit.  So for instance we would probably not try to package OpenMM for PyPi.

[Tom Goddard]

I made a ChimeraX wheel from site-packages/chimerax on macOS and tried running non-gui scripts to save images, January 2021. Needed to make a ten to twenty small changes so ChimeraX modules could work without running app initialization code.

I compiled an OSMesa version on macOS and got it working to save images, and made a wheel for easy install in Mac ChimeraX.

I've attached example scripts that work with the PyPi package. One opens a PDB, prints info like number of atoms and saves an image. Another opens a EMDB map, saves GLTF and saves an image. I also attached the setup.py file used to make the chimerax wheel.

[Tom Goddard]

Pip setup.py file to create chimerax wheel.

[Tom Goddard]

Example Python script using ChimeraX wheel to open map, save gltf, save image.

[Tom Goddard]

Example python script using ChimeraX wheel to open a PDB and save an image.

[Tom Goddard]

It is not clear how to handle the 50 ChimeraX PyPi dependencies, probably don't want to require them all. Current test only uses a minimal set (numpy, tinyarray, pyopengl, pillow). Pip allows named optional sets of dependencies which may be useful to include bigger sets.

Some dependencies are not in PyPi like OpenMM and ffmpeg, and how to handle those also needs consideration.

[Zach Pearson]

Similarly to #4663 (Apple Silicon port), since I've been tasked with the CZI grant which includes PyPi packaging, I'm reassigning this ticket to myself, though not CC'ing Tom this time as chimera-programmers is already CC'd. As in the other ticket, any comments are always welcome.

[Tristan Croll]

Just thought of this while watching a presentation by Paul Emsley on Coot 1.0 (a complete rebuild compared to previous versions, with modern graphics and lighting) - one of the slides demonstrated how Coot could be imported from Blender's Python console and used to directly generate and contour a section of crystallographic map, and deposit it in the Blender scene for further workup. Seems like the sort of thing this ticket could readily enable.

[goddard@…]

That is the kind of use a ChimeraX PyPi package would enable.  Of course ChimeraX would have to be pip installed in Blender's Python environment.

[Zach Pearson]

Since Qt6 has been winding down and a universal build is around the corner, I've been putting more thought into this. I plan to give a ~1 hour presentation on 26 May (thanks Elaine for swapping time slots with me) on how this effort interacts with the following tickets:

Report problematic version number checking to pip (Ticket #3403)
Automate uploading wheels to the Toolshed (Ticket #3058)
Move pip dependencies into bundles (Ticket #3890)
Install PyPi packages directly within ChimeraX (Ticket #4762)
Why doesn’t devel install install PyPi dependency? (Ticket #5157)
Support pyproject.toml (Ticket #6434)

and how I think the native-packaging branch on GitHub can, with minimal effort, more or less close all of them.

[Zach Pearson]

Code changes to enable the test suite:

At Eric's request: ​load_libarrays automatically
​dist monitor's init method doesn't need bundle_info.xml so make it optional for the library

I also ​experimented with hiding most of arrays from consumers

Test cases can be found here:
​https://github.com/RBVI/ChimeraX/tree/develop/wheel/tests

[Greg Couch]

Not actually tied to a ChimeraX release. But might become easier with Python 3.11, which should be in ChimeraX 1.7.

[Zach Pearson]

A macOS-universal2 0.1 wheel has been published to PyPI.

[Zach Pearson]

Not a 1.7 blocker.