OpenMM ToolShed bundle

[Tristan Croll]

As discussed in our meeting in March (if there's already a ticket on this, I apologise). If I'm to start supporting multiple Linux platforms with ISOLDE, I'll need OpenMM built with the correct compiler for each platform - I'm now making quite extensive use of its C++ API, and really have no other choice but to continue doing so to maintain decent performance. Since I'm also extensively using ChimeraX's C++ API, trying to juggle different compilers and maintain ABI compatibility with the precompiled OpenMM distribution would be a nightmare.

There are other good reasons to want a ToolShed bundle: I recently contributed some optimisations to OpenMM's Python API to improve simulation start-up times quite dramatically, but there's no telling when there will be a new official release allowing it to be merged into ChimeraX. Also, compiling a bundle would allow you to standardise on a specific CUDA version, if you're planning on using CUDA elsewhere in ChimeraX.

Would be fantastic if this was possible by 0.9, but I understand that's likely to be pushing it. I'm personally completely swamped with deadlines for the coming week, but could discuss/help after.

[Tom Goddard]

We won't have time to look into OpenMM builds for ChimeraX 0.9.

As a first step we should list what C++ compilers are being used for ChimeraX modules like the atomic library, and list which C++ compilers are used for the standard OpenMM Conda builds.

Also maybe Greg has some knowledge about C++ ABI compatibility that relates to whether we really need to do custom builds.

If you require modifications to OpenMM that can't be integrated by the official OpenMM developers fast enough, that is a separate problem.

These needs are driven by ISOLDE and so it may be that you will have to make all the needed OpenMM builds because we don't have the resources to work on it. It is a good bit of ongoing maintenance work for anyone who takes it on.

[Tristan Croll]

I'm starting to look into this semi-seriously now. In theory, as a short-term approach it should be fairly straightforward to pull the precompiled conda packages from ​https://anaconda.org/omnia-dev/openmm/files and repackage their contents for ChimeraX. The immediate problem I see, though, is that essentially all of its internal Python code is written with import statements expecting simtk to be at the top of the import tree, e.g.:

from simtk.openmm.app import Topology

As far as I'm aware, bundle_builder doesn't currently allow this - the bundle would be installed subordinate to chimerax. Any thoughts on how best to deal with that?

[Tristan Croll]

Would it be breaking the rules for the bundle to add its own top-level directory to the PYTHONPATH?

[tic20@…]

I have the same issue with ParmEd. I need to have it available in order 
to provide users with a reasonably friendly workflow for parameterising 
new ligands (phenix.elbow --amber will now provide .frcmod and .mol2; 
ParmEd then converts those to an OpenMM ffXML) but it's only present on 
the PyPI as a source tarball.

A new thought: what if the (Python standard) wheel files for 
dependencies like this were packaged inside the ToolShed bundle? Then 
`toolshed install` could just install them using the standard setuptools 
installer as the first step.

On 2019-07-03 12:09, ChimeraX wrote:

[Tom Goddard]

Conrad handles Toolshed and build system issues.

[Tom Goddard]

I think the aim should be to allow installing prereqs like OpenMM in site-packages, not site-packages/chimerax.

[Tristan Croll]

<bump>

I’m giving a webinar for SBGrid on September 24, and it would be great to be able to demonstrate a straightforward ligand parameterisation pipeline. But for that I need to be able to provide a precompiled build of ParmEd as part of ISOLDE (as previously discussed, it’s only available on PyPI as a source tarball).
 

[Tristan Croll]

Just wanting to bump this again, since it's becoming a thorn in my side. The "easy" solution I'd like to reiterate/expand upon is that bundles be allowed to include a directory (e.g. "prereqs") containing any "vanilla" Python wheels to be installed by pip before ChimeraX goes ahead and installs the main bundle. That seems like it should be fairly simple to accomplish without too much code complexity?

[Tristan Croll]

The OpenMM team say they're happy to help with this (particularly with adding to their Conda build pipeline to include Linux builds that match ChimeraX's GCC version). They suggest opening a ticket on their tracker at ​https://github.com/openmm/openmm/issues to coordinate.

[Tom Goddard]

It is not entirely clear what the objectives are in making OpenMM a bundle versus what we have now where it is built into the ChimeraX core distribution. Here is what I gather from the preceding comments.

1) If the ChimeraX C++ compiler and OpenMM C++ compilers are different on LInux then ISOLDE can't easily use both. The compilers are compatible right now so this is not a n issue. The latest OpenMM is 7.4.2. When OpenMM 7.5 comes out Tristan says it will require a newer compiler than what ChimeraX is using on Linux. We don't need a bundle to assure the same compilers, we just need an OpenMM build using the ChimeraX C++ compiler. We may be updating our C++ compilers on Linux -- Greg Couch would have to advise on that.

2) There was some comment about a bundle helping with CUDA versioning which I don't understand. Right now OpenMM offers libraries for different CUDA versions and we choose one.

3) Specific ISOLDE patches could be compiled into an OpenMM bundle. Tristan would be responsible for this. Our goal on ChimeraX is not to compile any third party packages, just use the standard distributions. It is way too much work especially for packages with dependencies like OpenMM for us to automate that. We did a lot of compiling third party packages with Chimera, and will avoid that approach unless absolutely essential.

4) Using an OpenMM bundle would allow it to be updated through toolshed without needing a new ChimeraX. Not clear if this is worth the effort.

[Tristan Croll]

If we stick with the current approach, the only time it will be possible 
to incorporate any major improvements to OpenMM (whether in its core or 
in the Python API) will be at significant version-number changes to 
ChimeraX. It would be nice to be able to have a faster development cycle 
than that. We've had a few times in the past where I've made enabling 
changes to the OpenMM Python code, which have then been patched into the 
ChimeraX OpenMM installation while waiting for them to become part of 
the next OpenMM release. I don't think that's really going to be a 
viable approach past 1.0.

What if the OpenMM team could be convinced to build pip-style wheel 
files alongside their Conda builds? Would it be possible for the 
ToolShed to host those alongside the ChimeraX-specific bundles?

On 2020-06-08 16:57, ChimeraX wrote:

[goddard@…]

If you want to patch OpenMM more frequently than ChimeraX releases with just Python patches you could have an OpenMM bundle that depends on a Conda OpenMM package and then applies your custom patches.  Currently a ChimeraX bundle can list a PyPi dependency but not a Conda dependency.  Perhaps Toolshed could be enhanced to handle Conda dependencies.

[Tristan Croll]

That's an option. In any case, I think it would be a very good idea to 
open the discussion with them. In general I've found them to be very 
happy to help, and they're quite professional in their approach to 
things.

On 2020-06-08 17:42, ChimeraX wrote:

[goddard@…]

There are two issues we want to solve: 1) Get OpenMM distributions for Python 3.8, and 2) Get OpenMM distributions that are compatible with whatever C++ compiler we are going to use on Linux.  Python 3.8 was released 8 months ago (October 2019).  Perhaps Tristan can open an issue on OpenMM github requesting Python 3.8 releases and you could say it is for ISOLDE and ChimeraX.  For the Linux compilers we need Greg Couch to say what the ChimeraX Linux compiler update situation will be.  I understand from Tristan that OpenMM 7.5 needs C++11 (is that right?) that is not supported with the current Linux compilers ChimeraX is using.

I'm reassigning to Greg to comment on the Linux compiler situation.

[goddard@…]

Another issue is 3) What should we do about all the different CUDA versions?  Do we just support one CUDA version.  Tristan has the most experience using CUDA -- what is the best solution?  Can a user have many different versions of CUDA installed without conflicts, or only one version?  If they can have only one version of CUDA installed does that mean every app that uses CUDA has to support a large range of CUDA versions?  Sounds insane.

[Tristan Croll]

That's not actually a huge issue. CUDA is only the first option by 
default in Linux (in Windows it requires Visual Studio for the 
just-in-time kernel compilation, which makes it a non-starter for all 
but the most enthusiastic - and it's significantly slower to get started 
compared to OpenCL anyway). If it can't find CUDA, it falls back to 
OpenCL which is quite insensitive to the CUDA version.

In Linux it is possible to have multiple CUDA versions installed - the 
default install location is versioned, and the version desired as 
default is symlinked to /usr/local/cuda. If the user does get it set up 
correctly then ISOLDE will use it, but that's going to be limited to 
only the most enthusiastic and savvy users. It's generally no big deal - 
CUDA simulations do run a bit faster than OpenCL ones, but the 
performance difference isn't really enough to matter.

On 2020-06-10 17:38, ChimeraX wrote:

[Greg Couch]

Won't make 1.1

[Tom Goddard]

Want to update ChimeraX to Python 3.8 or possibly Python 3.9 due out in October 2020.

The latest OpenMM 7.4.2 is still not available for Python 3.8 on Anaconda:

​https://anaconda.org/omnia/openmm/files

But OpenMM 7.5 beta does have Python 3.8 versions on the development site

​https://anaconda.org/omnia-dev/openmm/files

Tristan suggests we try the 7.5 beta. But supposedly it uses a newer C++ compiler than our gcc 4.X linux ChimeraX build compiler. Since Tristan's ISOLDE links against both ChimeraX atomic C++ libraries and OpenMM C++ he would like them to use C++ compilers with compatible ABIs.

Greg can advise on whether the Linux ChimeraX builds can use a newer C++ compiler that supports the C++-11 Tristan says is needed by OpenMM 7.5.

Tristan suggests we try to

[goddard@…]

For further emails on this topic please CC the ChimeraX ticket so we have an easy to find record of the discussion.

[Greg Couch]

It shouldn't be difficult to switch new a newer compiler. But we will need to match what compiler OpenMM is using. We're currently using devtoolset-3 on CentOS 7, but that doesn't seem to be available anymore. devtoolset-7-gcc is gcc 7.3.1, devtoolset-8-gcc is gcc 8.3.1, and devtoolset-9-gcc is gcc 9.1.1. All of those are layered on top of system version of gcc (4.8.5), so the ABI is still gcc 4.8.5. If OpenMM isn't using one of the devtoolsets, then we will need to compile it ourselves.

[Tristan Croll]

 From Jaime Rodríguez-Guerra on the OpenMM team. I've asked him if he's 
OK with me adding him to the CC list on this ticket.



Our last development builds have been using the conda compilers. You can 
check the specific versions per platform [here], but in a nutshell

- Linux x64: GCC 7 as available in CentOS 6
- Linux PPC: GCC 8 as available in CentOS 7
- MacOS: Clang 10
- Windows: VS 2017

You can get the conda package using these metapackages: 
https://github.com/conda-forge/compilers-feedstock

On 2020-09-17 19:19, ChimeraX wrote:

[Tristan Croll]

Added Jaime Rodriguez-Guerra (from the Chodera lab, responsible for OpenMM builds on Conda) to the CC list.

[jaime.rodriguez-guerra@…]

Hi all!

We are in the process of transitioning from our good ol' Omnia
infrastructure to today's gold-standard Conda Forge. There are some rough
edges with how they handle OpenCL and how we did historically, so we are
working on that with one of their core devs. Anyway, this means OpenMM 7.5
will probably be released through conda-forge. To ease this transition, we
have been trying their infrastructure for a while, which involves using
their own compilers [1]. As mentioned before, this is GCC 7.x on Linux x64.
Glibc is CentOS 6's (i.e. 2.12). The compilers packages provide their own
libgcc and friends, which are runtime dependencies. This ensures the whole
conda forge ecosystem is consistent wrt ABI, thanks to the globally applied
pinnings [2]. In the case of OpenMM, this also means fftw is pinned to v3.

With respect to the devtoolset versions, Jonathan Helmus from conda-forge
clarifies:

that uses the "new" C++11 ABI. This can cause incompatibilities with the
conda-forge compilers which defaults to this ABI.

Hope this helps!

[1]
https://conda-forge.org/docs/maintainer/infrastructure.html#compilers-and-runtimes
[2]
https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/master/recipe/conda_build_config.yaml

On Thu, Sep 17, 2020 at 9:12 PM ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
wrote:

[Tom Goddard]

OpenMM 7.5 beta builds are only available for Linux and Windows, not macOS. I am working on moving ChimeraX to Python 3.8 today, may take a few more days. Would be useful to have Python 3.8 OpenMM on Linux, Windows and macOS.

[Tom Goddard]

Last comment was wrong. OpenMM 7.5 beta builds are available from macOS, there were 7 pages of files and I only saw the first page. I will try the 7.5 beta versions in ChimeraX.

[Tom Goddard]

Discussion of OpenMM Python 3.8 c++ library issues on Linux in ticket #3756

[Greg Couch]

Won't make 1.3.

[Greg Couch]

We now track the openMM binary distributions on conda-forge, ​https://anaconda.org/conda-forge/openmm.