PyPi dependencies not installed by "devel install".

[mulvaney@…]

Command devel install of a bundle does not install PyPi dependencies unless the bundle version is incremented since the previous install.

Begin forwarded message:

From: Thomas Mulvaney
Subject: Re: ChimeraX-TEMPy and PySide2
Date: November 30, 2020 at 1:07:30 PM PST
To: Tom Goddard
Cc: Greg Couch

Hi Tom and Greg,

Thanks for the advice on sharing a wheel for beta-testers. That makes a lot of sense.

I’ve managed to figure out when the dependency issue surfaces.

If a plugin has been installed using the usual “devel install …” method and /then/ a dependency is added to its bundle_info.xml the new dependencies are not downloaded.

It seems to be that the toolshed reinstall pathway does not install dependencies.

I can get around this for now by uninstalling the plugin and then doing a devel install whenever
I add a package. This may in fact be expected behaviour in which case I apologise for the alarm,
but it did make me rather confused.

The rough process I followed is below.

Cheers,

Tom

——————————

Start with a clean slate by deleting the caches:

rm -rf ~/Library/Caches/ChimeraX/
rm -rf ~/Library/Application\ Support/ChimeraX

From my package directory I run:

$ CHIMERAX —-nogui —-exit —cmd “devel install . exit true”

The output contains the following relevant lines:

Executing: toolshed install ./dist/ChimeraX_Tempy-0.7.0-py3-none-any.whl
Successfully installed ChimeraX-Tempy-0.7.0
Installed ChimeraX-Tempy (0.7.0)

I then add numba and biopython as a dependency to the bundle_info.xml, and rerun:

$ CHIMERAX —-nogui —-exit —cmd “devel install . exit true”

The output contains the following relevant lines:

Executing: toolshed install ./dist/ChimeraX_Tempy-0.7.0-py3-none-any.whl reinstall true

Check to see if numba or biopython was added to ~/Library/Application\ Support/ChimeraX/1.1/site-packages/ but they are not present.

If however I start over, clearing the cache again or just using the “toolshed unininstall” tool, the dependency is installed and the output of “devel install …” lists the extra dependencies:

Executing: toolshed install ./dist/ChimeraX_Tempy-0.7.0-py3-none-any.whl
Successfully installed ChimeraX-Tempy-0.7.0 biopython-1.78 llvmlite-0.34.0 numba-0.51.2
Installed ChimeraX-Tempy (0.7.0)

On Mon, Nov 30, 2020 at 6:20 PM Tom Goddard <goddard@…> wrote:
Hi Tom,

That sounds good. We have not switched the ChimeraX daily builds to PySide2 but I hope to do that later this week. But you should release for ChimeraX 1.0 and 1.1 which is what most people have using PyQt5. I agree it should be very easy to make a PySide2 release later, probably just changing imports of PyQt5 to PySide2.

For TEMPy dependencies on biopython an numpy or others from PyPi I have not heard of "devel install ." failing to install these. I would expect log output to say something like "installing numpy -- requirement already satisfied. installing biopython ..." Is there any output about what happened? If not, how do you know it didn't install? You use the TEMPy plugin and an import fails? Greg Couch works on the bundle install tools so I have cc'ed him. But I think we need more info from you.

For beta-testers installing I think I would not have them run devel install. Instead give them a wheel file TEMPy*.whl and use the ChimeraX command "toolshed install TEMPy*.whl" and that should install the PyPi dependencies that are listed in the wheel. If not it is a bug and we need a test case. Maybe you can give us the TEMPy*.whl that fails to install the PyPi dependencies.

Tom

On Nov 30, 2020, at 2:34 AM, Thomas Mulvaney <mulvaney@…> wrote:

Hi Tom,

I'm Maya Topf's Master's student, Tom, and am working on a new revised ChimeraX-TEMPy plugin.

The existing plugin in the toolshed doesn't work with versions newer than ChimeraX 0.5 AFAIK but I have an updated version which works with ChimeraX >=1.0.

You mentioned in an email to Joe Beton, in Maya's group that you were thinking of moving to PySide2. It looks like a trivial change on our end and I am happy to make the changes when the time comes. Is this likely to be in 1.2?

Hopefully I'll update the plugin in the toolshed sometime this week.

One final question:

When I run "devel install ." in a project where the bundle_info.xml file lists additional python dependencies for example biopython or numpy, sometimes these dependencies are installed and sometimes they are not. I haven't been able to figure out why this is the case.

It would be useful to get a grasp on how to make this work so I can get beta testers to install the plugin.

Kind regards,

Tom

[Tom Goddard]

Begin forwarded message:

From: Greg Couch
Subject: Re: ChimeraX-TEMPy and PySide2
Date: November 30, 2020 at 2:23:46 PM PST
To: Thomas Mulvaney , Tom Goddard

Did you change the version number when you changed the dependencies? If you change the version number then the dependency changes should take effect. If not, that is a bug. And it is a bug, see next paragraph.

There's an ongoing discussion on the Python pip mailing list about this. I've asserted that developers don't want to keep changing the version number while developing the next release of a package. But that view isn't shared by everyone. So to be able to reinstall package with the same name and version as an installed package, ChimeraX is using pip install's "--force-reinstall --no-deps" options. The bug is that ChimeraX is not checking that the version is the same before asking for a reinstall, and later requiring a reinstall even though the versions are different. I'm working on a fix.

-- Greg

[Tom Goddard]

Begin forwarded message:

From: Thomas Mulvaney
Subject: Re: ChimeraX-TEMPy and PySide2
Date: December 1, 2020 at 3:41:57 AM PST
To: Greg Couch
Cc: Tom Goddard

Hi Greg,

Yes, I did try updating the version number when I modified the dependencies as well, but that also didn't work.

I must admit I'm new to python and am even less familiar with the packaging ecosystem. Thanks for the insight!

Cheers,

Tom

[Greg Couch]

Make it so both the bundle's name *and* its version have to be the same before giving "--force-reinstall --no-deps" to pip install.