Tube helices broken on Windows, Scipy cannot import _uarray

[goddard@…]

The following bug report has been submitted:
Platform:        Windows-10-10.0.18362
ChimeraX Version: 1.0 (2020-05-07)
Description
Helix tubes (Presets / Cylnders) is broken on Windows because a scipy import fails.  Probably need to update or rollback scipy to fix this.

Log:
UCSF ChimeraX version: 1.0rc202005070439 (2020-05-07)  
© 2016-2020 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open 4hus format mmcif fromDatabase pdb

4hus title:  
Crystal structure of streptogramin group A antibiotic acetyltransferase VatA
from Staphylococcus aureus in complex with virginiamycin M1 [more info...]  
  
Chain information for 4hus #1  
---  
Chain | Description  
A B C | Virginiamycin A acetyltransferase  
  
Non-standard residues in 4hus #1  
---  
CL — chloride ion  
EDO — 1,2-ethanediol (ethylene glycol)  
NA — sodium ion  
PEG — di(hydroxyethyl)ether  
PGE — triethylene glycol  
SO4 — sulfate ion  
VIR — virginiamycin M1  
  

> preset cartoons/nucleotides cylinders/stubs

Changed 0 atom styles  
Preset expands to these ChimeraX commands:

    
    
    show nucleic
    hide protein|solvent|H
    surf hide
    style (protein|nucleic|solvent) & @@draw_mode=0 stick
    cartoon
    cartoon style modeh def arrows t arrowshelix f arrowscale 2 wid 2 thick 0.4 sides 12 div 20
    cartoon style ~(nucleic|strand) x round
    cartoon style (nucleic|strand) x rect
    cartoon style protein modeh tube rad 2 sides 24 thick 0.6
    cartoon style nucleic x round width 1.6 thick 1.6
    nucleotides stubs

  
Traceback (most recent call last):  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\core\triggerset.py", line 130, in invoke  
return self._func(self._name, data)  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\atomic\structure.py", line 1562, in
_update_graphics_if_needed  
s[i]._update_graphics_if_needed()  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\atomic\structure.py", line 300, in
_update_graphics_if_needed  
self._create_ribbon_graphics()  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\atomic\structure.py", line 553, in _create_ribbon_graphics  
ribbons_drawing.compute_ribbons(self)  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\atomic\ribbon.py", line 586, in compute_ribbons  
_make_ribbon_graphics(structure, self)  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\atomic\ribbon.py", line 128, in _make_ribbon_graphics  
centers = _arc_helix_geometry(coords, xs_mgr, displays, start, end, geometry)  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\atomic\ribbon.py", line 957, in _arc_helix_geometry  
hc = HelixCylinder(coords[start:end])  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\atomic\sse.py", line 185, in __init__  
self._straight_optimize()  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\atomic\sse.py", line 398, in _straight_optimize  
opt = OptLine(self.coords, centroid, axis)  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\chimerax\atomic\sse.py", line 25, in __init__  
from scipy.optimize import minimize  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\scipy\\__init__.py", line 156, in <module>  
from . import fft  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\scipy\fft\\__init__.py", line 76, in <module>  
from ._basic import (  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\scipy\fft\\_basic.py", line 1, in <module>  
from scipy._lib.uarray import generate_multimethod, Dispatchable  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\scipy\\_lib\uarray.py", line 27, in <module>  
from ._uarray import *  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\scipy\\_lib\\_uarray\\__init__.py", line 114, in <module>  
from ._backend import *  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\scipy\\_lib\\_uarray\\_backend.py", line 15, in <module>  
from . import _uarray # type: ignore  
ImportError: DLL load failed: The specified module could not be found.  
  
Error processing trigger "graphics update":  
ImportError: DLL load failed: The specified module could not be found.  
  
File "C:\Program Files\ChimeraX-May-7-2020\bin\lib\site-
packages\scipy\\_lib\\_uarray\\_backend.py", line 15, in  
from . import _uarray # type: ignore  
  
See log for complete Python traceback.  
  




OpenGL version: 3.3.0 NVIDIA 442.19
OpenGL renderer: GeForce GTX 1080 Ti/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation
Manufacturer: Gigabyte Technology Co., Ltd.
Model: Z270X-Gaming K7
OS: Microsoft Windows 10 Home (Build 18362)
Memory: 17,129,693,184
MaxProcessMemory: 137,438,953,344
CPU: 8 Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz"

[Tom Goddard]

There are two previous reports of this same bug.

[Greg Couch]

Duplicates are #3122 and #3211.

I can't reproduce this. Both my development build, and today's candidate release work without problems. Please double check that CHIMERAX\bin\Lib\site-packages\scipy\_lib\_uarray\_uarray.cp37-win_amd64.pyd exists. And assuming it exists, copy the _uarray*.pyd to /tmp/_uarray.dll and then send me the output of "ldd /tmp/_uarray.dll". I'm hoping for something to be missing.

[Tom Goddard]

Retested with today's release candidate same error.

open 4hus
menu Presets / Cylinder

Hard to believe that does not reproduce for you. _uarray*.pyd file exists, appears to have access permissions, ldd output follows.

$ ldd /tmp/_uarray.dll

ntdll.dll => /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffa5dfe0000)
KERNEL32.DLL => /cygdrive/c/WINDOWS/System32/KERNEL32.DLL (0x7ffa5d860000)
KERNELBASE.dll => /cygdrive/c/WINDOWS/System32/KERNELBASE.dll (0x7ffa5af70000)
_uarray.dll => /tmp/_uarray.dll (0x7ffa56a80000)
ucrtbase.dll => /cygdrive/c/Windows/System32/ucrtbase.dll (0x7ffa5b220000)
msvcp140.dll => /cygdrive/c/Windows/System32/msvcp140.dll (0x7ffa44360000)
vcruntime140.dll => /cygdrive/c/Windows/System32/vcruntime140.dll (0x7ffa4e930000)

$

[Tom Goddard]

Here is what I get importing _uarray from the shell after starting today's release candidate

Jupyter QtConsole 4.7.3

Current ChimeraX session available as 'session'.

Python 3.7.7 (tags/v3.7.7:d7c567b08f, Mar 10 2020, 10:41:24) [MSC v.1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.14.0 -- An enhanced Interactive Python. Type '?' for help.

from scipy import _uarray

---

ImportError Traceback (most recent call last)
<ipython-input-1-68cc928b5970> in <module>

---

C:\Program Files\ChimeraX-May-14-2020\bin\lib\site-packages\scipy\init.py in <module>

154 # This makes "from scipy import fft" return scipy.fft, not np.fft
155 del fft

--> 156 from . import fft

C:\Program Files\ChimeraX-May-14-2020\bin\lib\site-packages\scipy\fft\init.py in <module>

74 from future import division, print_function, absolute_import
75

---> 76 from ._basic import (

77 fft, ifft, fft2, ifft2, fftn, ifftn,
78 rfft, irfft, rfft2, irfft2, rfftn, irfftn,

C:\Program Files\ChimeraX-May-14-2020\bin\lib\site-packages\scipy\fft\_basic.py in <module>

---

2 import numpy as np
3
4
5 def _x_replacer(args, kwargs, dispatchables):

C:\Program Files\ChimeraX-May-14-2020\bin\lib\site-packages\scipy\_lib\uarray.py in <module>

25 from uarray import _Function
26 else:

---> 27 from ._uarray import *

28 from ._uarray import _Function
29

C:\Program Files\ChimeraX-May-14-2020\bin\lib\site-packages\scipy\_lib\_uarray\init.py in <module>

112 """
113

--> 114 from ._backend import *

115
116 version = '0.5.1+5.ga864a57.scipy'

C:\Program Files\ChimeraX-May-14-2020\bin\lib\site-packages\scipy\_lib\_uarray\_backend.py in <module>

13 import inspect
14 import functools

---> 15 from . import _uarray # type: ignore

16 import copyreg # type: ignore
17 import atexit

ImportError: DLL load failed: The specified module could not be found.

[Tom Goddard]

Here is what the scipy/_lib/_uarray directory contains

/cygdrive/c/Program Files/ChimeraX-May-14-2020/bin/lib/site-packages/scipy/_lib/_uarray:
total used in directory 84 available 800351116
drwxrwx---+ 1 Administrators None 0 May 14 10:57 .
drwxrwx---+ 1 Administrators None 0 May 14 10:57 ..
-rwxr-x---+ 1 Administrators None 1543 May 14 02:09 LICENSE
-rwxr-x---+ 1 Administrators None 4583 May 14 02:09 init.py
drwxrwx---+ 1 Administrators None 0 May 14 10:57 pycache
-rwxr-x---+ 1 Administrators None 11697 May 14 02:09 _backend.py
-rwxr-x---+ 1 Administrators None 43008 May 14 02:09 _uarray.cp37-win_amd64.pyd
-rwxr-x---+ 1 Administrators None 1255 May 14 02:09 setup.py

[Greg Couch]

My ldd output is slightly different:

$ ldd /tmp/uarray.dll
        ntdll.dll => /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffa7f9e0000)
        KERNEL32.DLL => /cygdrive/c/WINDOWS/System32/KERNEL32.DLL (0x7ffa7f740000)
        KERNELBASE.dll => /cygdrive/c/WINDOWS/System32/KERNELBASE.dll (0x7ffa7d290000)
        uarray.dll => /tmp/uarray.dll (0x7ffa7b4b0000)
        ucrtbase.dll => /cygdrive/c/Windows/System32/ucrtbase.dll (0x7ffa7d960000)
        msvcp140.dll => /cygdrive/c/Windows/System32/msvcp140.dll (0x7ffa71220000)
        vcruntime140.dll => /cygdrive/c/Windows/System32/vcruntime140.dll (0x7ffa712c0000)
        vcruntime140_1.dll => /cygdrive/c/Windows/System32/vcruntime140_1.dll (0x7ffa7b4a0000)

See the additional vcruntime140_l.dll. So the bug is that you need a newer version of the Visual C++ runtime. I thought there was a ticket for putting the correct C++ runtime in the Windows installer, but I can't find it.

[Greg Couch]

Got to ​https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads, download and install vc_redist.x64.exe and see if that fixes the problem.

[goddard@…]

I don't want to install that on my devel machine.  My Windows 10 is up to date, updated today.

[Greg Couch]

It won't hurt your devel machine, it just adds newer version of existing runtimes and is backwards compatible. For this bug, it looks like it will need to be added to the Windows installer before the 1.0 release. You can wait to test it until it is in the installer, but it needs to be tested before 1.0 comes out.

[Greg Couch]

Tom, can add the output of the cygwin command:

regtool get /HKLM/SOFTWARE/Microsoft/DevDiv/vc/Servicing/14.0/RuntimeMinimum/Version

to this ticket. On my system it is 14.24.28127.

[Tom Goddard]

$ regtool get /HKLM/SOFTWARE/Microsoft/DevDiv/vc/Servicing/14.0/RuntimeMinimum/Version
14.16.27033

[Greg Couch]

An updated ChimeraX Windows installer, that embeds the Visual C++ runtime installer, fixed the problem on Scooter's Windows computer. That installer is temporarily available at plato.cgl.ucsf.edu:~gregc/chimerax-installer.exe. Would you like to test it?

I should note that we have recommended installing the Visual C++ runtime in the Windows Notes section of on the ChimeraX download page since the beginning, but the version changed when we updated the compiler. And in those notes we also said that we planned to embed the runtime installer in the ChimeraX installer.

[goddard@…]

If it works on Scooter's machine seems ready to put into the daily builds, and next time I start my Windows machine (happens every several days) I'll give it a try.

[Greg Couch]

Windows release candidates will now include the Visual C++ runtime and it is conditionally installed.

[Tom Goddard]

I tested the May 16 Windows build and tube helices worked.

But installing ChimeraX now asks me to restart the computer. That is very annoying. Is it possible to have the installer not ask to restart the computer? I did not restart and ChimeraX still found the new library and worked correctly without restart.

If the installer has to ask to restart the computer I think the best fix for the broken SciPy is to install a working SciPy that does not require a new C++ runtime. This is a bug in the SciPy PyPi package, and we should not degrade the ChimeraX installation experience to work around an easily fixed SciPy problem.

[Greg Couch]

I agree that the installer asking to restart the computer is annoying. The restarting of the computer is due to the C++ runtime installer updating some system files. I had tried to prevent the restart by passing the /norestart option to the C++ runtime installer, but that was not sufficient. Found an installer configuration option to turn off the asking for a restart.