Segger obsolete import

[Tristan Croll]

The following bug report has been submitted:
Platform:        Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-centos-7.8.2003-Core
ChimeraX Version: 1.1rc202009012033 (2020-09-01 20:33:45 UTC)
Description
Import needs updating in segger.

Log:
UCSF ChimeraX version: 1.1rc202009012033 (2020-09-01)  
© 2016-2020 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> isolde tutorial

> open 6mhz

Summary of feedback from opening 6mhz fetched from pdb  
---  
notes | Fetching compressed mmCIF 6mhz from
http://files.rcsb.org/download/6mhz.cif  
Fetching CCD AOV from http://ligand-expo.rcsb.org/reports/A/AOV/AOV.cif  
  
6mhz title:  
Vanadate trapped Cryo-EM Structure of E.coli LptB2FG Transporter [more
info...]  
  
Chain information for 6mhz #1  
---  
Chain | Description  
A B | Lipopolysaccharide export system ATP-binding protein LptB  
F | Lipopolysaccharide export system permease protein LptF  
G | Lipopolysaccharide export system permease protein LptG  
  
Non-standard residues in 6mhz #1  
---  
AOV — adp orthovanadate  
  

> open 1924 fromDatabase emdb

Opened emd_1924.map, grid size 160,160,160, pixel 2.24, shown at level 1.74,
step 1, values float32  

> close #2

> open 6mhz

6mhz title:  
Vanadate trapped Cryo-EM Structure of E.coli LptB2FG Transporter [more
info...]  
  
Chain information for 6mhz #2  
---  
Chain | Description  
A B | Lipopolysaccharide export system ATP-binding protein LptB  
F | Lipopolysaccharide export system permease protein LptF  
G | Lipopolysaccharide export system permease protein LptG  
  
Non-standard residues in 6mhz #2  
---  
AOV — adp orthovanadate  
  

> open 9124 fromDatabase emdb

Opened emd_9124.map, grid size 192,192,192, pixel 1.23, shown at level 0.0368,
step 1, values float32  

> volume #3 level 0.06071

> volume #3 level 0.07505

> volume #3 level 0.06489

> ui tool show "Segment Map"

Segmenting emdb 9124, density threshold 0.064891  
Showing emdb 9124.seg - 0 regions, 0 surfaces  
Traceback (most recent call last):  
File "/opt/UCSF/ChimeraX/lib/python3.7/site-
packages/chimerax/segger/segment_dialog.py", line 399, in _segment  
self.Segment()  
File "/opt/UCSF/ChimeraX/lib/python3.7/site-
packages/chimerax/segger/segment_dialog.py", line 2133, in Segment  
smod = self.SegmentAndGroup(show, group)  
File "/opt/UCSF/ChimeraX/lib/python3.7/site-
packages/chimerax/segger/segment_dialog.py", line 2179, in SegmentAndGroup  
self.SmoothAndGroup ( smod, task )  
File "/opt/UCSF/ChimeraX/lib/python3.7/site-
packages/chimerax/segger/segment_dialog.py", line 2326, in SmoothAndGroup  
smod.smooth_and_group(numit, sdev, targNRegs, csyms, task)  
File "/opt/UCSF/ChimeraX/lib/python3.7/site-
packages/chimerax/segger/regions.py", line 527, in smooth_and_group  
from chimerax.map.filter import gaussian  
ModuleNotFoundError: No module named 'chimerax.map.filter'  
  
ModuleNotFoundError: No module named 'chimerax.map.filter'  
  
File "/opt/UCSF/ChimeraX/lib/python3.7/site-
packages/chimerax/segger/regions.py", line 527, in smooth_and_group  
from chimerax.map.filter import gaussian  
  
See log for complete Python traceback.  
  




OpenGL version: 3.3.0 NVIDIA 450.51.06
OpenGL renderer: TITAN Xp/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation
Manufacturer: Dell Inc.
Model: Precision T5600
OS: CentOS Linux 7 Core
Architecture: 64bit ELF
CPU: 32 Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.10GHz
Cache Size: 20480 KB
Memory:
	              total        used        free      shared  buff/cache   available
	Mem:            62G        4.3G         49G        119M        9.0G         57G
	Swap:          4.9G          0B        4.9G

Graphics:
	03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP102 [TITAN Xp] [10de:1b02] (rev a1)	
	Subsystem: NVIDIA Corporation Device [10de:11df]	
	Kernel driver in use: nvidia
PyQt version: 5.12.3
Compiled Qt version: 5.12.4
Runtime Qt version: 5.12.9
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    Babel: 2.8.0
    backcall: 0.2.0
    blockdiag: 2.0.1
    certifi: 2020.6.20
    chardet: 3.0.4
    ChimeraX-AddH: 2.1.1
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.1
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.6.1
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AxesPlanes: 2.0
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 1.0.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.0
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.0
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.0
    ChimeraX-Clipper: 0.15.0
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-CommandLine: 1.1.3
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.1rc202009012033
    ChimeraX-CoreFormats: 1.0
    ChimeraX-coulombic: 1.0.1
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-DataFormats: 1.0
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1
    ChimeraX-DistUI: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.0
    ChimeraX-Hbonds: 2.0
    ChimeraX-Help: 1.0
    ChimeraX-HKCage: 1.0
    ChimeraX-IHM: 1.0
    ChimeraX-ImageFormats: 1.0
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0
    ChimeraX-ISOLDE: 1.0.2
    ChimeraX-Label: 1.0
    ChimeraX-LinuxSupport: 1.0
    ChimeraX-ListInfo: 1.0
    ChimeraX-Log: 1.1.1
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Map: 1.0.1
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.0
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 1.1
    ChimeraX-MDcrds: 2.0
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.0
    ChimeraX-mmCIF: 2.2
    ChimeraX-MMTF: 2.0
    ChimeraX-Modeller: 1.0
    ChimeraX-ModelPanel: 1.0
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.0
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0
    ChimeraX-OpenCommand: 1.2.1
    ChimeraX-PDB: 2.1
    ChimeraX-PDBBio: 1.0
    ChimeraX-Phenix: 0.1
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0
    ChimeraX-PubChem: 2.0
    ChimeraX-Read-Pbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.0
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.2
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SeqView: 2.2
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.0
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.0
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.0.4
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.0
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.0
    ChimeraX-ToolshedUtils: 1.0
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.2.3
    ChimeraX-uniprot: 2.0
    ChimeraX-ViewDockX: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.3
    comtypes: 1.1.7
    cxservices: 1.0
    cycler: 0.10.0
    Cython: 0.29.20
    decorator: 4.4.2
    distlib: 0.3.1
    distro: 1.5.0
    docutils: 0.16
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    graphviz: 0.14.1
    h5py: 2.10.0
    html2text: 2020.1.16
    idna: 2.10
    ihm: 0.16
    imagecodecs: 2020.5.30
    imagecodecs-lite: 2020.1.31
    imagesize: 1.2.0
    ipykernel: 5.3.0
    ipython: 7.15.0
    ipython-genutils: 0.2.0
    jedi: 0.17.2
    Jinja2: 2.11.2
    jupyter-client: 6.1.3
    jupyter-core: 4.6.3
    kiwisolver: 1.2.0
    line-profiler: 2.1.2
    lxml: 4.5.1
    MarkupSafe: 1.1.1
    matplotlib: 3.2.1
    msgpack: 1.0.0
    netifaces: 0.10.9
    networkx: 2.4
    numexpr: 2.7.1
    numpy: 1.18.5
    numpydoc: 1.0.0
    objgraph: 3.4.1
    openvr: 1.12.501
    packaging: 20.4
    parso: 0.7.1
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 7.1.2
    pip: 20.2.2
    pkginfo: 1.5.0.1
    prompt-toolkit: 3.0.7
    psutil: 5.7.0
    ptyprocess: 0.6.0
    pycollada: 0.7.1
    pydicom: 2.0.0
    Pygments: 2.6.1
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PyQt5-commercial: 5.12.3
    PyQt5-sip: 4.19.19
    PyQtWebEngine-commercial: 5.12.1
    python-dateutil: 2.8.1
    pytz: 2020.1
    pyzmq: 19.0.2
    qtconsole: 4.7.4
    QtPy: 1.9.0
    RandomWords: 0.3.0
    requests: 2.24.0
    scipy: 1.4.1
    setuptools: 49.4.0
    sfftk-rw: 0.6.6.dev0
    six: 1.15.0
    snowballstemmer: 2.0.0
    sortedcontainers: 2.2.2
    Sphinx: 3.1.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 1.0.3
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.4
    suds-jurko: 0.6
    tables: 3.6.1
    tifffile: 2020.6.3
    tinyarray: 1.2.2
    tornado: 6.0.4
    traitlets: 5.0.0
    urllib3: 1.25.10
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.34.2

[Tom Goddard]

The Linux build for Segger is not the correct Segger in release candidate distributions. It is including a very old Segger instead of the currently checked in code. This is only on Linux. The Mac ChimeraX release candidate has the current Segger.

This is some problem with the ChimeraX nightly build system on Linux. I see on plato.cgl.ucsf.edu in /usr/local/projects/chimerax/builds/repo/ that the Segger wheel has an old date (from July)

-rw-r--r-- 1 chimera ferrin 230106 Jul 13 16:02 ChimeraX_Segger-1.0-py3-none-any.whl

while all other wheel have Sep 4, 2020 date. I don't know why Segger is not being updated. It appears it was built last night from current source during the centos-7 build, but apparently it did not get copied to the right place. Reassigning to Greg Couch who knows how the Linux builds are done.

[Greg Couch]

Weird. When I do a "make install" like the daily build script does, it puts the Segger wheel in build/sync/python-only as expected. But the output from the daily build shows that the rsync from that directory omits the Segger wheel.

[goddard@…]

The bundles/segger/Makefile is a weird one in that it does not import ../Makefile.bundle.  I am suspicious that is behind the problem.

[Greg Couch]

The Segger version in the builds shouldn't have been an issue. The repo directory on plato is a copy of the wheels in the build. The reason why the Segger version was so old was because the wheels in the candidate release were overwriting the wheels from the daily build. If the candidate build were done first, we wouldn't have observed this bug. I've changed the build script to only copy over the daily build wheels. (The plato repo is there to make it easier for CGL developers to release new bundles from the daily build on the production toolshed. It is also possible to upload new bundles.)

Giving back to Tom to fix bundles/segger/Makefile in the release candidate to point to a newer release. It is currently seggerx_2020_07_13.tar.gz.

[Tom Goddard]

Ok, thanks for explaining the repo directory on plato. The date on the segger wheel in the repo directory is still unexpected since the wheel should be built every night and should have had the Sept 4 date. But I agree that is a separate (although probably related) problem.

The seggerx_2020_07_13.tar.gz is the correct tarball for the release candidate and it does not have the error that is in the release candidate reported in this bug. The Linux release candidate includes an older Segger than either of the versions in the prereqs directory on plato. So the Linux build is somehow using a squirreled away old wheel. Hopefully you know where the wheel is coming from. With the convoluted linux build where part is done on centos-7 and part on another VM I don't know how this works.

The wrong segger is also in the Linux tar ball for the current release candidate.

I tried to check whether the wrong segger is in the linux daily build but the generic tar ball does not contain anything in site-packages and is less than 1 Mbyte in size, which must be some other build error.

[Greg Couch]

The linux builds work by building everything in /tmp/new-directory in a CentOS 7 singularity image. Then it saves all of the wheels, including the core wheel, and the include files. Then, for each linux platform (so CentOS 7 again), a singularity image for that platforms is used to build the prereqs, and then the wheels and include files from the CentOS 7 build are installed. This way,all of the Linux ChimeraX binaries have bundles built with the same compiler, so third-party bundles only have to target one Linux ABI.

The daily generic linux build failed because it didn't find curl. I've improved the linux build error output. And I've added curl and wget to the generic Linux build image.

[Greg Couch]

Haven't figured this out yet. The candidate build has the wrong version of segger, but if I do it by hand, it get the right version. Do you have some time this afternoon, Wednesday, to zoom with me and go over the build process? I'm hoping by having to explain all of the steps, I'll figure out the problem.

[goddard@…]

I have a theory.  Does the centos-7 build just untar the current source code over the previous night's source?  Because that would leave stuff from the previous night's build and "make clean" might not do a good enough job to clean it up.  In fact with my new "curl" fetch of prereqs including segger I think I did not put in clean rules everywhere to get rid of the prereq tar balls and untarred versions.  In any case, I need to fix that and make sure clean removes that stuff.

[Greg Couch]

I thought of that theory, but the build is done in a fresh directory.

[Greg Couch]

Also thought the it might be cached in ~chimera/.cache/pip. But it's not.

[Greg Couch]

Also noticed that the Segger version number isn't changing. All of the wheels are 1.0.

[goddard@…]

Yes, maybe the Segger version should be bumped on each update.  But I think none of the ChimeraX bundles get there version number changed for bug fixes and yet they update correctly.  I think Segger must be special because of its custom Makefile.

[Tom Goddard]

I see what is going on. Two mistakes in Segger, first the segger tarball contains dist/*.whl instead of just the source code. And second it untars the segger tar ball twice, the second time after the build made the new wheel. I'll figure out how this is happening and fix it.

[Tom Goddard]

Fixed.

Segger tar balls on plato mistakenly contained wheel files and the July tar ball did not even contain the wheel corresponding to the source code but instead an older wheel. I fixed the tar balls. Also I fixed the Makefile so it does a clean before making new tar balls. And I fixed the Makefile so it does not untar more than one time. That last problem happened because the untarred seggerx directory had a time stamp older than the fetched tar ball.