OpenMMException: Error compiling kernel: Compile Server Error

[fasseli.coulibaly@…]

The following bug report has been submitted:
Platform:        macOS-10.15.7-x86_64-i386-64bit
ChimeraX Version: 1.2.2 (2021-05-08 05:48:20 UTC)
Description
Tutorial "intro to cryo-EM model rebuilding in iSOLDE": 
isolde sim start sel 
(same error for own data before so decided to check if this was related to my data).

Log:
UCSF ChimeraX version: 1.2.2 (2021-05-08)  
© 2016-2021 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> ui tool show ISOLDE

> set selectionWidth 4

Done loading forcefield  

> isolde demo cryo_em_intro modelOnly true startIsolde false

6out.pdb title:  
Asymmetric focused reconstruction of human norovirus gi.1 norwalk strain VLP
asymmetric unit In T=3 symmetry [more info...]  
  
Chain information for 6out.pdb #1  
---  
Chain | Description  
A B C | capsid protein VP1  
  
6out.pdb title:  
Asymmetric focused reconstruction of human norovirus gi.1 norwalk strain VLP
asymmetric unit In T=3 symmetry [more info...]  
  
Chain information for 6out.pdb  
---  
Chain | Description  
1.2/A 1.2/B 1.2/C | capsid protein VP1  
  
Loaded cryo-EM demo: PDB ID 6out, EMDB ID 20205  

> open 20205 fromDatabase emdb

Summary of feedback from opening 20205 fetched from emdb  
---  
note | Fetching compressed map 20205 from
ftp://ftp.ebi.ac.uk/pub/databases/emdb/structures/EMD-20205/map/emd_20205.map.gz  
  
Opened emdb 20205 as #2, grid size 98,112,104, pixel 1.07,1.07,1.07, shown at
level 0.287, step 1, values float32  

> open 20205 fromDatabase emdb

Opened emdb 20205 as #3, grid size 98,112,104, pixel 1.07,1.07,1.07, shown at
level 0.287, step 1, values float32  

> clipper associate #2 toModel #1

Opened emdb 20205 as #1.1.1.1, grid size 98,112,104, pixel 1.07,1.07,1.07,
shown at step 1, values float32  

> hide #!3 models

> isolde start

> ui dockable false ISOLDE

> set bgColor white

> hide #!1.1 models

> show #!1.1 models

> addh

Summary of feedback from adding hydrogens to 6out.pdb #1.2  
---  
notes | Termini for 6out.pdb (#1.2) chain A determined from SEQRES records  
Termini for 6out.pdb (#1.2) chain B determined from SEQRES records  
Termini for 6out.pdb (#1.2) chain C determined from SEQRES records  
Chain-initial residues that are actual N termini: /B THR 9  
Chain-initial residues that are not actual N termini: /A ASP 29, /C ASP 29  
Chain-final residues that are actual C termini: /A SER 520  
Chain-final residues that are not actual C termini: /B ALA 519, /C ALA 519  
Missing OXT added to C-terminal residue /A SER 520  
1438 hydrogen bonds  
Adding 'H' to /A ASP 29  
Adding 'H' to /C ASP 29  
/B ALA 519 is not terminus, removing H atom from 'C'  
/C ALA 519 is not terminus, removing H atom from 'C'  
11439 hydrogens added  
  

> hide HC

> isolde restrain ligands #1

> select #1

22945 atoms, 23062 bonds, 1720 residues, 14 models selected  

> isolde sim start sel

Traceback (most recent call last):  
File
"/Applications/ChimeraX-1.2.2.app/Contents/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-
packages/chimerax/core/triggerset.py", line 130, in invoke  
return self._func(self._name, data)  
File "/Users/fasselic/Library/Application Support/ChimeraX/1.2/site-
packages/chimerax/isolde/cmd/cmd.py", line 95, in _sim_start_cb  
isolde.start_sim()  
File "/Users/fasselic/Library/Application Support/ChimeraX/1.2/site-
packages/chimerax/isolde/isolde.py", line 2780, in start_sim  
sm.start_sim()  
File "/Users/fasselic/Library/Application Support/ChimeraX/1.2/site-
packages/chimerax/isolde/openmm/openmm_interface.py", line 674, in start_sim  
sh.start_sim()  
File "/Users/fasselic/Library/Application Support/ChimeraX/1.2/site-
packages/chimerax/isolde/openmm/openmm_interface.py", line 1601, in start_sim  
self._prepare_sim()  
File "/Users/fasselic/Library/Application Support/ChimeraX/1.2/site-
packages/chimerax/isolde/openmm/openmm_interface.py", line 1561, in
_prepare_sim  
s = self._simulation = app.Simulation(self.topology, self._system,  
File
"/Applications/ChimeraX-1.2.2.app/Contents/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-
packages/simtk/openmm/app/simulation.py", line 105, in __init__  
self.context = mm.Context(self.system, self.integrator, platform,
platformProperties)  
File
"/Applications/ChimeraX-1.2.2.app/Contents/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-
packages/simtk/openmm/openmm.py", line 2671, in __init__  
_openmm.Context_swiginit(self, _openmm.new_Context(*args))  
simtk.openmm.OpenMMException: Error compiling kernel: Compile Server Error.  
  
Error processing trigger "frame drawn":  
simtk.openmm.OpenMMException: Error compiling kernel: Compile Server Error.  
  
File
"/Applications/ChimeraX-1.2.2.app/Contents/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-
packages/simtk/openmm/openmm.py", line 2671, in __init__  
_openmm.Context_swiginit(self, _openmm.new_Context(*args))  
  
See log for complete Python traceback.  
  




OpenGL version: 4.1 ATI-3.10.21
OpenGL renderer: AMD Radeon Pro 5300M OpenGL Engine
OpenGL vendor: ATI Technologies Inc.Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro16,1
      Processor Name: 6-Core Intel Core i7
      Processor Speed: 2.6 GHz
      Number of Processors: 1
      Total Number of Cores: 6
      L2 Cache (per Core): 256 KB
      L3 Cache: 12 MB
      Hyper-Threading Technology: Enabled
      Memory: 16 GB
      Boot ROM Version: 1554.120.19.0.0 (iBridge: 18.16.14663.0.0,0)

Software:

    System Software Overview:

      System Version: macOS 10.15.7 (19H1217)
      Kernel Version: Darwin 19.6.0
      Time since boot: 2 days 12:18

Graphics/Displays:

    Intel UHD Graphics 630:

      Chipset Model: Intel UHD Graphics 630
      Type: GPU
      Bus: Built-In
      VRAM (Dynamic, Max): 1536 MB
      Vendor: Intel
      Device ID: 0x3e9b
      Revision ID: 0x0000
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1

    AMD Radeon Pro 5300M:

      Chipset Model: AMD Radeon Pro 5300M
      Type: GPU
      Bus: PCIe
      PCIe Lane Width: x8
      VRAM (Total): 4 GB
      Vendor: AMD (0x1002)
      Device ID: 0x7340
      Revision ID: 0x0043
      ROM Revision: 113-D3220E-190
      VBIOS Version: 113-D32207P1-019
      Option ROM Version: 113-D32207P1-019
      EFI Driver Version: 01.A1.190
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1
      Displays:
        Color LCD:
          Display Type: Built-In Retina LCD
          Resolution: 3072 x 1920 Retina
          Framebuffer Depth: 30-Bit Color (ARGB2101010)
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: No
          Connection Type: Internal
        LG FULL HD:
          Resolution: 1920 x 1080 (1080p FHD - Full High Definition)
          UI Looks like: 1920 x 1080 @ 60 Hz
          Framebuffer Depth: 30-Bit Color (ARGB2101010)
          Display Serial Number:             
          Mirror: Off
          Online: Yes
          Rotation: Supported
          Automatically Adjust Brightness: No
          Connection Type: DVI or HDMI
          Adapter Firmware Version: 7.36

Locale: (None, 'UTF-8')
PyQt5 5.15.2, Qt 5.15.2
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    appnope: 0.1.2
    Babel: 2.9.1
    backcall: 0.2.0
    blockdiag: 2.0.1
    certifi: 2020.12.5
    cftime: 1.4.1
    chardet: 3.0.4
    ChimeraX-AddCharge: 1.0
    ChimeraX-AddH: 2.1.6
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.1
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.13.3
    ChimeraX-AtomicLibrary: 3.1.3
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AtomSearchLibrary: 1.0
    ChimeraX-AxesPlanes: 2.0
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 1.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.5.2
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.1
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.1
    ChimeraX-Clipper: 0.16.1
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.2.1
    ChimeraX-CommandLine: 1.1.4
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.2.2
    ChimeraX-CoreFormats: 1.0
    ChimeraX-coulombic: 1.1.1
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0
    ChimeraX-DataFormats: 1.1
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1.2
    ChimeraX-DistUI: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.0
    ChimeraX-Hbonds: 2.1
    ChimeraX-Help: 1.1
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.0
    ChimeraX-ImageFormats: 1.1
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ISOLDE: 1.2.2
    ChimeraX-Label: 1.0
    ChimeraX-ListInfo: 1.1.1
    ChimeraX-Log: 1.1.2
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.1
    ChimeraX-Map: 1.0.2
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.0
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 1.2.1
    ChimeraX-MDcrds: 2.2
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.1
    ChimeraX-mmCIF: 2.3
    ChimeraX-MMTF: 2.1
    ChimeraX-Modeller: 1.0.1
    ChimeraX-ModelPanel: 1.0.1
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.1
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0.1
    ChimeraX-OpenCommand: 1.5
    ChimeraX-PDB: 2.4.1
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0.1
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0.1
    ChimeraX-PubChem: 2.0.1
    ChimeraX-ReadPbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.4
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SeqView: 2.3
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.0
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.0.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.3.1
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.1
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.0.1
    ChimeraX-ToolshedUtils: 1.2
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.7.4
    ChimeraX-uniprot: 2.1
    ChimeraX-UnitCell: 1.0
    ChimeraX-ViewDockX: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.3
    comtypes: 1.1.7
    cxservices: 1.0
    cycler: 0.10.0
    Cython: 0.29.21
    decorator: 5.0.7
    distlib: 0.3.1
    docutils: 0.16
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    h5py: 2.10.0
    html2text: 2020.1.16
    idna: 2.10
    ihm: 0.17
    imagecodecs: 2020.5.30
    imagesize: 1.2.0
    ipykernel: 5.3.4
    ipython: 7.18.1
    ipython-genutils: 0.2.0
    jedi: 0.17.2
    Jinja2: 2.11.2
    jupyter-client: 6.1.7
    jupyter-core: 4.7.1
    kiwisolver: 1.3.1
    line-profiler: 2.1.2
    lxml: 4.6.2
    lz4: 3.1.0
    MarkupSafe: 1.1.1
    matplotlib: 3.3.2
    matplotlib-inline: 0.1.2
    msgpack: 1.0.0
    netCDF4: 1.5.4
    networkx: 2.5
    numexpr: 2.7.3
    numpy: 1.19.2
    numpydoc: 1.1.0
    openvr: 1.14.1501
    packaging: 20.9
    ParmEd: 3.2.0
    parso: 0.7.1
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 7.2.0
    pip: 21.0.1
    pkginfo: 1.5.0.1
    prompt-toolkit: 3.0.18
    psutil: 5.7.2
    ptyprocess: 0.7.0
    pycollada: 0.7.1
    pydicom: 2.0.0
    Pygments: 2.7.1
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PyQt5-commercial: 5.15.2
    PyQt5-sip: 12.8.1
    PyQtWebEngine-commercial: 5.15.2
    python-dateutil: 2.8.1
    pytz: 2021.1
    pyzmq: 22.0.3
    qtconsole: 4.7.7
    QtPy: 1.9.0
    RandomWords: 0.3.0
    requests: 2.24.0
    scipy: 1.5.2
    setuptools: 50.3.2
    sfftk-rw: 0.6.7.dev1
    six: 1.15.0
    snowballstemmer: 2.1.0
    sortedcontainers: 2.2.2
    Sphinx: 3.2.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 1.0.3
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.4
    suds-jurko: 0.6
    tables: 3.6.1
    tifffile: 2020.9.3
    tinyarray: 1.2.3
    tornado: 6.1
    traitlets: 5.0.5
    urllib3: 1.25.11
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.36.0
    wheel-filename: 1.3.0

[Tristan Croll]

Something is going wrong with the just-in-time compilation of OpenMM's OpenCL kernels. I'm afraid there could be various reasons for this (all of which I'm afraid are outside of my control). Could be as simple as your /tmp directory being full, but I'm afraid the more likely explanation is a bug in your OpenCL driver. I've had one other report of the same issue - there the user realised the same problem was happening across multiple OpenCL programs, and was fixed by upgrading to Big Sur. If you don't want to go that far, you might just consider checking that all your drivers are up to date. It might also be worth trying "isolde set gpuDeviceIndex 1" to see if forcing the simulations to run on your other GPU helps.

One other thing that *might* help diagnose (I can't think of a good way to deliberately trigger an OpenCL error on my machine to check)... start ChimeraX from the terminal via the following command:

CL_LOG_ERRORS=stdout /Applications/ChimeraX-1.2.2.app/Contents/bin/ChimeraX

... which *might* print some more verbose error messages to the terminal.

[fasseli.coulibaly@…]

Thanks for the quick answer!

Somehow I managed to get it to work for a session. I thought it was simply
the "isolde set gpuDeviceIndex 1" that did the trick but I can't reproduce
that now. Running the "verbose" command I get:

OpenCL internal error: [CL_DEVICE_NOT_AVAILABLE] : OpenCL Error : Error:
Build Program driver returned (10015)
OpenCL internal error: OpenCL Warning : clBuildProgram failed: could not
build program for 0x1024500 (Intel(R) UHD Graphics 630) (err:-2)
OpenCL internal error: [CL_BUILD_ERROR] : OpenCL Build Error : Compiler
build log:
Compile Server Error.

Upgrading to Big Sur might be an option but I'm concerned with breaking
other installed programs that I need over the coming couple weeks.


On Mon, 31 May 2021 at 22:55, ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
wrote:

[Tristan Croll]

Thanks for digging more deeply. StackOverflow is filling up with lots of reports of the same CL_DEVICE_NOT_AVAILABLE problem recently... according to the first answer at https://stackoverflow.com/questions/67318465/rendering-ciimage-results-in-a-blank-image-on-macos-10-15-7-19h1030 it's only a problem with MacOS Catalina (10.15):

-----------
Catalina Security Update 2021-002 breaks OpenCL. clBuildProgram always fails with a CL_BUILD_PROGRAM_FAILURE, even with basic kernels, and clGetProgramBuildInfo returns an empty string, not providing any clue what's wrong. The security update release notes don't mention OpenCL at all, but this regression is directly caused by it. The corresponding Security Update for Mojave does not have this regression, and neither does Big Sur 10.3. Only on Catalina it is broken now. Filed a bug report. Please do so as well.
------------
________________________________
From: ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
Sent: 02 June 2021 02:26
To: Tristan Croll <tic20@cam.ac.uk>; fasseli.coulibaly@monash.edu <fasseli.coulibaly@monash.edu>
Subject: Re: [ChimeraX] #4702: OpenMMException: Error compiling kernel: Compile Server Error

#4702: OpenMMException: Error compiling kernel: Compile Server Error
------------------------------------------+---------------------------
          Reporter:  fasseli.coulibaly@…  |      Owner:  Tristan Croll
              Type:  defect               |     Status:  feedback
          Priority:  normal               |  Milestone:
         Component:  Third Party          |    Version:
        Resolution:                       |   Keywords:
        Blocked By:                       |   Blocking:
Notify when closed:                       |   Platform:  all
           Project:  ChimeraX             |
------------------------------------------+---------------------------

Comment (by fasseli.coulibaly@…):

 {{{
 Thanks for the quick answer!

 Somehow I managed to get it to work for a session. I thought it was simply
 the "isolde set gpuDeviceIndex 1" that did the trick but I can't reproduce
 that now. Running the "verbose" command I get:

 OpenCL internal error: [CL_DEVICE_NOT_AVAILABLE] : OpenCL Error : Error:
 Build Program driver returned (10015)
 OpenCL internal error: OpenCL Warning : clBuildProgram failed: could not
 build program for 0x1024500 (Intel(R) UHD Graphics 630) (err:-2)
 OpenCL internal error: [CL_BUILD_ERROR] : OpenCL Build Error : Compiler
 build log:
 Compile Server Error.

 Upgrading to Big Sur might be an option but I'm concerned with breaking
 other installed programs that I need over the coming couple weeks.


 On Mon, 31 May 2021 at 22:55, ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
 wrote:


 }}}

--
Ticket URL: <https://plato.cgl.ucsf.edu/trac/ChimeraX/ticket/4702#comment:4>
ChimeraX <http://www.rbvi.ucsf.edu/chimerax/>
ChimeraX Issue Tracker

[Tristan Croll]

Just doing some spring cleaning and closing out old tickets. Hopefully this is no longer an issue?