Problem with multiprocessing scripts

[Andrei Istrate]

The following bug report has been submitted:
Platform:        macOS-10.16-x86_64-i386-64bit
ChimeraX Version: 1.2.dev202101090324 (2021-01-09 03:24:40 UTC)
Description
There is a problem with using multiprocessing in scripts ran by ChimeraX after the update from Python 3.7 to Python 3.8 on mac.

The problem comes from the requirement to protect the “entry point” of the program by using if __name__ == '__main__'

Like this:

from multiprocessing import Process

def slave(a):
    print(a)

if __name__ == '__main__':
    thread_list = []
    for i in range(4):
        t = Process(target=slave, args=(i,))
        thread_list.append(t)
    for thread in thread_list:
        thread.start()
    for thread in thread_list:
        thread.join()

Which does not work beacause the scripts are executed in a sandbox



Log:
UCSF ChimeraX version: 1.2.dev202101090324 (2021-01-09)  
© 2016-2020 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> help help:contact.html

> ui tool show Shell

Populating font family aliases took 868 ms. Replace uses of missing font
family "Monospace" with one that exists to avoid this cost.  




OpenGL version: 4.1 ATI-4.2.13
OpenGL renderer: AMD Radeon Pro 560 OpenGL Engine
OpenGL vendor: ATI Technologies Inc.Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro14,3
      Processor Name: Quad-Core Intel Core i7
      Processor Speed: 2.9 GHz
      Number of Processors: 1
      Total Number of Cores: 4
      L2 Cache (per Core): 256 KB
      L3 Cache: 8 MB
      Hyper-Threading Technology: Enabled
      Memory: 16 GB
      System Firmware Version: 429.60.3.0.0
      SMC Version (system): 2.45f4

Software:

    System Software Overview:

      System Version: macOS 11.1 (20C69)
      Kernel Version: Darwin 20.2.0
      Time since boot: 20 days 6:21

Graphics/Displays:

    Intel HD Graphics 630:

      Chipset Model: Intel HD Graphics 630
      Type: GPU
      Bus: Built-In
      VRAM (Dynamic, Max): 1536 MB
      Vendor: Intel
      Device ID: 0x591b
      Revision ID: 0x0004
      Automatic Graphics Switching: Supported
      gMux Version: 4.0.29 [3.2.8]
      Metal Family: Supported, Metal GPUFamily macOS 2

    Radeon Pro 560:

      Chipset Model: Radeon Pro 560
      Type: GPU
      Bus: PCIe
      PCIe Lane Width: x8
      VRAM (Total): 4 GB
      Vendor: AMD (0x1002)
      Device ID: 0x67ef
      Revision ID: 0x00c0
      ROM Revision: 113-C980AJ-927
      VBIOS Version: 113-C9801AU-A02
      EFI Driver Version: 01.A0.927
      Automatic Graphics Switching: Supported
      gMux Version: 4.0.29 [3.2.8]
      Metal Family: Supported, Metal GPUFamily macOS 2
      Displays:
        Color LCD:
          Display Type: Built-In Retina LCD
          Resolution: 2880 x 1800 Retina
          Framebuffer Depth: 24-Bit Color (ARGB8888)
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: Yes
          Connection Type: Internal
        BenQ PD3200U:
          Resolution: 5120 x 2880 (5K/UHD+ - Ultra High Definition Plus)
          UI Looks like: 2560 x 1440 @ 60.00Hz
          Framebuffer Depth: 30-Bit Color (ARGB2101010)
          Display Serial Number: S9H01523019 
          Mirror: Off
          Online: Yes
          Rotation: Supported
          Automatically Adjust Brightness: No
          Connection Type: Thunderbolt/DisplayPort
          Television: Yes

Locale: (None, 'UTF-8')
PySide2 version: 5.15.2
Qt version: 5.15.2
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    appnope: 0.1.2
    Babel: 2.9.0
    backcall: 0.2.0
    blockdiag: 2.0.1
    certifi: 2020.12.5
    cftime: 1.3.0
    chardet: 3.0.4
    ChimeraX-AddH: 2.1.3
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.1
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.10.4
    ChimeraX-AtomicLibrary: 1.1.1
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AtomSearchLibrary: 1.0
    ChimeraX-AxesPlanes: 2.0
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 1.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.2
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.1
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.1
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-CommandLine: 1.1.3
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.2.dev202101090324
    ChimeraX-CoreFormats: 1.0
    ChimeraX-coulombic: 1.0.3
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-DataFormats: 1.1
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1
    ChimeraX-DistUI: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.0
    ChimeraX-Hbonds: 2.1
    ChimeraX-Help: 1.0
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.0
    ChimeraX-ImageFormats: 1.1
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0
    ChimeraX-Label: 1.0
    ChimeraX-ListInfo: 1.1
    ChimeraX-Log: 1.1.1
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.1
    ChimeraX-Map: 1.0.2
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.0
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 1.1
    ChimeraX-MDcrds: 2.2
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.0
    ChimeraX-mmCIF: 2.2
    ChimeraX-MMTF: 2.0
    ChimeraX-Modeller: 1.0
    ChimeraX-ModelPanel: 1.0
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.0
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0
    ChimeraX-OpenCommand: 1.4
    ChimeraX-PDB: 2.2
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0.1
    ChimeraX-PubChem: 2.0.1
    ChimeraX-Read-Pbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.3
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SeqView: 2.3
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.0
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.0.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.2.2
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.0
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.0
    ChimeraX-ToolshedUtils: 1.1
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.3.2
    ChimeraX-uniprot: 2.1
    ChimeraX-ViewDockX: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.3
    comtypes: 1.1.7
    cxservices: 1.0
    cycler: 0.10.0
    Cython: 0.29.21
    decorator: 4.4.2
    distlib: 0.3.1
    docutils: 0.16
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    h5py: 2.10.0
    html2text: 2020.1.16
    idna: 2.10
    ihm: 0.17
    imagecodecs: 2020.5.30
    imagesize: 1.2.0
    ipykernel: 5.3.4
    ipython: 7.18.1
    ipython-genutils: 0.2.0
    jedi: 0.17.2
    Jinja2: 2.11.2
    jupyter-client: 6.1.7
    jupyter-core: 4.7.0
    kiwisolver: 1.3.1
    line-profiler: 2.1.2
    lxml: 4.5.2
    lz4: 3.1.0
    MarkupSafe: 1.1.1
    matplotlib: 3.3.2
    msgpack: 1.0.0
    netCDF4: 1.5.4
    networkx: 2.5
    numexpr: 2.7.2
    numpy: 1.19.2
    numpydoc: 1.1.0
    openvr: 1.14.1501
    packaging: 20.8
    ParmEd: 3.2.0
    parso: 0.7.1
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 7.2.0
    pip: 20.3.1
    pkginfo: 1.5.0.1
    prompt-toolkit: 3.0.10
    psutil: 5.7.2
    ptyprocess: 0.7.0
    pycollada: 0.7.1
    pydicom: 2.0.0
    Pygments: 2.7.1
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PySide2: 5.15.2
    python-dateutil: 2.8.1
    pytz: 2020.5
    pyzmq: 20.0.0
    qtconsole: 4.7.7
    QtPy: 1.9.0
    RandomWords: 0.3.0
    requests: 2.24.0
    scipy: 1.5.2
    setuptools: 50.3.2
    sfftk-rw: 0.6.7.dev1
    shiboken2: 5.15.2
    six: 1.15.0
    snowballstemmer: 2.0.0
    sortedcontainers: 2.2.2
    Sphinx: 3.2.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 1.0.3
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.4
    suds-jurko: 0.6
    tables: 3.6.1
    tifffile: 2020.9.3
    tinyarray: 1.2.3
    tornado: 6.1
    traitlets: 5.0.5
    urllib3: 1.25.11
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.36.0
    wheel-filename: 1.2.0

[Greg Couch]

That is by design. ChimeraX, the application, needs to protect itself. And it injects the current session in to the global session variable.

The simple solution is to change your code:

if __name__ == '__main__' or __name__.startswith("ChimeraX_sandbox_"):
    main()

If you install your package in ChimeraX's Python (pip install --user ...), then you can write a shell script to invoke it:

#!/bin/sh
CHIMERAX=--where ChimeraX was installed--
app="$CHIMERAX/bin/ChimeraX"
if [ -e "${app}-console" ]
then
        app="${app}-console"
fi
if test $# -gt 0
then
        exec "$app" -m PACKAGE "$@"
else
        exec "$app" -m PACKAGE
fi

[Andrei Istrate]

Dear Greg,

Thank you for looking into this.
I tried to run this sample code:

from multiprocessing import Process


def slave(a):
    print(a)

if __name__ == '__main__' or __name__.startswith("ChimeraX_sandbox_"):
    thread_list = []
    for i in range(4):
        t = Process(target=slave, args=(i,))
        thread_list.append(t)
    for thread in thread_list:
        thread.start()
    for thread in thread_list:
        thread.join()
using both your suggestions

/Applications/ChimeraX_Daily_2.app/Contents/bin/ChimeraX --nogui --script "/Users/andrei/PycharmProjects/toy_module/toy_module/toy_module.py”


and


/Applications/ChimeraX_Daily_2.app/Contents/bin/ChimeraX -m toy_module.toy_module


in both cases I get this error:

/Applications/ChimeraX_Daily_2.app/Contents/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ChimeraX_main.py: Unknown argument '--'
usage: /Applications/ChimeraX_Daily_2.app/Contents/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ChimeraX_main.py [--debug] [--devel] [--exit] [--nogui] [--nocolor] [--help] [--lineprofile] [--listioformats] [--offscreen] [--silent] [--nostatus] [--start <tool name>] [--cmd <command>] [--script <python script and arguments>] [--notools] [--safemode] [--stereo] [--uninstall] [--usedefaults] [--version] [--qtscalefactor <factor>] [--toolshed preview|<url>] or Python command line arguments (e.g., -m module_name [args] or -c command [args])


The error is printed for each child process, four times in this case.

Thank you,
Andrei