RFE: Remote display on a compute cluster

[alphataubio@…]

The following bug report has been submitted:
Platform:        macOS-14.4.1-arm64-arm-64bit
ChimeraX Version: 1.8rc202405240417 (2024-05-24 04:17:52 UTC)
Description
[FEATURE REQUEST] remote display on a compute cluster

im hitting the size limit of what datasets i can process and visualize on my little mac mini with 16gb ram. i tried opening a 10gb dcd trajectory but ChimeraX.app blew up to 24gb of memory usage. im not complaining, i accept that's a reasonable amount of resources to open a trajectory of that size. what im doing now is postprocessing 1 out of 10 (or 1 out of 100 frames) using mdanalysis in python then bringing that small sliced .dcd into chimerax on my home computer for visualization.

However, im trying to render a 2 minute movie for youtube at 60 frames/sec and 4k resolution. This is well beyond the capacity of my mac mini. I would guess im not the only user that's dealing with large datasets generated from simulations on clusters.

​https://www.cgl.ucsf.edu/chimerax/system_requirements.html(​https://www.cgl.ucsf.edu/chimerax/system_requirements.html) :

often provides unreliable or slow interactive performance when the rendering or display is not on the same computer that is running the application

not a problem when im on campus with a gigabit connection to the cluster(s).

We also do not support running Chimera or ChimeraX on a virtual machine, as configuring graphics drivers to use GPU acceleration on a virtual machine presents technical issues. The ChimeraX development team does not have the computer systems on which to test these unsupported configurations.

im willing to pitch in. i have access to cluster nodes with 256gb memory and four V100 GPUs with 32gb each. what are the technical issues ?

the main issue for me is actually requiring sudo access to install chimerax. ill never get root access to the cluster nodes (nor should i) i install all my stuff in ~/local.

OpenGL version: 4.1 Metal - 88
OpenGL renderer: Apple M1
OpenGL vendor: Apple

Python: 3.11.4
Locale: UTF-8
Qt version: PyQt6 6.6.1, Qt 6.6.1
Qt runtime version: 6.6.3
Qt platform: cocoa
Hardware:

    Hardware Overview:

      Model Name: Mac mini
      Model Identifier: Macmini9,1
      Model Number: G12N1LL/A
      Chip: Apple M1
      Total Number of Cores: 8 (4 performance and 4 efficiency)
      Memory: 16 GB
      System Firmware Version: 10151.101.3
      OS Loader Version: 10151.101.3

Software:

    System Software Overview:

      System Version: macOS 14.4.1 (23E224)
      Kernel Version: Darwin 23.4.0
      Time since boot: 11 days, 20 hours, 53 minutes

Graphics/Displays:

    Apple M1:

      Chipset Model: Apple M1
      Type: GPU
      Bus: Built-In
      Total Number of Cores: 8
      Vendor: Apple (0x106b)
      Metal Support: Metal 3
      Displays:
        DELL U3011:
          Resolution: 2560 x 1600
          UI Looks like: 2560 x 1600 @ 60.00Hz
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Rotation: Supported


Installed Packages:
    alabaster: 0.7.16
    appdirs: 1.4.4
    appnope: 0.1.4
    asttokens: 2.4.1
    Babel: 2.15.0
    beautifulsoup4: 4.12.3
    blockdiag: 3.0.0
    blosc2: 2.0.0
    build: 1.2.1
    certifi: 2023.11.17
    cftime: 1.6.3
    charset-normalizer: 3.3.2
    ChimeraX-AddCharge: 1.5.17
    ChimeraX-AddH: 2.2.6
    ChimeraX-AlignmentAlgorithms: 2.0.2
    ChimeraX-AlignmentHdrs: 3.5
    ChimeraX-AlignmentMatrices: 2.1
    ChimeraX-Alignments: 2.12.7
    ChimeraX-AlphaFold: 1.0
    ChimeraX-AltlocExplorer: 1.1.1
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.1
    ChimeraX-Atomic: 1.57
    ChimeraX-AtomicLibrary: 14.0.5
    ChimeraX-AtomSearch: 2.0.1
    ChimeraX-AxesPlanes: 2.4
    ChimeraX-BasicActions: 1.1.2
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 2.4.6
    ChimeraX-BondRot: 2.0.4
    ChimeraX-BugReporter: 1.0.1
    ChimeraX-BuildStructure: 2.12.1
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.2.6
    ChimeraX-ButtonPanel: 1.0.1
    ChimeraX-CageBuilder: 1.0.1
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.4
    ChimeraX-ChangeChains: 1.1
    ChimeraX-CheckWaters: 1.4
    ChimeraX-ChemGroup: 2.0.1
    ChimeraX-Clashes: 2.2.4
    ChimeraX-ColorActions: 1.0.5
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.5.6
    ChimeraX-CommandLine: 1.2.5
    ChimeraX-ConnectStructure: 2.0.1
    ChimeraX-Contacts: 1.0.1
    ChimeraX-Core: 1.8rc202405240417
    ChimeraX-CoreFormats: 1.2
    ChimeraX-coulombic: 1.4.3
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0.1
    ChimeraX-DataFormats: 1.2.3
    ChimeraX-Dicom: 1.2.4
    ChimeraX-DiffPlot: 1.0
    ChimeraX-DistMonitor: 1.4.2
    ChimeraX-DockPrep: 1.1.3
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ESMFold: 1.0
    ChimeraX-FileHistory: 1.0.1
    ChimeraX-FunctionKey: 1.0.1
    ChimeraX-Geometry: 1.3
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.1.1
    ChimeraX-Hbonds: 2.4
    ChimeraX-Help: 1.2.2
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.2
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ItemsInspection: 1.0.1
    ChimeraX-IUPAC: 1.0
    ChimeraX-Label: 1.1.10
    ChimeraX-ListInfo: 1.2.2
    ChimeraX-Log: 1.1.6
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.9.1
    ChimeraX-Map: 1.2
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0.1
    ChimeraX-MapFilter: 2.0.1
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1.1
    ChimeraX-Markers: 1.0.1
    ChimeraX-Mask: 1.0.2
    ChimeraX-MatchMaker: 2.1.3
    ChimeraX-MCopy: 1.0
    ChimeraX-MDcrds: 2.7.1
    ChimeraX-MedicalToolbar: 1.0.2
    ChimeraX-Meeting: 1.0.1
    ChimeraX-MLP: 1.1.1
    ChimeraX-mmCIF: 2.14.1
    ChimeraX-MMTF: 2.2
    ChimeraX-Modeller: 1.5.16
    ChimeraX-ModelPanel: 1.5
    ChimeraX-ModelSeries: 1.0.1
    ChimeraX-Mol2: 2.0.3
    ChimeraX-Mole: 1.0
    ChimeraX-Morph: 1.0.2
    ChimeraX-MouseModes: 1.2
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nifti: 1.2
    ChimeraX-NMRSTAR: 1.0.2
    ChimeraX-NRRD: 1.2
    ChimeraX-Nucleotides: 2.0.3
    ChimeraX-OpenCommand: 1.13.4
    ChimeraX-PDB: 2.7.5
    ChimeraX-PDBBio: 1.0.1
    ChimeraX-PDBLibrary: 1.0.4
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0.1
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.1.1
    ChimeraX-PubChem: 2.2
    ChimeraX-ReadPbonds: 1.0.1
    ChimeraX-Registration: 1.1.2
    ChimeraX-RemoteControl: 1.0
    ChimeraX-RenderByAttr: 1.4.1
    ChimeraX-RenumberResidues: 1.1
    ChimeraX-ResidueFit: 1.0.1
    ChimeraX-RestServer: 1.2
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 4.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.5.1
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0.2
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0.1
    ChimeraX-Segmentations: 3.0.11
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.11.2
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0.1
    ChimeraX-Shortcuts: 1.1.1
    ChimeraX-ShowSequences: 1.0.3
    ChimeraX-SideView: 1.0.1
    ChimeraX-Smiles: 2.1.2
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.16.5
    ChimeraX-STL: 1.0.1
    ChimeraX-Storm: 1.0
    ChimeraX-StructMeasure: 1.2.1
    ChimeraX-Struts: 1.0.1
    ChimeraX-Surface: 1.0.1
    ChimeraX-SwapAA: 2.0.1
    ChimeraX-SwapRes: 2.5
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-TaskManager: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.1.2
    ChimeraX-ToolshedUtils: 1.2.4
    ChimeraX-Topography: 1.0
    ChimeraX-ToQuest: 1.0
    ChimeraX-Tug: 1.0.1
    ChimeraX-UI: 1.39.1
    ChimeraX-uniprot: 2.3
    ChimeraX-UnitCell: 1.0.1
    ChimeraX-ViewDockX: 1.4.2
    ChimeraX-VIPERdb: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0.1
    ChimeraX-vrml: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0.2
    ChimeraX-WebServices: 1.1.4
    ChimeraX-Zone: 1.0.1
    colorama: 0.4.6
    comm: 0.2.2
    contourpy: 1.2.1
    cxservices: 1.2.2
    cycler: 0.12.1
    Cython: 3.0.10
    debugpy: 1.8.1
    decorator: 5.1.1
    docutils: 0.20.1
    executing: 2.0.1
    filelock: 3.13.4
    fonttools: 4.51.0
    funcparserlib: 2.0.0a0
    glfw: 2.7.0
    grako: 3.16.5
    h5py: 3.11.0
    html2text: 2024.2.26
    idna: 3.7
    ihm: 1.0
    imagecodecs: 2024.1.1
    imagesize: 1.4.1
    ipykernel: 6.29.2
    ipython: 8.21.0
    ipywidgets: 8.1.2
    jedi: 0.19.1
    jinja2: 3.1.4
    jupyter-client: 8.6.0
    jupyter-core: 5.7.2
    jupyterlab-widgets: 3.0.10
    kiwisolver: 1.4.5
    line-profiler: 4.1.2
    lxml: 5.2.1
    lz4: 4.3.3
    MarkupSafe: 2.1.5
    matplotlib: 3.8.4
    matplotlib-inline: 0.1.7
    msgpack: 1.0.8
    nest-asyncio: 1.6.0
    netCDF4: 1.6.5
    networkx: 3.3
    nibabel: 5.2.0
    nptyping: 2.5.0
    numexpr: 2.10.0
    numpy: 1.26.4
    openvr: 1.26.701
    packaging: 23.2
    ParmEd: 4.2.2
    parso: 0.8.4
    pep517: 0.13.1
    pexpect: 4.9.0
    pillow: 10.3.0
    pip: 24.0
    pkginfo: 1.10.0
    platformdirs: 4.2.2
    prompt-toolkit: 3.0.43
    psutil: 5.9.8
    ptyprocess: 0.7.0
    pure-eval: 0.2.2
    py-cpuinfo: 9.0.0
    pycollada: 0.8
    pydicom: 2.4.4
    pygments: 2.17.2
    pynmrstar: 3.3.4
    pynrrd: 1.0.0
    PyOpenGL: 3.1.7
    PyOpenGL-accelerate: 3.1.7
    pyopenxr: 1.0.3401
    pyparsing: 3.1.2
    pyproject-hooks: 1.1.0
    PyQt6-commercial: 6.6.1
    PyQt6-Qt6: 6.6.3
    PyQt6-sip: 13.6.0
    PyQt6-WebEngine-commercial: 6.6.0
    PyQt6-WebEngine-Qt6: 6.6.3
    python-dateutil: 2.9.0.post0
    pytz: 2024.1
    pyzmq: 26.0.3
    qtconsole: 5.5.1
    QtPy: 2.4.1
    RandomWords: 0.4.0
    requests: 2.31.0
    scipy: 1.13.0
    setuptools: 69.5.1
    setuptools-scm: 8.0.4
    sfftk-rw: 0.8.1
    six: 1.16.0
    snowballstemmer: 2.2.0
    sortedcontainers: 2.4.0
    soupsieve: 2.5
    sphinx: 7.2.6
    sphinx-autodoc-typehints: 2.0.1
    sphinxcontrib-applehelp: 1.0.8
    sphinxcontrib-blockdiag: 3.0.0
    sphinxcontrib-devhelp: 1.0.6
    sphinxcontrib-htmlhelp: 2.0.5
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.7
    sphinxcontrib-serializinghtml: 1.1.10
    stack-data: 0.6.3
    superqt: 0.6.3
    tables: 3.8.0
    tcia-utils: 1.5.1
    tifffile: 2024.1.30
    tinyarray: 1.2.4
    tornado: 6.4
    traitlets: 5.14.2
    typing-extensions: 4.12.0
    tzdata: 2024.1
    urllib3: 2.2.1
    wcwidth: 0.2.13
    webcolors: 1.13
    wheel: 0.43.0
    wheel-filename: 1.4.1
    widgetsnbextension: 4.0.10

[Zach Pearson]

You can try X11 forwarding with XQuartz: ​https://www.cyberciti.biz/faq/apple-osx-mountain-lion-mavericks-install-xquartz-server/

[Greg Couch]

XQuartz doesn't solve the 3D graphics problem (the 3D graphics is still done locally, and while remote 3D graphics can work, it often doesn't). For 3D graphics it helps to convert the 3D data into an image with VirtualGL on the remote Linux server. For speed, VirtualGL needs a 3D graphics card (and driver) installed on the server. There are also remote desktop applications that might work (some incorporate VirtualGL) -- but make sure you try before you buy. Your cluster might already have a preferred remote desktop application.

[Tom Goddard]

I think you will find you are much better off on the Mac Mini. A 2 minute movie is 120 seconds and at 60 frames per second that is 7200 frames. For a 100,000 atom simulation you have about 1.2 Mbytes of coordinates per frame (assuming 32-bit float) which would be about 8 Gbytes of memory. I'm not sure if ChimeraX will be that efficient with memory and might convert to 64-bit float. Then your 16 Gbytes of memory is too little. Trying to visualize large science data with 16 Gbytes of data is a bit absurd given that upgrading your Mac Mini to 64 GB costs about $150. I don't think 4 antique V100 GPUs (released 2017) are going to help you even a little. First ChimeraX can only use one. Your Mac Mini probably has not much worse GPU performance and much faster SSD drive disk than spinning disk network storage on the typical compute cluster. So I'd say your Mac Mini would likely render the movie as fast as a cluster V100.

if you really wanted to speed it up, any more modern desktop (Mac with more GPU cores, Windows or Linux with a $300 gaming graphics card) would likely be a factor of a few times faster. Given that you will only render a few times, I don't think a factor of a few makes much difference. It will likely take hours or overnight no matter what machine you use.

Another thing to consider is if you are going to try to show molecular surfaces or something else that takes a lot of computation per frame then the graphics speed is irrelevant because it will be bottlenecked by the CPU speed.