ChimeraX Medical Imaging and Virtual Reality Progress Report

Tom Goddard and Elaine Meng
Feb 25, 2019

Here are improvements made to ChimeraX for viewing DICOM medical imaging data on desktop displays and with virtual reality headsets. These new features were added December 15, 2018 to February 28, 2019 under a contract with Medical Science and Computing and the National Institute of Allergy and Infectious Diseases.

(1) Tutorial for viewing DICOM image data in VR. Elaine Meng developed a tutorial covering many of the improvements described below.

(2) ChimeraX DICOM Reference. Contains detailed descriptions of many of the features described below.

(3) Read DICOM files. ChimeraX can open DICOM image series.

(4) Appearance presets. A new volume command option "appearance" allows setting standard brightness and color curves taken from Horos, for example, volume #1 appearance "Airways II".

(5) Windowing and leveling VR mode. Added a new VR hand controller mode and mouse mode to do windowing (changing width of range of data values shown) and leveling (shifting range of data values shown).

(6) Data planes VR mode. Added a new VR hand controller mode and mouse mode to move slice plane through data set. Also allows moving 3-orthogonal planes.

(7) New toolbar buttons for image display. Added seven new image toolbar icons to control the appearance of image data: show single plane, show orthogonal planes, show full data, use axis aligned planes when rendering, use view aligned planes when rendering, airways brightness profile, default brightness profile.

(8) Inspecting image values. Hovering the mouse over a 2D image slice reports the grid index and data value in the 3D image.

(9) VR user interface improvements.

Colormap on GPU Colormap on CPU

(10) Better quality image rendering. Render 3d images as a stack of planes perpendicular to the view direction ("view aligned planes"). The older method used a stack of planes aligned with the data axes and switched planes when the data is rotated causing visible jumps in brightness particularly for data where z plane spacing is much larger than xy spacing.

(11) Realtime color, brightness and transparency changes in VR. Allowed mapping image values to colors on the graphics processor (GPU) instead of the CPU for smooth interactive windowing and leveling in VR. Old slower CPU method caused severe flicker in VR. This also improved appearance reducing noise by interpolating the image data values on the GPU. See images comparing GPU coloring to CPU coloring.

(12) Realtime image cropping in VR. Allow keeping all image data on graphics processor so that cropping to show subregions or moving a slab, or moving orthogonal planes through the data is fast and flicker-free in VR. Formerly only the shown part of the image was kept on the GPU.