NIAID SOW Proposed deliverables (based on 2 FTE), January 2020

August 1, 2020 - February 28 , 2021.

Original ​Google doc version. Earlier SOW draft. Extra 2.5 FTE deliverables.

1. Extensions to ChimeraX to support the NIH 3D pipeline
   1. Support the conversion of existing NIH 3D Print Exchange Chimera scripts to ChimeraX, advising on ChimeraX syntax for capabilities where available and noting missing features in ChimeraX where comparable features don’t exist
      1. Investigate using Smart Display
      2. Discuss Python vs. ChimeraX command line
   2. Implement essential missing features discovered above, including:
      1. PubChem fetch
      2. Additional import formats, including mol and sdf
      3. PDB biounit fetch #3515
   3. Sequence conservation coloring in ChimeraX
   4. Fetch sequence annotations (UniProt, domains, disease-associated mutations) (Eric)
      1. Automatic labels
   5. Enhancements to X3D export format (Greg)
      1. Look at GLTF export improvements (support for both per vertex coloring and texture coloring)
   6. Read and visualize segmentation models (see deliverable 2) (Tom)

2. Significantly enhance ChimeraX’s segmentation capabilities for 3d electron microscopy, light microscopy, and medical imaging
   1. Interactive SimpleITK use in ChimeraX (Tom)
   2. Allow loading, visualizing, creating, measuring and saving segmentations. (Tom)
   3. Support new EMDB-SFF segmentation file format from the EM Databank (Tom). #3639 in progress
   4. NOTE: some example data sets would be useful - AI Detected Lesions in lungs. Phil will get some data to TomG. Could also look at some data sets from SimpleITK. Bryan Hansen and Beth Fisher could also get some data sets

3. Medical imaging
   1. Metadata browser for DICOM files (Tom)
   2. Support radiologist collaborator needs by adapting ChimeraX to better suit standard workflows and to provide critical missing features identified by our collaborators (Tom)
      1. Need to get a collaborator
      2. Les Folio / Ron Summers
      3. New NIAID section focused on lung imaging
      4. Roberto Rodriguez Rubio, MD at UCSF director of ​Skull Base lab, interested in VR. Emphasis on dissections, no medical imaging.

4. Improve multi-person VR beyond work accomplished in 2019 SOW
   1. Localizable connection server solution - deliver a rendezvous server that can easily be deployed by institutions or enterprises. (Conrad/Greg)
      1. Fix cxconference TLS security #3629
      2. Get TLS certificate for plato cxconference server (Greg)
      3. Make cxconference work with VR #2892, #2893, #3498, #3512
      4. Greg to help containerize cxconference
5. Human Biomolecular Atlas Program (HuBMAP) multiscale visualization
   1. Engage with the HuBMAP group to evaluate the potential for ChimeraX extensions to support exploration and analysis of HuBMAP data, particularly as those extensions align with NIAID areas of interest. This will include determining the programmatic interfaces to access the HuBMAP data and initial efforts to scope the task of supporting the various HuBMAP data formats.
      1. Discuss needs with Meghan McCarthy and Darrell Hurt
6. Outreach
   1. Workshops, Training and presentations
   2. Instructional material and tools documentation.
      1. Detailed instructions for all features shall be provided in a user manual
      2. Written user guides and tutorials shall be available as an HTML page
7. Administration
   1. A project schedule, implementation plan, and documentation plan will be provided
   2. Monthly status reports detailing progress on each subtask will be delivered

Additional Requests from Users

1. Adding Coulombic coloring (Eric) #3596 (Elaine: this is one of the essential missing features mentioned in 1b above)
   1. 90% cases covered
2. Colada export for preparing models for PathogenAR cellphone app (Tom) #485
3. Add support for SDF V3000 (Eric)

Meeting Notes

9/24/2020

Darrell Hurt, Phil Cruz, Meghan McCarthy, Scooter Moris, Greg Couch, Tom Goddard, Elaine Meng, Eric Pettersen

1. NIH 3D pipeline status
   1. they're working on the architecture, backend, using GLTF (.glb) rather than X3D
   2. Darrell: Philip will use ChimeraX as executable rather than reaching into python
   3. Scooter: how about REST interface approach with Json; are they interested?
   4. Darrell: makes sense, but need Philip to weigh in

2. VR meeting
   1. TomG, Phil, Victor tried 3-way meeting to work on firewall issues.
   2. Direct connection to TomG's home computer worked.
   3. AWS firewall mis-configured, test failed, will try again.
   4. Found several bugs in multiuser VR, created tickets. #3735, #3736, #3737, #3738, #3739, #3740, #3755.
   5. Discussed naming meetings using a rendezvous service, simplifying AWS setup, providing free AWS service hosted by NIAID or UCSF.

3. V3000 SDF support from Eric
   1. Phil is happy, allows them to jettison OpenBabel
   2. Elaine put this in Change Log dated Sept 15

4. macOS Big Sur
   1. TomG mentions ChimeraX fails to on macOS Big Sur beta, patched in daily build
   2. Phil says they aren't even at Catalina yet

5. VR hand tracking
   1. Darrell asks about VR hand-tracking, shows video of putting on gloves, PPE
   2. TomG has not looked at hand-tracking since the Leap device has poor tracking

10/08/2020

Darrell, Meghan, Phil, Philip, Kristen, Victor; Scooter, Greg, Elaine, Eric, TomG

• ChimeraX REST interface and Json output
  • Eric demo with GUI but can be done with script in nogui mode
  • "remotecontrol" to start REST, put URL shown in the ChimeraX Log into web browser; the resulting page allows entering commands and getting json replies, if any
  • currently Json output is implemented for "open" and "info" commands
  • would be useful for "info chain" to report polymer type in addition to whether there are any polymers
  • maybe we can provide a prototype python script to start rest, open structure, get information on chains and their types
  • Eric will improve user-error reporting in json

• VR experiment 1 wk ago with 4-way: Phil, Meghan, Victor, TomG via AWS
  • found more bugs, including "ghosts" (now fixed)
  • TomG recently added (Sep 30) features: new input model type ​photo (2D image, e.g. for showing a paper figure in VR) and new ​move picked model mouse mode to move models w/o having to select them first
  • TomG plans to add remembering the face image, name, and hand-cone color previously specified for VR meeting
  • worked pretty well on cheapest AWS tier
  • perhaps NIAID could provide the hub
  • Greg wonders about security; TomG mentions ssh tunneling, Scooter mentions generated password

10/22/2020

Darrell, Meghan, Phil, Philip, Kristen, Victor; Greg, Elaine, Eric, TomG

• Scooter on vacation for a month, sailing home from Mexico, returns Nov 22 (Elaine: this is approximate, weather-dependent)

NIH 3D pipeline:

• Working on converting their scripts to ChimeraX
  • Joined github repository for scripts set up by Philip; it seems that 2-factor authentication will not be a problem for our day-to-day work
  • Got 3 scripts from Phil Cruz
  • Eric: only few issues with the 2 shorter scripts
    • ChimeraX does not write VRML; not absolute requirement, can convert glb to vrml in separate step
    • ChimeraX "graphics quality" can be used to adjust numbers of triangles for STL output
    • no incremental radius adjustment a la Chimera vdwdefine plus/minus; Eric will supply python to do this
    • no combine, why was it used? Phil: to handle biological assemblies that were multiple models. Eric: now the fetch gets the assemblies as single models
  • Philip asks about taking all the file reading and branching out of the ChimeraX scripts and instead have several separate ChimeraX scripts that are called; however, parsing with ChimeraX has several advantages
  • Phil: actually may want to combine small-molecule and macromolecule scripts into one to reduce user error; distinction is presence of "standard residues" (biopolymer chains)
  • Darrell: how about also combining with volume script, have ChimeraX identify if upload is map? Eric: should be doable, at least if filename has standard suffix
  • Darrell: will also need to distinguish which database goes with an identifier for fetch purposes
  • Phil: if REST, is a hybrid approach possible?
  • Eric has improved error reporting in JSON

Potential topics not covered:

• VR meetings: firewalls, slow networks, 3 or 4 participants
  • Wrote documentation for setting up AWS server added to meeting command web page
  • David Liou has set up NIAID AWS server, will test tomorrow
  • Working on improved handling of low-bandwidth connections
  • Adding meeting command proxy options for simpler use of AWS