NTT DATA UCSF NIAID SEB Subcontract 2021 Task Order #2

March 1, 2021 - February 28, 2022 from ​Google Doc, continuation of NETE contract (Aug 2020 - Feb 2021).

1. General ChimeraX improvements to support NIAID (Eric: 20%)

2. Support the NIH 3D pipeline development, including any changes to ChimeraX to support ongoing development. (Eric: 15%)

3. Improve multi-person VR (Tom: 35%)
   1. Create video tutorials in how to use multi-person VR
   2. Hold VR office hours to train researchers to use VR capabilities
   3. Allow participants to add models from local files during a VR session
   4. Set up VR meeting servers for Africa, Europe, Asia

4. Human Biomolecular Atlas Program (HuBMAP) multiscale visualization (Eric: 30%)
   1. Develop ChimeraX tool to browse and select HuBMAP datasets for loading

5. Enhance ChimeraX’s segmentation capabilities
   1. Investigate increased use of simple ITK to support segmentation (Tom: 15%)
   2. Investigate utilizing web services to support more complex machine learning-based segmentation algorithms (Tom: 15%)
   3. Investigate reading and displaying segmentation format(s) available from imaging databases (tissue microscopy, cancer imaging data, etc.) (Tom: 10%)

6. DICOM (Tom: 25%, Eric: 5%)
   1. Improve ability to visualize and analyze medical images in VR with ChimeraX
   2. Develop browser-style interface to show DICOM dataset contents and associated metadata such as sample characteristics

7. Outreach
   1. Instructional material and tools documentation. (20%)
      1. Detailed instructions for all features shall be provided in a user manual
      2. Written user guides and tutorials shall be available as an HTML page

8. Administration (10%)
   1. Submit monthly written reports of accomplishments

Meeting Minutes

3/12/2021

Phil, Philip, Darrell (2nd half); Elaine, Eric, TomG

Problem reported last time with strut disconnection in X3D exports has been fixed by Greg. Phil will test it.

Phil will try viral capsids with both a single combined surface and separate surfaces. Each may be preferred in different situations (e.g. coloring looks better with separate surfaces but they may cause 3D printing problems).

Use "graphics triangles" to report the number of triangles for better idea of export file size.

3D pipeline developers are thinking about parallelization of postprocessing. Multiple outputs for a single user input structure, may need to organize them by purpose (e.g. printing) in the interface or ask the user about purpose and then present the appropriate outputs.

We have been adding more GUIs to ChimeraX: volume tools, color key, etc. but still many more to go: dock prep, etc.

Phil asks about copy/combine functionality, not yet in ChimeraX. In the case of where the same model is split and then put back together, can instead use ChimeraX mouse mode rotate/translate selected atoms only.

Phil and Meghan want to know more about the SPOKE project and its data sources. Elaine will send them some links.

Darrell relaying concerns from people at Rocky Mt Lab that our interactions with them and with this NIAID in-house group might conflict (?) We aren't sure what this means.

Darrell suggests TomG or bigger ChimeraX team may visit NIH Bethesda for in-person collaboration and training similar to previous Chimera workshop at NIH.

Phil wants to try Virtual Desktop wireless VR with Quest 2 to PC with ChimeraX.

Next Wed 6 AM Phil will be training new postdoc in Mali to use VR. TomG can join VR meeting if desired. Will use Africa RENU VR server.

Phil attended VR in Public Health conference this week. VR will be in doctor's offices soon and has billable codes for insurance.

3/24/2021

Phil, Philip, Darrell; Elaine, Eric, TomG, Greg

Any progress on NIH3D pipeline as pertains to ChimeraX?

• Philip has incorporated the chimeraX unified py file into the workflow, says ChimeraX download from UCSF is slow. Error codes 64 (faulty usage) and 70 (internal software error) plus "expect trouble" (using gui. To avoid this, need to use nogui mode via --offscreen startup flag)
• Script calls sys.exit. Greg says zero/nonzero should be all you see, must be something with your workflow, and nonzero = error.
• NIH3D pipeline will be using cloud computing, currently not using systems with graphics hardware.
• offscreen rendering is currently slower, uses CPU rather than GPU. Can OSMESA use multiple CPU threads? Greg and possibly others will look into it.
• ambient lighting is slower, script already does size-dependent lighting
• Scooter: next step could be performance testing. Darrell's seen very slow examples with Chimera. Philip: let's try with ChimeraX, could well be different. Scooter: should try those specific examples, Phil will try to find them in the logs.

Did you want to know anything else about SPOKE?

• Phil: it was related to meeting with NASA re drug discovery to treat radiation effects
• NASA folks are combining lots of data types related to radiation effects (e.g. from exposure in space, nuclear accidents, etc.), including CT

VR stuff

• Phil reports problematic ChimeraX VR meeting with Mali last week; may have been internet connectivity to name server.
• TomG encourages use of "report a bug" and looping him in to meetings, in case he could help.
• Darrell: latency to Uganda was measured at ~350 milliseconds.
• Phil: how best to specify meeting proxy server? TomG: use Access menu in the Options of the GUI, but you have to use a command (meeting access? meeting settings?) to add a new server to the menu.
• TomG and Willan made a 60 minute long two-person VR video.
• we added a new ChimeraX videos page, includes some VR
• Video tutorials can play in ChimeraX help browser, at least on Mac, tested by Elaine.

4/8/2021

Phil, Philip, Meghan; Elaine, Eric, TomG, Scooter

Meghan: we've talked with an AR/VR group (​imisi3d.com) in Nigeria. They have downloaded and tried ChimeraX, and will do a remote meeting with NIAID folks, who will show them medical image data.

Phil: I have a collaboration with an investigator here to examine how two protein structures might interact, VR really helps with maintaining multiple constraints simultaneously. Also used HADDOCK protein-protein docking. Now have a compelling model of how hemoglobin and eNOS might fit together, will help guide peptide competition experiments. Steven Brooks et al., manuscript is in medarchive, will be submitted to Circulation.

Discussion of vaccines for influenza and covid-19 made by fusing viral proteins or their fragments to ferritin.

They have showed ChimeraX VR to "new guy" Wilbert van P and deputy director Jill Harper.

Tom G: would like to make a video showing multiperson AR/VR on SARS-CoV2 spike mutant(s). Would be great if Fauci could do it! Meghan: or get Fauci to do voiceover. Have been discussing similar with Barney (?)

ChimeraX 1.2 release is slated for end of month.

Tom G: will be working with Rodgers in Uganda to solve Mali VR meeting connection issues. SLAC also has firewall connection issues.

4/22/2021

Meghan, Philip, Darrell; Eric, TomG, Greg, Scooter

Network security

• Discussion of network breaches effecting NIH and UCSF.

NIH 3D giving users time estimates

• Good for users to know if job will takes 1 minute or 30 minutes
• Philip says another project (nefali?) output progress log, but no time estimates
• Tom says useful to get percent complete estimates for longest running step

X3D ChimeraX including meta data

• Greg Couch asked about including per-residue meta-data in X3D
• Meghan says want meta data for data attribution in GLB

Barney's covid-19 Chimera session file

• Meghan trying to use Chimera on spike model, Chimera session from Phil Cruz (on vacation), to help Barney (virus researcher)
• Chimera molecular surfaces failed on Windows
• Did not find a way to get it into ChimeraX.

Web 3D viewers

• For fastest 3d model network transmission, DRACO algorithm/format useful.
• Kristen Browne is working on a few 3D web viewers and should give a demo.
• Babylon.js web viewer, can drag and drop GLTF files for testing.

5/6/2021

Phil, Philip, Meghan, Darrell; Scooter, Elaine, Eric, Greg

Use of PDB or other databases' metadata in pipelines

For example, Eric's script uses the suggested contour level from an EMDB file. Meghan: another example is to look in the header to distinguish a virion from other viral proteins (Phil: this is in a record in the header) and direct them to different pipeline branches. Greg: metadata in scene export formats is which graphical object corresponds to which atom(s), but other metadata is best obtained from original data file, e.g. mmCIF/PDF.

Scooter: so you want to use existing metadata to influence pipeline processing... what about writing metadata into the GLTF/GLB? Phil: yes, maybe something like the structural hierarchy, overall complex -> different proteins -> chain descriptions, and which graphical objects correspond to each, similar to what Greg was describing. Greg: we would need to know what tags are required for the intended application (Enduvo, Phil says it's like powerpoint for VR, used in Uganda for training content, includes 2D assets like slides and 3D assets, movies, sound, evaluation questions).

Kristin is developing the Enduvo content with NIAID. We would like to talk directly to Kristin and Enduvo, primarily Greg to discuss what tags would be needed. Meghan and Phil: we will discuss with them and try to organize a collective meeting. Darrell: not sure which of these tasks are worth ChimeraX development. Need to strive for only what we think is highest value to both NIAID and ChimeraX user community. ChimeraX team can weigh in on what seems to be within our reasonable scope of effort. Scooter: seems worthwhile for export from ChimeraX to be showable in 3D web viewers. Meghan's idea: maybe have ChimeraX button to put the scene on NIH 3D, which would allow people to view it via the web there. Could have some limited interactivity using hierarchy metadata, e.g. turning chains on and off. Scooter: sounds like a great idea! Meghan: tradeoff between amount of metadata/interactivity in web viewer vs. file size, at some point (e.g. individual atom level) one should just use ChimeraX directly instead. (Meghan: conversely, going the other direction, could also have links from entries in NIH 3D to download the corresponding ChimeraX session files.)

Meghan likes the viewer and annotation tool in ​https://www.biodigital.com

Scooter: useful poster on glTF: ​https://kcoley.github.io/gLTF

Next time May 20: Kristin will be at the meeting Time after that June 3: maybe discuss PDB50 issues with Tef