NIAID Guidehouse/MSC contract May 1, 2023 - Feb 29, 2024

1. General ChimeraX improvements to support NIAID.
   1. Provide support for an NIAID internal app store
   2. Specific improvements to assist NIAID personnel to transition away from the unsupported legacy Chimera program, e.g.:
      1. More GUIs (notably, copy/combine, 2D labels)
   3. Investigate and improve ChimeraX usability for very wide displays and touch screens
      1. Specific focus on the BioViz lab wall display
   4. Work with Enduvo to improve exporting ChimeraX scenes in various formats to their environment.
2. Support the NIH 3D pipeline development, including any changes to ChimeraX to support ongoing development.
   1. Put presets on github
      1. Improve presets documentation
   2. Share code interacting with ChimeraX and NIH3D on github
   3. Continuing support for NIH3D as needed
3. Extend virtual reality support
   1. Investigate molecular viewer for standalone headsets such as Quest 2.
   2. Investigate use of pass-through video with the Quest Pro for multi-person sessions.
   3. Implement a VR ergonomic toolbar and Model panel user interface.
   4. Migrate ChimeraX from SteamVR to OpenXR, the current API standard for cross-platform virtual reality.
4. Explore pedagogical benefits of ChimeraX in VR vs. flat screen
   1. Develop a single web page describing ChimeraX VR capabilities and link to existing tutorials, how-tos, equipment advice.
   2. Describe use cases where VR is advantageous.
   3. Collaborate with the University of Indiana (Katy Borner), UCSF, and NIAID to conduct a task analysis comparing VR vs. flat screen for understanding biological macromolecules
5. Medical Images
   1. Improve DICOM reader by including more data types such as segmentations, and making it more robust by testing against the NCI TCIA repository.
   2. Add ability to “fetch” studies from TCIA
   3. Add support for NifTI format
   4. Add support for NRRD files
   5. Improve VR experience for medical images
      1. Easier manipulation of windowing and leveling, especially for complex curves
   6. General usability improvements for using ChimeraX with medical images driven by TCIA data
6. Outreach
   1. Instructional material and tools documentation.
      1. Detailed instructions for all features shall be provided in a user manual. ii.Written user guides and tutorials shall be available as HTML pages.
      2. Create videos demonstrating new capabilities.
      3. Present webinar and workshop tutorials to train users on existing and new capabilities.
   2. Create video tutorials for how to use multi-person VR.

Attendance at meetings or workshops as required by NIAID

7. Administration
   1. Submit monthly written reports of accomplishments

1/11/2024

Scooter, Elaine, Eric, Zach; Darrell, Phil, Kristen, Meghan, Bhinnata

Extra meeting presumably to discuss the trip to NIAID. Scooter and Zach scheduled their flights and will stay at the Even hotel. Tom G is sick now so we don't know what he's done, but we are planning for those 3 people.

Phil: our outreach folks are looking into getting space at the clinical center for our workshop and will get back to us soon. We also need to decide which events/topics. First day: just us, an intense day. Second day: VR pop-up at clinical center (both molecular and medical, interactive and standalone) as well as other workshops. Third day: more focused interactions in bioviz lab. Scooter: for the second day (assuming the clinical FAES space is available that day) pop-up in morning, other workshops in afternoon. First day and second day might get reversed depending on space availability.

Scooter: we could also work with specific labs on their ChimeraX and Cytoscape issues on the second day. Phil: so the ball's in our court to identify these specific labs with issues in ChimeraX, medical imaging, Cytoscape. Scooter: Cytoscape is subsidiary, lower priority than ChimeraX. For workshops: ChimeraX in general, AlphaFold, cryoEM applications (can probably use the materials Tom G just presented in the Macro course at UCSF), medical imaging. Or maybe we should focus on medical imaging for the smaller groups on the second day. Zach: that sounds reasonable.

Phil: For workshops, I'm thinking (1) ChimeraX general (2) cryoEM-focused (3) medical image-focused. Scooter: maybe 2-hr on the general one, then split into two groups for the others as 1 hr each. Phil: I don't have a good idea about the time required, so whatever you think. Elaine: for the broad one see the Jan 2024 entry in the ChimeraX tutorials page. Phil: will these be hands-on, people following along? Scooter: yes, I think that would be better but would take more time. Phil: would be good to do it in the Bioviz lab with headsets. Scooter: depends on number of people, 12 or fewer. Meghan: maybe at FAES (the pop-up?) would be better for medical imaging to get more interest from the clinical crowd. Scooter: maybe have one VR, invite interested parties to visit the bioviz lab the next day. We're arriving Tues, to be there W-F. Meghan: will try to get the FAES space Wed or Thurs.

Meghan: our guy from the infrastructure team is trying to figure out how to push ChimeraX to all machines with some Windows config manager, is there any documentation on how to do this? Scooter, Greg: no, but put them in touch with Greg to help work it out.

1/4/2024

Scooter, Elaine, Eric, Greg, TomG; Darrell, Meghan, Phil, another guy in that room (not sure about name), Mike Bopf, Kristen

Meghan: so, are you going to come visit? Phil: tentative dates in Feb? Scooter: the week of Feb 20 (Tues, Mon 19th is holiday)? Meghan: avoid Dulles, BWI is probably best. Kristen: I pick the cheapest of the 3 (another is Reagan aka DCA). Try not arrive at rush hour. Scooter: round trip airfare $523 SFO-BWI. Kristen: hotel about $150/night, that may be the per diem. Hilton is $122 currently.

• ​https://www.kayak.com/hotels/Hilton-Washington-DCRockville-Hotel--Executive-Meeting-Ctr,Rockville-c35029-h8634-details/2024-02-20/2024-02-23/2adults?psid=hgCkL0s7Io&pm=daybase#overview
• ​https://www.kayak.com/hotels/EVEN-Hotel-Rockville,Washington-DC-Area,an-IHG-Hotel,Rockville-c35029-h735283-details/2024-02-20/2024-02-23/2adults?psid=hgCkL0s7Io&pm=daybase#overview

Darrell: Government per diems and housing allowances: ​https://www.gsa.gov/travel/plan-book/per-diem-rates TomG: UC per diem is higher, >$200.

Scooter: we can aim for 3 days. TomG, Zach, maybe myself. Greg also interested but depends on budget. Darrell: will do a quick calculation for 3 days, 3 or 4 nights. 3 people 4 nights may run ~$5K. Should be OK for as many people as want to come, as long as we can keep the total below $10K. Scooter: wow! Will get back to you on which people.

Darrell Hurt to Everyone (1:35 PM) Cost targets:

• Flight: ~$600
• Hotel: ~$140
• Meals & Incidentals: $79/day; 75% of rate for first and last day
• Additional transport: $200 total; try to share

TomG: I would suggest 3 days 4 nights, Tuesday arrival Saturday departure. Scooter: seems OK. Meghan: avoid workshop Friday, let's do Friday here, leave Wed/Thu for workshop/campus stuff. We should at least meet up Tuesday eve.

Scooter: we ordered a 4070TI to evaluate here. Meghan, Phil: you don't need to bring any hardware on the trip. Phil: well, maybe if you have a VR laptop could bring that, but it's not required. We already have two. Meghan: maybe your visit can be part of our grand opening of this viz lab. Kristen: maybe NIH library can help advertise a workshop.

Scooter: will need to know, what kinds of workshops: cryoEM-specific? Intro to ChimeraX? Darrell: cryoEM-specific would be the most exciting, many at NIAID would be interested. Scooter: can include Phenix UI stuff. Kristen: the intro would be of interest to fellows and students. Phil: should advertise NIH-wide. TomG: other topics we had in mind are medical imaging (Meghan indicates interest) and VR. Meghan: could also think about the pedagogical study. If we do have a general training, want to have enough people in person. Don't want to waste our time together if most of the people would be attending by Zoom anyway.

Kristen: Mike and I could use some time for NIH3D stuff with TomG and/or Eric. Mike: we've been having issues with spaces in filenames. TomG: probably need to add quotes around the pathname. Mike: I tried things along those lines but was not successful. Let's cover details in email. Eric probably won't be there in person but can Zoom in.

Phil, Scooter: let's meet next week. 1/11/2024. Scooter: we'll work on personnel and budget specifics. Meghan: I'll look into workshop specifics like publicity.

12/21/2023

Scooter, Elaine, Eric, Greg, TomG, Zach; Darrell, Kristen, Phil, Bhinnata

Scooter: are we aiming for 2FTEs in the new SOW? Darrell: Yes. However, last year the increase in cost was unexpected, which we were told was an accounting problem from previous years. Would not easily absorb another increase like that. Scooter: no, it should not be a substantial cost increase, just the standard COL increase. Will keep a careful eye on the amounts next time around. Darrell: we are planning for a 10% cut in NIAID funding, although the exact amount is TBD. Our first goal is to maintain personnel, and we are including you in that personnel bucket. We believe another year of support should be possible, but we are still under a CR and cannot provide certainty at this time. Scooter: we'll go ahead with 2FTE until/unless told otherwise. Darrell: yes.

Darrell: One question about VR: was there something about a bug with our new expensive graphics and a workaround? TomG: yes, involved mirroring from Desktop display... workaround was to use SteamVR's mirroring instead of the other. Phil: well, we still get this problem intermittently regardless of the workaround, an outline like a skyline flashing near the bottom of the display. TomG: maybe we can test this here by putting our 4090 card (currently used with Linux) with a Windows machine. Phil: we have 4070's with laptops, 4070TI with desktop. TomG: SteamVR vs. OpenXR? Different headsets? Phil: haven't tested those yet. Scooter: Does this need to be fixed before new SOW? Darrell: not necessarily, could be in new SOW. TomG: depends on whether we can reproduce this problem here. Scooter: we can buy a 4070TI. Darrell: we're grateful for the "white glove" service with out specific problems!

Phil: want to bring up something from when the CIT (IT for NIH) leadership team came to our lab. We set up a scene in ChimeraX and handed the controls/headset to somebody else and somehow everything got messed up. We need a button or keystroke to temporarily disable the controls before we hand them over to the person. Would like that in the SOW. Elaine: also a similar issue is saving/restoring scenes. Not yet implemented, though. Scooter: that's in our R01.

Darrell: the radiological society meeting (RSNA conference) we went to was really huge, like 40,000 people. We were demonstrating segmenting in VR with ChimeraX, and people couldn't believe that the software is free for noncommercial use. (shows slide with comparison among medical imaging viewers... Elaine: can I copy this? Darrell: I can send it to you) We continue to be interested in this area because you provide a unique set of capabilities. Scooter: we're also looking at voice input. Intend to write a paper soon on our medical image-related capabilities. Darrell: may be worth submitting this to the RSNA conference. Presenting there is considered a feather in your cap by the community maybe even more than the paper itself. You have to submit fairly far ahead of the conference. TomG: is it always in Chicago? Darrell: Yes. Phil: always the weekend after Thanksgiving.

Darrell: one of the items in the current SOW is supporting wide displays. Just wanted to mention our video wall is two 4Ks next to each other, where each is an aspect ratio that is standard to some extent. So that work will be valuable to others, not just us.

Scooter: re scheduling a visit to your lab soon. Can we make it happen in January or February? Darrell: I saw something like $15K allotted for 3 people, seems high. Scooter: we'll go with our actual expenses and it should be much less than that. Darrell: The week of Feb 19-23 has opened up since our retreat was canceled. First week of Feb some of us were traveling to visit another person. Scooter: Feb 19-23 is mostly open, I'll get together some numbers. Phil: the preceding week (Feb 12->) would also work. Darrell: oh, Feb 19 is holiday, maybe Feb 12-16 or Feb 20-23. We could advertise a seminar and/or set up demos in the atrium. Scooter: we can bring some Quest3s and also show interactive VR with medical imaging. Darrell: would like to have a nice dinner too. Scooter: I wouldn't be able to make if it Feb 12-16, will be in Copenhagen. Darrell: with the contract ending it will be important to get the expenses submitted promptly. I can recommend a hotel in walking distance, but Nada may have advice about other possibilities. Darrell: It’s the Hilton in Rockville, MD: 1750 Rockville Pike, Rockville, MD 20852. Via email: I think we had a great conversation during today’s meeting. I confirm this task order with the changes we talked about in the meeting as well as travel to Maryland during the week of February 19-23 (although some flexibility on those dates is okay, too).

NIH3D: Kristen: Phil, this was an entry that had DNA only that we were talking about: ​https://wfe-app-testing.threed.niaiddev.net/entries/3DPX-020164 ... if no protein (nucleic only), we should omit the Coulombic and secondary structure displays. Rainbow is single color, unclear why. Also: nucleotide representations should be same color as ribbon. Elaine, Eric: those are probably bugs, will look into it.

Phil: SteamLink is much more robust than using AirLink. They had a demonstration with 7 Quest headsets using both SteamLink and AirLink. Trying 8 fails because there are only 7 80MHz wide channels on the 5 GHz wifi band.

Tom: Could use a wifi-6e router which uses the 6 GHz band.

Darrell: The 6 GHz band supports 14 80MHz channels.

Phil: The BioViz lab has 12 VR workstations so would like to be able to handle up to 12 wifi connections, so 6 GHz wifi may do it.

12/7/2023

Scooter, Elaine, Eric, Greg, TomG, Kristen, Phil, Meghan

• Radiology Conference
  • Several VR Radiology apps being showed
    • Fancy, slick interfaces, but they all were laggy
  • Lots of AI
    • Lots of automated segmentation
      • Total segmentator (has a pip package)
  • Contact: UCSF Center for Advanced CD+ Technologies
  • Shafkat Anwar -- E-Mail
  • Tatiana Kelil -- Co-Director for Radiology
  • Alexander Lin -- Evaluating one of the other VR systems
  • Investigate putting a 2D view below the model at the same time

11/9/2023

Elaine, Eric, Greg, Tom G; Kristen, Darrell, Phil, Bhinnata, Andi

TomG and Phil discuss Quest3 improvements over earlier versions. Zach is in Kansas now for the Phenix meeting.

​Radiological Society of North America meeting is Nov 26-30. After our previous meeting (Oct 26), Elaine sent Phil and Meghan e-mail about current ChimeraX DICOM features and the test data we've been using. Zach would be the one to advise about showing this stuff in VR specifically, but he's currently out of town presenting at another meeting.

Phil: I've been testing the latest DICOM tools in VR, some issues:

(1) Noticed that windowing/leveling with VR controller isn't updating in histogram in VR, only on the monitor. TomG: yes it doesn't know when to update the panel in VR; doing it continually may be too computationally expensive and cause flicker. Phil: may be less important with pass-thru, can view monitor. TomG: it wouldn't be hard to change the code for more panel updates and see how it affects performance. Eric: maybe there could be a button assignment to toggle panel updates.

(2) thresholds were not saved in sessions. TomG: that would be very surprising. They should be saved. What version are you using? Tom & Elaine: maybe Zach has code that recognizes DICOM and automatically resets thresholds. Phil: I can explicitly set them with the "volume" command. I can find out the values with "volume settings" TomG: manual adjustment should log the corresponding command. You can also use "buttonpanel" to make a custom GUI with buttons that run commands.

Phil: the RIDER lung data with the tumor shows up nicely in VR. With the right thresholds, it seems to even have an outline. Elaine: maybe there is a slightly denser capsule around the tumor.

When Zach is back (next week) maybe he can advise in more detail about showing the DICOM Segmentations tool in VR. Elaine: you can open the DICOM that already has segmentations, and edit them. Erase bits, add bits, etc.

Phil, re NIH3D: how to visualize struts, discussed previously with Eric. Halfbond coloring looks better than a separate color, let's use halfbond in the presets.

Discussion of per-vertex coloring. TomG: can you suggest a VRML viewer in which to test this?

• Phil: instantplayer on Mac ​https://www.instantreality.org/downloads/
• Kristen: this one does recognize vertex coloring: ​https://3dviewer.net/
• Darrell: web3d consortium validation tool ​https://www.web3d.org/news-story/x3d-validator-available-x3d-scenes

TomG: can you send me a per-vertex-colored VRML that works with 3D printing? Phil, others: yes

Darrell: is there possibility of your traveling, using the contract (ending Feb 2024)? Also, where do you go to show ChimeraX, and do you have advice?

TomG: maybe Zach and/or I could come to see you and your facilities. We are open to that. On the second question, we haven't done so many booth exhibits, more like presentations or lectures at meetings. The main advice is to keep it simple because time is limited, and show data relevant to the audience's research when possible. Lack of internet access is often a problem. More bang for our buck lately has been showing ChimeraX locally to UCSF labs with people who in turn attend more conferences elsewhere and spread the word. TomG: the standalone VR is very effective.

Darrell: I've been thinking of our future travel, so if you do think of particular conferences that may be good opportunities, let us know. Kristen: VizBi. Darrell: what about Siggraph? Greg: maybe not so much molecular graphics these days. TomG: used to go to Biophysical Society, but seemed too large. Gordon Conference on cryoEM was a better size. Prefer smaller, more specialized conferences. Elaine: NIAID/NIH3D website has a broader scope, perhaps.

TomG: Scooter is probably the best to advise, he travels and goes to more conferences than the rest of us. Scooter told me VizBi is for the biologists, BioViz for computer scientists developing visualization tools.

Next meeting? Not Thanksgiving. Phil may try to connect with Zach next week. TomG: I'm available any time to Zoom. Phil: Great!

10/26/2023

Elaine, Eric, Greg, Scooter, Tom G; Kristen, Meghan, Phil, (Bhinnata?)

Phil: just installed new LookSee, I like using the triggers instead of grip buttons. Discussion of inconvenient things about setup with Meta.

Scooter: we ordered a Quest 3. Phil: we tried them but didn't order yet. Passthrough/resolution much better, room setup easier. Tom: graphics is supposed to be 2.6x faster, am hoping to handle larger models.

Phil: LookSee color matching not great. Tom: I have a ticket on our LookSee github with some issues, including this poor color matching. Has to do with color space. Phil: I would also like to preload a series of scenes and have a button to traverse them. Currently if I add scenes, they're not aligned as they were in ChimeraX. Tom: I may add an option to only move/scale scene as a whole, otherwise models will not be kept in alignment. Phil: what is the ordering of scenes? Tom: currently alphabetical. Phil: that order is probably best, I can name them according to the order I want.

Tom: I got vr meeting working with OpenXR. Currently have both SteamVR and OpenXR in ChimeraX, may keep both in for a while. Scooter: may be better to drop SteamVR. Phil: it seems to be more robust if we use a nonVR computer to start the meeting. When to you plan the next release? Tom: something like end of year, hope to branch next week.

Scooter: big downtime next week to upgrade cluster.

Phil: will the meeting server be down? Tom: No, it's on AWS. They are going to start charging by the hour for a public IP, but currently it would just be a few dollars more per month.

Meghan: we're preparing for the Radiological Society of North America meeting (large, ~60K people) in a couple weeks, November 26-30, will credit your group for VR etc. Still not sure what we'll be showing other than NIH3D features. The booth is not that big, and the needed setup was very expensive. Do you have suggested data?

Elaine: The Download DICOM tool allows you browse all the publicly available TCIA datasets. We've been using examples from RIDER lung in our tutorials and documentation images, but there are plenty of other ones you could use.

Scooter: we need collaborators for DICOM. Kristen: would like collaborate with Johns Hopkins in our VR study, since that could bring in surgeons and radiologists.

Meghan: we appreciate your documentation, the LookSee instructions were easy to follow. Tom: I just changed it all!

Meghan: if you could get together some suggestions related to DICOM, it would be useful. Elaine: sure, I could do that, especially since Zach is away right now. Just be aware that some of these tools are currently bleeding edge (i.e. have lots of bugs and are in flux). I have no idea how the DICOM data works in VR and with LookSee.

Kristen: not much with NIH3D, are waiting until after we do more testing with the latest presets.

Scooter: think about whether we should schedule a visit since it's a possibility in our current contract, which goes through Feb 2024. Also could start thinking about next SOW... it's not too early. VR, DICOM, machine learning... Phil: maybe can discuss at our next meeting, or at least the next one with Darrell. Scooter: I won't be at the next meeting (I'll be in Germany), so maybe in a month. Elaine: that's Thanksgiving. Meghan: let's figure out a day where both Darrell and Scooter can be there. Scooter: Dec 7?

9/28/2023

Elaine, Eric, Greg, Zach, Tom G; Phil, Kristen, Andi, Bhinnata

Tom G is at the Meta Connect event. Tried Meta Quest 3, but hand-controller tracking may be poor. Does not have eye tracking (feature of Quest Pro). Pass-through very high resolution, pass-through video frame rate still not that high (10/s?). Meeting has been emphasizing mixed-reality applications. Has real-time depth mapping, can show mesh to represent the physical room.

Phil: no updates with BioViz Lab currently

Kristen: NIH3D issues:

• New preset is not putting out hydropathy surface any more, some error message about atom patches. Eric will investigate.
• Which daily build should be used with the latest set of presets? Eric: any after pseudobond sides option was added. Won't have another production release until maybe November. Phil got errors with new presets using previous production versions of ChimeraX, and a 3-wk-old daily build, but we think it was because he needed a newer daily for this pseudobond sides option.
• Want another preset with lighter-weight surfaces (fewer triangles) just for viz, while keeping the smoother ones for print. Parameters should be the same except for grid spacing. Phil will try to come up with some guidance for grid spacing.

Phil described some of the things they did in Africa. Uganda has mostly Vive Pro headsets plus two Quest 2 headsets. NIH campus bioviz lab no longer uses Vive Pro because all base stations were sent to Uganda. Mali has two Quest 2 headsets but not much experience using them. Uganda computer science has been using Quest headsets for astronomy visualization. After Uganda, went to Capetown South Africa where Uganda and Mali researchers presented their work (talks and posters).

Discussion of Unity fees. Andi asked if Kristen or Tom consider using Unreal Engine instead of Unity. Runtime fees for Unity apply only with more than 200,000 installs, so not likely to ever effect us.

8/24/2023

Zach, Tom G., Tom F., Eric, Kristen, Andi, Darrell, Scooter, Phil, Victor, Elaine, Greg, Bhinnata, Meghan

Meghan & Darrell show off BioViz lab. 13 VR workstations, LED touch-sensitive wall display, snazzy floor.

Lab event went well. Thursday 2 groups came through; David Chen VR presentation on video wall. Seen directly by many people in the room but also by others via Zoom and in remote conference rooms. They have a recording of a practice version of the presentation. Had 10 people in a VR meeting with separate computer hosting. All used Quest 2 headsets. Had to tape cables to the headsets for reliability. Worked great. Friday the headsets were used separately, with a lab person helping each. Felt it was better this way because there were more people in the lab than expected and therefore it was noisy, which made communication in a multi-person meeting difficult. The lab meeting was part of a conference and known well ahead of time, but the lab was only actually ready the day before! Now that it’s ready, they will advertise its availability.

DAE files slow to load (2 minutes to get VR set up). GLB was almost instantaneous, but didn’t work in meeting. T.G. had determined it was because GLB fails to save in session, and fixed it several days ago now. Phil(?)/Dave(?) says gzipped DAE/GLB files don’t load despite being documented to do so [worked for me in later testing, maybe missing final .gz suffix?].

Phil has been investigating presets and requests:

• H-bonds and struts be 16-sided
• Small-molecule bonds be 32-sided
• No sharp boundaries on surfaces
• 16 sides for ribbon cross sections

Tom F. asked if maybe those settings could be printer dependent. Phil said he just looked at meshes. Darrell noted that most printers print objects that are a foot cubed or less, so there won’t be a lot of variance in scale.

3D print questions on our mailing lists could be forwarded to Phil.

The NIH folks need to decide among themselves how much they want ChimeraX to do and how much is post-processed, e.g. low-res representations generated directly or via decimation of a high-res rep? Kristen noted that low-res reps can look good in visualization software due to smoothing from lighting. There was disagreement between Phil and Kristen as to how crude the preset output was, but Kristen was likely using an older version of the presets than Phil. Greg mentioned that maybe they just wanted ChimeraX to output ideal primitives and they would work from there, but Darrell said they wouldn’t want to have to deal with tessellation issues. Scooter mentioned that additional presets would not be hard to add. Eric added that the main issue is ensuring the NIH and CGL are on the same page about the specifications for such a preset.

Kristen said that there may need to be more small-molecule presets, because that’s the main category where one size definitely does not fit all due to the wide variety of such molecules.

Hundreds of thousands of files need to be rerun with the new workflows.

Dave Chen requested Phong lighting for volumetric rendering. T.G. noted that that (and many other modes of lighting for volumetric rendering) is covered in the R01.

Phil noted that applying a surface preset repeatedly increases the triangle count (“graphics triangles #1”).

Andi got funding for a hackathon for the Human Reference Atlas in the Biovisualization Lab, 10/18-19, and needs a template for the Statement of Work. Meghan: Scooter? LOL. Scooter noted that the SoW is different for academics.

Tom G. gave an update on OpenXR. It works, but was harder than anticipated to get going (2 weeks). OpenXR is very verbose. The code needs shakedown testing and T.G. is thinking of putting it in the daily build alongside the SteamVR version. Then he would test it internally and when it seems possibly okay then have the NIH folks test it. Meghan was excited by the possible additional reliability provided by not having SteamVR be yet another possible point of failure. There is no information as to whether Apple Vision Pro will work with OpenXR (T.G. rates it unlikely) but it supposedly going to work with Unity. Kristen’s Pathogen AR app is in Unity. Meghan noted that T.G.’s LookSee app worked well and is well documented.

Darrell and Meghan will be visiting Rocky Mountain Labs in October and wondered if T.G. was interested in also going. T.G. seemed possibly amenable and volunteered Zach as well.

Meghan and Phil noted that 5 machines in the lab had bad stuttering in VR and they all had 4070TI graphics cards, and offered to send one such card for testing. The 4000 series cards are not supported by Quest/Meta. We have a 4090 card, but it’s on a Linux box and it’s not clear if it can be switched to the Dell Optiplex 7000 due to power issues. Ultimately decided that NIH would try the OpenXR version to see if that remedied the problem, and if not then ship the card.

Next meeting will be next week, but could be cancelled if no discussion topics emerge.

(8/3,8/17 meetings cancelled)

7/20/2023

Phil, Bhinnata, Victor (at end); Zach, TomG, Eric, Elaine

Phil: a couple of things about using ChimeraX in the Bioviz lab. Tried to use VR meeting with Quest 2 headsets but had problems: airlink disconnections (link cables not much better, easily jiggle loose), problems joining meeting. Later investigation found timeouts in Steam and in headset. Setting Steam timeout to longer time (in Startup/Shutdown settings) made most issues go away, but sometimes Steam forgets this and it gets reset back to 30 seconds. After resetting everything got 6 headsets working as compared to only 1 in the previous session. Our SOP from now on will be to check the timeout settings of both the headset and SteamVR beforehand.

TomG: if you get mysterious ChimeraX errors even if you find out they are due to these non-ChimeraX timeouts, please report them so we can try to improve the error messages.

TomG: next ChimeraX release should remove dependence on SteamVR (will use OpenXR). Hoping to do it in the fall although the release may be later near the end of the year.

Phil: if I save a session after "vr on" has been used (even if turned off before saving), that session won't open on M1 mac. Even have this problem sometimes on Intel mac. TomG: it's probably trying to import the SteamVR module.

Phil: Dave Chen may have contacted you about some DICOM issue. We were confused about Dave Chen, but Phil says that his problem was fixed. Dave Chen is working with irradiated minipigs.

Phil: Another that we ran into was in reading certain sets of DICOM files, e.g. some of the individual slice files don't have .dcm file extensions and so ChimeraX won't open them. Would be nice if we didn't have to add the filename extensions. Zach: will open a ticket.

TomG: I encourage use of the standalone headset approach, is limited in some ways, but would be more reliable for demonstrations. Phil: I did get LookSee going and tried the minipig data at home. TomG: the passthrough video is helpful and works with standalone. Do report any problems, as it hasn't been tested that much. Phil: as you warned, can't view huge systems with too many triangles.

Victor: have multiple devices that cannot use vr meeting, cannot reach (ping) the UCSF server chimerax.meeting.net. Phil: it was working this morning, or at least from other machines. TomG: I can't ping it from my computer here either (request timed out), will have to investigate further. I don't have much experience with AWS. Maybe there are transient outages. Phil: I could start a meeting on my mac just now, seems fine. Victor: do I have to use some specific network settings in ChimeraX? TomG: no specific settings are required. TomG: I can see Phil's current meeting. Victor: mine says no meeting found.

Conclusion: some problem with the specific machine, as Phil and TomG can engage with this server.

TomG: you don't have to use this server, it is just a convenience to (1) create a meeting name (2) provide a hub that is not firewalled.

TomG: we recently updated some packages in the daily build, causing many unforeseen problems. May be worth trying 1.6 release instead of daily build.

Phil: issue with VTK model not saved in session. TomG: will doublecheck and fix in daily build.

Phil: will LookSee save volume models (transparent volume rendering style, not surfaces)? TomG: yes, will make a stack of rectangles. TomG: make sure you use a recent version of ChimeraX, there was problem earlier where volume renderings weren't included in saved GLTF. However, the performance of volume style rendering with Quest 2 headset is not great, slow (low size limit). TomG: Hopefully will be better in the Quest 3.

Eric: did you try the sharp-boundaries preset? Phil: want to discuss with Kristen (at a conference now) before we decide what to do.

7/6/2023

Phil, Kristen Browne, Meghan, Darrell, Andi Bueckle, Ziv Yaniv; Zach, TomG, Eric, Elaine, Scooter, Selina

Eric: uploaded a new NIH3D bundle. Kristen: Phil was going to test some changes before we implemented. Eric: I sent you a wheel (not on toolshed) to evaluate the presets with and without sharp boundaries, and color fromatoms. Phil: haven't looked at that yet, sorry, when was the email? Kristen: 6/30, will forward it.

Phil: also need the newer ChimeraX with surface speedup. Kristen: we will update that at the same time we update the presets. Phil: I'll get the newer ChimeraX for my testing purposes.

Scooter: lots of DICOM progress, Zach will demo... we also have an intern Selina looking at machine-learning methods for segmenting medical images. Two methods currently: nnUNET, and another new collaboration with Johns Hopkins folks developing a method named something like universal clip (?) Kristen: is that for CT? Scooter: yes, potentially other modalities as well.

Darrell: we have image segmentation experts on staff, I would like to invite Ziv Yaniv

Zach demo: showing RIDER lung CT. Ziv complained that the patient was face down in the initial view, said that clinicians always say that. Zach demo: the dicom view layout options and Segmentation tool with "paintbrush" type interface. Ziv says clinicians complained that the automated segmentations were useless. Phil is very interested in what the VR interface will be. Ziv: it will also be important how the rendering is done, because visualizing the data is the basis of manual segmentation. TomG: we have simple windowing/leveling controls in VR, but there is a lot that could be improved. Ziv: take a look at existing work and use it where you can, the complex presets in other programs such as 3D Slicer and Horos.

back to machine learning... Scooter: the Johns Hopkins group has a large image set. Ziv asks about the design: docker? Scooter: more likely web services. Ziv: you may be interested in ​https://monai.io/model-zoo.html Scooter and Selina: the universal model group already supports >20 tissues, with data from many different sources. Ziv: modality? CT. Ziv: one of the common problems is generalizability, the broader the dataset(s) the better.

Discussions of AR vs. VR, and "natural" interfaces w/o hand controllers.

6/15/2023

Phil, Kristen Browne, Meghan, Darrell, Bhinnata, Victor Starr Kramer; Zach, TomG, Eric, Elaine, Greg

"Extra" meeting to figure out suitable settings for the desired number of surface triangles (smoothness vs. file size, calculation time, etc.) for NIH3D pipeline.

Phil: established that the requirements for NIH3D and Enduvo are so different that they should just be in separate workflows

Phil: turns out the discrepancies between 3DPX (3D print exchange with Chimera) and NIH3D (new pipeline with ChimeraX) were due to different procedure/rules for grid size calculation. Is this in the scripts or presets?

Eric: presets, but we need to decide on appropriate size limits for switches in resolution, or caps on the number of triangles or vertices. Calculations for the biggest structures can be extremely slow. Eric: Tunable parameter is grid size. Another possibility is to go to gaussian surfaces instead of molecular surfaces. TomG: Gaussian surfaces have two adjustable parameters, resolution and grid size. We also noticed that calculation time is not linear in the number of atoms.

Phil: to me, long calculations are OK as along as the user's expectations are managed. Darrell: I agree, the expectations for a website like NIH3D can be different than for an interactive program like ChimeraX. Still, we don't want ridiculously huge files that many will not be able to handle. Kristen: in situations where we have to re-run everything, multiday calculations are less feasible. May also need to warn users specifically that processing will take a long time, and have some kind of indicator to us as pipeline developers that the process is still running/proceeding rather than hung.

Elaine: the hard part is the nitty gritty of setting the rules and boundaries. Darrell: we will need to work on that on our end.

Kristen: can we change these parameters? Elaine: they are command options. Phil: I see the preset-generated commands and then make variations on those commands and try them. TomG: I would edit the python in the bundle files. Try it on a set of test cases and evaluate the results. Eric: we should address the problem systems, make sure the appropriate grid size is used (wrong sizes found by Phil), have Kristen apply decimation and see the results, see if TomG can speed up surface calculation, and decide on sharpBoundaries true or false. Phil: advantage to false is smaller files. Eric: maybe true for visualization presets, false for printing presets. Kristen: I should take a look at decimation for both sharp true and sharp false, see what happens to those boundaries.

Phil: we're only talking about surfaces so far. I also want to think more about ribbon cross-section (number of sides, etc.).

Discussion of lighting ("smoothing" using surface normals). Kristen: should we show the smoothing in the viewer, or is that misleading when the model will be printed? Similarly, what if model is too big for VR use? How to manage expectations and give the right level of detail for the intended use? Darrell: we should discuss amongst ourselves outside of this meeting with the UCSF folks. TomG: that mammoth example was interesting, giving some choices for level of resolution, but that site did not give file sizes, which was problematic. People need to understand what they are getting, to avoid frustration. Darrell: maybe we could offer a printable version and a VR-ready version. Eric: In most cases we do offer two versions, or at least specific settings suitable for printing. I'm thinking sharpboundaries true except for the larger printable models.

Phil: for testing, could you make an additional set of presets same as existing except with opposite sharpBoundaries settings? Can send wheels directly instead of putting on toolshed. Where are the presets loaded? Eric: use command "info path"... Phil: found it. TomG: after editing, restart ChimeraX.

Phil: do you have a feel for maximum triangles for VR? TomG: Quest2 or QuestPro 900,000 without passthrough, 700,00 with passthrough. Higher-end card 20 million triangles. Kristen: I want to make scenes with lots of proteins. TomG: I'll send you a link about somebody who was making spectacular multiprotein scenes.

6/8/2023

Phil, Kristen Browne, Meghan, Bhinnata(?); Zach, TomG, Eric, Elaine, Greg

Phil will be away on vacation (Europe) next time

3D pipeline issues:

• Kristen had one rendering that came up blank, Phil did not test that case yet

• Kristen: did we want to increase quality on small molecules too? Phil did not get to that yet

• Kristen: EMDB 10067 was not using recommended contour level. Phil was going to investigate that one further and if needed, make a ticket for it.

• Phil: there were some updates for the disconnected pseudobond issue. Eric: submitted new version of NIH3D bundle with preset that makes missing segment pseudobonds go between CA atoms.

• Phil: regarding quality of surface models, did some test prints of 4 molecules of different sizes, 3DPX (old Chimera) vs. NIH3D (new ChimeraX). Still need to investigate changing grid size and sharpBoundaries parameter. Old ones from Chimera are smoother. Tom: We need to know exactly which commands/options were used to make the 3DPX surfaces. Phil: we were using the "grid" option for more robust surfaces in Chimera. TomG: in that case, ChimeraX should be able to generate the same surfaces if we give it the right option values, since it's the same algorithm. Another important factor is whether hydrogens were added. TomG: sharpBoundaries significantly increases the number of vertices. Megan sent pictures for 6eee, and TomG and Elaine both compared Chimera and ChimeraX grid surfaces, spacing 2.179, and both looked bad/chunky. Phil: I will send data files, commands that were used, etc. How to send? email? Eric: how did you count vertices? Phil: I load the files into Blender. I used to do it in Meshlab. Kristen: different software will give you different counts.

5/25/2023

Phil, Darrell, Kristen Browne, Andi Bueckle; Scooter, Zach, Selina Liu, TomG, Eric, Elaine

Scooter describes Alex Pico's recent Wikipathways work

Kristen: only one new thing for NIH3D since last time; pdbe biounit url was broken, didn't fall back to pdb. Original ticket was about obsolete entries, should make a new ticket for failure to fall back.

There was an issue with vertex colors in gltf, ChimeraX default gltf save behavior was changed to not include them for single-color objects. Need to get daily build.

Another issue was higher-resolution surfaces from Chimera than ChimeraX. Phil made a table comparing # vertices etc. for several example structures of different sizes, we're not sure why there are so many more triangles from the NIH3D pipeline than in ChimeraX directly. Maybe the sharp boundaries parameter? None of their colorings are per atom, only per chain, by ESP, or by MLP. So they could turn that option off (sharpBoundaries false). [I saved an image of his table but couldn't figure out how to insert it here]

Phil didn't get a chance to look at the pseudobonds-not-connecting issue.

Andi's writing up something but it's not yet ready for discussion.

TomG: maybe the study should use the standalone VR approach with Quest 2. Andi: is this available as a build? TomG: Yes. Take a look at ChimeraX presentations on this, e.g. demo1 demo2 demo3

Scooter introduces Selina

Scooter says the contract was adjusted down for the 2 months time lost

Phil: back to VR, one of the structural biology groups here is asking for headset recommendations. Tethered HP (Windows mixed reality?) seems robust but controller buttons are quite different and not recognized by ChimeraX currently. The worst is that that one that brings up the menu is not recognized. TomG: we have Windows mixed reality setup and it was working, but I haven't used it for a few years, maybe API changed somehow and/or you have a different version. Phil: positions of the buttons don't map correctly, both A and B depress same thing on virtual representation, thumbstick and menu button not recognized at all. Only grip and trigger and A/B (melded) are recognized.

TomG: I'll try our Windows mixed reality and get back to you.

Phil: obvious recommendation would be RiftS except we can't get those anymore, whereas one can still get HP Reverbs. TomG: this group already has the expensive PC? Phil: yes. Andi: we have the tracking stations on tripods, which makes them easy to move around.

TomG: We hope to get ChimeraX working with OpenXR, so it wouldn't be dependent on SteamVR.

TomG: I found that devices couldn't "see" each other on guest wifi, only the secure one. This may be an issue for other institutions. Phil: We are still working through wifi issues in the bioviz lab, headsets need 80mHz and router is not providing that. Andi: too hard fighting with institutional IT, I just have to use cables everywhere. Phil: Meghan is doing a great job coordinating the hardware setups here.

Discussion of airlink connections.

5/11/2023

Phil, Scooter, Zach, Renata, Greg, Tom, Eric, Elaine, Kristin, Meghan

Mehgan: Dmitry at Seattle doing VR work including with clinical applications

Zach: Nifty & NRRD support. TCIA lists ChimeraX as an analysis tool -- Federick labs still sending bugs (lots of interest)

Eric: pseudo-bond error (pseudo bonds don't always connect to the ribbon). Phil is going to review ticket again and see how important it is

Elaine: introduction

Greg: introduction, work on small molecules

Tom: VR standalone, Quest Pro broken, Quest Pro very effective. Discussion about the resolution of the camera in the Quest Pro

Phil: Confusion about Mac install (does Universal installer install M1?)

Kristin: High priority NIH3D issues

• No color modules should always use materials, not vertex coloring (small molecules use vertex colors). Possibly the GLTF writer could potentially switch from vertex coloring to material coloring if the color is all the same
• Resolution of surface models is inconsistent between chimera and chimerax. Chimera is higher resolution.
• Issue with shading and 3D printing (possibly drop normal vectors)