= Medical Science Computing contract Ideas = Feb 13, 2019 == Background == What developments should we propose for a March 2019 one year subcontract with Medical Science Computing? Meghan !McCarthy and Darrell Hurt say it can be broader than medical imaging and VR. Meghan mentioned next gen NIH 3D print exchange including VR/AR with those models and in-browser visualization (specifically the WebXR preliminary standard) and tutorials. She also mentioned cytoscape 3d and VR viz. Darrell said though that the focus should be VR. There was some discussion of the NIAID interest in vacine development and molecular visualization and analysis. == General Considerations == Should we propose new areas like web VR or augmented reality or machine learning? VR has uses for demonstrations, collaboration, and analysis. Where should we focus? ChimeraX has VR capabilities for molecular structure, electron microsocopy, optical microscopy, medical imaging. Which application area should we focus on? == Relation to our other VR projects == What do I want to do with VR in the next year? Would like to move it beyond demonstrations, advance its use for collaboration an analysis in other labs. The main ingredient needed is collaborators with specific problems. Maybe the most promising chance for that is in cryoEM atomic model building with Yifan and Adam. I need to get Dyche and Max Krummel setup up for optical microscopy VR. Also I have more to do on binding site VR for Koda-Kimble award. MSC does not have end-users so they are not the collaborators. Proposed projects need specific collaborators who will be using the tools. == What does NIAID want? == A key missing part of the picture is what exactly the MSC contract from NIAID is funding. A subcontract would of course need to be aligned with the goals of the primary contract. If we don't know those details then we may propose things that are not in the scope of what can be funded. == Potential application areas == == 1) Medical imaging and optical microscopy visualization == === 1a) Segmentation using VR === Use [http://www.simpleitk.org Simple ITK] in ChimeraX. [wiki:HurtITK Darrell Hurt says] NIAID has hired SimpleITK team. Slicer [https://www.youtube.com/watch?v=BJoIexIvtGo&feature=youtu.be grow from seeds] segmentation, cardiac CT Slicer [https://www.youtube.com/watch?v=0at15gjk-Ns&feature=youtu.be scissor projection tool], masking to sphere (volume eraser) for hip ball joint Machine learning segmentation methods for medical images and [https://www.allencell.org/label-free-determination.html cell microscopy]. Need to investigate medical imaging methods that could be used Segger watershed segmentation, especially hand mode, an example illustrating method is [http://www.cgl.ucsf.edu/chimera/data/ncsm-nov2012/microscopy.html termite gut]. Tools to adapt to VR: - Volume eraser - Hide dust - Show only largest connected blobs - Loop tracing - Masking to surface to create binary segmentation === 1b) Medical imaging and optical microscopy visualization === - [https://www.allencell.org/photorealistic-cells.html Photorealistic lighting] from Allen Cell Science. - Presets for coloring [https://en.wikipedia.org/wiki/Hounsfield_scale#Value_in_parts_of_the_body different tissues] * Horos - Smart initial coloring (bones, different organs or tissue types) - Machine learning to [https://www.allencell.org/label-free-determination.html color organelles in cells]. [https://www.alleninstitute.org/what-we-do/cell-science/news-press/press-releases/machine-learning-technique-predict-human-cells-organization-published-nature-methods Video]. == 2) Molecular Modeling and Graphics in VR == === 2a) Building molecular models in cryoEM using VR === - ISOLDE in VR - Rotamers - Build structure - Clashes - OpenMM and MD tools * parameterization * charges - Interactive local energy minimization === 2b) Molecular Analysis in VR === * Presets * Full menus * Render/select by attribute (+ define attribute) * Residue conservation * ''Worms; ribbon color blending'' * Selection improvements * Multi-level selection-only undo/redo (left/right arrow keys) * More levels to selection hierarchy (including glycosylation level) * Axes/planes/centroids * ESP surface coloring * PQR charge info * Add Charge equivalent * Tool GUIs (more usable in VR) * AddH * !MatchMaker * !Clashes/Contacts * FindHBond * Chimera 1 session migration * Thermal ellipsoids * Rotamers * Build Structure == 3) NIH 3D == === 3a) Web VR visualization === - Primarily for communication and education - AltPDB. ChimeraX exports glTF scene used with !AltSpace to provide Web VR collaborative molecular PDB display (ribbons, surfaces, atoms). - [https://www.stevenabbott.co.uk/practical-solubility/Peptides-VR.php Peptide solubility Web VR app]. Solid phase protein synthesis, Steve Kent, Steven Abbott - Could ChimeraX offer web browser connection for multiperson VR viewing-only mode? Or just export to an AltSpace or other social VR site with one button click? === 3b) 3D printing needs === - Color printing format requirements - Reduced level of detail controls - Direct export from ChimeraX == 4) VR Development == === 4a) User Interface Development === Task specific user interfaces - Drug binding analysis user interface - Medical imaging - Optical microscopy - cryoEM modelling Speech recognition (controlled vocabulary and grammar) === 4b) Training === Mixed reality video capture shows person and data. - For tutorials and explaining research results to public. How-To screen capture videos Multiperson VR sessions for remote training, possibly group training. Written tutorials and user documentation Help desk === 4c) Multiperson VR === Remote and same room collaborative VR discussions. Allow any participant to bring in new data.