Opened 4 years ago
Closed 3 years ago
#5954 closed enhancement (fixed)
Add Phenix douse tool to add waters
Reported by: | Tom Goddard | Owned by: | Eric Pettersen |
---|---|---|---|
Priority: | high | Milestone: | |
Component: | Structure Editing | Version: | |
Keywords: | Cc: | Zach Pearson, Tom Goddard | |
Blocked By: | Blocking: | ||
Notify when closed: | Platform: | all | |
Project: | ChimeraX |
Description
As part of the Phenix R24 grant we will be allowing ChimeraX to call Phenix tools. The first one we are trying is phenix douse which places water molecules in a give cryoEM map near an atomic model. Douse returns a new atomic model with all the original atoms plus some waters added. We will make a command and a gui interface for this. It will assume the user has installed Phenix and should allow the user to specify the phenix location if it cannot be found automatically.
Pavel Afonine on the Phenix team developed douse and will work with us on this.
We want the ChimeraX tools to invoke Phenix tools via a REST service so that it can get JSON return values (file paths, error messages, calculated numeric quantities) without having to parse the typical command-line output from phenix tools. Billy Poon has developed an initial REST server for phenix that we can use to call douse, and other tools.
Change History (8)
comment:1 by , 4 years ago
comment:2 by , 4 years ago
I made the ChimeraX phenix douse command work with maps with non-zero origin by shifting the input atom coordinates and then shifting back the output coordinates with the waters.
I also improved the water display showing just the waters and nearby residues, and the map near those waters as a transparent surface. Might be nice to add hbonds to the waters to aid assessment.
It would be nice to have a list of the waters in a gui to click on each to focus the view on it to be able to assess if it is correct. A gui should also allow easy deletion of incorrect waters. It will also need an easy way to adjust the map contour level. The command could output an html table of waters to the log with clickable links to focus on each. Map contour level when focusing on each water could be auto chosen to show the water density.
Also need to make this use Billy's phenix REST server. And will want ChimeraX to run the job in the background with a task panel showing running jobs, and that allows stopping jobs, and that can show jobs that are still running or completed after ChimeraX is quit and restarted.
Lots to do to make this work well.
comment:3 by , 4 years ago
Made phenix douse command find which predicted waters match input waters and which are new, and which input waters were not predicted, and put links in the logs to select these different groups of waters.
comment:4 by , 4 years ago
Made phenix douse command keep input waters but not add second shell waters. The keepInputWater option controls whether to preserve input waters. I added a separate flag farWater (default false) which allows adding waters that are only near other waters. The new code keeps the same residue numbers of all the input waters and the new waters have residue numbers starting 1 higher than the highest residue number. The keep_input_water option in Phenix doesn't handle these cases, it always finds second shell waters. So to not find them (which I think is the most common use case) I have to run Phenix douse without keep waters and then ChimeraX adds back in the input waters, replacing any overlapping predicted waters.
comment:5 by , 4 years ago
I made the phenix douse command able to run using the phenix rest server. There are various problems. 1) How to know when the rest server is actually ready to accept connections. Currently I look for an output line "Started Flask app" which will fail hanging ChimeraX if this line is not output -- bad solution. 2) Starting the phenix rest server creates 6 subprocesss (server and 5 workers) that are tricky to kill when ChimeraX exits. I had to get the process group and explicitly kill them in an an atexit handler. Will that work on Windows? 3) Somehow the phenix rest server logs messages about server running and status queries to the shell where chimerax started even though the subprocess call redirects stdout and stderr. Not sure how it is doing that. 4) Starting phenix rest server when one is already running gives a Python error about an undefined variable. Will report that to Billy.
With all these complications of running douse with the phenix rest server I have not seen any advantages over running douse directly as a subprocess. Need to discuss why we going this complicated route.
comment:6 by , 4 years ago
Cc: | added |
---|---|
Component: | Volume Data → Structure Editing |
Owner: | changed from | to
comment:7 by , 4 years ago
Here are a few example test cases I have used.
Baby test case
open 1a0m open 1a0m from eds phenix douse #2 near #1
That runs in maybe 10 seconds and opens a new model with the predicted waters.
That is x-ray data and douse is tuned to cryoEM and the Pavel says douse is not intended for X-ray maps! So a more realistic 15 minute run is apoferritin cryoEM at 1.5 Angstroms
open 11122 from emdb open 6z9f phenix douse #1 near #2
This douse run currently hangs ChimeraX for 15 minutes and would be a typical run time, so backgrounding and task management are a pretty high priority for this project.
comment:8 by , 3 years ago
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
I made a ChimeraX command "phenix douse" (not yet in daily builds) to run douse as a subprocess while Billy got the REST server working.
I encountered a problem that Phenix does not read the MRC floating point origin, treating it as zero. So the ChimeraX code will need to shift the atomic model coordinates it writes out for douse to use this coordinate system where the map origin is 0,0,0, then when it gets the atomic model result back it can shift its coordinates back to align with the map in ChimeraX.