Logged AlphaFold command link non-functional

[Tristan Croll]

The following bug report has been submitted:
Platform:        Linux-3.10.0-1160.25.1.el7.x86_64-x86_64-with-glibc2.17
ChimeraX Version: 1.3.dev202107240347 (2021-07-24 03:47:26 UTC)
Description
The "open #{model} from alphafold" command leaves a non-functional link in the file history.

Log:
> alias preview_toolshed toolshed url https://cxtoolshed-
> preview.rbvi.ucsf.edu; toolshed reload available

> alias production_toolshed toolshed url https://cxtoolshed.rbvi.ucsf.edu;
> toolshed reload available

> isolde shorthand
    
    
    Initialising ISOLDE-specific command aliases:
    Alias	Equivalent full command
    -------------------------------------------------
    st	isolde step {arguments}
    aw	isolde add water {arguments}
    awsf	isolde add water {arguments} sim false
    al	isolde add ligand {arguments}
    aa	isolde add aa $1 sel {arguments}
    ht	isolde mod his sel {arguments}
    so	setattr sel atoms occupancy {arguments}
    ab	isolde adjust bfactors {arguments}
    ss	isolde sim start sel
    rt	isolde release torsions sel {arguments}
    rd	isolde release distances sel {arguments}
    pf	isolde pepflip sel
    cf	isolde cisflip sel
    cbb	color bfactor {arguments}
    cbo	color byattr occupancy {arguments}
    cbc	color bychain; color byhet
    cs	clipper set contourSensitivity {arguments}
    

  
UCSF ChimeraX version: 1.3.dev202107240347 (2021-07-24)  
© 2016-2021 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open #1 format mmcif fromDatabase AlphaFold

UniProt identifiers must be 6 characters long  

> open 6zpl format mmcif fromDatabase pdb

6zpl title:  
Inward-open structure of human glycine transporter 1 in complex with a
benzoylisoindoline inhibitor, sybody Sb_GlyT1#7 and bound Na and Cl ions.
[more info...]  
  
Chain information for 6zpl #1  
---  
Chain | Description | UniProt  
A B | Sodium- and chloride-dependent glycine transporter 1 | SC6A9_HUMAN  
C | Endoglucanase H | GUNH_HUNT2  
E F | Sybody Sb_GlyT1#7 |  
  
Non-standard residues in 6zpl #1  
---  
CL — chloride ion  
NA — sodium ion  
QET —
[5-fluoranyl-6-(oxan-4-yloxy)-1,3-dihydroisoindol-2-yl]-[5-methylsulfonyl-2-[2,2,3,3,3-pentakis(fluoranyl)propoxy]phenyl]methanone  
  

> open #1 fromDatabase AlphaFold

Summary of feedback from opening #1 fetched from AlphaFold  
---  
warnings | UniProt sequence identifier not specified in file for chains /E,/F  
AlphaFold database does not have models for 1 UniProt ids P16218 (chains C)  
notes | 5523 atoms, 706 residues, atom bfactor range 20.9 to 98.7  
Matchmaker 6zpl, chain B (#1) with AlphaFold P48067, chain B (#), sequence
alignment score = 2847.1  
RMSD between 279 pruned atom pairs is 1.009 angstroms; (across all 530 pairs:
3.612)  
  
5523 atoms, 706 residues, atom bfactor range 20.9 to 98.7  
Matchmaker 6zpl, chain A (#1) with AlphaFold P48067, chain A (#), sequence
alignment score = 2844.1  
RMSD between 283 pruned atom pairs is 1.001 angstroms; (across all 518 pairs:
3.557)  
  
Chain information for UniProt P48067 chain B #2.1  
---  
Chain | Description  
B | Sodium- and chloride-dependent glycine transporter 1  
  
Chain information for UniProt P48067 chain A #2.2  
---  
Chain | Description | UniProt  
A | Sodium- and chloride-dependent glycine transporter 1 | SC6A9_HUMAN  
  
Opened 2 AlphaFold chain models  




OpenGL version: 3.3.0 NVIDIA 465.19.01
OpenGL renderer: NVIDIA TITAN Xp/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation
Manufacturer: Dell Inc.
Model: Precision T5600
OS: CentOS Linux 7 Core
Architecture: 64bit ELF
Virutal Machine: none
CPU: 32 Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.10GHz
Cache Size: 20480 KB
Memory:
	              total        used        free      shared  buff/cache   available
	Mem:            62G        9.2G         38G        537M         14G         52G
	Swap:          4.9G          0B        4.9G

Graphics:
	03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP102 [TITAN Xp] [10de:1b02] (rev a1)	
	Subsystem: NVIDIA Corporation Device [10de:11df]	
	Kernel driver in use: nvidia
Locale: ('en_GB', 'UTF-8')
PyQt5 5.15.2, Qt 5.15.2
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    Babel: 2.9.1
    backcall: 0.2.0
    blockdiag: 2.0.1
    certifi: 2021.5.30
    cftime: 1.5.0
    chardet: 4.0.0
    ChimeraX-AddCharge: 1.1.4
    ChimeraX-AddH: 2.1.10
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.1
    ChimeraX-AlphaFold: 1.0
    ChimeraX-AltlocExplorer: 1.0
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.27.1
    ChimeraX-AtomicLibrary: 4.0
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AtomSearchLibrary: 1.0
    ChimeraX-AxesPlanes: 2.0
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 1.1.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.5.2
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.1
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.1
    ChimeraX-Clipper: 0.17.0
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.3.2
    ChimeraX-CommandLine: 1.1.4
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.3.dev202107240347
    ChimeraX-CoreFormats: 1.0
    ChimeraX-coulombic: 1.3
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0
    ChimeraX-DataFormats: 1.2
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1.4
    ChimeraX-DistUI: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.1
    ChimeraX-Hbonds: 2.1
    ChimeraX-Help: 1.1
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.1
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ISOLDE: 1.3.dev32
    ChimeraX-ItemsInspection: 1.0
    ChimeraX-Label: 1.1
    ChimeraX-LinuxSupport: 1.0
    ChimeraX-ListInfo: 1.1.1
    ChimeraX-Log: 1.1.4
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.1
    ChimeraX-Map: 1.1
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 1.2.1
    ChimeraX-MDcrds: 2.4
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.1
    ChimeraX-mmCIF: 2.3
    ChimeraX-MMTF: 2.1
    ChimeraX-Modeller: 1.0.2
    ChimeraX-ModelPanel: 1.1
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.1
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0.2
    ChimeraX-OpenCommand: 1.6.2
    ChimeraX-PDB: 2.4.4
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0.1
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-Phenix: 0.3
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0.1
    ChimeraX-PubChem: 2.1
    ChimeraX-ReadPbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-Sample: 0.1
    ChimeraX-SaveCommand: 1.4.1
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.4.1
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.1
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.6
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.1
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.1
    ChimeraX-ToolshedUtils: 1.2
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.10.1
    ChimeraX-uniprot: 2.1
    ChimeraX-UnitCell: 1.0
    ChimeraX-ViewDockX: 1.0.1
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-Voyager: 0.1
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.4
    comtypes: 1.1.10
    cxservices: 1.0
    cycler: 0.10.0
    Cython: 0.29.23
    decorator: 4.4.2
    distlib: 0.3.1
    distro: 1.5.0
    docutils: 0.17.1
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    h5py: 3.3.0
    html2text: 2020.1.16
    idna: 2.10
    ihm: 0.20
    imagecodecs: 2021.4.28
    imagesize: 1.2.0
    ipykernel: 5.5.5
    ipython: 7.23.1
    ipython-genutils: 0.2.0
    jedi: 0.18.0
    Jinja2: 2.11.3
    jupyter-client: 6.1.12
    jupyter-core: 4.7.1
    kiwisolver: 1.3.1
    line-profiler: 3.3.0
    lxml: 4.6.3
    lz4: 3.1.3
    MarkupSafe: 1.1.1
    matplotlib: 3.4.2
    matplotlib-inline: 0.1.2
    msgpack: 1.0.2
    netCDF4: 1.5.6
    networkx: 2.6.2
    numexpr: 2.7.3
    numpy: 1.21.0
    numpydoc: 1.1.0
    OpenMM: 7.6.0
    openvr: 1.16.801
    packaging: 21.0
    ParmEd: 3.2.0
    parso: 0.8.2
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 8.2.0
    pip: 21.1.1
    pkginfo: 1.7.0
    prompt-toolkit: 3.0.19
    psutil: 5.8.0
    ptyprocess: 0.7.0
    pycollada: 0.7.1
    pydicom: 2.1.2
    Pygments: 2.9.0
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PyQt5-commercial: 5.15.2
    PyQt5-sip: 12.8.1
    PyQtWebEngine-commercial: 5.15.2
    python-dateutil: 2.8.1
    pytz: 2021.1
    pyzmq: 22.1.0
    qtconsole: 5.1.0
    QtPy: 1.9.0
    RandomWords: 0.3.0
    requests: 2.25.1
    scipy: 1.6.3
    setuptools: 57.0.0
    sfftk-rw: 0.7.0.post1
    six: 1.16.0
    snowballstemmer: 2.1.0
    sortedcontainers: 2.4.0
    Sphinx: 4.0.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 2.0.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.5
    suds-jurko: 0.6
    tables: 3.6.1
    tifffile: 2021.4.8
    tinyarray: 1.2.3
    tornado: 6.1
    traitlets: 5.0.5
    urllib3: 1.26.6
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.36.2
    wheel-filename: 1.3.0

[Tom Goddard]

A "non-functional link" is not very descriptive. Your log does not show an example of what happens. One scenario is I click the file history thumbnail that runs the command
"open #1 from alphafold" and if there is no model #1, it indeed does not work. If I do have a model #1 open it is working for me.

To fix the above problem I think it is very sensible that "open #1 from alphafold" should not go into the file history. There are other cases that will fail in the same way, for instance, opening an MTZ file and specifying the associated atomic structure to associate with "open myfile.mtz structure #1". If no model #1 is opened that isn't going to work.

[Tom Goddard]

Eric, I'd like "open #1 from alphafold" not to go into file history. Do you have any creative ideas how I should do that? I see the open code already does some gruesome hack where if it sees " in browser" at the end of the fetch message it decides not to put it in history. I could similarly exclude from history when the message ends with "AlphaFold chain models". But I am not keen to extend the use of such code.

[Eric Pettersen]

I might be able to add an 'in_file_history' attr to the fetch type registration. Will require investigation.

[Eric Pettersen]

Oh, you only want things like "open #1 from alphafold" to not go in the history, but "open P12347 from alphafold" to go into the history? Is that right? If so, will require more thought.

[goddard@…]

Yes, I want the fetch by uniprot id in history but the fetch with model number not in history, and both use the same fetch method.  I think "open #1 from alphafold" is really weird syntax for the open command.  Not sure I like it.

[Eric Pettersen]

Okay, I think my plan is to add an 'in_file_history' keyword to open/save command providers which defaults to True, but that image saving and HTML opening would set to False. That keyword would also be settable in the session.open_command.open_data() call to override on a case-by-case basis, which is what you would use to suppress the "open #1" syntax getting into the file history. Probably will get to it tomorrow.

As an amusing aside, the " in browser" hack doesn't work anymore because the status message changed!

[goddard@…]

Sounds reasonable except the part about how I would use it for AlphaFold.  I don't call open_command.open_data().  I call fetch.fetch_file() to get the AlphaFold mmCIF models.

[Eric Pettersen]

You call session.open_command.open_data() twice in fetch_alphafold.py

[goddard@…]

I don't call open_data() at all in fetch_alphafold.py.  I am missing something.

[Eric Pettersen]

Okay, the changes we discussed have been pushed. You can now supply in_file_history=False to the open_data() call as appropriate.

The actual problem which is the subject of this ticket remains, which needs feedback from Tristan I think.

[Tristan Croll]

No, it sounds like that will fix the problem. In my initial report, that first “open #1 format mmcif fromDatabase AlphaFold” command arose from clicking a link labelled “#1” in the file history before opening any models. If I understand correctly this change will stop that link from appearing in the first place. Incidentally, it’ll also help prevent filling up the history with unimportant entries when running a script over many models...

[goddard@…]

There has been file history code for years that prevents putting into file history the files opened in scripts.  Is that not working?

[Tristan Croll]

I have a habit of running bespoke scripts straight from the console rather than via the command line...
________________________________
From: ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
Sent: 03 August 2021 20:38
To: goddard@cgl.ucsf.edu <goddard@cgl.ucsf.edu>; Tristan Croll <tic20@cam.ac.uk>
Cc: pett@cgl.ucsf.edu <pett@cgl.ucsf.edu>
Subject: Re: [ChimeraX] #4985: Logged AlphaFold command link non-functional

#4985: Logged AlphaFold command link non-functional
------------------------------------+-------------------------
          Reporter:  Tristan Croll  |      Owner:  Tom Goddard
              Type:  defect         |     Status:  assigned
          Priority:  normal         |  Milestone:
         Component:  Logging        |    Version:
        Resolution:                 |   Keywords:
        Blocked By:                 |   Blocking:
Notify when closed:                 |   Platform:  all
           Project:  ChimeraX       |
------------------------------------+-------------------------

Comment (by goddard@…):

 {{{
 There has been file history code for years that prevents putting into file
 history the files opened in scripts.  Is that not working?
 }}}

--
Ticket URL: <https://www.rbvi.ucsf.edu/trac/ChimeraX/ticket/4985#comment:14>
ChimeraX <https://www.rbvi.ucsf.edu/chimerax/>
ChimeraX Issue Tracker

[goddard@…]

How do you runs scripts from the "console"?  What do you mean by the "console"?  Do you mean the Python shell, or do you mean the unix shell that started ChimeraX.  I guess you mean Python shell.  How do you run a script from the Python shell, execfile()?  import?  just type or copy and paste the Python code?  ChimeraX only knows it is running a script if the ChimeraX open command is used to open a *.py or *.cxc file.

[Tristan Croll]

Sorry - yes, I meant the Python shell. And often if it’s just for some one-off idea that isn’t worth turning into a reusable script I’ll just input some simple loop directly into it.

More generally, though, I can think of various reasons why a tool might want to open/save things without logging to history. The little Phenix bundle, for example - it writes little fragments of model/map to give to Phenix for its ligand fitting; those are deleted once it’s done. Likewise, the model generated by opening the docked ligand only exists for as long as is needed to copy it into the working model. So I can see this in_file_history argument getting a bit of use there.

Actually, that raises a question: will this option also be available for saving data? Or is it already available and I’ve just missed it?

[Tom Goddard]

I don't understand how to use the open_data() in_file_history argument to exclude "open #1 from alphafold" from the history. The alphafold fetching code calls open_data() to open mmCIF models but those are already not put into file history (not sure how that magic works). What code is needed to prevent the typed open command from going into the history?

[Tom Goddard]

I am leaning towards the opinion that the syntax "open #1" is just too weird and just be rescinded. Maybe "alphafold match #1" would be the right command for doing this operation. The open command takes database identifiers as the argument. Extending to derive those identifiers from an already opened model is confusing, and does not follow the pattern of every other "open" command usage.

[Eric Pettersen]

That would help. Closer examination of the code reveals that models created by open_data() are never placed in file history. To get your old scheme working, the Info() object returned by run_provider() would need its in_file_history attribute set appropriately. Unfortunately, unlike any other provider, the value for in_file_history for your provider is data dependent, which would be ugly to implement (though possible).

If you are in fact going to split off the '#1' syntax into a separate command, I will just remove the spurious 'in_file_history' arg to open_data.

[Tom Goddard]

I have eliminated the "open #1 from alphafold" syntax in favor of "alphafold match #1".

Eric you can remove the in_file_history option.

[Tom Goddard]

Fixed.

Got rid of the "open #1 from alphafold" syntax which was overloading the open command with too much magic that was not simply opening a data file.

[Eric Pettersen]

Tristan, FYI you can prevent your "script" files from going into the file history by embedding your code in a "with session.in_script:" block.

[Tristan Croll]

Awesome tip - thanks!
________________________________
From: ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
Sent: 04 August 2021 18:34
Cc: goddard@cgl.ucsf.edu <goddard@cgl.ucsf.edu>; pett@cgl.ucsf.edu <pett@cgl.ucsf.edu>; Tristan Croll <tic20@cam.ac.uk>
Subject: Re: [ChimeraX] #4985: Logged AlphaFold command link non-functional

#4985: Logged AlphaFold command link non-functional
------------------------------------+-------------------------
          Reporter:  Tristan Croll  |      Owner:  Tom Goddard
              Type:  defect         |     Status:  closed
          Priority:  normal         |  Milestone:
         Component:  Logging        |    Version:
        Resolution:  fixed          |   Keywords:
        Blocked By:                 |   Blocking:
Notify when closed:                 |   Platform:  all
           Project:  ChimeraX       |
------------------------------------+-------------------------

Comment (by Eric Pettersen):

 Tristan, FYI you can prevent your "script" files from going into the file
 history by embedding your code in a "with session.in_script:" block.

--
Ticket URL: <https://www.rbvi.ucsf.edu/trac/ChimeraX/ticket/4985#comment:23>
ChimeraX <https://www.rbvi.ucsf.edu/chimerax/>
ChimeraX Issue Tracker