command-line breaks when a command is a prefix of another command (was: make install: '_Defer' object has no attribute 'function')

[Tristan Croll]

I *think* this has been triggered because I defined a set of commands clipper xxx ... for which the existing clip command is a prefix. Anyway, when I run make app-install I get, at the tail end of the build:

Executing: toolshed install './dist/ChimeraX_Clipper-0.2.1-cp36-cp36m-linux_x86_64.whl'
unloading module chimerax.clipper
Successfully installed ChimeraX-Clipper-0.2.1
Installed ChimeraX-Clipper (0.2.1)
'_Defer' object has no attribute 'function'

exit
Unknown command: exit
Unknown command: exit
100.00% done: Finished initialization
UCSF ChimeraX version: 0.8 (2018-12-06)
cmd> 

With no traceback I have no idea where the offending _Defer.function call is (I believe it should be replaced with ...proxy), but adding the following to commands.cli._Defer is a successful band-aid:

    @property
    def function(self):
        return self.proxy

[Tristan Croll]

Looks like I'm going to have to find a different front-end command than clipper... The previously-mentioned band-aid fix gets the build working again, but when I start ChimeraX the entire command system is broken:

open 6hn5

Unknown command: lighting default 
Unknown command: lighting default

[Greg Couch]

Hi Tristan,

Could you send me the bundle_info.xml file for the Clipper bundle? I've found other bugs, but not that one yet.

-- Greg

[Greg Couch]

And the bundle's init.py file please. Might be an interaction between them.

[tic20@…]

Here you go. I've also attached the cmd.py defining the commands. The 
line:

     <ChimeraXClassifier>ChimeraX :: Command :: cxclipper :: General :: 
Command-line control of Clipper plugin for symmetry and map 
visualisation</ChimeraXClassifier>

was originally:

     <ChimeraXClassifier>ChimeraX :: Command :: clipper :: General :: 
Command-line control of Clipper plugin for symmetry and map 
visualisation</ChimeraXClassifier>

... with __init__.py and cmd.py changed to suit. Adding the 'clipper 
...' commands triggered the bug, and renaming them to 'cxclipper ...' 
made it go away.

Happy to send a full source tarball if you need it.


On 2018-12-11 01:41, ChimeraX wrote:

bundle_info.xml​

__init__.py​

cmd.py​

[tic20@…]

Added by email2trac

[tic20@…]

Added by email2trac

[tic20@…]

Added by email2trac

[Tristan Croll]

Whatever you did, the problem appears to be gone in today's build. Clipper now builds without issue with commands named as clipper .... I notice that the warning that previously appeared in the bash shell (something along the lines of "'clip' command is a prefix of 'clipper'") is gone. What is the general consensus on this situation? "clipper" would certainly be the most natural top-level command to contain Clipper-specific commands, but I'm ok with switching over to something else (I was running with "cxclipper") if you prefer.

[Tristan Croll]

... or not. Not sure why it stopped happening for a while, but it popped up again. By forcing toolshed.info.register() to raise errors rather than logging them as warnings, I was able to drill down closer to the issue (see traceback below).

There are actually two issues. The first, in core.commands.cli.add_subcommand() is the source of the original "'_Defer' object has no attribute 'function'" error. In cli.register():

    if isinstance(function, _Defer):
        cmd_desc = function
    else:
        cmd_desc.function = function
        if cmd_desc.synopsis is None:
            msg = 'Command "%s" is missing a synopsis' % name
            if logger is None:
                print(msg)
            else:
                logger.warning(msg)

... so something's not right there. The bigger issue, though, is in toolshed.info.register():

        try:
            self._register_commands(logger)
        except Exception as e:
            logger.warning(str(e))

... so the ValueError arising from the prefix clash only gets logged as a warning, but stops registration of all commands beyond that point. So, safe to say that it's *not* currently possible to have two registered commands where one is a prefix of the other. Will adjust my code accordingly.

NOTE: Traceback (most recent call last):  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/commands/cli.py", line 1894, in add_subcommand  
    _check_autocomplete(word, self.subcommands, name)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/commands/cli.py", line 1709, in _check_autocomplete  
    raise ValueError("'%s' in '%s' is a prefix of an existing command '%s'" %
(word, name, key))  
ValueError: 'clip' in 'clip' is a prefix of an existing command 'clipper'  
  
During handling of the above exception, another exception occurred:  
  
Traceback (most recent call last):  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/runpy.py", line 193, in
_run_module_as_main  
    "__main__", mod_spec)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/runpy.py", line 85, in
_run_code  
    exec(code, run_globals)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/ChimeraX_main.py", line 807, in <module>  
    exit_code = init(sys.argv)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/ChimeraX_main.py", line 544, in init  
    check_available=opts.get_available_bundles)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/toolshed/__init__.py", line 1283, in init  
    _toolshed = Toolshed(*args, **kw)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/toolshed/__init__.py", line 341, in __init__  
    self.reload(logger, check_remote=check_remote,
rebuild_cache=rebuild_cache)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/toolshed/__init__.py", line 405, in reload  
    self._installed_packages)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/toolshed/installed.py", line 77, in register_all  
    bi.register(logger)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/toolshed/info.py", line 239, in register  
    raise e  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/toolshed/info.py", line 237, in register  
    self._register_commands(logger)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/toolshed/info.py", line 268, in _register_commands  
    cli.delay_registration(ci.name, cb, logger=logger)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/commands/cli.py", line 1839, in delay_registration  
    register(name, None, _Defer(proxy_function), logger=logger)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/commands/cli.py", line 2003, in register  
    _parent_info.add_subcommand(words[-1], name, cmd_desc)  
  File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/commands/cli.py", line 1896, in add_subcommand  
    if cmd_desc is None or not isinstance(cmd_desc.function, Alias):  
AttributeError: '_Defer' object has no attribute 'function'  
  

BUG: AttributeError: '_Defer' object has no attribute 'function'  
  
File "/opt/UCSF/ChimeraX-daily/lib/python3.6/site-
packages/chimerax/core/commands/cli.py", line 1896, in add_subcommand  
if cmd_desc is None or not isinstance(cmd_desc.function, Alias):  
  
 _See log for complete Python traceback._  

[Greg Couch]

To handle clip vs. clipper, we will turn it into a warning and we will register the standard commands first, so you won't see the warning.

[Greg Couch]

Can reproduce by registering two commands in a bundle and the second one is a prefix of the first.

[Greg Couch]

FYI, the symptoms are gone, and there are better warnings in the toolshed.info.register() code for cases when something does go wrong. The fix where was to not check if cmd_desc is not None, but that isinstance(cmd_desc, CmdDesc) before checking that the cmd_desc.function is an Alias or not.

The clip vs clipper issue still needs work.

[Greg Couch]

clip vs clipper will be handled by having the standard commands bundle be a prerequisite so it is registered first. In the mean time, it will show up as a warning if clipper happens to be registered before clip.