Bug in unit testing

[Tristan Croll]

Not sure why I was suddenly hit with this, other than that I'm currently cleaning out obsolete/duplicate code. In any case, I suddenly started hitting the following traceback in make app-install

  File "/home/tic20/apps/chimerax/lib/python3.6/site-packages/setuptools/command/test.py", line 67, in loadTestsFromModule
    def __init__(self, fget):
  File "/home/tic20/apps/chimerax/lib/python3.6/unittest/suite.py", line 24, in __init__
    self.addTests(tests)
  File "/home/tic20/apps/chimerax/lib/python3.6/unittest/suite.py", line 60, in addTests
    self.addTest(test)
  File "/home/tic20/apps/chimerax/lib/python3.6/unittest/suite.py", line 47, in addTest
    raise TypeError("{} is not callable".format(repr(test)))
TypeError: None is not callable

After some digging, this seems to boil down to a clear bug in unittest that occurs if/when a submodule is traversed more than once. In site-packages/setuptools/command at line 26:

    def loadTestsFromModule(self, module, pattern=None):
        """Return a suite of all tests cases contained in the given module

        If the module is a package, load tests from all the modules in it.
        If the module has an ``additional_tests`` function, call it and add
        the return value to the tests.
        """
        if module in self._visited:
            return None
        self._visited.add(module)

The None gets carried through, until in unittest/suite.py it finally breaks:

    def addTest(self, test):
        # sanity checks
        if not callable(test):
            raise TypeError("{} is not callable".format(repr(test)))
        if isinstance(test, type) and issubclass(test,
                                                 (case.TestCase, TestSuite)):
            raise TypeError("TestCases and TestSuites must be instantiated "
                            "before passing them to addTest()")
        self._tests.append(test)

I think the easy monkeypatch would be to add

    if test is None:
        return

at the beginning of addtest().

What beats me is: why hasn't this been hitting everyone?

[Tristan Croll]

OK, one mystery solved. I'd deleted validation.py from my source tree, and renamed the directory validation_new to replace it. But it turns out that without a make clean step, an old cached version of validation.py was still being installed. Obviously unittest (understandably) didn't like having a .py file and a directory with the same name.