Webcam command broken by pyside2

[goddard@…]

The following bug report has been submitted:
Platform:        macOS-10.15.7-x86_64-i386-64bit
ChimeraX Version: 1.2.dev202012112210 (2020-12-11 22:10:09 UTC)
Description
webcam command no longer works on Mac.  Tests show it worked in Dec 2 daily build that used PyQt5 and broke in Dec 3 daily build that uses PySide2.


Log:
UCSF ChimeraX version: 1.2.dev202012112210 (2020-12-11)  
© 2016-2020 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open 1a0m format mmcif fromDatabase pdb

1a0m title:  
1.1 angstrom crystal structure of A-conotoxin [TYR15]-epi [more info...]  
  
Chain information for 1a0m #1  
---  
Chain | Description  
A B | α-conotoxin [TYR15]-epi  
  
Non-standard residues in 1a0m #1  
---  
NH2 — amino group  
  

> webcam true

set view finder  
Camera "FaceTime HD Camera (Built-in)" pixel formats (Invalid) are not
supported by webcam command (ARGB32,YUYV)  




OpenGL version: 4.1 ATI-3.10.18
OpenGL renderer: AMD Radeon Pro Vega 20 OpenGL Engine
OpenGL vendor: ATI Technologies Inc.Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro15,3
      Processor Name: 8-Core Intel Core i9
      Processor Speed: 2.4 GHz
      Number of Processors: 1
      Total Number of Cores: 8
      L2 Cache (per Core): 256 KB
      L3 Cache: 16 MB
      Hyper-Threading Technology: Enabled
      Memory: 32 GB
      Boot ROM Version: 1037.147.4.0.0 (iBridge: 17.16.16610.0.0,0)

Software:

    System Software Overview:

      System Version: macOS 10.15.7 (19H2)
      Kernel Version: Darwin 19.6.0
      Time since boot: 7 days 22:11

Graphics/Displays:

    Intel UHD Graphics 630:

      Chipset Model: Intel UHD Graphics 630
      Type: GPU
      Bus: Built-In
      VRAM (Dynamic, Max): 1536 MB
      Vendor: Intel
      Device ID: 0x3e9b
      Revision ID: 0x0002
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1

    Radeon Pro Vega 20:

      Chipset Model: Radeon Pro Vega 20
      Type: GPU
      Bus: PCIe
      PCIe Lane Width: x8
      VRAM (Total): 4 GB
      Vendor: AMD (0x1002)
      Device ID: 0x69af
      Revision ID: 0x00c0
      ROM Revision: 113-D2060I-087
      VBIOS Version: 113-D20601MA0T-016
      Option ROM Version: 113-D20601MA0T-016
      EFI Driver Version: 01.01.087
      Automatic Graphics Switching: Supported
      gMux Version: 5.0.0
      Metal: Supported, feature set macOS GPUFamily2 v1
      Displays:
        Color LCD:
          Display Type: Built-In Retina LCD
          Resolution: 2880 x 1800 Retina
          Framebuffer Depth: 24-Bit Color (ARGB8888)
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: No
          Connection Type: Internal

Locale: (None, 'UTF-8')
PySide2 version: 5.15.2
Qt version: 5.15.2
Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    appnope: 0.1.2
    Babel: 2.9.0
    backcall: 0.2.0
    biopython: 1.78
    blockdiag: 2.0.1
    certifi: 2020.11.8
    cftime: 1.3.0
    chardet: 3.0.4
    ChimeraX-AddH: 2.1.3
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.1
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.10
    ChimeraX-AtomicLibrary: 1.1
    ChimeraX-AtomSearch: 2.0
    ChimeraX-AtomSearchLibrary: 1.0
    ChimeraX-AxesPlanes: 2.0
    ChimeraX-BasicActions: 1.1
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 1.1
    ChimeraX-BondRot: 2.0
    ChimeraX-BugReporter: 1.0
    ChimeraX-BuildStructure: 2.2
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.1
    ChimeraX-ButtonPanel: 1.0
    ChimeraX-CageBuilder: 1.0
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.1
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.1
    ChimeraX-ColorActions: 1.0
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-CommandLine: 1.1.3
    ChimeraX-ConnectStructure: 2.0
    ChimeraX-Contacts: 1.0
    ChimeraX-Core: 1.2.dev202012112210
    ChimeraX-CoreFormats: 1.0
    ChimeraX-coulombic: 1.0.2
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-DataFormats: 1.1
    ChimeraX-Dicom: 1.0
    ChimeraX-DistMonitor: 1.1
    ChimeraX-DistUI: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0
    ChimeraX-FunctionKey: 1.0
    ChimeraX-Geometry: 1.1
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.0
    ChimeraX-Hbonds: 2.1
    ChimeraX-Help: 1.0
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.0
    ChimeraX-ImageFormats: 1.1
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0
    ChimeraX-Label: 1.0
    ChimeraX-ListInfo: 1.1
    ChimeraX-Log: 1.1.1
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Map: 1.0.2
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.0
    ChimeraX-Markers: 1.0
    ChimeraX-Mask: 1.0
    ChimeraX-MatchMaker: 1.1
    ChimeraX-MDcrds: 2.1
    ChimeraX-MedicalToolbar: 1.0.1
    ChimeraX-Meeting: 1.0
    ChimeraX-MLP: 1.0
    ChimeraX-mmCIF: 2.2
    ChimeraX-MMTF: 2.0
    ChimeraX-Modeller: 1.0
    ChimeraX-ModelPanel: 1.0
    ChimeraX-ModelSeries: 1.0
    ChimeraX-Mol2: 2.0
    ChimeraX-Morph: 1.0
    ChimeraX-MouseModes: 1.0
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0
    ChimeraX-OpenCommand: 1.4
    ChimeraX-PDB: 2.2
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0
    ChimeraX-PickBlobs: 1.0
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.0.1
    ChimeraX-PubChem: 2.0.1
    ChimeraX-Read-Pbonds: 1.0
    ChimeraX-Registration: 1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-ResidueFit: 1.0
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.3
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SeqView: 2.3
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0
    ChimeraX-Shortcuts: 1.0
    ChimeraX-ShowAttr: 1.0
    ChimeraX-ShowSequences: 1.0
    ChimeraX-SideView: 1.0
    ChimeraX-Smiles: 2.0.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.2.2
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-Struts: 1.0
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0
    ChimeraX-SwapRes: 2.0
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Tempy: 0.6.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.0
    ChimeraX-ToolshedUtils: 1.1
    ChimeraX-Tug: 1.0
    ChimeraX-UI: 1.3.1
    ChimeraX-uniprot: 2.1
    ChimeraX-ViewDockX: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0
    ChimeraX-WebServices: 1.0
    ChimeraX-Zone: 1.0
    colorama: 0.4.3
    comtypes: 1.1.7
    cxservices: 1.0
    cycler: 0.10.0
    Cython: 0.29.21
    decorator: 4.4.2
    distlib: 0.3.1
    docutils: 0.16
    filelock: 3.0.12
    funcparserlib: 0.3.6
    grako: 3.16.5
    h5py: 2.10.0
    html2text: 2020.1.16
    idna: 2.10
    ihm: 0.17
    imagecodecs: 2020.5.30
    imagecodecs-lite: 2020.1.31
    imagesize: 1.2.0
    ipykernel: 5.3.4
    ipython: 7.18.1
    ipython-genutils: 0.2.0
    jedi: 0.17.2
    Jinja2: 2.11.2
    jupyter-client: 6.1.7
    jupyter-core: 4.7.0
    kiwisolver: 1.3.1
    line-profiler: 2.1.2
    lxml: 4.5.2
    lz4: 3.1.0
    MarkupSafe: 1.1.1
    matplotlib: 3.3.2
    MolecularDynamicsViewer: 1.1
    msgpack: 1.0.0
    netCDF4: 1.5.4
    networkx: 2.5
    numexpr: 2.7.1
    numpy: 1.19.2
    numpydoc: 1.1.0
    openvr: 1.14.1501
    packaging: 20.7
    parso: 0.7.1
    pexpect: 4.8.0
    pickleshare: 0.7.5
    Pillow: 7.2.0
    pip: 20.3
    pkginfo: 1.5.0.1
    prompt-toolkit: 3.0.8
    psutil: 5.7.2
    ptyprocess: 0.6.0
    pycollada: 0.7.1
    pydicom: 2.0.0
    Pygments: 2.7.1
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 2.4.7
    PySide2: 5.15.2
    python-dateutil: 2.8.1
    pytz: 2020.4
    pyzmq: 20.0.0
    qtconsole: 4.7.7
    QtPy: 1.9.0
    RandomWords: 0.3.0
    requests: 2.24.0
    scipy: 1.5.2
    Send2Trash: 1.5.0
    setuptools: 50.3.2
    sfftk-rw: 0.6.7.dev1
    shiboken2: 5.15.2
    six: 1.15.0
    snowballstemmer: 2.0.0
    sortedcontainers: 2.2.2
    Sphinx: 3.2.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 2.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 1.0.3
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.4
    suds-jurko: 0.6
    tables: 3.6.1
    tifffile: 2020.9.3
    tinyarray: 1.2.3
    tornado: 6.1
    traitlets: 5.0.5
    urllib3: 1.25.11
    wcwidth: 0.2.5
    webcolors: 1.11.1
    wheel: 0.36.0
    wheel-filename: 1.2.0

[Tom Goddard]

Problem appears to be that my VideoCapture class which is derived from QAbstractVideoSurface never has its supportedPixelFormats() method called. In the same code with PyQt5 that overridden method is called 3 times. With PySide2 it is never called and the error starting webcam says the only pixel format is Invalid_Format.

This is possibly this PySIde2 bug that titled "Overridden method of QAbstractVideoSurface do not get called", opened in 2018, priority P2 but not resolved.

​https://bugreports.qt.io/browse/PYSIDE-794

[Tom Goddard]

I made a simple test program for starting the webcam that works in PyQt5 and fails in PySIde2 apparently because it does not call the overridden QAbstractVideoSurface supportedPixelFormats() method. I added a comment and this test program to the Qt bug report on this issue

​​https://bugreports.qt.io/browse/PYSIDE-794

It may be that PySide2 is picky about the method function signature. The method has an optional argument. I tried several variations and none were called. It seems this is probably a Shiboken2 issue. If the Qt bug gets no action I could look at some other case of a Qt pure virtual method in a Qt class and see how Shiboken2 handles the override. Maybe the problem is there are few cases of pure virtual methods so it just is not handled. I tried instantiating QAbstractVideoSurface and it gave an error saying it is an abstract class that cannot be instantiated.

[Tom Goddard]

Closer study of pyside bug 794 shows a patch was made to PySide2 in 5.11 that fixed the equivalent problem with QMediaPlayer.setVideoOutput(QAbstractVideoSurface*), but the patch was soon retracted because it broke another overloaded variant setVideoOutput(QGraphicsVideoItem*) which resulted in another bug report pyside-864.

​https://bugreports.qt.io/browse/PYSIDE-864

Details of that bug show that before the patch only the QGraphicsVideoItem* argument of the 3 possible (QAbstractVideoSurface, QVideoWidget, QGraphicsVideoItem) could possibly work. That is clear from the shiboken generated code

​https://bugreports.qt.io/secure/attachment/78826/pyside864_generated.cpp.txt

The exact same situation applies to QCamera.setViewFinder() which can take one 3 arguments (QAbstractVideoSurface, QVideoWidget, QGraphicsVideoItem). I don't know what the current Qt 5.15 shiboken generated code looks like but the QAbstractVideoSurface case is missing from typesystem_multimedia_common.xml

​https://code.qt.io/cgit/pyside/pyside-setup.git/tree/sources/pyside6/PySide6/QtMultimedia/typesystem_multimedia_common.xml

and is almost sure not to generate the right code for that case. Specifically the other overload cases in typesystem_multimedia_common.xml for QCamera.setViewFinder() make sure the QObject * argument that shiboken gets is upcasted to the correct type, either QAbstractVideoSurface *, QVideoWidget *, or QGraphicsVideoItem *.

Compiling PySIde2 and Shiboken2 from scratch and looking at the generated files would surely reveal the problem. It could be that shiboken now generates the correct code and only the original patch that was retracted needs to be reapplied (actually a QCamera.setViewFinder() version since the original patch only handle QMediaPlayer.setVideoOutput()). Or it could be that shiboken still generates the wrong code even for the two cases QGraphicsVideoItem and QVideoWidget and a fix to shiboken would be needed.

[Tom Goddard]

I compiled PySide2 and Shiboken2 5.15.2 in order to see what code was generated for QCamera.setViewFinder(). It is the same wrong code mentioned in pyside bug 864 from Qt 5.11 long ago. The basic problem seems to be that it decides any QObject * argument must be a QGraphicsVideoItem *. Here's the relevant bit of code

    // Overloaded function decisor
    // 0: QCamera::setViewfinder(QAbstractVideoSurface*)
    // 1: QCamera::setViewfinder(QGraphicsVideoItem*)
    // 2: QCamera::setViewfinder(QVideoWidget*)
    if ((pythonToCpp = Shiboken::Conversions::isPythonToCppPointerConvertible(reinterpret_cast<SbkObjectType *>(SbkPySide2_QtCoreTypes[SBK_QOBJECT_IDX]), (pyArg)))) {
        overloadId = 1; // setViewfinder(QGraphicsVideoItem*)
    } else if ((pythonToCpp = Shiboken::Conversions::isPythonToCppPointerConvertible(reinterpret_cast<SbkObjectType *>(SbkPySide2_QtMultimediaTypes[SBK_QABSTRACTVIDEOSURFACE_IDX]), (pyArg)))) {
        overloadId = 0; // setViewfinder(QAbstractVideoSurface*)
    }

I've attached the generated file this came from qcamera_wrapper.cpp. In order to fix this the QGraphicsVideoItem check needs to be fixed. I have not yet tracked down where that type information is specified. The QAbstractVideoSurface type check looks correct so I should find out how that argument type is being specified and see how QGraphicsVideoItem does not follow the same pattern.

[Tom Goddard]

Shiboken2 generated QCamera wrapper code for Qt 5.15.2, file pyside-setup/pyside3_build/py3.8-qt5.15.2-64bit-release/pyside2/PySide2/QtMultimedia/PySide2/QtMultimedia/qcamera_wrapper.cpp

[Tom Goddard]

The difference between QGraphicsVideoItem and QAbstractVideoSurface is that the former is in a different module QtMultimediaWidgets while QAbstractVideoSurface and QCamera are defined in QtMultimedia. There is no SBK_QGRAPHICSVIDEOITEM_IDX in QtMultimedia shiboken generated files. QGraphicsVideoItem is instead listed using a "forward declaration" in pyside-setup/sources/pyside2/PySide2/QtMultimedia/typesystem_multimedia_forward_declarations.xml and apparently that is not working correctly for deciding which overloaded method to call. QVideoWidget is also not identified correctly in the setViewFinder() overloaded method choosing code and it belongs to QtMultimediaWidgets also.

I'd guess this situation of handling overloaded arguments from other Qt modules happens a fair bit, so there must be a way to handle it in shiboken2. Should probably track down a less obscure case where it is handled correctly and see how the source annotates the types to get it to work correctly.

[Tom Goddard]

I gave the details of the wrong shiboken2 generated code in bug report pyside-794 and made a plea for a knowledgable shiboken2 developer to fix it.

​https://bugreports.qt.io/browse/PYSIDE-794

[Tom Goddard]

Fixed.

Reverted from PySide2 to PyQt5.