RealSense camera freezes

[goddard@…]

The following bug report has been submitted:
Platform:        Windows-10-10.0.18362
ChimeraX Version: 1.1.dev202006070045 (2020-06-07 00:45:17 UTC)
Description
RealSense camera freezes, RGB image does not update, although depth updates.  Happens quickly at 1920,1080 or 960,540 but works at 640x480 all at 30 frames per second.  Works at 6 frames per second at 1920,1080 at least in a 3 minute test.

Get same freezes in RealSense Viewer application and there it gives USB Cam Overflow error message.  Reports online suggest this may be because the USB3 connection cannot deliver enough current to the device.  That is why it works at slower frame rates where presumably less current is needed.

Log:
UCSF ChimeraX version: 1.1.dev202006070045 (2020-06-07)  
© 2016-2020 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open 1a0m format mmcif fromDatabase pdb

1a0m title:  
1.1 angstrom crystal structure of A-conotoxin [TYR15]-epi [more info...]  
  
Chain information for 1a0m #1  
---  
Chain | Description  
A B | α-conotoxin [TYR15]-epi  
  
Non-standard residues in 1a0m #1  
---  
NH2 — amino group  
  

> realsense true size 1920,1080

RealSense camera model #2  

> windowsize 1920 1080

color image type uint8 (1080, 1920, 3)  
depth image type uint16 (1080, 1920) mean 1026.4552734375 min 0 max 14099  
color intrinsics width: 1920, height: 1080, ppx: 956.966, ppy: 539.739, fx:
1383.16, fy: 1383.42, model: 2, coeffs: [0, 0, 0, 0, 0]  
color fov [69.52584838867188, 42.64519500732422]  
depth intrinsics width: 1920, height: 1080, ppx: 956.966, ppy: 539.739, fx:
1383.16, fy: 1383.42, model: 2, coeffs: [0, 0, 0, 0, 0]  
depth fov [69.52584838867188, 42.64519500732422]  
extrinsics color to depth rotation: [1, 0, 0, 0, 1, 0, 0, 0, 1]  
translation: [-0, -0, -0]  




OpenGL version: 3.3.0 NVIDIA 442.19
OpenGL renderer: GeForce GTX 1080 Ti/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation
Manufacturer: Gigabyte Technology Co., Ltd.
Model: Z270X-Gaming K7
OS: Microsoft Windows 10 Home (Build 18362)
Memory: 17,129,693,184
MaxProcessMemory: 137,438,953,344
CPU: 8 Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz"
PyQt version: 5.12.3
Compiled Qt version: 5.12.4
Runtime Qt version: 5.12.8

[Tom Goddard]

The only solution is probably to have an adequate USB 3 connection either from the computer or a powered external hub. Cable length can also matter. I was using a 6 foot USB-C to USB-C cable on my home Windows PC. Did not try the 3 foot cable the camera comes with, might have helped. IR projector was off.

[Tom Goddard]

I got a powered usbc hub to see if low current is the cause of the freezes but I have not tested yet (even though I got it several months ago). Model StarTech HB31C3A1CS,

[Tom Goddard]

The powered USB hub did not prevent freezes.

Trying to use the RealSense D435 in ChimeraX at 960x540 with Oculus Rift headset froze the RealSense camera on about 5 tries each in less than 10 seconds. The Oculus Rift is using 3 tracking cameras (two usb3, one usb2) plus a usb connection to the headset and is known to put high demands on USB bandwidth. I then tried without the hub and 6 foot cable, same freezing. I then switched to the Vive Pro headset and successfully used the D435 for abut 15 minutes without freezes. These tests were on my home Windows machine with D435 plugged into the one USBC port on that computer.

So it appears that USB bandwidth is the problem causing the freezes. Possibly can reduce problems by finding what other USB jacks are on the same USB controller as the USBC jack and not use those. If that still has problems at 1920x1080, getting a dedicated USBC card might solve the problem.

[Tom Goddard]

In #4235 I tried a newer librealsense version 2.41 instead of 2.35 and it had the same freezes, even at 640x480. Looking at my home windows motherboard specs Gigabyte Z270X-Gaming K7 I see the USBC and the red USBA next to it are on the same controller. A usb extender to an Oculus camera was on the red USBA. I disconnected camera from the red USBA and then the RealSense worked without freezes at 960x540 (about 15 minute test) and 1920x1080 (about a 3 minute test). So maybe the USBC controller needs to have no other devices connected. But the first time I started the RealSense after removing the Oculus camera it froze immediately, so it could be these results are unreliable.