Vive Tracker for RealSense camera not aligned with Oculus headsets

[Tom Goddard]

Doing mixed reality video capture with VR and a RealSense depth sensing camera with a Vive Tracker on the camera so its position is known requires the headset be one that is tracked by the Vive/Valve lighthouse base stations. That assures that the coordinate system of the Vive Tracker is the same as that of the headset. If instead an Oculus headset (e.g. Rift S or Quest 2) is used then the headset has a different coordinate system than the headset.

There are supposedly ways to align the two coordinate systems, for example, such as the OPenVR-SpaceCalibrator software

​https://github.com/pushrax/OpenVR-SpaceCalibrator

The readme for that project and various reports online suggest that the alignment goes bad for Quest 2 inside-out tracking quite often.

Without using the Vive Tracker to know the position and orientation of the RealSense camera it is hard to manually calibrate the position. If the Quest 2 coordinate somehow drift over time, then the calibration will drift resulting in the hand cones and models and all computer rendering drifting relative to the person in the video.

The easiest solution to all these problems is to limit mixed reality capture to use with the Vive Pro headsets.

[Tom Goddard]

In my testing with Quest 2 I could not even get the Vive Tracker to appear in the SteamVR status window. Apparently it needs a SteamVR configuration file setting changed "activateMultipleDrivers: true" which is a setting that OpenVR-SpaceCalibrator makes.

I think this tracker problem is best solved by waiting for Oculus to come out with their own trackers. Not sure when that might happen.