IMU-Optical Intercalibration

Node ID: imuOpticalIntercalibration · Role: Sink · Realtime config: no

Description

Estimates the rotation alignment between an IMU and an optical tracking sensor. Run this to compute the calibration quaternion, then apply results to the IMU-Optical Fusion filter.

Algorithm notes

What the node estimates

This is not a recursive tracking filter: it is a batch estimator that runs on demand, accumulates a buffer of synchronized pose pairs from two sources, and solves a closed-form alignment problem over that buffer. It holds no continuously updated state that you can read as a pose; its “state” is a growing set of accepted orientation samples plus the counters shown in the status feed, and its output is a single calibration transform published once the solve succeeds.

Two modes are available. In IMU-to-Optical mode the result is the rotation that carries vectors expressed in the IMU body frame into the optical body frame. It is produced in the convention expected by the fusion node’s alignment setting, so it can be applied directly with no inversion or conjugation. Internally each side is first turned into an orientation stream, the rotation deltas of both sides relative to the first accepted sample are converted to rotation vectors, and the best-fit rotation between the two vector sets is recovered by an SVD alignment without centering. Only rotation is estimated; translation stays zero.

In Hand-Eye mode both inputs are full poses and the node solves the classic AX = XB problem between relative motions of the two sources, returning the complete rigid transform, rotation and translation, between the two attached frames. Rotation is solved from the log maps of paired relative motions, translation from a least-squares system built on those same motions, followed by a residual-based inlier pass and a second solve.

How the inputs are used

The node stays idle until it receives an explicit start command; data arriving before that is discarded, and a start always clears previously accumulated samples. In IMU-to-Optical mode the IMU stream is integrated by an internal orientation fusion that must first settle on gravity, so the first samples after start contribute nothing. The optical stream provides the reference orientation and is only used for its rotation; its position is ignored. Reverse mode inverts the incoming optical pose before use, for the case where the marker or fiducial is static and the camera moves with the IMU.

In Hand-Eye mode either input side may be an optical pose or an already-fused pose, and each side is identified by which input it is wired to. Both sides are used with their original timestamps, which is what allows samples from different sources to be paired.

Progress is published periodically on the status channel: total received samples per side, accepted pose count, per-axis coverage, skip counts for fast motion and timestamp mismatch, and the good-pair count of the most recent solve attempt. When the solve succeeds the node stops itself, reports finished, and emits the result. It is a sink, so nothing is forwarded on the data path.

Parameter groups

Sample admission gates decide which incoming samples are trustworthy: an angular-rate ceiling drops IMU samples taken during fast motion, a maximum inter-stream delay drops pairs whose timestamps disagree, and per-stream delay offsets let you compensate a known constant latency on either side. A minimum angular distance between consecutive accepted samples prevents the buffer from filling with near-duplicate poses while the sensor is at rest.

Coverage and completion gates control when a solve is attempted at all: a minimum total accepted pose count, a minimum number of accepted poses whose dominant rotation axis is X, Y and Z respectively, and a minimum number of pairs that survive consistency checking. All three must be satisfied before a result can be produced.

Consistency and robustness gates decide which pairs enter the solve: an absolute limit on how far the two sources may disagree about how much they rotated between two samples, and, in Hand-Eye mode, a maximum accepted rotation residual on the final fit. Hand-Eye additionally only uses relative motions whose rotation magnitude is roughly between 17 and 97 degrees.

Mode and routing settings select the calibration type and, for Hand-Eye, which source is treated as side A and which as side B.

Tuning and failure modes

Move deliberately and in discrete steps: rotate to a new attitude, pause, then move again. Continuous sweeping causes most samples to be rejected by the angular-rate gate and biases the ones that survive.

Exercise all three axes. The most common stall is heavy yaw and pitch with almost no roll, which blocks the per-axis gate indefinitely; watch the per-axis counters rather than the total count.

Beware the progress illusion: the total accepted pose count can reach its target and stay there while the good-pair count remains below its threshold, so the calibration never finishes. That means the two streams disagree about rotation magnitude. If the ratio of their reported rotation angles is roughly constant, suspect a gyro scale or unit error. If the disagreement grows with faster motion, suspect uncompensated latency between the streams and adjust the per-stream delay offsets.

High timestamp-mismatch counts point to clock skew or pipeline lag rather than to bad geometry; widen the delay tolerance only after checking that the two sources really share a time base, since a loose tolerance pairs stale samples and silently degrades accuracy.

If the solve completes but the answer is wrong by several degrees on one axis, the buffer was under-sampled on that axis. Raise the per-axis minimum and repeat rather than accepting the result. In Hand-Eye mode, motions that are all about the same axis leave the transform under-determined, and translation in particular requires rotation about at least two distinct axes with genuine positional displacement; pure in-place rotation yields a usable rotation but an unreliable translation. Repeated failures with plenty of samples usually mean the residual gate is rejecting a poor fit, which is the intended behavior; loosen it only if you can accept a correspondingly less accurate transform.

Inputs / Outputs

• Inputs: Imu, Optical, OpticalA (Optical) - Optical A, OpticalB (Optical) - Optical B, FusedPoseA (FusedPose) - FusedPose A, FusedPoseB (FusedPose) - FusedPose B
• Outputs: (none)

Config aliases

ImuOpticalIntercalibrationFilter, imuOpticalIntercalibration

Properties

The following fields are exposed in the node’s Properties panel in the UI. Types map to UI widgets (e.g. string → text input, number → numeric, boolean → toggle, select → dropdown, json → JSON editor).

Example node definition

A full node definition in config.json looks like the block below. Paste it under the sinks key of your config, keyed by the node’s instance name (any identifier; it doesn’t have to match the node ID).

{
  "sinks": {
    "imuOpticalIntercalibration": {
      "inputEndpoints": [
        "inproc://imu_data",
        "inproc://optical_data",
        "inproc://opticalA_data",
        "inproc://opticalB_data",
        "inproc://fusedPoseA_data",
        "inproc://fusedPoseB_data"
      ],
      "inputDataFilter": [
        "Imu",
        "Optical",
        "Optical",
        "Optical",
        "FusedPose",
        "FusedPose"
      ],
      "settings": {
        "type": "imuOptical"
      }
    }
  }
}

Field reference

Connections can also be declared at the top-level connections array when settings.explicitConnections is true, as an alternative to filling inputEndpoints on each node.