Inside-Out Fusion

Node ID: insideOutFusion · Role: Filter · Realtime config: no

Description

Complementary filter that fuses inside-out tracking (e.g. VR headset SLAM) with external optical tracking (e.g. OptiTrack) to produce a globally referenced, high-frequency pose.

Algorithm notes

How it works

This node combines two tracking sources into one pose. The headset’s own inside-out tracking supplies fast, smooth motion but slowly drifts away from the room. The external optical system supplies a drift-free absolute position but is slower and noisier. The node takes the fast part of the inside-out motion and adds it to the slow part of the optical position, so the output moves instantly with the user and still stays anchored in the room. Orientation comes from the inside-out source, rotated into the room frame by an alignment that is continuously pulled toward the optical reference. The node also learns, while running, how the two systems differ in scale, in rotation and in timing, and corrects for that.

What you must configure

Inside-Out Sender ID must match the sender name of the headset stream. Everything else arriving as optical data is treated as the external reference, so this one string decides which stream is which. Get it wrong and the node will fuse the wrong pair.

If you already have a hand-eye calibration between the optical target and the headset, enter it as Optical Orientation Offset and Optical Position Offset. Leave them at their defaults if you do not - the node then aligns the frames by itself.

How it starts

Output begins as soon as inside-out data arrives, but until the first optical sample has been paired with it the position is not yet referenced to the room. Scale and latency learning wait for Auto-Calib Delay (s) after start, and then need real movement - at least a few centimetres of travel - before they converge. With Auto-Calib Save/Load on, a converged calibration is reloaded at the next start, so subsequent sessions come up already correct.

Tuning

IO High-Pass Weight is the one to look at first. It smooths fast motion but delays the output; 0.85 costs roughly 60 ms. With a modern headset, use 0 to 0.4.
Orientation Blend Weight raises how fast heading follows the optical reference. Increase it if orientation drifts visibly; decrease it if the optical jitter shows up in the view.
Optical Low-Pass Weight closer to 1 gives a smoother, slower absolute reference. Enable Adaptive Optical Pull-In if large offsets after startup or a dropout take too long to resolve.
Turn on Use IO Height when the optical system’s vertical is the weaker of the two.
Set Translation Scale Factor by hand only if you disable Enable Scale Auto-Calibration.

Troubleshooting

Position never leaves the headset’s own frame - Inside-Out Sender ID does not match the headset stream, so no stream is being used as the external reference.
The scene lags behind head motion - IO High-Pass Weight is too high; lower it toward 0.
The object drifts sideways in proportion to how far you move - scale or frame mismatch; keep Enable Translation Scaling, Enable Scale Auto-Calibration and Enable Rotation Refinement on and move around for a while so they can converge.
Position jumps or snaps when the optical system is briefly occluded - enable Inside-Out Fallback, and raise Optical Timeout (s) if short gaps trigger it too eagerly.
Auto-calibration never converges - not enough movement, or Auto-Calib Delay (s) is set too long. Lower Auto-Calib Forget Factor for a steadier estimate, raise it to adapt faster.
Calibration looks wrong after a hardware change - turn Auto-Calib Save/Load off for one run so the stale stored values are not reloaded.

Inputs / Outputs

Inputs: FusedPose, Optical
Outputs: FusedPose

Config aliases

InsideOutFusion, insideOutFusion

Required feature

imu_optical_fusion

Properties

Each row shows the setting’s label in the node’s Properties panel and, in code, its key in the node’s settings object in the config file. A setting omitted from the config file uses the default shown here.

Property	Type	Default	Description
Inside-Out Sender ID `insideOutSenderId`	string	`insideOut`	Sender id marking which incoming optical stream is the inside-out (SLAM/headset) source. The other optical stream is treated as the external absolute reference. Both inputs arrive as OpticalData and are distinguished by this id.
IO Low-Pass Weight `ioLpWeight`	number	`0.999`	Low-pass smoothing factor (0-1) used to estimate the slow drift of the inside-out position. The estimate is subtracted to isolate the high-frequency motion. Closer to 1 = slower drift estimate, so more of the inside-out signal counts as fast motion.
IO High-Pass Weight `ioHpWeight`	number	`0.85`	Smoothing factor (0-1) applied to the high-pass (fast-motion) component of the inside-out position before it is added to the optical reference. Higher values smooth the fast motion more but delay the position output (group delay ~ frame_dt * w/(1-w), e.g. 0.85 at 90 Hz = ~63 ms). With a smooth SLAM source, 0-0.4 is recommended.
Optical Low-Pass Weight `optLpWeight`	number	`0.999`	Low-pass smoothing factor (0-1) on the external optical position, which provides the drift-free absolute reference. The fused position is this low-passed optical plus the high-passed inside-out motion. Closer to 1 = smoother, slower-responding absolute reference.
Orientation Blend Weight `oriBlendWeight`	number	`0.02`	Per-update slerp weight pulling the inside-out alignment quaternion toward the optical reference orientation. Small values (0.02) correct orientation drift slowly and smoothly; larger values follow the optical reference faster but pass through more jitter.
Use IO Height `useIOHeight`	boolean	`false`	Take the vertical (height) component of position from the inside-out source directly instead of the complementary blend. Useful when the inside-out tracker has good vertical stability and the optical reference does not.
Use IO Horizontal `useIOHorizontal`	boolean	`true`	Blend the inside-out high-frequency motion into the horizontal position. When off, horizontal position follows the external optical reference directly with no inside-out contribution.
Prediction Time Modifier `predictionTimeModifier`	number	`0.02`	Look-ahead time in seconds used to forward-predict the pose, compensating for downstream latency. 0 disables prediction.
Inside-Out Fallback `fallbackToInsideOut`	boolean	`true`	When the optical reference stops arriving for longer than Optical Timeout, keep tracking from the inside-out source alone with the position offset frozen at dropout, so the transition is seamless. On recovery the pose blends back to the optical reference at the Optical Low-Pass rate.
Optical Timeout (s) `opticalTimeoutSec`	number	`0.5`	Time in seconds without optical data before the filter declares a dropout and switches to the inside-out fallback.
Enable Translation Scaling `translationScalingEnabled`	boolean	`true`	Apply a scale factor to the inside-out translations (about their slow mean) so short-term motion matches the optical reference scale. Uses the auto-calibrated factor once scale auto-calibration has converged, otherwise the manual Translation Scale Factor.
Translation Scale Factor `translationScaleFactor`	number	`1.0`	Manual scale factor for the inside-out translations, used while Translation Scaling is enabled and no auto-calibrated estimate is available. E.g. 1.012 enlarges inside-out motion by 1.2%.
Enable Scale Auto-Calibration `autoCalibEnabled`	boolean	`true`	Continuously estimate the translation scale factor and the residual alignment rotation between the inside-out and optical positions from recent motion (recursive displacement regression). Scale is applied when Translation Scaling is enabled; rotation when Rotation Refinement is enabled.
Enable Rotation Refinement `rotationRefinementEnabled`	boolean	`true`	Use the auto-calibrated residual rotation for mapping inside-out translations into the optical frame, correcting small cross-calibration errors that cause lateral drift proportional to head motion. The rotation is only updated when recent motion spans more than one direction.
Adaptive Optical Pull-In `adaptiveOptLpEnabled`	boolean	`false`	Scale the optical low-pass correction rate with the current disagreement between the fused output and the optical reference: large offsets (startup, fallback recovery) resolve quickly while small residuals keep the slow, smooth correction.
Enable Latency Auto-Calibration `latencyAutoCalibEnabled`	boolean	`true`	Estimate the time offset between the optical and inside-out streams by cross-correlating their speed profiles, then time-shift sample pairing for the alignment and scale estimates. Needs no common time base, only that both streams are timestamped on the same receive clock.
Auto-Calib Delay (s) `autoCalibDelaySec`	number	`5.0`	Delay in seconds after startup before auto-calibration begins, allowing both tracking sources to stabilize first.
Auto-Calib Forget Factor `autoCalibForgetFactor`	number	`0.001`	Per-sample exponential forgetting of the recursive scale regression. Higher values weight recent motion more heavily, adapting faster at the cost of stability.
Auto-Calib Save/Load `autoCalibSave`	boolean	`true`	Persist the converged scale and latency calibration to disk and reload it on the next start, so the filter resumes warm instead of recalibrating from scratch.
Optical Orientation Offset `opticalOrientationOffset`	quaternion	`{"w":1.0,"x":0.0,"y":0.0,"z":0.0}`	Static rotation applied to incoming optical pose before fusion. Set from a hand-eye calibration so the optical reference (e.g. DTrack) lines up with the inside-out frame (e.g. ALVR native pose). When non-identity, the online orientation alignment is locked to identity; scale and latency auto-calibration remain active.
Optical Position Offset `opticalPositionOffset`	vector3	`{"x":0.0,"y":0.0,"z":0.0}`	Static translation applied to incoming optical pose after the orientation offset (p’ = q_off * p + t_off). Paired with opticalOrientationOffset to bring the optical reference into the inside-out frame.

Example node definition

A minimal entry in config.json looks like the block below. Paste it under the sinks key, keyed by an instance name of your choice. All settings are optional: anything not listed under settings uses the default from the Properties table above. See The configuration file for how the sections and endpoint wiring work.

{
  "sinks": {
    "insideOutFusion": {
      "dataEndpoint": "inproc://insideOutFusion_data",
      "inputEndpoints": [
        "inproc://fusedPose_data",
        "inproc://optical_data"
      ],
      "inputDataFilter": [
        "FusedPose",
        "Optical"
      ],
      "settings": {}
    }
  }
}

Loading documentation…