dynamic-mani VIVE HEADING / CAMERA EXTRINSICS

Guides

On this page

Calibration

Updated: 2026-10-01

Back to the main usage guide

  1. Vive heading calibration
  2. Camera extrinsic calibration

#1. Vive heading calibration

#Run the program

Run the calibration program while the teleoperation dataflow is stopped:

bash
uv run calibrate
# Tianji: use either arm to establish the shared room heading.
uv run calibrate --config configs/tianji.yml --side right
# Optionally select the same tracker serial used for teleoperation.
uv run calibrate --serial LHR-XXXXXXXX

Start SteamVR first. The program polls the real Vive tracker and displays the configured robot at its default (idle.arm_q) pose. It opens no robot connection and sends no actuator targets. If the tracker is disconnected, the window waits and reconnects automatically when it becomes available.

#Select the live reference

  1. Hold your mounted tracker/hand in the direction represented by the displayed default pose. Choose the tracker axis pointing along the hand: T cycles the orange forward arrow. E cycles the corresponding cyan tool axis. The upper triad shows the live wrist orientation; X is red, Y green, Z blue.
  2. Hold still and press Space in the preview window to select the current live frame. The reference stays fixed while tracking continues.
  3. Move the wrist a little. The lower target triad shows the position and orientation produced by the shared teleop mapper with the new heading; the robot model remains at the default pose. Check forward, sideways, up, and wrist rotation. R clears the reference; Space selects again.
  4. Press Enter to save and close. Q, Escape, or closing the window cancels. Restart teleoperation to load the saved heading.

#Choose the forward axes

The forward arrows identify corresponding physical directions; they need not already overlap on screen before capture, since correcting their heading difference is the purpose of calibration. Defaults are wrist-bus local +X and tool local +Z (along the fingers) for both UR5e and Tianji. These are coordinate axes, not a claim about how your tracker is mounted. For a different mounting, select the appropriate signed axis with T, or start with --tracker-axis=-y. Use --ee-axis=+x to change the tool axis at startup. Choose directions with a clear horizontal component; a nearly vertical arrow has no reliable heading and cannot be used as the reference.

#Saved calibration

Saving updates only teleop.wrist_to_robot_rpy_deg in the selected rig YAML, preserving other settings and comments. A rollback copy is written beside it as <config>.before-heading-calibration. The solver adjusts yaw and retains any existing roll/pitch alignment. With the normal Z-up rig alignment, vertical motion stays vertical. Tianji's two arms share this room alignment; --side chooses which arm supplies the reference, and does not change the rig's active arm mode or tracker serials.

This calibration determines movement directions. Engaging still pins the current wrist pose to the default tool pose; resuming pins it to the held pose. Those clutch operations retain the saved heading. Simulation and hardware load the same calibration through the common mapper. Recalibrate after changing the SteamVR room heading or the operator's reference facing direction.

A single pose can determine heading only when the chosen tracker/tool axes represent the same direction. It does not estimate an arbitrary tracker mount or wrist pivot offset. Missing, invalid, stale, and nearly vertical reference samples are rejected. If the rig YAML changes while the window is open, reopen the program before saving so its default pose and alignment match the config.

#2. Camera extrinsic calibration

Mount the ChArUco board rigidly to the end effector and keep both cameras fixed. Capture an arm-only session using the hand-eye recording profile. After stopping the hand-eye-calib recording, run:

bash
uv run dmani-record calibrate out/recordings/<session> --episode 1
# An episode path and the shortcut work too:
uv run dmani-calibrate out/recordings/<session>/episodes/episode_000001

The default board is 12 × 9 squares, 30 mm squares, 22 mm markers, using DICT_5X5_100. Set --board-size X Y --square-mm S --marker-mm M --dictionary NAME for another board. Physical sizes must match the printed board.

Calibration scans only zed_1_left and zed_2_left, samples good board poses, and uses measured arm_joint_state, the captured URDF, and recorded native intrinsics/distortion. It jointly fits camera → robot-base transforms and the board mount, with per-camera timing corrections. --no-time-offset disables timing fitting. A closed session, sealed episode, and matching URDF hash are required.

Results go to a new out/calibrations/<session>_episode_<number>/ directory (or --output NEW_DIR): native PNG samples, annotated images, corresponding end-effector poses, .npy transforms, calibration_summary.json, and a dark calibration_report.html. Existing output directories are never overwritten.

#Preview camera extrinsics with UR5e

Preview the saved camera poses together with the UR5e model and recorded arm motion:

bash
# Preview the default saved calibration and open the browser.
uv run dmani-extrinsics --browser
# Or select your newly fitted calibration.
uv run dmani-extrinsics --calibration out/calibrations/<result> --browser

The viewer opens at http://127.0.0.1:8769. It needs the source recording and displays the two calibrated cameras alongside the recorded UR5e. This is an offline preview; it does not connect to or move the physical robot.

Transforms map raw left-eye optical coordinates into the recorded robot base frame; SDK rectified images may need a frame conversion. Validation uses held-out images from the same take and reports internal consistency. The command runs offline without starting hardware.

#Worked calibration example

For the supplied 2026-10-01 recording, run from the repository root:

bash
uv run dmani-record calibrate out/recordings/2026-10-01_17-17-57.324266-0700 --episode 1 --output out/calibrations/zed-left-example
uv run dmani-extrinsics --calibration out/calibrations/zed-left-example

Use a new --output directory if that example has already been run. Open out/calibrations/zed-left-example/calibration_report.html for the image gallery and numeric results. The verified default run selected 12 / 33 fit images for ZED 1 / ZED 2 and reserved 18 / 65 validation images. Held-out reprojection RMS was 0.313 / 0.294 px, with fitted delays of 15.3 / 11.1 ms. These measure consistency within the same recording.