Stereo depth for an indoor rover: dual IMX296 global-shutter cameras on a Pi 5 with Hailo-8
  • Python 60.5%
  • HTML 39.5%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Bethany 1779a78810 aruco: object marker sheet, and arm-camera intrinsic calibration
Plain ArUco for monocular object localisation, so the ChArUco board is
out of the picture and the whole dictionary is free: 12 markers at 40 mm,
ids 0-11, three per row on A4. Verified by round trip at 39.93 mm against
40.00 nominal.

calib_cam.py calibrates the arm camera, which is the hard blocker --
estimatePoseSingleMarkers needs K and dist, and the gimbal's deg/us is
unpinned for the same missing reason.

The board is propped and stationary while the GIMBAL does the waving. A
propped board cannot be motion-blurred and the sweep is repeatable, which
is strictly better than hand-holding. It hunts coarsely for the board
first, then sweeps a fine grid around wherever it found it.

Captures at 1296x972 rather than 640x480: corner localisation error is
roughly constant in pixels, so the wider frame is directly a more
accurate calibration, and marker decoding benefits later too.

Detection is findChessboardCornersSB with NO legacy fallback, deliberately
-- calibrate.py's lesson that the two detectors disagree on corner order
applies verbatim here.

Documented in the docstring: re-propping the board between sweeps is not
optional. Rotating the camera about its own centre with the board fixed
gives views related by a homography, which constrains intrinsics only
weakly. The sweep buys image-position coverage (pins distortion);
re-propping buys pose diversity (pins focal length and principal point).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 18:51:52 -07:00
calib aruco: object marker sheet, and arm-camera intrinsic calibration 2026-08-27 18:51:52 -07:00
gimbal aruco: object marker sheet, and arm-camera intrinsic calibration 2026-08-27 18:51:52 -07:00
.gitignore Initial commit: hardware survey, telemetry notes, calibration tooling 2026-08-26 21:43:39 -07:00
aruco-explainer.html Add gimbal servo driver and camera-feedback travel-limit finder 2026-08-27 17:14:37 -07:00
CLAUDE.md calib: ChArUco board and marker sheet generators 2026-08-27 18:25:55 -07:00
pi5-rail-telemetry.html Initial commit: hardware survey, telemetry notes, calibration tooling 2026-08-26 21:43:39 -07:00
README.md Initial commit: hardware survey, telemetry notes, calibration tooling 2026-08-26 21:43:39 -07:00

stereo-vision

Stereo depth perception for an indoor wheeled rover: two IMX296 global-shutter cameras on a 68 mm baseline, driven by a Raspberry Pi 5 with a Hailo-8 accelerator.

Status: hardware verified end to end; calibration tooling written and validated; no calibration has been produced yet. Producing one is the next step and blocks everything downstream — SGBM and every learned matcher alike need rectified pairs.

Layout

Path
CLAUDE.md hardware survey, rig geometry, prior art, and the traps. Read first.
calib/ calibration target generator, headless capture, and calibration + checks
pi5-rail-telemetry.html how to read the Pi 5 PMIC's rails and the rest of the board telemetry

Calibrating

python3 calib/make_target.py --page letter     # or --page a4
# print at 100%, glue FLAT to something rigid, then MEASURE it

ssh pi@freddie
cd ~/stereo-calib
python3 capture_pairs.py --out calib_data --pairs 24
python3 calibrate.py --in calib_data --span-x <measured> --span-y <measured>

Pass the board spans you physically measured, never the nominal values — print scaling goes straight into your depth scale. The two axes are taken separately because printers scale the carriage and paper-feed directions differently.

calibrate.py self-checks four things: reprojection error, rectified vertical residual, baseline against the known 68 mm, and left/right eye ordering via the sign of T[0]. make_synthetic.py generates a known-truth dataset so the pipeline can be validated without touching hardware.