Skip to content

Ivan/feat/cameracalibrator go2#2123

Merged
leshy merged 71 commits into
mainfrom
ivan/feat/cameracalibrator_go2
May 19, 2026
Merged

Ivan/feat/cameracalibrator go2#2123
leshy merged 71 commits into
mainfrom
ivan/feat/cameracalibrator_go2

Conversation

@leshy
Copy link
Copy Markdown
Contributor

@leshy leshy commented May 17, 2026

builds upon april tag camera calibrator (#2107) makes it usable with go2

  • Pubsub topic string registry - very quickly vibed, we need string -> Pubsub subscribable topic function for specifying topics via cli, config etc, will iterate on this, let us ignore for now
  • Live topic source. --source topic --topic <proto>:<channel> subscribes to a running pubsub stream
  • Fisheye distortion model. --distortion-model {plumb_bob,fisheye} (default plumb_bob for back-compat)
  • New Go2 front camera calibration. dimos/robot/unitree/go2/front_camera_720.yaml, fit with the new fisheye path and consumed by GO2Connection via CameraInfo.from_yaml. Moves out of params/ and into the go2 package directory so it ships as a packaged resource
  • Removes references to base_link from marker detector module - it just needs to know about world -> optical_frame
  • Passes emit_every arg to global mapper reduces speed to 1-2hz for global map just to chill a bit
  • Switches go2 sensor data to system timestamps having annoying issues aligning frames (image from webrtc with sensor data from go2, go2 onboard timestamps are also very flakey depending on a firmware)
  • Fixes dimos topic echo - was broken for a while
uv run dimos cameracalibrate --source topic --topic "jpeg_lcm:/color_image" --cols 9 --rows 6 --square-size-m 0.02 --out ./camera_info.yaml
2026-05-17_14-24

leshy and others added 30 commits May 9, 2026 22:01
Switch from mypy-ignore to types-reportlab>=4.5.0 (matches reportlab 4.5
in deps), matching the project's pattern for the other ~15 types-* packages.
The stubs immediately caught a real bug — Canvas.setKeywords expects str |
None, not list[str].
Add a top-level `pytest.importorskip("cv2.aruco")` if not already present so CI without the extra skips, not errors.
Comment thread dimos/robot/unitree/connection.py Outdated
Comment thread dimos/perception/fiducial/marker_tf_module.py
Comment thread dimos/perception/fiducial/marker_tf_module.py
Comment thread dimos/protocol/tf/tf.py Outdated
Comment thread dimos/utils/cli/cameracalibrate/cameracalibrate.py
Comment thread dimos/protocol/tf/tf.py
- tf.get warning block was indexing args[0]/args[1] unconditionally and
  crashed with IndexError when callers used keyword arguments. Fall back to
  kwargs lookup so the warning fires correctly either way.
- test_make_pubsub_transport_jpeg_shm_uses_JpegShmTransport: the Python
  turbojpeg package imports fine without the native libturbojpeg.so, so
  importorskip didn't help. Probe TurboJPEG() and skip on RuntimeError.
@dimensionalOS dimensionalOS deleted a comment from greptile-apps Bot May 18, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 18, 2026

Greptile Summary

This PR extends the camera calibrator to support a live pubsub topic source (--source topic) and a fisheye/equidistant distortion model, then consumes both in a fresh Go2 front-camera calibration file. It also simplifies MarkerTfModule to do a direct world → optical TF lookup, switches Go2 sensor data to system timestamps, and fixes the dimos topic echo command.

  • Fisheye calibration pipeline: adds DistortionModel.fisheye that routes through cv2.fisheye.calibrate, writes ROS equidistant to YAML, and undistorts corners in estimate_marker_pose before calling solvePnP — the analytical SOLVEPNP_IPPE_SQUARE solver is preserved for both paths.
  • Topic frame source: _capture_frames_from_topic subscribes via the new pubsub/registry.py, feeds frames into the shared _interactive_capture loop, and tears down the transport in a finally block.
  • Go2 plumbing: GO2Connection now loads front_camera_720.yaml (equidistant, 4 coefficients) via importlib.resources.as_file, and odometry/lidar/video streams stamp frames with time.time() to improve cross-stream alignment.

Confidence Score: 4/5

Safe to merge with awareness that the duplicate-frame-on-SPACE bug in the topic source remains unaddressed and will silently inflate calibration view counts.

The fisheye calibration path, direct world→optical TF lookup, and Go2 YAML resource loading all look correct. The outstanding concern is that _capture_frames_from_topic._next() never clears latest_frame[0] after returning it, so rapid SPACE presses before a new frame arrives can silently add the same frame to accepted multiple times.

dimos/utils/cli/cameracalibrate/cameracalibrate.py (_next() frame-clearing logic), dimos/protocol/tf/tf.py (warning log level)

Important Files Changed

Filename Overview
dimos/utils/cli/cameracalibrate/cameracalibrate.py Major refactor: adds DistortionModel enum, fisheye calibration path via cv2.fisheye.calibrate, topic frame source with threading, and shared _interactive_capture loop. Duplicate-frame-on-SPACE issue for topic source (latest_frame not cleared) remains from previous review.
dimos/perception/fiducial/marker_tf_module.py base_link hop removed in favour of direct world→optical lookup; fisheye undistortion path added with proper ≥4-coeff guard and cv2.fisheye.undistortPoints before solvePnP; SOLVEPNP_IPPE_SQUARE preserved for both paths.
dimos/protocol/tf/tf.py get() signature made explicit (fixes previous IndexError); identity transform short-circuit added for same-frame lookups; logger.warning() on every failed lookup may be too noisy at frame rates.
dimos/protocol/pubsub/registry.py New pubsub URI registry mapping proto:topic strings to transport instances; well-tested.
dimos/robot/unitree/go2/connection.py Replaces hard-coded pinhole intrinsics with CameraInfo.from_yaml; correctly uses importlib.resources.as_file() for zip-safe resource access.
dimos/robot/cli/topic.py Fixes topic echo by importing the message submodule directly rather than the parent package.
dimos/robot/unitree/go2/front_camera_720.yaml New Go2 front camera calibration with equidistant distortion model, 4 Kannala-Brandt coefficients, camera_name set to go2_front_camera_720p.

Reviews (14): Last reviewed commit: "Merge branch 'main' into ivan/feat/camer..." | Re-trigger Greptile

Comment thread dimos/robot/unitree/connection.py Outdated
Comment thread dimos/robot/unitree/go2/blueprints/smart/unitree_go2.py
Comment thread dimos/protocol/tf/tf.py Outdated
Comment thread dimos/robot/unitree/connection.py
paul-nechifor
paul-nechifor previously approved these changes May 19, 2026
@leshy leshy enabled auto-merge (squash) May 19, 2026 05:28
@leshy leshy merged commit c66009e into main May 19, 2026
20 checks passed
@leshy leshy deleted the ivan/feat/cameracalibrator_go2 branch May 19, 2026 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants