🚧 Work in progress: this documentation is still being written and reviewed. Check details against the source code.

ZED camera backend

ZED backend using streamed RGB images and registered point clouds. It estimates marker centers but not marker orientation; unavailable depth can leave a returned marker at the default zero position.

beambot.vision.camera.zed · Source

Perception and mapping · Guide · Symbol index

How it is used

VisionEngine selects this backend for streamed ZED images and registered depth clouds. detect_markers() temporarily subscribes for frames, detects image-space ArUco corners, and samples depth at each marker center. create_client() exists only to match the camera interface.

The backend does not estimate marker orientation or synchronize image/cloud acquisition. Missing depth can leave a returned marker at the default zero position; this is not a full substitute for Zivid’s native 3D marker capture.

Start here

Callable	Purpose
detect_markers	Detect ArUco corners in the streamed image and look up the center pixel in the available point cloud.
create_client	Satisfy the shared backend interface without creating a capture service client.

Functions

detect_markers()

Detect ArUco corners in the streamed image and look up the center pixel in the available point cloud.

Function · Source

detect_markers(
    client,
    node: Node,
    marker_ids: list[int] | None = None,
    dictionary: str = 'aruco4x4_50',
    timeout: float = 10.0,
    settle_time: float = 0.0,
) -> DetectionResult

Parameter	Meaning
client	Unused compatibility argument; ZED uses subscriptions.
node	ROS node for capture, spinning, and logging.
marker_ids	Allowed integer IDs; None accepts all detected IDs. An empty list accepts none.
dictionary	aruco4x4_50, aruco5x5_100, or aruco6x6_250; unrecognized names fall back to 4x4_50.
timeout	Polling budget in seconds passed to _grab_latest().
settle_time	Unused compatibility argument; no settling sleep is applied.

Returns: DetectionResult with marker ID/Pose pairs and the image timestamp. No image returns an empty result with no timestamp. Every marker orientation is identity; absent/invalid depth can leave position at (0,0,0).

Errors: Missing images or markers return empty results and log messages. Conversion/OpenCV/point-cloud errors are not caught.

Side effects: Creates temporary image/cloud subscriptions through _grab_latest(), spins the node, and logs detections; does not move the arm.

create_client()

Satisfy the shared backend interface without creating a capture service client.

Function · Source

create_client(node: Node)

Parameter	Meaning
node	Unused ROS node argument retained for interface compatibility.

Returns: None.

Errors: No local failure path.

Internal callbacks and helpers

_grab_latest()

Subscribe temporarily and spin the node until an image and, when requested, a cloud arrive or the polling budget ends.

Internal helper · Source

_grab_latest(
    node: Node,
    timeout: float = 5.0,
    need_cloud: bool = True,
) -> tuple[Image | None, PointCloud2 | None]

Parameter	Meaning
node	ROS node used to create subscriptions and passed to rclpy.spin_once().
timeout	Seconds converted to int(timeout * 10) polling iterations of up to 0.1 seconds each.
need_cloud	Whether to subscribe to the registered cloud and wait for it as well as an image.

Returns: Tuple (Image or None, PointCloud2 or None). Image and cloud are received independently; timestamps are not synchronized here.

Errors: Missing messages remain None. Subscription/spin errors propagate; created subscriptions are destroyed in finally after polling.

Side effects: Creates temporary subscriptions on the ZED image/cloud topics and spins callbacks on the supplied node.

Referenced in source (1)

• detect_markers — beambot.vision.camera.zed

_grab_latest.on_image()

Store the latest message for the enclosing wait loop.

Callback · Source

on_image(msg: Image)

Parameter	Meaning
msg	Received ROS Image message.

Returns: None.

Errors: No local validation or exception handling.

Side effects: Replaces received_image[0]; no copy or timestamp matching is performed.

Referenced in source (1)

• _grab_latest — beambot.vision.camera.zed

_grab_latest.on_cloud()

Store the latest message for the enclosing wait loop.

Callback · Source

on_cloud(msg: PointCloud2)

Parameter	Meaning
msg	Received ROS PointCloud2 message.

Returns: None.

Errors: No local validation or exception handling.

Side effects: Replaces received_cloud[0]; no copy or timestamp matching is performed.

Referenced in source (1)

• _grab_latest — beambot.vision.camera.zed