🚧 Work in progress: this documentation is still being written and reviewed. Check details against the source code.

Zivid camera backend

Zivid triggered capture and native marker/sample-ROI detection. The helpers rely on an already running executor to deliver service futures and image callbacks; their waits do not spin the node. Pre-capture timestamps approximate acquisition timing for later TF lookup.

beambot.vision.camera.zivid · Source

Perception and mapping · Guide · Symbol index

How it is used

VisionEngine uses this backend for triggered native 3D marker detection and marker-relative sample pickup. capture_2d() also supplies BGR images to the spincoater detectors. Choose the entry point for the required output: image, marker poses, or sample pickup pose.

The owning node needs an active executor while these functions wait for service replies and image callbacks. Sample-ROI positions are projected onto the marker plane with configured thickness; they are not measured sample-depth points.

Start here

Callable	Purpose
create_client	Create the native Zivid marker-capture client.
detect_markers	Request native 3D marker detection and attach a pre-capture timestamp to the returned poses.
detect_sample_roi	Capture a native 3D anchor marker plus a color image, segment a nearby sample, and project its pickup pixel onto the marker plane.
capture_2d	Trigger /capture_2d, then return a newly received color image as a BGR array.

Functions

create_client()

Create the native Zivid marker-capture client.

Function · Source

create_client(node: Node)

Parameter	Meaning
node	ROS node that owns the service client.

Returns: A CaptureAndDetectMarkers client for /capture_and_detect_markers.

Errors: Client creation errors propagate.

Side effects: Creates a ROS client; this function does not wait for service readiness or trigger capture.

detect_markers()

Request native 3D marker detection and attach a pre-capture timestamp to the returned poses.

Function · Source

detect_markers(
    client,
    node: Node,
    marker_ids: list[int] | None = None,
    dictionary: str = 'aruco4x4_50',
    timeout: float = 45.0,
    settle_time: float = 0.0,
) -> DetectionResult

Parameter	Meaning
client	Existing CaptureAndDetectMarkers client, normally from create_client().
node	ROS node used for its clock and logger; another executor must complete the future.
marker_ids	IDs to request and retain. None expands to the configured dictionary’s supported range; an explicit empty list is passed through.
dictionary	Zivid marker dictionary name. It also selects the ID range used when marker_ids is None; unknown names use a 50-ID fallback range.
timeout	Total deadline in seconds across readiness, settling, and service completion.
settle_time	Optional pre-capture sleep in seconds, limited by the remaining deadline.

Returns: DetectionResult containing requested ID/Pose pairs and the timestamp sampled just before capture. Service failures return an empty result with capture_stamp=None; successful detection with no matches retains the pre-capture timestamp.

Errors: Readiness failures, service timeouts/exceptions, and success=false are logged and return an empty result. No future cancellation occurs on timeout.

Side effects: Calls a physical camera service, optionally sleeps, and logs measured positions.

detect_sample_roi()

Capture a native 3D anchor marker plus a color image, segment a nearby sample, and project its pickup pixel onto the marker plane.

Function · Source

detect_sample_roi(
    node: Node,
    tag_id: int,
    strategy: str = 'farthest_edge',
    edge_inset_mm: float = 0.0,
    dictionary: str = 'aruco4x4_50',
    timeout: float = 45.0,
    params: SampleRoiDetectionParams | None = None,
) -> tuple[Pose, TimeMsg] | None

Parameter	Meaning
node	ROS node whose executor can deliver image and service callbacks.
tag_id	ArUco anchor ID requested from the native marker service.
strategy	Sample pickup strategy forwarded to detect_sample_in_roi().
edge_inset_mm	Requested inset from the selected sample edge/corner, in millimetres.
dictionary	Native marker dictionary name sent to Zivid.
timeout	Total deadline in seconds; the post-service image wait is also limited to 20 seconds.
params	SampleRoiDetectionParams controlling marker size, segmentation, ROI, and sample thickness; None creates defaults.

Returns: (camera-frame Pose, pre-capture Time) or None. The returned position includes sample thickness; orientation is identity.

Errors: Service failures, missing markers/images, invalid marker-corner geometry, absent samples, and failed plane projection return None with logs. Image/algorithm exceptions can propagate; temporary subscriptions and clients are always cleaned up after creation.

Side effects: Triggers camera capture, receives an image, waits for subscription matching, discards pre-trigger frames, and logs results. It does not command robot motion.

capture_2d()

Trigger /capture_2d, then return a newly received color image as a BGR array.

Function · Source

capture_2d(node: Node, timeout: float=15.0) -> np.ndarray | None

Parameter	Meaning
node	ROS node with an executor able to service the reentrant subscription/client callbacks.
timeout	Total nonnegative deadline in seconds covering service readiness, connection settling, service completion, and image wait.

Returns: BGR NumPy image, or None for readiness, service, response, or image timeout/failure.

Errors: Service-call exceptions are caught and logged. Image conversion errors propagate. The temporary subscription and client are destroyed in finally.

Side effects: Creates temporary ROS entities, waits up to 0.5 seconds for subscription matching, discards an older received frame, and triggers physical camera capture without the 3D projector.

Internal callbacks and helpers

_wait_for_future()

Poll an externally serviced future until it finishes or the monotonic deadline expires.

Internal helper · Source

_wait_for_future(future, timeout: float, poll_interval: float=0.01) -> bool

Parameter	Meaning
future	Object exposing done(); another executor/thread must complete it.
timeout	Maximum wait in seconds, added to time.monotonic().
poll_interval	Sleep duration in seconds between completion checks.

Returns: True if the future is done, including an already completed future; False if it is still pending at the deadline. It does not inspect success or retrieve the result.

Errors: Exceptions from done() or sleep propagate; invalid negative sleep intervals are not validated.

Side effects: Blocks the current thread; does not spin ROS or cancel a timed-out future.

Referenced in source (3)

• capture_2d — beambot.vision.camera.zivid
• detect_markers — beambot.vision.camera.zivid
• detect_sample_roi — beambot.vision.camera.zivid

capture_2d.on_image()

Retain the most recently delivered image for the enclosing capture helper.

Callback · Source

on_image(msg: Image)

Parameter	Meaning
msg	ROS Image from /color/image_color.

Returns: None.

Errors: No local validation.

Side effects: Replaces received[0]; the outer helper clears that slot before triggering capture.

Referenced in source (1)

• capture_2d — beambot.vision.camera.zivid

detect_sample_roi.on_image()

Store an image for sample detection after the marker capture.

Callback · Source

on_image(msg: Image)

Parameter	Meaning
msg	ROS Image received on /color/image_color.

Returns: None.

Errors: No local validation.

Side effects: Replaces received_image[0]; the outer workflow clears pre-capture contents.

Referenced in source (1)

• detect_sample_roi — beambot.vision.camera.zivid