🚧 Work in progress: this documentation is still being written and reviewed. Check details against the source code.

MCP and the direct agent

EROBS exposes robot tools to language-model clients through Model Context Protocol (MCP). A generic ROS server handles ROS communication; a custom beamline server adds EROBS tools. The language model chooses tools; the ROS action servers and device stack perform the resulting work.

The two MCP servers

Server	Responsibility	Location
Generic ROS MCP server	ROS communication, including action-goal dispatch	External process declared in .mcp.json
Custom beambot server	Robot/pose inspection, perception, task-building helpers and beamline-specific operations	src/beambot/mcp_server/beambot_mcp_server.py

The custom server is not the sole task executor. For example, a helper can construct task JSON that is then submitted through the generic server’s action interface. Other tools capture images or change saved data directly. The MCP reference lists each tool’s inputs and effects.

.mcp.json describes local processes and arguments. The generic server runs ros-mcp==3.1.0 through isolated uvx, with fastmcp<4 and ROS_DEFAULT_TIMEOUT=60. The custom server resolves the checkout with git rev-parse --show-toplevel, sources Jazzy and that checkout’s installation overlay, then starts its source script with python3. Start the client from inside the intended checkout and export BEAMBOT_BEAMLINE_CONFIG for that deployment.

RobotAgent searches for .mcp.json from the current directory upward (at most ten levels).

The custom server lives outside the installed beambot Python package directory. CMake does not install mcp_server/, so the documented MCP process layout requires a source checkout.

The startup wrappers add rosbridge to the ROS stack. utils/start_mcp.sh also records a bag; utils/start_mcp_nobag.sh omits that recorder. Those wrappers start processes, while an MCP client’s configuration starts its MCP servers. Keep the distinction when diagnosing “server unavailable.”

The direct RobotAgent

beambot.agent supplies a direct API-to-MCP loop reused by the CLI and GUI chat. The source entry point is python -m beambot.agent; __main__.py supports an interactive conversation or a one-shot request passed as command-line text.

The agent uses an Anthropic-compatible client to connect to the NSLS-II Hermes gateway. It reads:

Setting	Meaning
AIFAPIM_API_KEY	Required API credential for the gateway
HERMES_ENDPOINT	Gateway base URL; defaults to the configured Hermes endpoint
BEAMBOT_MODEL	Model identifier override; the source supplies a default

Set credentials in the execution environment and keep them out of source control.

For each configured server, the agent starts the stdio process, initializes an MCP session, lists its tools and converts their schemas into API tools.

If a server connection fails, the agent logs the failure and continues to the others. Startup can therefore succeed with only some of the expected tools available. Check the available tools before attributing a missing capability to the model.

robot_operation.md is loaded next to robot_agent.py whenever a RobotAgent is constructed. It is shared runtime input for the CLI, GUI chat and repository operation skill. Do not relocate it to the documentation site or replace it with a web link. The reviewed CMake file installs the Python package with ament_python_install_package but has no explicit installation rule for this Markdown file. Verify its presence when using an installed package. setup.sh imports ROS repositories and runs rosdep; it does not explicitly install the agent’s anthropic and mcp Python dependencies. Install these Python dependencies separately before starting the agent.

GUI Plan and Run modes

The GUI composes the same agent with local tools and a tool filter. agent_modes.py defines the behavior, AgentBridge connects the async agent to Qt signals, and the main window owns the task queue and execution path.

Mode	Queue behavior
Plan	The agent can propose or clear tasks. The human reviews the queue and starts execution.
Run	The agent can also call the local execute_queue tool.

Both modes filter generic ROS motion-dispatch tools so queue execution follows the GUI’s existing path.

Plan mode can still capture camera images. Both modes are experimental; their tool filters are not hardware safety interlocks.