Skip to main content

TraceHouse Agent Overview

The TraceHouse™ agent (robot-agent) is a Rust binary that runs on the robot's onboard computer and streams telemetry into TraceHouse. It is the carrier: it receives distributed-trace spans from the RobotOps Trace SDK over a local socket and forwards them, and independently collects logs, metrics, TF, and MCAP. It is designed to be low-overhead, resilient, and safe to run on production robots.

What it collects​

CategoryDetails
Distributed tracesOTLP spans received from the RobotOps Trace SDK over a local Unix socket (/run/robotops/trace.sock) and forwarded to TraceHouse
ROS2 topicsAutomatically discovers and subscribes to all topics using configurable capture strategies
ROS2 logsCaptures the /rosout topic for all ROS2 node logs
System logsIntegrates with journald for host-level log collection
System metricsCPU, memory, disk, network quality (latency, jitter, packet loss), temperature, battery, and process list
TF transformsTransform tree snapshots with deduplication (~99% storage savings for stationary robots)
MCAPRecords telemetry to MCAP for offline buffering and replay

:::note Trace source: the SDK, not the RMW shim Traces now come from the SDK running inside your nodes, not from the deprecated passive rmw_robotops RMW shim. The agent's log / metric / TF / MCAP collection is unchanged — only the trace ingestion path moved to OTLP over the local socket. :::

Architecture​

The agent is a standalone Rust binary installed via Debian package and managed by systemd. It:

  1. Sources the ROS2 environment at startup to discover the running ROS2 graph
  2. Subscribes to all topics using the configured capture strategy (adaptive, rate-limited, sampled, etc.)
  3. Buffers data locally in a 500 MB FIFO buffer at /var/tmp/robot_agent/buffer for offline resilience
  4. Streams to the backend over gRPC when connectivity is available
  5. Enters offline mode automatically if the API key is missing or the backend is unreachable — data is preserved and uploaded when the connection is restored

Production readiness​

The agent is built for production robot deployments:

  • Offline mode: No data loss during network outages — the FIFO buffer holds up to 500 MB (configurable) and uploads automatically when connectivity is restored
  • Exponential backoff: Retries failed backend connections with jitter to avoid thundering-herd
  • Heartbeat monitoring: Periodic pings detect API key revocations and backend unavailability at runtime
  • Minimal footprint: Rust binary with no runtime allocations in hot paths; TF deduplication prevents storage runaway on stationary robots
  • Layered configuration: Embedded defaults → YAML file → environment variables (highest priority), making it container and Kubernetes-native

Cross-platform​

Supports amd64 and arm64 architectures. Runs on any Ubuntu 24.04 Noble robot regardless of whether it's an x86 server, a Raspberry Pi 5, a Jetson, or a custom arm64 board.

Installation​

See the Quickstart for the one-line install command, or System Requirements for architecture and OS details.

What's next​