Skip to content

ReSim Agent

Introduction

An important part of testing embodied AI is hardware-in-the-loop testing or HiL. This is testing that involves communicating with real hardware. HiL testing complements simulation-based testing and is typically run less frequently such as when changes are merged to a default branch, or before a release.

The ReSim Agent is designed to facilitate HiL testing by integrating with your existing suite of tests, allowing HiL tests to be triggered and the results to be analyzed using the same tools as simulation-based testing.

The ReSim Agent works by polling ReSim for tasks so there is no need to open ports or adjust firewall rules to allow inbound traffic.

Prerequisites

The ReSim Agent is currently tested against Linux on amd64 and requires Docker (version 27 or later).

Note

The user running the Agent needs permission to use the Docker API. In some environments this means adding the user to the docker group.

If you can do docker run hello-world you're ready to go.

Installation

Download the latest release from our repository at github.com/resim-ai/agent

Note that the Agent is open-source so you can inspect the code before you run it.

You can also install the Agent directly using Go:

go install github.com/resim-ai/agent

Configuration

The Agent loads configuration from ~/resim/config.yaml.

The minimal configuration you need is as follows:

name: robot-arm-1
pool-labels:
  - arm
username:
password:

The name of the agent will be used to identify which agent instance ran a particular job (although uniqueness is not currently enforced.)

Setting pool-labels as above will configure the agent to request jobs from ReSim that have been put into the pool called arm. This is a way to group jobs, for example based on the hardware they require. This value is a list, and the agent will receive jobs with any of the configured labels.

Info

Pool labels that begin with the prefix resim are reserved, so please ensure your pool labels do not have this prefix.

A username and password for use by your agents will be provided by ReSim.

Run the binary and it will start polling ReSim for work.

By default, the agent logs to stdout and to ~/resim/agent.log

Network Mode

By default, your containers will be run as part of the default bridge network, which matches the Docker default.

From this network, containers can reach services on the same network (or the internet), but not contact anything else running on the host.

If you need to communicate with services running on the same host, set the following option in the agent configuration file:

docker-network-mode: host

This is equivalent to docker run --net=host.

Privileged Mode

By default, your containers will not be run with elevated privileges, which matches the Docker default.

If you would like your containers to be run with elevated privileges, set the following option in the agent configuration file:

privileged: true

This is equivalent to docker run --privileged.

Other Options

See the repo README file for a full list of configuration options.

Build Images

The ReSim Agent uses the Docker daemon on the host to pull and launch images.

If the image repository requires authentication, make sure the host is authenticated before running the agent by running docker login.

For automatic authentication, the agent supports the AWS ECR credential helper.

To enable its use, ensure credentials/configuration are available in ~/.aws and that ~/.docker/config.json is configured as required.

Launching Batches

You can now launch batches to be run by the agent, by specifying a matching pool label:

resim batch create --project robot-arm --build-id 4b50f4f9-3b86-49a6-84b2-7813fd4bbcbe --experience-tags hardware-validation --pool-labels arm

Note that when creating a batch, the pool labels are used as an AND, that is, an agent will only run the batch if it is running with all of the labels on the batch.

This is to enable selecting agents with certain capabilities, without having to create unique labels for each permutation.

The status and results of the batch will be presented in ReSim as normal. In the near future, ReSim will highlight when a batch was run on an agent.

Designing Experiences

The workflow we anticipate is that the agent is running on the hardware to be tested, or that the hardware is accessible over the network from the container that will run your experience. The experience container will send commands to the hardware and receive data back which it will process as required and store in preparation for the metrics step.

Experience input data is currently expected to be built into the image, or accessible via S3.

Running in the Background

Here is an example systemd unit file for running the agent as a systemd service:

[Unit]
Description=ReSim Agent
After=network-online.target

[Service]
ExecStart=/usr/local/bin/agent # Or other path as required
Restart=always
User=ubuntu # Replace with the user that will be running the agent

[Install]
WantedBy=multi-user.target