Build Images

In order to run our experiences against our system we need a system build available as a container image hosted somewhere we can access.

Prerequisites

For your build image to be usable in ReSim, it needs to be hosted in an image registry that we are granted permission to access.

Uploading the image to a registry is documented below, but you should ensure you have followed Resim Data Access to set up an AWS ECR repository to work with ReSim.

Furthermore, you must have permission to push to that registry.

If you're using an IAM user, the easiest way to get the needed permissions is to attach the AWS-managed IAM policy called AmazonEC2ContainerRegistryPowerUser to the user. You can do this through the AWS browser UI (see "Adding permissions by attaching policies directly to the user") or using the CLI:

aws iam attach-user-policy --policy-arn "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser" --user-name <name of user>

If you're using an IAM role instead, it's a similar operation.

If you'd prefer to be stricter with your permissions, you really only need the permissions from the policy here rather than all of those in AmazonEC2ContainerRegistryPowerUser.

If there is a repository policy on the repository already, check it doesn't explicitly deny access by the IAM identity (user or role) you're using.

Entrypoint

The first step in creating a build image is to identify a shell command that you would like to run in order to test your software. If your software test involves multiple different processes, this is fine, but you may want to write a shell script or another piece of software to correctly initialize and run them. For instance, if you are using ros2 with gazebo, you might write a launch file for this and then your test command might be a call to a script containing something like:

#!/bin/bash

# run_sim_test.sh

# Source our ros2 distribution
source /opt/ros/<distro>/setup.bash

# Navigate to, build, and source the ros2 package we're building
cd <workspace-path>
colcon build
source install/setup.bash

# Launch our test sim
ros2 launch my_package launch_test_sim.py

But really, any script which your docker container can execute will suffice. Once you have identified such a command, your goal is to create a Dockerfile which builds an image that has everything it needs to run your test command. Then, that command should be denoted as the ENTRYPOINT of that Dockerfile.

Inputs and Outputs

It is typically desirable to run the same Build with a variety of different Experiences (sets of input files). Therefore, the ReSim worker will mount a volume containing the inputs for each run at /tmp/resim/inputs/. Using the ros2 example above, the launch_test_sim.py launch file might reference ROS2 Parameter YAML files stored in /tmp/resim/inputs/. You would then put these YAML files in an Amazon S3 bucket and add them as experiences as described here. Again, there's nothing special about using ros2 in this case and any file stored correctly in the experiences bucket will be available there.

The ReSim platform additionally places a file called test_config.json in the /tmp/resim/ directory. This JSON contains the following fields:

{
  "experienceID": "the unique id of the experience",
  "experienceName": "the name of the experience",
  "experienceLocation": "the location of the experience",
  "branchName": "the name of the branch",
  "buildVersion": "the version of the build running",
  "buildID": "the unique id of the build running",
  "systemName": "the name of the system the build running belongs to",
  "systemID": "the unique id of the system"
}

This config file can be helpful for determining the experience location, if the experience is not stored in S3 but within the build image, by reading /tmp/resim/test_config.json

When the test command is run, it will produce some sort of valuable outputs that can be accessed through the ReSim app. Any such outputs (e.g. console logs or rosbags) should be stored in the /tmp/resim/outputs/ directory. If this is done correctly, you will be able to download them using a link in the ReSim app.

Developing an Image

For an example of a very simple build image please refer to this directory which contains directions and example scripts to create and test run a build image. In this case, we use ReSim's open source simulator to run a simple scripted simulation of two quadcopter drones.

The example uses a build script to build our test executable and then uses Docker's COPY command in its Dockerfile to move the resulting executable into the finished image. The general rule is that the Docker image must contain everything your test needs to run except for the the inputs (i.e. the experience).

In order to develop a working test image, a good workflow is generally to iterate on it locally. This involves the following steps:

1. Inputs and Outputs

Create local inputs/ and outputs/ folders with the inputs/ folder containing a set of representative input files. If you are doing this from within a Docker container, creating volumes and copying content to them is probably better. Here's how you would do that:

# Create a helper container to copy content over
docker container create \
    --name tmp \
    --volume test_inputs:/inputs \
    busybox

# Copy test data to the volume
docker cp path/to/test/inputs tmp:/inputs

# Remove the helper container
docker rm tmp

2. Build

Assuming your Dockerfile is in a folder called runner/, build your Docker image using:

docker build runner/ -t test-runner-image:latest

Note that the name test-runner-image here is arbitrary. You can use whatever name you would like as long as you're consistent when developing because the image will be re-tagged with docker tag before it's pushed.

3. Run

Try out a run with your Docker image using:

docker run \
  --volume /absolute/path/to/inputs:/tmp/resim/inputs \
  --volume /absolute/path/to/outputs:/tmp/resim/outputs \
  test-runner-image:latest

The --volume flags here ensure that the inputs and outputs are available to the container at /tmp/resim/inputs and /tmp/resim/outputs. Note that /absolute/path/to/inputs and /absolute/path/to/outputs are paths on the host system. If you are iterating within a Docker container, you would use the Docker volumes approach mentioned in step 1. In this case, you would instead run:

docker run \
  --volume test_inputs:/tmp/resim/inputs \
  --volume test_outputs:/tmp/resim/outputs \
  test-runner-image:latest

And getting your outputs back would look like:

# Create a helper container to copy content back:
docker container create \
    --name tmp \
    --volume test_outputs:/outputs \
    busybox

# Copy outputs from the volume
docker cp tmp:/outputs .

# Remove the helper container
docker rm tmp

4. Iterate

If any step is unsuccessful, make modifications and return to step 2.

Pushing

Once the build image is created and working, the next step is to push it to your AWS Elastic Container Registry (set up according to ReSim Data Access) like so:

docker tag test-runner-image <AWS account number>.dkr.ecr.<AWS region>.amazonaws.com/test-runner-image:<commit-sha>
aws ecr get-login-password --region <AWS region> | docker login --username AWS --password-stdin <AWS account number>.dkr.ecr.<AWS region>.amazonaws.com
docker push <AWS account number>.dkr.ecr.<AWS region>.amazonaws.com/<ecr-name>:<build ID>

Note the use of a commit SHA here. This is what we recommend as a unique ID associated with your software version if you're using git for version control. You can get a short form of the current commit hash by running:

git rev-parse --short HEAD

On a successful push, the command will report something like:

The push refers to repository [<AWS account number>.dkr.ecr.<AWS region>.amazonaws.com/<ecr-name>]
a9950558cded: Pushed
29563d2ab359: Pushed
aa5d6a7236b9: Pushed
...
bce45ce613d3: Pushed
latest: digest:
sha256:aa47e2d0c6dcc5d48d3c9b8c9c8a57433b2d241c27ef92aa53de885683f06487 size: 7480

If you instead get something like the following, it is an indicator either that the ECR repository URL (account number or repository name) you are trying to push to is incorrect, or that you don't have the required permissions to push to it. Check the Prerequisites for additional info.

The push refers to repository [<AWS account number>.dkr.ecr.<AWS region>.amazonaws.com/<ecr-name>]
a9950558cded: Retrying in 7 seconds
29563d2ab359: Retrying in 7 seconds
aa5d6a7236b9: Retrying in 7 seconds
...
bce45ce613d3: Waiting
EOF

You might want to automate these build and push steps by adding them to a new or existing CI pipeline. See our articles on Pushing Build Images from CI for how.

External links

AWS ECR authentication documentation

Creating a Project and a System

Every build is associated with a project and a system. If you don't already have a project, you can create one with the CLI:

resim projects create \
    --name "my-project" \
    --description "My demo project"

For convenience, select that project:

resim projects select "my-project"

If you do not already have a System, it can also be created. A system will also specify the resource requirements needed to run a build:

resim systems create --project "my-project" --name "my-system" \
    --description "A system, which we want to create builds for" \
    --build-vcpus 4 --build-memory-mib 16384 --build-gpus 0

Creating a Branch

Each build is also associated with a branch. The branch represents the particular change you're testing. A build can be thought of as one implementation of that change. A given branch may have many builds associated with it.

You can manually create a branch with the CLI:

resim branches create \
    --name "my-branch" \
    --type CHANGE_REQUEST

It's often easier to use the --auto-create-branch flag when creating a build which is the approach taken below.

The ReSim app supports three types of branch, for three common git workflows:

CHANGE_REQUEST: a pull or merge request, a short-lived branch
MAIN: a long-lived, protected branch that most change requests will land to
RELEASE: a release of a piece of software

The --auto-create-branch flag will create a CHANGE_REQUEST branch, but if the branch is named main it will make it a MAIN branch.

Creating a Build

Now that the build image has been pushed to your elastic container registry, the final step is to formally create the build by informing the ReSim app as to its location. This is done using the ReSim CLI. We assume that we already have a project called my_project and a branch called my_branch. These are required for build creation, so if the project or branch you want to use does not yet exist, see above. After authenticating we can create a new build like so:

resim builds create \
    --image "<AWS ID>.dkr.ecr.<region>.amazonaws.com/<ecr-name>:<commit-sha>" \
    --version "<commit-sha>" \
    --description "My build which runs a sim!" \
    --branch "my-branch" \
    --project "my-project" \
    --system "my-system" \
    --auto-create-branch

The --project, --branch, and --system flags will accept either a name or an ID. When using --auto-create-branch, one must also pass a branch name using --branch. If you have selected a project via the CLI, you can omit the --project for convenience.

This command should return a message containing the UUID of the created build, and it should now be browsable through the ReSim app.

Note that the image itself requires the full tagged URI, since we allow an arbitrary version to be stored that need not reflect the image tag.