CI

Introduction

This document is intended to help you build and push build images to an image registry from a CI service so that ReSim can use the images to run tests.

Currently ReSim supports authenticating against, and pulling images from, an AWS ECR, Dockerhub or Google Artifact Registry repository. If you use a different image registry provider and would like ReSim to integrate with it, please get in touch.

The example workflows provided are for GitLab and GitHub but almost any CI service should be able to push to ECR. If you have a suggested service to add to this list, or that you would like help configuring to push build images, please reach out to ReSim.

Note

If you already have well-established processes for pushing images build in a CI/CD environment, this guide is not required. Instead, follow the Next Steps described below.

Prerequisites

• We recommend (although it's not mandatory) that you have built, pushed and tested an image from a local machine before setting up a CI pipeline to do so. See Build Images for guidance.

• You need to have set up an image repository that ReSim can access, as described in Resim Data Access. You will need the repository URI or: your AWS account number, the AWS region in which the account was created, and the repository name.

• Your ReSim contact should have provided you with a client ID and client secret. These are the credentials your CI service will use to authenticate with ReSim.

• You will need a project in the ReSim app with which to associate your builds. See Projects.

Creating an IAM user with AWS ECR Permissions

Your CI pipeline will need to authenticate with AWS to push images. A thorough exploration of AWS configuration is outside the scope of this article, but as a starting point if you do not have existing AWS identities that your CI service uses, you can create an IAM user with the required permissions.

You will need to create a user, create security credentials for the user, and assign the AmazonEC2ContainerRegistryPowerUser managed IAM policy to the user.

If you have the AWS CLI installed and configured, we recommend using it rather than the browser UI for this task. See the Command line interface section below.

If you don't have the AWS CLI set up, you can create the user in the AWS browser interface.

Browser interface (AWS Console)

Following along with the AWS documentation as required, you need to:

1. Create a user, named ci-user or any other name you choose. You do not need to enable console access for this user.
2. Create credentials in the form of an access key and secret access key. Store these locally in a password manager or other secure location, they are equivalent to a username and password for this CI user.
3. Assign permissions to the user so it can push to ECR. Following the instructions to "add permissions by attaching policies directly to the user", assign the AmazonEC2ContainerRegistryPowerUser IAM policy. This policy is managed by AWS.

Command line interface

Create a new user:

$ aws iam create-user --user-name ci-user
{
  "User": {
      "Path": "/",
      "UserName": "ci-user",
      "UserId": "AIDAWRXCWKBZSLQOSKX4V",
      "Arn": "arn:aws:iam::<AWS account number>:user/ci-user",
      "CreateDate": "2023-08-10T18:47:20+00:00"
  }
}

Create an access key pair:

Note

Store the access key and secret access key from the following command somewhere secure - they are static credentials equivalent to a username and password. We will add them to your CI service's secrets configuration later.

$ aws iam create-access-key --user-name ci-user
{
  "AccessKey": {
      "UserName": "ci-user",
      "AccessKeyId": "AKIAWRXCWKBZVEXAMPLE",
      "Status": "Active",
      "SecretAccessKey": "kKEZbvjb6UV/jEFPEED5sdcSiJUGeezDuEXAMPLE",
      "CreateDate": "2023-08-10T18:47:53+00:00"
  }
}

Attach a policy to the new user which will allow it to push to ECR repositories.

$ aws iam attach-user-policy --policy-arn "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser" --user-name ci-user

Testing permissions

Once you have created a user, generated credentials for it, and assigned permissions to it following one of the above methods, you may wish to test the configuration locally before configuring your CI service.

To log in to your ECR repository as the CI user:

AWS_ACCESS_KEY_ID=<CI user access key> AWS_SECRET_ACCESS_KEY=<CI user secret key> aws ecr get-login-password --region <AWS region> | docker login --username AWS --password-stdin <AWS account number>.dkr.ecr.<AWS region>.amazonaws.com

Then pull and tag an example image and push it to your repository:

docker pull public.ecr.aws/docker/library/hello-world
docker tag public.ecr.aws/docker/library/hello-world <AWS account number>.dkr.ecr.<AWS region>.amazonaws.com/<repository name>:hello-world
docker push <AWS account number>.dkr.ecr.<AWS region>.amazonaws.com/<repository name>:hello-world

Try pulling the image:

Note

This will not pull any data because the image already exists locally, but it will confirm that the image exists in the remote repository.

docker pull <AWS account number>.dkr.ecr.<AWS region>.amazonaws.com/<repository name>:hello-world

Log out from ECR in your local environment:

docker logout <AWS account number>.dkr.ecr.<AWS region>.amazonaws.com

You have now confirmed that the CI user you've created can manage images in your ECR repository. (For more information about working with ECR images, see https://docs.aws.amazon.com/AmazonECR/latest/userguide/images.html)

Next Steps

The next step is to configure your CI system to build and push images and run tests in ReSim:

• GitHub
• GitLab