GitHub

In this example, you have a GitHub repository which contains the application you would like to test.

Note that you will need a ReSim project. You can create a project with the ReSim CLI.

This example uses our GitHub action.

Prerequisites

Referring to GitHub's documentation about setting repository secrets as required, navigate to your repository's Settings and go to the "Secrets and variables" page in the Security section.

Add the following secrets:

• AWS_ACCESS_KEY_ID - set the value of this to the access key generated when you set up your ECR repository for use with ReSim
• AWS_SECRET_ACCESS_KEY - set the value of this to the secret access key generated when you set up your ECR repository for use with ReSim
• SLACK_WEBHOOK - set this to a Slack webhook URL (see the Slack documentation), if you want nightly summaries

Also add one of these sets of secrets. Your ReSim contact will provide you with either a client ID and secret, or a username and password:

• RESIM_CLIENT_ID and RESIM_CLIENT_SECRET, or
• RESIM_USERNAME and RESIM_PASSWORD

Add a variable containing your ECR registry URL:

• ECR_REGISTRY_URL - for example 123456789.dkr.ecr.us-east-1.amazonaws.com

Workflow

Now you need to create a GitHub workflow by adding a file to .github/workflows/ in your repository.

Below is an example of a workflow that builds a Docker image, pushes it to an ECR repository and launches a batch in ReSim to test it.

If you use this example, make sure to replace the following placeholder values:

• <your build command> - any commands that need to run before the Docker build, e.g. bazel build
• <your image name> - replace with the name of the ECR repository, e.g. the name of the image/application
• <your project name> - replace with the name of your ReSim project
• <your system name> - replace with the name of your ReSim system
• <your test suite> - the test suite you want to test against (see Test Suites)

name: build-and-test
on:
  pull_request:
  push:
    branches:
      - "main"
  schedule:
    - cron: '0 0 * * *'

jobs:
  build_and_test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Build
        run: <your build command>

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Log in to ECR
        uses: docker/login-action@v2
        with:
          registry: ${{ vars.ECR_REGISTRY_URL }}
          username: ${{ secrets.AWS_ACCESS_KEY_ID }}
          password: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

      - name: Prepare Docker metadata
        id: docker_meta
        uses: docker/metadata-action@v5
        with:
          images: |
            ${{ vars.ECR_REGISTRY_URL }}/<your image name>
          tags: |
            type=sha

      - name: Build and push image
        id: docker_build
        uses: docker/build-push-action@v5
        with:
          context: .
          file: ./Dockerfile
          push: true
          tags: ${{ steps.docker_meta.outputs.tags }}

      - name: Launch Batch in ReSim
        uses: resim-ai/action@v1
        id: launch_batch
        with:
          client_id: ${{ secrets.RESIM_CLIENT_ID }}
          client_secret: ${{ secrets.RESIM_CLIENT_SECRET }}
          project: <your project name>
          system: <your system name>
          image: ${{ steps.docker_meta.outputs.tags }}
          test_suite: <your test suite>
          # You may use experience names or tags instead
          # experiences: <your experience names>
          # experience_tags: <your experience tags>
          metrics-build-id: <your metrics build id>
          ## If you would like the action to post a link to the results on PRs:
          comment_on_pr: true
          github_token: ${{ secrets.GITHUB_TOKEN }}
      - name: Fetch ReSim CLI
        if: github.event_name == 'schedule'
        run: |
            curl -L https://github.com/resim-ai/api-client/releases/latest/download/resim-linux-amd64 -o resim-cli
            chmod +x resim-cli
      - name: Wait for completion and send Slack summary
        if: github.event_name == 'schedule'
        env:
          PROJECT_ID: ${{ steps.launch_batch.outputs.project_id }}
          BATCH_ID: ${{ steps.launch_batch.outputs.batch_id }}
          RESIM_CLIENT_ID: ${{ secrets.RESIM_CLIENT_ID }}
          RESIM_CLIENT_SECRET: ${{ secrets.RESIM_CLIENT_SECRET }}
          SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
        run: |
          # the default wait time is one hour. If you think your batch will reliably take longer, you may need to edit this
          ./resim batch wait --project "$PROJECT_ID" --batch-id "$BATCH_ID" || exit_status=$?
          ./resim batch get --project "$PROJECT_ID" --batch-id "$BATCH_ID" --slack | \
          curl -X POST -H "content-type: application/json" --data @- $SLACK_WEBHOOK
          exit "${exit_status:-0}"