Adding Experiences

To test our system we need some experiences.

Prerequisites

Before creating an S3 experience, you need to have an experiences AWS S3 bucket set up as outlined in ReSim Data Access. You will also need access to this S3 bucket.

Storing files

Each experience should be stored in a unique path containing the input files. So for an S3 bucket, each experience should have a unique prefix, such as s3://my-experiences-bucket/project_hal9000/experiences/experience_56ccce99/. Anything contained in that prefix will be treated as the same experience. We recommend using a directory structure that represents your ReSim projects. For example:

📂 .
├── 📂 project_a
│  └── 📂 experiences
│     └── 📂 experience_56ccce99
│        └── ⚙️ test_file
│
└── 📂 project_b
   └── 📂 experiences
      └── 📂 experience_679bef80
         └── ⚙️ test_file

Local experiences

It may be the case that experience data lives in the same repository as your system-under-test. For example, if it is tightly coupled to changes in the system. In order to support this use case any path may be supplied in the location field.

Experience Timeouts

Each experience has a timeout. This is the maximum amount of time that a build can run for a given experience. If the build does not finish running before the timeout is reached, it will be sent a SIGTERM and then a SIGKILL after a grace period.

The default timeout is 1 hour, but you can specify a different timeout when creating an experience.

Creating an experience

You can create new experiences like so:

resim experiences create \
    --project "my-project" \
    --name "Experience_56ccce99" \
    --description "In this experience, we test that our robot can navigate to the goal without colliding with any other agents." \
    --location "s3://my-experiences-bucket/project_hal9000/experiences/experience_56ccce99/" \
    --timeout "1h10m" \
    --systems "perception", "localization"

Important

Any location that points to an S3 bucket must be prefixed with s3:// rather than the https://... form.

This command should return a message containing the UUID of the created experience, and it should now be browsable through the ReSim app.

The final flag enables you to specify any systems that this experience is compatible with.

Updating an experience

You can add or remove a system from an experience if it's already been created like so:

resim experiences add-system \
    --project "my-project" \
    --system "<system name>" \
    --experience "Experience_56ccce99"

resim experiences remove-system \
    --project "my-project" \
    --system "<system name>" \
    --experience "Experience_56ccce99"

Similarly you can add and remove arbitrary tags:

resim experiences tag \
    --project "my-project" \
    --tag "<tag string>" \
    --experience "Experience_56ccce99"

resim experiences untag \
    --project "my-project" \
    --tag "<tag string>" \
    --experience "Experience_56ccce99"