Metrics Dashboards

Dashboards (under Analyze > Dashboards) aggregate data from your tests and batches across time, so you can see how your system, tests, or test suite are performing as a whole — not just within a single batch. If you've used Grafana, the mental model is the same: a dashboard is a named, refreshable view that pins a set of metric charts to a branch and a time window.

Dashboards are scoped to a single branch (typically main) and refresh on demand to pull in newly completed batches.

What can I answer with a dashboard?

A dashboard is useful any time the question spans more than one batch. Some examples:

• Pass rate trend — what percentage of tests have passed over the last 30 days, broken down by day?
• Build health — how does the failure rate of build N compare to build N−1?
• Flaky experiences — which experiences have the highest failure rate across the last week of runs?
• Coverage — how many distinct experiences has my test suite executed against in the last month?
• Regression watch — has the average value of a key performance metric (e.g. localization error, latency) drifted over time?

The common thread: the result needs data from multiple batches to be meaningful.

Topics

Dashboards are powered by the same SQL queries as regular metrics, but the scope of the query is wider — you're now looking at every job that ran on the branch within the dashboard's day range, not a single batch. Under the Topics tab you'll see all your data from tests along with the usual special topics:

• metadata: general info about the test such as the time it ran, the experience and build it ran with, its status, and more
• container_performance: the CPU and memory usage of each container, from each test
• test_length_seconds: how long each test took

The entry point for many dashboard queries is the metadata topic as it will allow you to do things such as:

• for each build of my system, what percentage of tests passed?
• which experiences fail the most often?
• over time, has my Nightly Regression test suite gotten slower, or faster?

See the metadata schema in the main metrics guide for the full column reference.

You will see these topics under the Topics tab, alongside data emitted from your tests:

How to create Dashboards

There are two ways to create a dashboard, and they're complementary — most users start with the first and graduate to the second once their dashboards stabilise.

From the webapp

The fastest path is to explore your data in the app. Navigate to Analyze > Dashboards, create a new dashboard scoped to a branch, and add metrics directly from the UI. Choosing a metrics set from your metrics config file is optional — you can start from scratch. You can browse the Topics tab to see what data is available, write queries against it, and iterate on charts until they look right. This is the best workflow when you're still figuring out what questions you want to answer.

From your metrics config file

Once a dashboard is doing useful work, you'll typically want to source-control it. Define the metrics in your .resim/metrics/config.resim.yaml file, group them into a metrics set, and create the dashboard once via the ReSim CLI:

Bash

resim dashboards create \
    --project "my project" \
    --name "Dashboard" \
    --branch main \
    --metrics-set "my dashboard" \
    --day-range 90

From then on, edits to the config flow into the dashboard via resim metrics sync. To pick up the changes, open the dashboard in the app and click Refresh data — no need to recreate it.

An example dashboard metric

Here's a metric you can drop into your config to get started. It plots daily pass rate over time using only the metadata topic — no emitted data required:

YAML

metrics:
  Daily Pass Rate:
    type: dashboard
    description: Percentage of jobs that passed, grouped by day.
    query_string: |
      SELECT
        'Pass Rate' AS group_name,
        date_trunc('day', time) AS "Day",
        100.0 * count_if(job_conflated_status = 'PASSED') / count(*) AS "Pass Rate (%)"
      FROM metadata
      GROUP BY date_trunc('day', time)
      ORDER BY date_trunc('day', time)
    template_type: system
    template: line

metrics sets:
  Branch Health:
    metrics:
      - Daily Pass Rate

A few things to note:

• Dashboard metrics use type: dashboard.
• The query runs against every job on the branch that falls within the dashboard's day range — the dashboard takes care of windowing.
• Grouping by date_trunc('day', time) produces one point per day. Switch to 'hour' or 'week' to change granularity.