Senior Test Engineer - Metrics Quality

Senior Test Engineer - Metrics Quality

Every decision Avride makes about its autonomous driving software merge or revert, ship or hold, this approach or that one — rests on a metric computed over a set of recorded scenes. Our QA organization is what stands between a metric and a decision made on a metric that was quietly wrong.

You will own the quality of the metrics themselves. Our data scientists build them; you decide whether they can be trusted, before anyone starts making release decisions with them. This is a specific and underrated craft.

A metric can be computed correctly, pass every unit test, and still be the wrong number for the decision it is meant to support. Finding that takes reading the implementation, understanding the product, and knowing what the people who rely on the number are actually trying to learn from it. The hard half of the job is not checking that a metric fires when it should.

It is finding the places where it stays silent and should not have — the failures that produce no number at all, and therefore no complaint, until someone ships on the strength of them.

Own the acceptance of new and changed metrics. Before a metric is used to decide whether a change is safe to ship, you decide whether it can be: what it should count, what it should not, and whether the implementation agrees with either. Build the case space.

For each metric, work out the full set of situations it has to handle — including all th