Hello folks! We’d like some help with terminology here at Sauce Labs. Specifically, we want to talk about Builds.
We introduced a Build reporting system several years ago, with the intent of letting customers group a series of tests together, to indicate they all ran at the same time. This gave customers easier reporting and matched how testing is usually done in a CI process.
This feature works great; The problem is the name. See, it’s pretty common for Build to refer to a specific version of the code under test, but not the tests ran against that code. For instance, if you have a quarterly release with a pre-determined release number, you might refer to that release number as the Build. Or, you might have several suites of independent tests, running against the same checkin, and only consider that checkin to be a Build.
So, we’re looking for a better, clearer term for:
A single execution of a group of tests against a specific version of a system ran at a single point in time
Not the version of the code being ran, or the release number, or which tests ran, but the actual test instances themselves, as a group. What would you call that?