Friday 17 December 2021

Defining good

Within non-trivial technical projects, it helps to have a common language to describe "good":

  • It enables cross-feature comparisons and prioritization.
  • It facilitates conversations between stakeholders, business and developers.

This page suggests a feature quality scale, along with how it can be applied.

Quality scale

A quality scale can be applied to:

  • Business goals, stakeholder needs, features, quality, schedule, maintenance, and more ...

User experience Description Requires
Qx Over-deliver * Functionality cannot be noticed or used by the user.
Q5 Delight * Anticipate user desires, and provide it. * Understand the user’s desires and passions.
Q4 Impress * Anticipate user unanticipated needs, and provide it. * Analyze the user behaviors and needs.
* Understand the product capabilities.
* Establish critical user journeys.
Q3 Satisfy * Meet user’s known wants. * Listen to the user’s asks.
Q2 Underperform * Under specified or poorly implemented.
* User experience includes many micro-frustrations.
* Meet purchasing authority’s specification.
* Cost saving implementation.
* Minimal testing.

Q1 Not practically functional * While the product “works” the experience is so poor that the user chooses to use something else if available.
Q0 Broken * Functionality doesn’t work at all.

Possible quality scale

Quality categories for each feature

For each feature, a project can define quality categories.

Feature Delight Impress Satisfy Underperform Not practically functional Broken
On/off button Works Fails
Waterproof to > 100m > 10m > 1m < 1m
Uptime > 99.999% > 99.99% > 99.9% > 99% > 90% < 90%

Example quality criteria

Note: Many features won't need the full scale.

Phase exit criteria

The importance of each quality criteria will vary depending upon:

  • Different products.
  • Different features.
  • Different development phases: alpha, staging for test, production/stable release.

Phase: Alpha release

Feature Priority Should Must Error tolerance
On/off button P1 Satisfy Satisfy Satisfy
Waterproof P3 Impress Underperform Broken
Uptime P2 Impress Satisfy Underperform

Example phase exit criteria

As requirements

The criteria can be converted into traditional requirements:

The device should be waterproof to 100m.
The device must be waterproof to 1m.
Example requirements

Bug severity levels

Terminology from the quality scale can be aligned with bug severity levels. Severity can be driven by impact to the user, or impact to the business.

Severity User Impact Business Impact
S4 Trivial * P3 feature underperforms * Person days to fix.
S3 Minor * P3 feature is broken
* P2 feature underperforms
* Person weeks to fix.
S2 Major * P1 feature is broken, with work-around
* P2 feature is broken, no work-around
* Person months to fix.
S1 Critical * P1 feature is broken. No work-around. * Upcoming releases blocked until fix provided.

Example severity scale

Quality Scale