Case 03 / AI outcomes · Product judgement
A fair reading of results
Designing how to present the results of an AI recommendation, while making the limits of the comparison part of the experience.
- My contribution
- UX lead for the outcomes experience and its frontend prototypes.
- Scope
- Measurement interpretation, discovery preparation and a focused dashboard experience.
- Concrete outcome
- A narrower first version with an explicit measurement constraint.
01 / The problem
A comparison can look more certain than it is.
The feature recommended a marketing channel. Its results needed to fit within an existing outcomes dashboard. The first version did not have a control group; the available comparison was a modelled alternative scenario.
A model estimate is different from a measured difference against a control group. I treated that distinction as a design requirement, rather than something to hide in the implementation details.
02 / The approach
Start with one metric and a clear explanation.
I worked with a product colleague to prepare customer discovery, investigated the available measurement basis and scoped the first experience to one metric within the existing dashboard.
I built a frontend prototype so colleagues could discuss the presentation in customer conversations. The scope was how to show and interpret the outcome, rather than improving the recommendation model itself.
Decision 01
Name the basis of the comparison
Make it clear when a baseline is estimated. The experience should help someone interpret the evidence without implying a controlled experiment.
Decision 02
Reduce scope before expanding the dashboard
One metric in the established structure created a focused starting point for feedback. The final metric and the discovery outcome were still open in the case notes.
Decision 03
Keep the work connected to the team
I focused on the experience and frontend prototype, working with the product colleague’s customer context. The work was subsequently handed over to a colleague.
Explore why the comparison matters
An estimate of an alternative scenario.
The reference is generated by a model. It can provide a comparison, but it is not an observed outcome from a control group. Label the estimate and explain what it represents.
This was the comparison available in the first version. A difference from this estimate should not be presented as experimentally measured incremental impact.
An educational comparison created for this portfolio. The control-group option is hypothetical; it was not available in the first version. No customer or performance data is shown.
03 / The outcome
A more explicit and testable starting point.
The scope was reduced to one metric, discovery was prepared and the lack of a control group became explicit input to the design. A frontend prototype made that direction tangible for the next conversation.
What I contributed
- Identified the distinction between the estimate and a measured control.
- Scoped the first experience to one metric in the existing dashboard.
- Prepared discovery and built a frontend prototype before handover.
What the evidence does not establish
There are no recorded measured user or business outcomes in this case. It does not establish that users understood the explanation, that discovery validated the design or that the feature generated incremental value.
04 / The next question
What would I want to learn next?
The next evidence would be whether users correctly explain the baseline in their own words and make appropriately qualified decisions from the results.
Continue exploring / Case 04
One system. Shared rules.