Weighted decision matrix: compare options and show uncertainty

A weighted decision matrix compares alternatives against explicit criteria, turns their performance into comparable scores and combines those scores using stated weights. Its value is making the trade-offs inspectable. The highest total is a recommendation under the model’s assumptions, rather than an objective fact about which option is best.

Start with requirements that cannot be traded away. Then define meaningful scoring scales, weight the differences those scales represent, calculate the totals and test whether plausible changes alter the ranking. Keep missing evidence visible throughout.

The worked example follows a fictional community group planning a mobile repair service. It must choose between a compact van, a larger van and a cargo trailer. All costs, performance scores and judgments below are invented for teaching; they are not quotations or actual vehicle assessments.

Define the decision and its non-negotiable conditions

The group’s decision is which transport option deserves further inspection before a limited service trial. It has not authorised a purchase. The options need to carry a folding bench and tools, reach agreed locations and fit the group’s storage arrangements.

List hard requirements separately from desirable performance. Any option that cannot be used safely within the agreed operating conditions is outside the comparison until that condition is resolved. A low purchase cost should not compensate for failing a requirement.

NASA’s decision-analysis guidance distinguishes mandatory from enhancing criteria and recommends discarding options that do not meet mandatory criteria. It also asks for operational definitions and attention to uncertainty that could alter a ranking. Those principles are useful here, while its mission-engineering context is different.

Use stakeholder mapping to identify whose work the criteria need to represent. The coordinator may see the budget, while the volunteer who loads equipment understands an access problem. Both perspectives matter; neither can replace the other with a confident score.

For less developed ideas, use early idea evaluation first. A matrix is premature when alternatives are only slogans and the team cannot explain their operating mechanism.

Choose criteria that represent distinct consequences

The fictional team selects four criteria: total cost over the trial period, ease of loading, access to planned locations and reliability of availability. Each needs a definition, an evidence source and a direction of preference.

Total cost includes the relevant hire or ownership cost and necessary equipment for this trial. It does not mean the advertised price alone. Loading covers the effort of moving the standard bench and tools in agreed conditions. Location access covers the planned route and parking arrangement. Availability concerns whether the option can be there when the service needs it.

Avoid overlapping criteria that count the same benefit repeatedly. “Low cost”, “affordability” and “budget fit” may be three labels for one consequence. The OECD/European Commission handbook on composite indicators discusses weights as value judgments and the risk of double counting overlapping indicators. Its country-ranking context differs from vehicle selection, but the warning about aggregation transfers.

Align criteria with the intended innovation strategy. If the service’s purpose is to reach small neighbourhood venues, access deserves examination. If the real purpose is transporting a large workshop, the options and criteria may need to change.

Anchor the scores in observable differences

Use a common preference scale, such as 0 to 100, with meaningful anchors for each criterion. Higher always means more preferred in this example. A scale should represent the team’s judgment of the difference, rather than simply rename “poor”, “average” and “good” as numbers.

The UK multi-criteria analysis manual describes performance matrices and linear-additive models. The calculation below is a simplified application. It assumes that the scores express meaningful preference differences and that the chosen trade-offs are acceptable.

For total cost, the team uses fixed hypothetical anchors of £15,000 at score 0 and £5,000 at score 100. Within that range, it deliberately chooses a linear value function:

Cost score = 100 × (£15,000 − option cost) ÷ (£15,000 − £5,000).

Thus a £9,000 cost scores 60; £12,000 scores 30; £6,000 scores 90. Linear scaling is an assumption about preference, not a financial law. If a budget boundary or diminishing benefit makes the relationship different, choose and explain another scale. Revisit out-of-range values rather than silently clipping them.

For loading, describe the endpoints and test the standard task. The lowest anchor represents the least preferred acceptable loading arrangement; the highest represents the most preferred arrangement for the same equipment and conditions. Do not use “unsafe” as a low-score endpoint after agreeing that safety is a gate.

Record the observation behind each score. In a real assessment, the group might rehearse loading, inspect route access and check provider commitments. In our invented calculation, the non-cost scores are stipulated illustrative preference values. They should not be mistaken for measured performance.

Weight the swing, not an abstract word such as importance

A weight depends on both the preference scale and how much its difference matters. “Safety is important” does not explain a weight if all eligible options meet the same safety requirement. The team needs to compare the improvement from the least preferred to the most preferred performance on each criterion.

HM Treasury’s MCDA guidance explicitly distinguishes swing weighting from weighting by relative importance alone. It considers the performance difference and the value stakeholders place on that difference, then checks sensitivity.

Use a simple pairwise discussion of defined swings. In the fictional team, the agreed cost swing receives a reference value of 100. Compared with that swing, the loading swing receives 75, location access 50 and availability 25. These values express hypothetical judgments about the anchored differences, not frequencies or empirical measurements.

The sum is 250. Dividing each by 250 produces weights of 0.40, 0.30, 0.20 and 0.10, or 40%, 30%, 20% and 10%. The weights total 100%.

A ranking of preferred swings alone would not supply their relative magnitudes. Nor is this short discussion a full Analytical Hierarchy Process. If pairwise judgments conflict, revisit them; do not let a spreadsheet conceal the disagreement. Six Thinking Hats can organise an earlier perspective discussion, but hat colours are not scoring criteria or weights.

Calculate the totals and show the contributions

Multiply each preference score by its weight, then add the contributions. Keep the component scores visible. The invented matrix uses the three cost scores calculated above and stipulates the other values for illustration.

Criterion and baseline weight	Compact van	Larger van	Cargo trailer
Cost: 40%	60	30	90
Loading: 30%	70	100	40
Location access: 20%	80	60	50
Availability: 10%	60	90	50
Weighted total	67	63	63

For the compact van: (0.40 × 60) + (0.30 × 70) + (0.20 × 80) + (0.10 × 60) = 24 + 21 + 16 + 6 = 67.

For the larger van, the contributions are 12 + 30 + 12 + 9 = 63. For the trailer, they are 36 + 12 + 10 + 5 = 63. The same total can conceal different strengths; the larger van and trailer are not equivalent service arrangements.

The compact van leads by four points under this model. That is not a probability of success, a percentage advantage or evidence of statistical significance. Inspect how much of the margin depends on uncertain scores or disputed trade-offs.

Change the weights before you trust the ranking

One volunteer argues that the loading swing matters more than the baseline discussion allowed. The team considers another defensible preference scenario: cost 20%, loading 50%, access 20% and availability 10%. These weights also total 100%.

Alternative	Baseline weights	Loading-priority weights	Meaning for the decision
Compact van	67	69	Loses the lead
Larger van	63	77	Becomes preferred
Cargo trailer	63	53	Its cost advantage matters less

The larger van’s revised calculation is 6 + 50 + 12 + 9 = 77. The compact van scores 12 + 35 + 16 + 6 = 69; the trailer scores 18 + 20 + 10 + 5 = 53. A priority change has altered the ranking without changing any vehicle score.

That finding exposes a decision about values, not a spreadsheet fault. Who can decide the trade-off? Which loading conditions are assumed? Does the volunteer’s concern reveal a new hard requirement? If assistance cannot be provided when needed, the issue may belong at the gate rather than in the weight discussion.

Sensitivity is not an invitation to keep adjusting weights until a favourite wins. Define plausible changes from real disagreement or uncertainty, preserve them and explain which ones matter.

Change uncertain scores as well as weights

Suppose the compact van’s loading score is uncertain. At the baseline weights, changing it from 70 to 55 reduces the total by 0.30 × 15 = 4.5 points, from 67 to 62.5. That is below both alternatives’ 63. A loading inspection could therefore change the recommendation.

Do not describe 55 to 70 as a confidence interval unless a justified statistical model produced it. Here it is an illustrative judgment range. The team needs to investigate what would make either endpoint plausible.

Assumption mapping can connect that uncertainty to the next action. The critical belief may be that a volunteer can load the standard bench in ordinary conditions. A bounded experiment or rehearsal should test that task, rather than ask which vehicle people prefer from a photograph.

Inspect dependencies too. A loading arrangement that needs extra paid help may change total cost. Altering only the loading score would understate the consequences. Scenarios should keep linked quantities coherent.

Make a recommendation with reasons and conditions

The fictional group postpones purchase and inspects loading with the compact and larger vans. It keeps the trailer in the comparison, while acknowledging that the initial totals do not justify treating its operating arrangement as identical to the larger van’s.

Use staged funding to authorise that inspection and estimate its cost. If the options themselves are too narrow, return to choosing an ideation method rather than refining scores for an incomplete option set.

Record the alternatives, hard requirements, scales, evidence, weights, arithmetic, sensitivity scenarios and decision owner. State why the next commitment follows. A premortem can then examine candidate implementation failures for the recommended arrangement, without turning the matrix into technical assurance.

Evidence and limitations of an additive matrix

Adding weighted scores allows strengths on one criterion to compensate for weaknesses on another. Use that only where the trade-off makes sense. Hard requirements remain separate. If preferences depend strongly on combinations of criteria, a simple additive model may be inappropriate.

The totals express judgments and evidence through a chosen model. They cannot repair missing alternatives, weak observations or an illegitimate decision process. Equal weights also embody a judgment; they do not remove weighting.

For consequential decisions, use suitable analytical expertise and inspect the model assumptions. For a bounded workshop decision, the practical benefit is a transparent record: what matters, how each option performs, which uncertainties could change the ranking and what the team will investigate next.