decision matrices (MADM) are a useful methodology for comparing multiple alternatives and selecting the choice that best fits your needs and budget. By evaluating a set of criteria for each option, you can be confident that you have a clear understanding of the decision space.
They are, however, often misinterpreted or misapplied. This article explains how to utilize multi-attribute decision matrices and avoid pitfalls commonly associated with their use. It also lays the groundwork for a different method that borrows important concepts from MADM without falling into its implicit traps.
A Motivating Example: Tent Selection
My family is in the market for a new tent. As such, we did what we usually do: we googled “best tent for car camping.” One of the first results was a GearLab article called “The Best Camping Tents | Tested and Ranked.”
In the article, GearLab rates 16 tents on a scale of 1 to 10 across five attributes. They weigh those attributes, and then rank the tents 1-16 based on the weighted scores. This is a straightforward example of a multi-attribute decision matrix.
The Purpose of MADM
MADM is often treated as a way for data to make a decision on behalf of a stakeholder. In the GearLab article, they recommend the single “best” tent based on their MADM findings. I want to emphasize that MADM does not make the decision; it informs it.
It can best be understood as a useful tool for structuring comparisons across all alternatives, eliminating clearly inferior options, and revealing the top contenders. Used appropriately, it helps decision-makers see the landscape of available choices rather than pointing them to a single “correct” choice.
When misused, it can steer a decision into the ground and leave the decision maker with a bad taste in their mouth about “data-driven” decision-making.
In short, MADM’s purpose is to give decision-makers a better grasp of their options, eliminate poor options, and present value propositions, not to automate the decision.
How to Properly Use MADM
Here is my basic guide to MADM:
- Identify the decision-maker, decision space, and attributes.
- Define the weights for each attribute.
- Collect the data and calculate the weighted scores.
- Plot the products against the price and find the efficient frontier.
- Present the findings and recommendations to the decision maker.
Briefly, I’ll describe each in a little more detail.
First, determine who the decision maker is. Are you doing this analysis for someone else’s decision, or for your own? For this example, let’s assume that it is for your own decision.
Defining the decision space is generally fairly straightforward. You need to know the type of item (such as a tent) being considered and identify the top n options. Be sure to fairly sample all options, not just the ones that come to mind first.
Then, assign several attributes. Come up with a list of things that might make the product more useful or valuable.
After you define the attributes, I recommend speaking with the decision-maker. Once you start talking to the decision-maker, ensure you use their priorities, not yours.
Rank the attributes by importance, and consider the tradeoffs. Tradeoff questions like “Would I trade an inch of headspace from 71 inches to 70 inches for a tent that is a little more wind-proof?” Then, assign attribute weights in accordance with these responses and place them in a table for later use. These will never be perfect, even when the analysis is for your own use.
Now you have something that looks like this.
| Criteria | Weight |
| Space and Comfort | 35% |
| Weather Resistence | 25% |
| Ease of Use | 15% |
| Family Friendliness | 15% |
| Quality | 10% |
Collecting the data can vary in difficulty. In this situation, it’s relatively straightforward. Search for each tent, go to “tech specs” to find most information, and reviews to find the rest. Record that data in your decision matrix. If it’s not straightforward, you may need to subjectively assign a value to each attribute, but be sure to define your criterion, or at least your general thinking, if you do this.
For the tents on GearLab, they rated each attribute on a scale of 1 to 10, as shown below.
Now, your decision matrix looks like this. Note that to keep the chart readable, I have omitted the “quality” attribute.
| Space | Weather Resistance | Ease of Use | Family Friendly |
| Zampire | 9.5 | 9 | 6 | 9 |
| Wawona | 9 | 8 | 7 | 9 |
| Base Camp | 9 | 8 | 6.5 | 8 |
| Aurora | 9 | 7 | 7 | 8 |
| Tungsten 4 | 7 | 8.5 | 9 | 7 |
| Bunkhouse 6 | 8 | 7 | 8 | 7 |
| Skydome 8 | 9 | 6 | 6 | 9 |
| Limestone | 7 | 9 | 8 | 5 |
| Alpha Breeze | 7 | 9 | 6 | 7 |
| T4 Hub | 7.5 | 7 | 8 | 7.5 |
| Wonderland | 7 | 8 | 7 | 7 |
| Wireless 6 | 7 | 7 | 8 | 8 |
| Zeta C6 | 8 | 6 | 10 | 6 |
| Sundome | 7 | 7 | 6 | 5 |
| TallBoy 4 | 6 | 7 | 7 | 5 |
| Coleman Cabin | 5 | 7 | 9 | 3 |
All that remains is to calculate the weighted scores. To do this, take the sum product of the weights and the values for each item. You now have your completed decision matrix. I have also included the price for reference.
| Tent | Price | Weighted Score |
| Zampire | $1,200.00 | 8.725 |
| Wawona | $550.00 | 8.45 |
| Base Camp | $569.00 | 8.225 |
| Aurora | $500.00 | 7.95 |
| Tungsten 4 | $399.00 | 7.775 |
| Bunkhouse 6 | $700.00 | 7.6 |
| Skydome 8 | $285.00 | 7.5 |
| Limestone | $429.00 | 7.45 |
| Alpha Breeze | $550.00 | 7.45 |
| T4 Hub | $430.00 | 7.4 |
| Wonderland | $429.00 | 7.35 |
| Wireless 6 | $270.00 | 7.3 |
| Zeta C6 | $160.00 | 7.2 |
| Sundome | $154.00 | 6.45 |
| TallBoy 4 | $170.00 | 6.25 |
| Coleman Cabin | $219.00 | 5.8 |
Next, plot the weighted score of each item against its price, orient yourself to the plot, and plot the efficient frontier:
From this, we can identify eight tents on the efficient frontier. Being on the efficient frontier means we cannot get a better weighted score at the same or lower price. This is the key insight MADM provides: identifying which options are strictly dominated and which involve meaningful trade-offs between quality and cost.
If this plot looks familiar, it is likely because you have seen a similar plot on a financial risk-return efficient frontier. One axis is something you want less of (price/risk), and the other is something you want more of (score/return).
| Tent |
Price |
Weighted Score |
| Sundome |
$154.00 |
6.450 |
| Zeta C6 |
$160.00 |
7.200 |
| Wireless 6 |
$270.00 |
7.300 |
| Skydome 8 |
$285.00 |
7.500 |
| Tungsten 4 |
$399.00 |
7.775 |
| Aurora |
$500.00 |
7.950 |
| Wawona |
$550.00 |
8.450 |
| Zampire |
$1,200.00 |
8.725 |
So which to recommend? If my budget is $600 and I want the highest-quality tent I can afford, I would opt for the North Face Wawona 6.
See here: I drew a line at the budget, then chose the first tent to the left of that line on the efficient frontier. I could do a similar thing if I had a “quality budget” and drew a line, then chose the first point on the efficient frontier above the line.
All that remains now is to present your findings to the decision-maker. When doing this, I recommend orienting them to the plot and pointing out and explaining the efficient frontier. Something as simple as “for each of these points, you cannot get a better rating for the same price” will suffice. Call attention to the highest-rated option. If you know their budget in advance, make the appropriate recommendation.
Note that if we use a ratio of the weighted score to price, we lose a lot of information and cannot determine which tent to choose. It is acceptable to include this information, but not necessary, as it sometimes tells a misleading story. For example, if a tent costs only $5 at a garage sale and is just as large as the best competitor, but leaks when it rains, it is not a real contender. However, the ratio would likely show it as the “best value” choice. For a similar reason, price should be kept separate from the attributes in MADM and used only as a constraint or tradeoff.
Conclusion
Now that you understand how MADM works, its shortcomings are easier to see. It has a tendency to overlook certain details in decision-making by generalizing everything into a single score and assuming linearity across all attributes (i.e., an increase from 70 inches to 71 inches is treated as equally valuable as an increase from 40 inches to 41 inches, which is probably not the case).
It’s essential to understand the mechanics of MADM to appreciate the improvement achieved by adopting this next method. In the second part of this two-part series, I will propose an alternative to MADM that preserves its strengths while yielding recommendations more closely aligned with decision makers’ priorities.
Author Note
If you enjoyed this, I write about analytical reasoning, decision science, optimization, and data science. I also share new work and related thoughts on LinkedIn.
Instagram Engineering
Shopify Engineering