In this guide
- 01Make the claim precise
- 02Trace original evidence
- 03Inspect method and transfer
- 04Choose a proportionate action
Before acting on a management claim, identify exactly what it says, trace it to its original evidence and ask whether the method supports that conclusion in a relevant setting. A plausible story, a chart and a citation can still leave the central claim unsupported.
The purpose is a better decision, not a competition to reject every source. Evidence can be useful while remaining limited. Match the strength of the commitment to what is known and make the most important uncertainty visible.
Make the claim precise enough to check
Imagine a hypothetical presentation saying that a new service arrangement dramatically reduced waiting. Before accepting it, ask which service, which users, which period and which definition of waiting. Ask whether the statement describes a measured difference or claims that the arrangement caused it.

Separate several possible claims. The recorded waiting time may have fallen. Users may report less frustration. More cases may be resolved. The service change may have caused one or more of those outcomes. Each statement needs appropriate evidence; none automatically follows from another.
The Center for Evidence-Based Management describes evidence as information supporting or contradicting a claim and emphasizes critical appraisal of its quality and applicability. That approach leaves room for research, organizational information and relevant experience, while asking what each can actually establish.
Write the claim in one sentence with a population, action or condition, outcome and time frame where relevant. If the presenter cannot clarify it, record the ambiguity. A claim that keeps changing when questioned is difficult to use as a basis for action.
Trace the claim to the evidence
Follow citations until you reach the underlying study, data or documented account. A consulting article quoting a book that refers to a survey is not three independent sources. Inspect the survey itself if the recommendation depends on its result.
Check the provenance of charts and percentages. What is counted, what is excluded and what is the denominator? Does the chart show all cases, only completed cases or a selected period? Are unresolved cases still in the queue? A shorter average among completed cases can coexist with a growing unresolved backlog.
The change-failure myth guide illustrates why a widely repeated management statistic needs a traceable origin and a clear definition. Repetition can make a statement familiar without improving its evidence.
Record the access you actually have. If only an abstract is available, say which judgement it permits and which would need the full text. Do not describe an uninspected appendix, model or sample as though you had checked it. Access limitations may justify a smaller claim or further inquiry.
Ask whether the design fits the question
CEBM-hosted critical-appraisal teaching material asks about a focused question, valid methods, importance and applicability. Its tools come from clinical evidence appraisal; the general questions can inform a management inquiry without turning a clinical checklist into a universal management score.
Different designs answer different questions. Interviews can illuminate experiences and possible mechanisms. A cross-sectional survey can describe relationships at one time. An appropriate comparison may strengthen a causal interpretation. None should be judged simply by whether its result agrees with the preferred recommendation.
Use an appraisal record:
| Question | What to inspect |
|---|---|
| What was asked? | Whether the research question matches the quoted claim |
| Who was included? | Selection, setting, nonresponse and exclusions |
| What was measured? | Definitions, instruments and timing |
| What was compared? | Alternative conditions and relevant differences |
| What could distort the result? | Bias, confounding, missing data and selective reporting |
| How uncertain is it? | Precision, variation and plausible alternative explanations |
| Does it transfer? | Differences between the study and the intended use |
This is an original inquiry aid, not a validated appraisal instrument. Use design-specific expertise when the decision depends on technical statistical or methodological questions.
Separate association, change and causation
In the fictional service presentation, waiting may fall after a change because demand falls, extra staff arrive or difficult cases are temporarily routed elsewhere. The new arrangement may help, but a before-and-after difference alone cannot distinguish these explanations.
The Magenta Book distinguishes evaluating implementation and outcomes from assessing impact. For a management decision, ask what comparison supports the proposed causal claim and which alternative explanations remain credible. Research design helps make that question answerable before collecting more information.
Do not demand a randomized experiment for every useful observation. A well-documented case can reveal an operational problem. The issue is the conclusion attached to it: an observed difficulty, a plausible mechanism, an association and an estimated causal effect have different evidential requirements.
Also examine measurement. If the system starts its waiting clock later after the change, the metric may improve without the person’s experience improving. Reliability and validity explains why consistency of a measure does not by itself establish that it supports the intended interpretation.
Read statistical language with care
The American Statistical Association’s statement on p-values explains that a p-value is not the probability that a hypothesis is true and does not measure an effect’s size or importance. A statistically significant result therefore does not answer whether a change is useful, substantial or transferable.
Ask for the estimated effect, uncertainty and practical consequence. Examine the original outcome and comparison rather than relying on the word significant in a summary. If several outcomes were examined, ask how the reported one was selected.
Do not infer that an inconclusive result proves no effect. It may reflect limited information, a poorly implemented intervention or an outcome measured at an unsuitable time. Those possibilities also do not establish that the proposal works. The appropriate conclusion may remain uncertain.
Choose an action that matches the evidence
Bring the appraisal back to the decision. What would be committed, what could go wrong and what further evidence is proportionate before that commitment? A reversible local trial and a large organization-wide programme should not automatically require the same preparation.

For the hypothetical queue, the next step might be to verify the timing definition, examine unresolved cases and compare the operating conditions across periods. If the claimed result survives that check, a further evaluation may still be needed before attributing the improvement to the service arrangement.
Use a decision memo to state the claim, evidence, limitation and proposed commitment together. If a small local test is appropriate, define the prediction, observations and decision condition through experiment design. Do not use the word pilot as permission to avoid those questions.
Keep uncertainty useful
An appraisal can conclude that a claim is supported within a boundary, plausible but unresolved, overstated or unsupported by the cited source. Explain the reason in ordinary language. Avoid attaching a precise confidence score when the method for producing it is unclear.
The service presentation in this guide is hypothetical and contains no measured result. The sources establish appraisal and evaluation principles; they do not validate this exact checklist or certify a particular management decision.
For the next recommendation you receive, select its strongest consequential claim. Write the original source, the method, the outcome and the main transfer limitation beside it. Then decide whether the evidence supports the requested action, a smaller inquiry or no further commitment. Preserve that reasoning so it can change when better evidence arrives.
Sources:
- Center for Evidence-Based Management (n.d.). What Is Evidence-Based Management?.
- Martin Downes / CEBM Evidence Live (2016). AXIS: Critical Appraisal of Cross-Sectional Studies.
- HM Treasury (2026). Magenta Book: Central Government guidance on evaluation.
- American Statistical Association (2016). ASA Statement on Statistical Significance and P-Values.



