In this guide
Match the claim to its support
  1. 01Make the claim precise
  2. 02Trace original evidence
  3. 03Inspect method and transfer
  4. 04Choose a proportionate action
An original appraisal route. Observation, association and causal attribution require different support.

Before acting on a management claim, identify exactly what it says, trace it to its original evidence and ask whether the method supports that conclusion in a relevant setting. A plausible story, a chart and a citation can still leave the central claim unsupported.

The purpose is a better decision, not a competition to reject every source. Evidence can be useful while remaining limited. Match the strength of the commitment to what is known and make the most important uncertainty visible.

Make the claim precise enough to check

Imagine a hypothetical presentation saying that a new service arrangement dramatically reduced waiting. Before accepting it, ask which service, which users, which period and which definition of waiting. Ask whether the statement describes a measured difference or claims that the arrangement caused it.

A researcher compares a large rising chart with the source records spread across a desk.
Figure 1. Make the claim precise enough to check.Illustration credits

Separate several possible claims. The recorded waiting time may have fallen. Users may report less frustration. More cases may be resolved. The service change may have caused one or more of those outcomes. Each statement needs appropriate evidence; none automatically follows from another.

The Center for Evidence-Based Management describes evidence as information supporting or contradicting a claim and emphasizes critical appraisal of its quality and applicability. That approach leaves room for research, organizational information and relevant experience, while asking what each can actually establish.

Write the claim in one sentence with a population, action or condition, outcome and time frame where relevant. If the presenter cannot clarify it, record the ambiguity. A claim that keeps changing when questioned is difficult to use as a basis for action.

Trace the claim to the evidence

Follow citations until you reach the underlying study, data or documented account. A consulting article quoting a book that refers to a survey is not three independent sources. Inspect the survey itself if the recommendation depends on its result.

Check the provenance of charts and percentages. What is counted, what is excluded and what is the denominator? Does the chart show all cases, only completed cases or a selected period? Are unresolved cases still in the queue? A shorter average among completed cases can coexist with a growing unresolved backlog.

The change-failure myth guide illustrates why a widely repeated management statistic needs a traceable origin and a clear definition. Repetition can make a statement familiar without improving its evidence.

Record the access you actually have. If only an abstract is available, say which judgement it permits and which would need the full text. Do not describe an uninspected appendix, model or sample as though you had checked it. Access limitations may justify a smaller claim or further inquiry.

Ask whether the design fits the question

CEBM-hosted critical-appraisal teaching material asks about a focused question, valid methods, importance and applicability. Its tools come from clinical evidence appraisal; the general questions can inform a management inquiry without turning a clinical checklist into a universal management score.

Different designs answer different questions. Interviews can illuminate experiences and possible mechanisms. A cross-sectional survey can describe relationships at one time. An appropriate comparison may strengthen a causal interpretation. None should be judged simply by whether its result agrees with the preferred recommendation.

Use an appraisal record:

QuestionWhat to inspect
What was asked?Whether the research question matches the quoted claim
Who was included?Selection, setting, nonresponse and exclusions
What was measured?Definitions, instruments and timing
What was compared?Alternative conditions and relevant differences
What could distort the result?Bias, confounding, missing data and selective reporting
How uncertain is it?Precision, variation and plausible alternative explanations
Does it transfer?Differences between the study and the intended use

This is an original inquiry aid, not a validated appraisal instrument. Use design-specific expertise when the decision depends on technical statistical or methodological questions.

Separate association, change and causation

In the fictional service presentation, waiting may fall after a change because demand falls, extra staff arrive or difficult cases are temporarily routed elsewhere. The new arrangement may help, but a before-and-after difference alone cannot distinguish these explanations.

The Magenta Book distinguishes evaluating implementation and outcomes from assessing impact. For a management decision, ask what comparison supports the proposed causal claim and which alternative explanations remain credible. Research design helps make that question answerable before collecting more information.

Do not demand a randomized experiment for every useful observation. A well-documented case can reveal an operational problem. The issue is the conclusion attached to it: an observed difficulty, a plausible mechanism, an association and an estimated causal effect have different evidential requirements.

Also examine measurement. If the system starts its waiting clock later after the change, the metric may improve without the person’s experience improving. Reliability and validity explains why consistency of a measure does not by itself establish that it supports the intended interpretation.

Read statistical language with care

The American Statistical Association’s statement on p-values explains that a p-value is not the probability that a hypothesis is true and does not measure an effect’s size or importance. A statistically significant result therefore does not answer whether a change is useful, substantial or transferable.

Ask for the estimated effect, uncertainty and practical consequence. Examine the original outcome and comparison rather than relying on the word significant in a summary. If several outcomes were examined, ask how the reported one was selected.

Do not infer that an inconclusive result proves no effect. It may reflect limited information, a poorly implemented intervention or an outcome measured at an unsuitable time. Those possibilities also do not establish that the proposal works. The appropriate conclusion may remain uncertain.

Choose an action that matches the evidence

Bring the appraisal back to the decision. What would be committed, what could go wrong and what further evidence is proportionate before that commitment? A reversible local trial and a large organization-wide programme should not automatically require the same preparation.

Two colleagues observe a service counter and queue while discussing possible explanations.
Figure 2. Choose an action that matches the evidence.Illustration credits

For the hypothetical queue, the next step might be to verify the timing definition, examine unresolved cases and compare the operating conditions across periods. If the claimed result survives that check, a further evaluation may still be needed before attributing the improvement to the service arrangement.

Use a decision memo to state the claim, evidence, limitation and proposed commitment together. If a small local test is appropriate, define the prediction, observations and decision condition through experiment design. Do not use the word pilot as permission to avoid those questions.

Keep uncertainty useful

An appraisal can conclude that a claim is supported within a boundary, plausible but unresolved, overstated or unsupported by the cited source. Explain the reason in ordinary language. Avoid attaching a precise confidence score when the method for producing it is unclear.

The service presentation in this guide is hypothetical and contains no measured result. The sources establish appraisal and evaluation principles; they do not validate this exact checklist or certify a particular management decision.

For the next recommendation you receive, select its strongest consequential claim. Write the original source, the method, the outcome and the main transfer limitation beside it. Then decide whether the evidence supports the requested action, a smaller inquiry or no further commitment. Preserve that reasoning so it can change when better evidence arrives.

Sources: