Sampling: choose whom to study and be clear about the limits

Sampling is the deliberate selection of people, cases or other units from a larger group. A defensible sample starts with the conclusion you want to make: about whom, in which setting and during which period? Only then can you choose a recruitment route and explain its limits.

For a survey estimating a population percentage, selection and nonresponse affect whether the respondents support that inference. For exploratory interviews, the aim can be to understand relevant differences, mechanisms or difficult cases. Assess that qualitative selection against the research purpose rather than expecting it to reproduce population proportions.

This guide uses a hypothetical public library reviewing its borrowing routes. The examples and counts are invented. It explains how to make a sampling decision reviewable, not how to declare any particular sample representative.

Define whom the conclusion is meant to cover

The library wants to know why some visitors seek help with self-checkout. “Library users” is too broad. Does it mean registered members, all visitors, people borrowing books or people who attempted the kiosk? Those groups overlap without being identical.

Statistics Canada’s sample-selection guide distinguishes the target population, survey population and frame. The target is the group the study is intended to describe. The survey population is the portion covered by the study’s operational scope, after any exclusions. The frame is the list or practical structure from which selection is made. Report differences between these groups even when the eventual respondent count is large.

For a descriptive study, our library defines adult visitors who attempted to borrow books at this branch during a named four-week period. Recruiting only at the kiosk excludes people who chose the staffed desk or could not attempt the kiosk. Recruiting from a membership email list excludes some people whose addresses are absent or no longer used.

Write the inclusion and exclusion rules before recruitment. If children, accompanying carers or non-borrowing visitors matter to another question, plan that question separately or revise the scope explicitly. Do not silently extend a conclusion to them after data collection.

A stakeholder map helps identify missing perspectives, including people who support a task. It is a useful starting inventory; it is not a probability sampling frame.

Choose between estimating and exploring

If the library needs the proportion of borrowing visitors requiring assistance, it needs a design that supports an estimate for that defined population. If it needs to understand what happens when borrowing becomes difficult, purposeful interviews or observations may be more useful.

Those goals can belong to one programme but should not be combined into a vague promise of representative insight. A carefully selected set of difficult cases might reveal an important failure pathway while intentionally overrepresenting that pathway. That makes the selection useful for explanation and unsuitable for a simple prevalence estimate.

Use customer interviews when the question concerns a recent experience and its context. Choose participants who can illuminate relevant differences: first use, repeated use, seeking assistance, choosing the staffed route or using an accessibility adjustment.

The Double Diamond allows a research question to evolve during discovery. If the purpose changes from explaining difficulty to estimating its extent, revisit the sampling design. The original recruitment route does not acquire a new inferential power simply because the project has moved to a decision stage.

Understand what probability selection offers

Statistics Canada’s probability-sampling chapter describes designs based on random selection with calculable selection probabilities. Equal probability is a feature of simple random sampling, not a requirement of every probability design.

A suitable frame and random selection can support population inference under the design and its assumptions. They do not guarantee a flawless sample. The frame can be incomplete, invitations may fail and selected people may not participate. Measurement can introduce error even when selection is sound.

Stratification can ensure planned selection from groups such as times of day or branches. Those groups must be defined before selection, and the resulting analysis must account for unequal selection where relevant. Deliberately inviting more evening visitors can be appropriate, but treating their raw responses as though everyone had the same selection chance can distort an overall estimate.

For the library, a formal visitor estimate could require a time-and-location design spanning relevant opening periods and a rule for repeated visitors. That is a specialist design task. The article does not promise that speaking to every tenth person at the kiosk describes everyone who attempts to borrow.

Random sampling and random assignment are different. Sampling selects who enters a study; assignment allocates participants or units to conditions in an experiment. One does not substitute for the other.

Use non-probability selection honestly

Statistics Canada’s non-probability chapter covers approaches such as convenience and quota selection and explains their limitations for generalisation. A volunteer link, available contacts or fixed quotas do not provide known probability selection merely because the respondents look diverse.

Quota recruitment can ensure that some categories appear in the study. It does not address every unmeasured difference between volunteers and non-volunteers. Matching respondents to broad age groups, for example, does not ensure that their borrowing difficulties match those of all visitors.

Purposeful qualitative recruitment has a different rationale. Select cases because they can answer the question. State why each variation matters and which experience is absent. The library may seek people who needed help retrieving a card, who received assistance from a carer and who chose the staffed desk despite familiarity with the kiosk.

The following original planning table separates possible purposes. It contains no measured library result.

Research purpose	Relevant selection logic	Limit to report
Estimate assistance among borrowing visitors	Design-based selection covering defined visitor periods	Frame, repeated visits, nonresponse and weighting
Explain why a task becomes difficult	Purposeful variation in relevant borrowing circumstances	Does not estimate population prevalence
Check a new instruction’s interpretation	Recruit likely users with varied experience and access needs	Responses concern the tested instruction and group
Hear operational handover problems	Include roles handling requests and exceptions	Staff accounts do not replace visitor accounts

Use survey design alongside selection planning. Even a carefully recruited respondent cannot supply a meaningful answer to a question whose categories do not fit their task.

Decide adequacy by the study’s purpose

There is no sample number that makes every project credible. A quantitative design depends on the estimate or effect required, variation, subgroup questions, design and desired precision. A qualitative design depends on what information the selected cases provide and how it will be analysed.

Malterud, Siersma and Guassora’s information-power paper proposes examining the study aim, sample specificity, established theory, dialogue quality and analysis strategy when considering qualitative sample size. It is a methodological proposal, accessed here through its abstract, rather than a numerical rule.

For the library, a tightly focused inquiry into one known handover can need different cases from a study comparing several borrowing routes and language contexts. Plan an initial round, analyse it and identify which material gap the next round would address.

Do not use saturation as a retrospective label for reaching the recruitment budget. Explain what the material covers, what remains unexplored and why further collection would or would not contribute to the question. Repeated general comments may conceal a missing type of experience.

An employee survey about change readiness likewise needs a specific scope. An organisation-wide average based mainly on day-shift responses cannot settle a question about an evening-shift dependency.

Keep a recruitment and nonresponse log

Record the recruitment route, who was eligible, invitations or approaches, participation, known refusals and other outcomes. Use categories appropriate to the design rather than merging everyone who did not complete into one uninterpretable group.

AAPOR’s Standard Definitions distinguish survey outcome categories and explain that response-rate information alone is insufficient to establish how much nonresponse error exists. A low rate can raise concern; a high rate does not prove that nonrespondents are irrelevant.

In an invented recruitment round, the library invites 20 people and receives 12 completed questionnaires. Completed divided by invited is 12/20, or 60%. That arithmetic is simple, but calling it a formal response rate requires the design’s eligibility and disposition rules. If some invitations reached ineligible people or completion depended on an unknown open link, the interpretation changes.

Look for consequential patterns in absence. Did evening invitations go out after the shift ended? Was the accessible format available? Did the invitation wording imply that only experienced kiosk users should respond? Change the route to address a defined gap rather than repeatedly reminding the people easiest to reach.

Keep participant identifiers and sensitive recruitment information under appropriate access arrangements. The public report usually needs the selection logic and its limitations, not identifiable explanations for someone’s refusal.

Examine coverage and selection before using weights

Coverage error concerns who can be reached through the frame. Nonresponse concerns selected units that do not provide the required information. Measurement error concerns what is recorded. These are different problems and may need different responses.

Weighting can address aspects of the design and observed differences where appropriate assumptions and information exist. It cannot automatically recover the experiences of people missing in an unmeasured way. A carefully weighted kiosk-only study still needs a defensible account of visitors who never approached a kiosk.

AAPOR’s survey best practices emphasise the frame, recruitment and transparent reporting as part of survey quality. Ask what information supports any adjustment and whether conclusions change under reasonable alternatives. Have a qualified analyst specify formal weights before reporting population estimates.

For qualitative work, a more useful response to a missing perspective is often another purposeful recruitment route. If all initial participants are confident readers, seek the relevant experience the question requires. That is an information decision, not an attempt to manufacture a balanced demographic portrait.

Report who the finding concerns

Describe participants through characteristics relevant to the question. “Visitors interviewed after attempting this branch’s kiosk” says more about the evidence than “a diverse group of users”. Add recruitment route, setting, dates, exclusions and meaningful gaps.

For exploratory findings, report the pattern and its supporting cases without turning counts into population percentages. “Several participants described asking staff where to place multiple books” remains an account of the study. “Most library visitors cannot use self-checkout” is a wider claim requiring a different design.

A World Café can develop shared understanding among participants, and Lean Coffee can surface the questions those attending want to discuss. Neither method makes attendees a representative sample of the wider organisation. Treat the conversation’s usefulness and its population coverage as separate judgements.

Connect the report to the decision named in research design. If the library has learned about one confusing step, it can investigate or revise that step while remaining honest about frequency. It does not need to exaggerate the sample to justify a proportionate next action.

Evidence and limitations

This guide introduces selection logic rather than providing a complete survey-design or weighting manual. Its institutional sources explain sampling and survey outcomes; the information-power paper provides a conceptual approach to qualitative adequacy. None certifies the invented library plan.

Representativeness is not a property established by sample size, a few matched characteristics or a random-looking invitation list. It depends on what the design and data allow you to say about the defined population.

Finish with a clear statement of the target, selection route, relevant variation, missing cases and allowed conclusion. That is useful whether the study seeks a population estimate or a deeper understanding of a particular task.