Why methods matter

Good research is built on a chain of methodological decisions.

Methodological rigour comes from coherence across the research process. The question, population, measurements, design, analysis and interpretation need to align with the inference the study is intended to support.

01 QuestionWhat needs to be answered?
02 Population & measurementWhat — and whom — to study?
03 DesignHow will evidence be generated?
04 Precision & validityWhat can distort the result?
05 ImplementationHow will the study work?
06 Analysis & communicationWhat does the evidence support?
01 / Start with the question

What exactly does the research need to answer?

Research should start with the question, not with a preferred method or available dataset. An initial idea needs to be developed into a clear, focused and answerable research question that can guide the rest of the study.

This means clarifying what the research is trying to understand or estimate, why the question matters, and whether its scope is realistic. From there, the question can be translated into specific objectives and, where appropriate, hypotheses and a study plan.

Clarify the problemWhat do we really want to know?
Define the objectiveWhat should the study estimate, compare or explain?
Assess relevance and feasibilityIs the question useful and realistically answerable?
Use the question as the anchorEvery later methodological decision should be traceable back to it.
The research question is the first methodological decision.
02 / Define what — and whom — to study

The population, sample and measurements determine what the study can tell us.

Once the question is clear, the next step is to define the population the study is intended to inform and the sample that will actually be studied. Selection criteria, sampling and recruitment all affect how confidently findings can be interpreted and applied.

At the same time, the study needs measurements that capture the concepts of interest with sufficient validity and reliability. Poorly chosen variables or instruments can limit a study even when the design and analysis are otherwise sound.

WHO
Define the target populationWho should the findings apply to?
Specify the study sampleWho will actually be included, and how will they be selected?
WHAT
Choose the right measurementsWhat variables, outcomes or instruments best represent the concepts of interest?
Consider validity and reliabilityDo the measurements capture the intended concept, and do they do so consistently?
Population and measurement choices define the scope and interpretation of the resulting estimates.
03 / Choose a design that can answer the question

Different questions require different designs.

Once the question, population and measurements are defined, the study design determines how the evidence will be generated. The choice between observational and experimental approaches — and between designs such as cohort, cross-sectional, case-control or randomised studies — should follow from what the research needs to establish.

A good design creates a credible path from the research question to the result. It should make the comparison, timing and source of evidence explicit, while balancing methodological strength with feasibility.

Match the design to the questionIs the aim to describe, compare, predict or estimate an effect?
Define the comparisonWhat is the relevant reference, control or counterfactual?
Consider timing and follow-upWhen and over what period should observations be made?
Balance rigour and feasibilityWhat design can answer the question well in the real setting?
The design determines which inferences the resulting data can support.
04 / Distinguish precision from validity

Random and systematic error affect different aspects of inference.

Every study is affected by error, but not all error is the same. Random error determines how precisely an effect or association can be estimated. Systematic error determines whether that estimate is centred on the truth at all.

Good methodological planning considers both: reducing random variability where possible, identifying potential sources of systematic distortion, and making the remaining uncertainty visible in the interpretation.

RANDOM ERROR

Precision

  • Sample size
  • Sampling variability
  • Random measurement error
  • Statistical uncertainty
SYSTEMATIC ERROR

Validity

  • Selection bias
  • Measurement bias
  • Confounding
  • Design and analytical assumptions
ROBUSTNESS

How much do the conclusions change when plausible assumptions or sources of error are challenged?

Precision reflects random uncertainty; validity depends on whether systematic error has distorted the estimate.
05 / Plan how the study will actually work

Implementation affects study validity and data quality.

Implementation problems in recruitment, measurement or data collection can compromise an otherwise appropriate design. The protocol therefore needs to be translated into procedures that can be applied consistently and as intended in the real setting.

This means thinking in advance about feasibility, ethics, operational processes, data quality and the controls needed to ensure that the study is conducted as intended.

Plan implementationHow will the study actually be carried out?
Protect data qualityHow will data completeness, consistency and integrity be maintained?
Anticipate operational problemsWhere are deviations, missing data or protocol failures most likely to arise?
Build in ethical and practical safeguardsCan the study be conducted responsibly, realistically and reproducibly?
The validity of the study depends partly on how closely implementation matches the planned design.
06 / Analyse, interpret and communicate

Analysis and interpretation should remain aligned with the research question and design.

Analysis should follow from the research question and study design, not be treated as a separate technical exercise at the end. A clear analytical strategy helps distinguish planned comparisons from exploratory findings, makes assumptions explicit and provides a defensible route from data to conclusions.

Interpretation then requires more than reporting statistical results. It means considering effect size, uncertainty, robustness, possible limitations and whether the findings actually support the claims being made. Communication should make that reasoning transparent to the intended audience.

Plan the analysisWhat analyses are needed to answer the research question?
Interpret, don’t just reportWhat do the estimates mean in context?
Respect uncertainty and limitationsWhat do the data support, and what remains uncertain?
Communicate clearlyHow should the findings be presented so that others can understand and use them?
The interpretation should not claim more than the design, data and analysis support.

METHODS95

Methodological coherence

Each methodological decision constrains what follows. Methodological coherence means maintaining a defensible link from the research question through design and analysis to an interpretation supported by the evidence.

Explore methodological services →