Building a Methodology

Every design decision locked before the data arrives

A study ran for four months. Data came in on schedule. Then the researcher began writing the methodology section — the part that explains what the study was designed to measure and why. Results that appeared went in. Outcomes with no signal were left out. A reviewer asked a plain question: which of these outcomes were primary? No answer predated the data. What existed was a methodology designed to explain the findings rather than to test them.

A pre-registered methodology closes that gap. It moves every consequential decision to a point in time when the data cannot influence it and creates a timestamped public record connecting the stated hypotheses to the analysis that was planned to test them. This book builds that record, one decision at a time, in the order the decisions have to happen.

Seven stages in the order they have to happen

• the-measurement-question — Tracing the full chain from intervention to observable signal and committing to primary and secondary outcomes in writing, before any data exists. Every downstream decision follows from which signal the study is committed to measuring.
• the-research-frontiers-map — A structured map of the field across four layers: established consensus, active debates where the field disagrees, speculative claims without strong evidence, and the specific gap the proposed study occupies. Shows where the new study sits and why the gap hasn’t been closed.
• the-gap-analysis — Confirming the gap identified in the frontier map holds under a structured search, then writing the contribution statement that names what the study adds — a claim about design and question, not a claim about results.
• the-label-architecture — The full taxonomy of labels the study will apply, the edge-case definitions that make the scheme consistent across annotators, and the inter-annotator agreement protocol that produces a reliability figure the methodology document can cite. Written before annotation begins, this is a commitment; produced after, it is a rationalization.
• the-causal-pathway — A four-link chain from intervention to observable label, plus verification predictions — specific, measurable expectations about what the data should show if the causal model is correct. Written before any data arrives. When the data diverges, the divergence is informative rather than explained away.
• the-methodology-document — A replication-ready protocol covering participant criteria, measurement procedure, analysis plan, exclusion criteria, and stopping rule. Every element was settled in the preceding five chapters; this assembles them into a single artifact an independent researcher could execute without further consultation.
• preregistration — Timestamping the hypotheses and analysis plan before data collection begins and depositing that record in a public repository so the sequencing — design before data — is verifiable by anyone. Registry selection, document structure, and the route for independent researchers without institutional affiliation.

A preview

When the researcher chose which outcomes to report, the choice operated the way post-hoc design operates: the methodology fitted itself to the results rather than committing to an expectation first. That is the gap a pre-registered methodology closes. It moves every consequential decision to a point in time when the data cannot influence it, and it creates a record, timestamped and public, that connects the stated hypotheses to the analysis that was planned to test them.

Claude does not make the commitment — it helps the researcher construct one precise enough to survive scrutiny.

Researchers who need a design that holds under review

Anyone running a measurement study who wants every design decision locked before data collection begins — so the methodology is a commitment, not a post-hoc explanation. The sequence applies to any empirical subject: a clinical trial, a behavioral experiment, a market analysis, or an evaluation of a model’s outputs. The pre-registered methodology this book produces is the starting point for the data collection, analysis, and verification work covered in Research Navigator (Book 18) and Writing White Papers (Book 19).

A longer excerpt is available to newsletter subscribers.

Buy Ebook on Amazon