Introduction to Research Design

• Aims to address big and smaller questions • To generate answers with external validity • To create strong designs that yield accurate, unbiased and relevant answers

Model, Inquiry, Data strategy and Answer strategy

Model and Inquiry

Data strategy and Answer Strategy

Represents our understanding of how the world works. - Speculations about causal relationships, correlations and sequences of events. - It defines units of study, conditions and outcomes.

Inquiry formulates research questions within the context of our model. Can be causal or descriptive.

Part of the Inquiry: The estimand represents the theoretical answer to the Inquiry (research question derived from our model)

Data strategy focuses on measuring and operationalizing elements of our model and inquiry. - Involves selecting units (sample vs population) - Conditions (observational vs experimental) - Measuring outcomes

Statistical or qualitative methods to summarise and interpret data (cleaning, transforming, estimating data.)

• Researchers use the MIDA framework to select research designs • Researchers can assess the dependencies across design elements by declaring each element (M,I,D,A) • Researchers are encouraged to consider real world implications of their design and adjust their strategies accordingly

Declaring a design involves identifying which components belong in M, I, D, and A of the MIDA framework.

Mapping ideas into MIDA isn’t always straightforward but is rewarding.

Designs can be declared in words or code, with statistical programming languages like R with the ‘DeclareDesign’ software.

Expressing a design in terms of MIDA allows for a deeper understanding of its properties

Diagnosis involves simulating a research design to understand potential outcomes and properties.

Simulation helps assess features like statistical power, biases, and error rates.

Summary statistics from a single run of a design, e.g., error or significance.

Summaries of diagnostic statistics across multiple simulations, e.g., bias or power.

Articulating what constitutes success and evaluating the likelihood of achieving it through simulation.

Risks include incorrect diagnoses and reliance on a narrow set of diagnosands.

Redesigning involves fine-tuning data and answer strategies to improve diagnosands.

Adjusting sample size, data collection methods, or analysis strategies to enhance design properties.

Redesign considers ethical, logistical, and financial limitations.

• Design holistically • Design agnostically • Design for purpose • Design early • Design often • Design to share