Questions to Ask a Researcher
Questions to Ask a Researcher
Rigorous prompts to understand research design, limitations, practical implications, and how to read results without hype.
1What real-world problem does this research aim to inform?
What real-world problem does this research aim to inform?
Anchors theory to practical impact and stakeholders.
2How was the study designed and why that method?
How was the study designed and why that method?
Connects design choice to validity and constraints.
3What are the key assumptions—and how did you test them?
What are the key assumptions—and how did you test them?
Surfaces hidden dependencies that can break findings.
4What sources of bias were most concerning?
What sources of bias were most concerning?
Improves critical reading of sampling and measurement.
5How big are the effects—and are they practically meaningful?
How big are the effects—and are they practically meaningful?
Distinguishes statistical from practical significance.
6What do the confidence intervals tell us?
What do the confidence intervals tell us?
Encourages uncertainty-aware interpretation.
7How might results differ in other populations or contexts?
How might results differ in other populations or contexts?
Tests generalizability and boundary conditions.
8What replication or follow-up work is most important?
What replication or follow-up work is most important?
Promotes scientific humility and next steps.
9What’s the most honest critique of your own study?
What’s the most honest critique of your own study?
Invites transparency and improves trust.
10How should practitioners apply (or not apply) these findings?
How should practitioners apply (or not apply) these findings?
Translates research into action with guardrails.
11What data or code are available for review?
What data or code are available for review?
Checks openness, reproducibility, and standards.
12How did you handle missing data and outliers?
How did you handle missing data and outliers?
Clarifies data hygiene and robustness.
13Were there preregistered hypotheses or analysis plans?
Were there preregistered hypotheses or analysis plans?
Reduces p-hacking and hindsight bias.
14Who funded the work—and any conflicts of interest?
Who funded the work—and any conflicts of interest?
Adds context for potential incentives or pressure.
15Which figure or table should a busy reader focus on?
Which figure or table should a busy reader focus on?
Directs attention to decision-relevant evidence.
16What’s the biggest way media might misinterpret this?
What’s the biggest way media might misinterpret this?
Prepares for clear, responsible communication.
17What related work should I read next?
What related work should I read next?
Builds a curated path through the literature.
18If a practitioner has one hour, what’s the best use of it?
If a practitioner has one hour, what’s the best use of it?
Bridges research and practice efficiently.
19What would you change if you could rerun the study?
What would you change if you could rerun the study?
Encourages iteration and better future designs.
20What open question are you most excited about now?
What open question are you most excited about now?
Ends with curiosity and future direction.
Want to learn more?
Understanding Research Without Hype
Want to learn more?
Understanding Research Without Hype
Ask About Limits and Use
Start With Design
Method and sampling shape everything downstream.
Quantify Uncertainty
Intervals tell you what’s known versus guessed.
Mind External Validity
Context determines whether results carry over.
Reading Checklist
Five Checks
Common Pitfalls
P-Value Worship
Look for effect sizes and intervals, not just p<0.05.
Overgeneralizing
Small or narrow samples rarely justify sweeping claims.