Jessica M. Utts, PhD Sources:
What Educated Citizens Should Know About Statistics and Probability
This 2003 paper in The American Statistician by Jessica Utts is the anchor citation for her statistical-literacy agenda. It identifies seven specific topics that every student of elementary statistics should learn and understand to function as an educated citizen, grounding each one in the kinds of statistical errors that recur in news coverage, medical advice, and policy debates. The paper was published as part of a special section on statistical literacy that Utts edited for the same journal issue.
Deeper dives — Utts:
Key findings
- The paper identifies seven specific topics that every elementary statistics student should understand to function as an educated citizen, derived from the kinds of statistical errors that recur in media coverage and public policy.1
- Utts frames the seven topics as conceptual reasoning tools rather than as mathematical formulas, prioritizing the recognition of error patterns over computational fluency.1
- Two of the seven topics, the distinction between statistical and practical significance and the non-equivalence of conditional probability and its inverse, map directly onto the most consequential statistical errors documented in clinical and journalistic settings.1
- The paper appeared as part of a special section on statistical literacy that Utts edited, with companion papers from other authors addressing complementary aspects of the literacy agenda.2
- The seven-topic framework has been incorporated into subsequent statistical-education guidelines (including discussions of GAISE revisions) and is a frequent reference point for introductory-course curriculum design.3
Overview
Utts’s 2003 paper grew out of her conviction that introductory statistics courses for non-specialists, the majority of statistics students, who will not become professional statisticians, were systematically misaligned with what those students actually needed to learn. Mathematical mechanics dominated the curriculum; conceptual reasoning about the kinds of statistical claims that arise in daily life received comparatively little attention. The paper’s argument is that the misalignment is fixable, but only if the curriculum is reorganized around a concrete and bounded list of conceptual targets, of which Utts proposes seven.1
Why Seven Topics, Not Six or Ten
The number seven is not theoretically privileged in Utts’s argument; it reflects the set of distinct conceptual failure modes she had documented in two decades of teaching and in her engagement with media misreporting of science. The paper does not claim the seven are exhaustive (other failure modes exist: misunderstandings of regression to the mean, of sampling variability, of multiple-comparison inflation). It claims that these seven are the highest-leverage targets, the ones whose acquisition most reduces the rate of consequential statistical error in citizen reasoning. Subsequent literature has occasionally proposed alternative or supplementary lists; Utts’s seven remain the most-cited single framework.1
The Seven Topics
Utts’s 2003 paper identifies the following seven topics that every educated citizen should understand:1 (1) when a cause-and-effect relationship can be inferred from data, and when it cannot; (2) the difference between statistical significance and practical significance; (3) the difference between not finding an effect and the power of the study, that is, the difference between absence of evidence and evidence of absence; (4) bias that can occur in surveys and sampling; (5) understanding that coincidences are not as coincidental as they appear in large samples; (6) the non-equivalence of conditional probability and its inverse (the prosecutor’s fallacy, the base-rate fallacy); and (7) knowing that “normal” in statistical terms is not equivalent to “average” or to “typical.”
Topics 1–3: Causation, Significance, and Power
The first three topics form a tightly coupled cluster about the inferential structure of empirical research. Topic 1 addresses the inference from observed correlation to causal mechanism, including the role of randomized assignment, confounding, and the limits of observational data. Topic 2 addresses what a small p-value does and does not tell us about effect magnitude. Topic 3 addresses what a non-significant result means in a study of given sample size, the Type-II concern that runs through Utts’s parapsychology work as well as her general literacy writing. The coupling reflects Utts’s broader methodological argument: the misuse of significance testing as a binary verdict is the single most consequential statistical error in the empirical sciences, and citizens who can recognize the error in news coverage are partially insulated from the policy and clinical consequences of it.1
Topics 4–5: Survey Bias and the Reality of Coincidence
Topic 4 addresses systematic bias in survey methodology: selection effects, non-response bias, question wording, and the difference between probability and convenience samples. Utts frames this as a literacy topic because survey results saturate news coverage and policy debate, and recognizing methodological flaws in a poll’s design is a higher-leverage skill than computing its margin of error. Topic 5 addresses the psychology of coincidence: in large samples, rare events are common, and the failure to apply this elementary observation drives confused reasoning about everything from cancer clusters to apparent psychic experiences to lottery patterns. Utts had documented the coincidence-misunderstanding pattern across multiple publications; the 2003 paper formalizes it as a citizen-literacy target.1
Topics 6–7: Conditional Probability and “Normal”
Topic 6 addresses the non-equivalence of P(A|B) and P(B|A). The base-rate fallacy in clinical screening, the prosecutor’s fallacy in courtroom statistics, and many media misrepresentations of risk all reduce to a failure to distinguish these two conditional probabilities. Utts treats this as the single statistical concept with the highest expected harm-per-misunderstanding in adult citizen life. Topic 7 addresses the colloquial-vs-statistical meaning of “normal”: in statistical usage, “normal” describes a distribution shape (the Gaussian/bell curve), not a moral or social judgment about typicality. The conflation of statistical and social senses of “normal” appears in news coverage about everything from clinical reference ranges to standardized test scores.1
Pedagogical Design and Curriculum Implications
The 2003 paper does not stop at identifying the seven topics; it argues for a corresponding pedagogical reorganization. Each topic should be introduced through concrete examples drawn from news coverage, clinical practice, or policy debate, not through abstract mathematical exercises. Computational fluency should serve conceptual understanding rather than substitute for it. Real data, drawn from contemporary sources the students recognize, should anchor each topic rather than constructed textbook examples.1
The “Future Doctor” Specialization (Baldi & Utts 2015)
The seven-topic framework was subsequently extended into a specialized curriculum proposal for future physicians in a 2015 paper by Baldi and Utts in The American Statistician, titled “What Your Future Doctor Should Know About Statistics: Must-Include Topics for Introductory Undergraduate Biostatistics.”4 The 2015 paper preserves the conceptual orientation of the 2003 framework but adds clinical-specific competencies: interpretation of sensitivity, specificity, and predictive value of diagnostic tests; the distinction between relative and absolute risk reduction in clinical trials; and the communication of uncertainty to patients. The two papers together establish a layered literacy framework: the seven citizen-level topics are the floor, and the clinical-specific extensions are what physicians additionally need on top.
Modern Context
The statistical-literacy program Utts’s 2003 essay outlined — what every educated citizen should know about statistics — connects directly to the foundational ASA program articulated by Katherine Wallman in her 1992 ASA presidential address “Enhancing Statistical Literacy: Enriching Our Society” (Journal of the American Statistical Association, 88(421), 1–8; 10.1080/01621459.1993.10594283), which framed statistical literacy as a civic capacity rather than a technical specialty. Garfield and Ben-Zvi’s Developing Students’ Statistical Reasoning: Connecting Research and Teaching Practice (Springer, 2008; 10.1007/978-1-4020-8383-9) provides the post-2000 research synthesis on how students develop the reasoning skills Utts’s seven-topic framework targets. Cobb’s 2007 critique of computational-procedure-centered introductory curricula (Technology Innovations in Statistics Education, 1(1); 10.5070/T511000028) provides the curriculum-design framing that has shaped subsequent reform.
Discussion: Why These Seven
The argument Utts makes for these seven topics, rather than a longer or shorter list, is pragmatic rather than theoretical. The seven represent the conceptual targets whose acquisition would, in her judgment, most reduce the rate of consequential statistical error in adult citizen life. The paper does not claim that other targets are unimportant; it claims that an introductory course attempting to teach everything teaches nothing well, and that prioritization is the appropriate response to time and attention constraints. Each of the seven maps to documented failure modes in media coverage, clinical practice, or policy reasoning.1
The Paper’s Place in Utts’s Broader Agenda
The 2003 paper sits at the center of Utts’s statistical-literacy agenda, which spans from her 1997 introductory textbook Seeing Through Statistics through her 2010 paper on media misrepresentation, her 2015 historical review of 175 years of statistics-education themes, her 2016 ASA Presidential Address, and her 2022 paper on integrating ethics into GAISE. The 2003 paper is the most-cited single anchor of this agenda; subsequent papers extend and refine the framework but consistently return to the seven topics as the literacy floor.1
Skeptical Critiques and Discussion
Critique 1: Citizen-literacy frameworks address symptoms rather than the upstream problem of how research is generated and reported
Skeptic source: A recurring critique of citizen-literacy frameworks, including Utts’s seven-topic list, is that the locus of statistical misuse is not the general public but scientists, journal editors, and journalists who misapply significance testing, and that teaching citizens to recognize errors does not address the upstream conditions that produce them.2
Response: Utts’s broader publication record addresses this critique directly. Her 2010 paper on media misrepresentation and her 2016 ASA Presidential Address engage the producer-side of the problem, and her 2022 paper on integrating ethics into GAISE addresses the training of future researchers, not just citizens. The 2003 paper itself is explicit that citizen literacy is a necessary but not sufficient component of a broader reform agenda. The argument is that a statistically literate public creates demand for better reporting and better science, while a statistically illiterate public provides no corrective check.1
Analysis. The critique identifies a real limitation; Utts’s response, that citizen literacy is necessary but not sufficient, addresses it but does not eliminate the residual question of which intervention most efficiently reduces statistical misuse in research and reporting. That empirical question remains open.
Critique 2: The seven-topic framework risks substituting one canonical list for engagement with the actual diversity of statistical reasoning
Skeptic source: A methodological concern about any bounded list of “essential” topics is that it can become canon, treated as exhaustive, and used to license neglect of conceptual targets the list omits (regression to the mean, sampling variability, multiple-comparison inflation, model-specification uncertainty, the distinction between exploratory and confirmatory analysis).
Response: The 2003 paper does not claim the seven are exhaustive; it claims they are the highest-leverage targets. Utts’s subsequent papers, particularly the 2015 historical review and the 2015 future-doctor paper, explicitly extend the framework with additional topics where clinical or scientific application demands them. The framework is best read as a literacy floor, not as a ceiling. The risk that downstream curricula treat the list as canon and stop extending it is a real concern, but it is a concern about reception rather than about the original argument.14
Analysis. Whether the seven-topic framework has been used as a floor or as a ceiling in actual introductory curricula is an empirical question on which the existing evaluation literature is mixed.
References
- Utts, J. (2003). What Educated Citizens Should Know About Statistics and Probability. The American Statistician, 57(2), 74–79. https://doi.org/10.1198/0003130031630 R001 [Utts 2003] ↩︎
- Utts, J. (2003). Special Section on Statistical Literacy. The American Statistician, 57(2), 73. https://doi.org/10.1198/0003130031658 R002 [Utts 2003] ↩︎
- Raman, R., Utts, J., Cohen, A. I., & Hayat, M. J. (2022). Integrating Ethics into the Guidelines for Assessment and Instruction in Statistics Education (GAISE). The American Statistician, 77(3), 323–330. https://doi.org/10.1080/00031305.2022.2156612 R003 [Raman 2022] ↩︎
- Baldi, B., & Utts, J. (2015). What Your Future Doctor Should Know About Statistics: Must-Include Topics for Introductory Undergraduate Biostatistics. The American Statistician, 69(3), 231–240. https://doi.org/10.1080/00031305.2015.1048903 R004 [Baldi 2015] ↩︎
Deeper dives — Utts:
See hub COI disclosure for subject-coauthorship transparency.