Utts’s 1988 Journal of Parapsychology paper laid the groundwork by tracing the historical entrenchment of p ≤ .05 as the definition of a “successful” experiment in parapsychology and psychology alike.¹ She demonstrated that this convention, rooted in historical practice rather than rational design, systematically disadvantages small-effect literatures: a study with N=50 detecting a d=0.3 effect has roughly 40% power, meaning 60% of exact replications will be “non-significant” even if the effect is perfectly real. The 1991 Statistical Science paper extended this argument with a survey of parapsychological meta-analyses, noting that the National Research Council’s 1988 review had explicitly ignored non-significant studies in its tally, a vote-counting error that Utts identified as methodologically indefensible.² She proposed that parapsychologists calculate power prior to running experiments, use confidence intervals and effect-size estimation alongside hypothesis tests, and consider Bayesian approaches as supplements to frequentist inference.¹

Vote-counting, tallying the number of studies that reach significance versus those that do not, is statistically known to be a biased and inconsistent estimator of the true effect, becoming more misleading as the number of studies increases when the true effect is small. Utts documented this problem explicitly in the context of parapsychology, noting that critics who cited a low percentage of “significant” studies as evidence against psi were applying a method that would also appear to disconfirm many well-established small-effect phenomena in medicine and psychology.² The alternative she advocated, formal meta-analysis computing a weighted mean effect size with confidence intervals, had by 1991 been applied to several parapsychological domains, and she summarized those results in the Statistical Science paper. She also cited the broader methodological casebook literature on meta-analysis as providing the framework for this approach.⁴

Utts applied a consistent effect-size benchmark across sciences: she treated effect sizes in the range of d ≈ 0.2–0.4 as small but scientifically meaningful when replicated across independent laboratories, consistent with standards applied in medical and psychological research. Her 1991 review found that several parapsychological meta-analyses reported effect sizes in this range, leading her to conclude that the aggregate evidence indicated an anomalous effect warranting explanation.² She was explicit that this conclusion was about the data, not about the mechanism: the proposed explanation (psi, broadly construed) remained contested, but the data pattern itself met the same evidentiary threshold she would apply to any other small-effect scientific domain. Her rejoinder to published commentaries on the Statistical Science paper reinforced this data/mechanism distinction in response to critics who conflated her statistical conclusions with metaphysical endorsement.⁵

The file-drawer problem (publication bias: unpublished null results inflate observed effect sizes in meta-analyses) is a specific artifact Utts addressed in her 1991 review. She applied fail-safe N analysis, calculating how many null studies would need to exist in file drawers to reduce the observed aggregate effect to chance, and found that for several parapsychological domains the required number was implausibly large relative to the known research output of the field.² This partially addressed but did not eliminate the publication-bias concern: fail-safe N is a conservative and widely criticized metric, and Utts acknowledged that more sophisticated bias-detection methods (such as funnel-plot regression) were not uniformly applied across the meta-analyses she reviewed. The artifact was therefore mitigated but not fully ruled out at the time of the 1991 paper.

Utts’s 1991 survey covered meta-analyses from multiple parapsychological paradigms available at the time of writing, including ganzfeld telepathy studies, remote viewing experiments, and random number generator (RNG) mind-matter interaction studies. For each domain she reported the aggregate effect size, the number of studies included, and the fail-safe N. She noted that the ganzfeld literature had been the subject of an extended methodological debate, including a specific exchange between Hyman and Honorton, and that a new series of autoganzfeld experiments had been designed specifically to address the methodological criticisms raised in that debate. The autoganzfeld results, available by 1991, showed effect sizes consistent with the earlier ganzfeld meta-analysis, which Utts treated as meaningful convergent evidence given that the new studies had addressed the specific artifact concerns (sensory leakage via physical isolation of sender and receiver; randomization via computer-controlled target selection) raised by critics.²

The 1993 Journal of Parapsychology collaborative paper, co-authored by Utts alongside Krippner, Braud, Child, Palmer, Rao, Schlitz, and White, addressed the role of meta-analysis in synthesizing demonstration research.³ The paper argued that parapsychology had accumulated a substantial body of demonstration-level evidence, consistent small effects across independent laboratories, but had not yet moved systematically to the hypothesis-testing phase in which specific mechanistic predictions are pre-specified and tested. This framing positioned meta-analysis as the appropriate tool for the demonstration phase, while calling for more rigorous pre-specification in future research. The paper also noted that multiple-comparisons inflation (testing many outcome variables without correction) and optional stopping (continuing data collection until significance is reached) were specific artifacts that hypothesis-testing designs needed to address explicitly, and that these artifacts were partially but not uniformly controlled in the existing demonstration literature.

The published discussion of Utts’s 1991 Statistical Science paper included commentary from multiple statisticians and critics. Her rejoinder addressed several specific objections: that the meta-analyses she surveyed were themselves methodologically heterogeneous (she acknowledged this but argued heterogeneity does not nullify aggregate evidence); that the file-drawer problem was more severe than her fail-safe N calculations suggested (she acknowledged the limitation of fail-safe N as a metric); and that her conclusion about an anomalous effect implied endorsement of a psi mechanism (she explicitly rejected this conflation, reiterating that the data claim and the mechanism claim are separable).⁷ The rejoinder also noted that critics who demanded a complete mechanistic explanation before accepting the data were applying a standard not uniformly applied in other sciences, aspirin’s mechanism was not fully understood when its clinical efficacy was accepted.

In a 2022 reflective paper in the Journal of Anomalistics, Utts described how she came to work with the parapsychological community as a statistician, noting that the welcoming nature of the community was a significant factor in her continued engagement.⁸ She held visiting positions at Stanford and SRI International in addition to her tenured positions at UC Davis and UC Irvine, which placed her in proximity to the remote viewing research programs of the 1980s and 1990s. Her engagement with parapsychology was consistently framed as a statistician applying her expertise to a contested empirical domain, not as a researcher with prior belief in psi phenomena.