The statistical argument for replication in psi research is correct but incomplete. Psi research studies people, not rocks — and behavioral science has principles about how phenomena distribute across populations, how measurement designs match those distributions, and how subject characteristics modulate effect sizes. The statistical argument by itself doesn’t engage these. This page does.

Deeper dives — Replication in Psi Research:

1. The Leonardo problem and the shoe-size problem

Suppose you wanted to study extreme creativity. To do this, you recruit 25,000 random adults from a general population, hand each one a brush and 30 seconds, and score their attempts against a panel of four reference images — one of which is a fragment of the Sistine Chapel ceiling. The mean hit rate against the Sistine Chapel target will be approximately 25% (chance, given the four-target panel). You would conclude that extreme creativity is not real.

This conclusion would be wrong, but the design — not the conclusion — is where the error lives. Extreme creativity is a trait-distributed phenomenon: it is concentrated in a small fraction of the population and is moderated by training, motivation, context, and inherited capacities. Averaged across random samples it produces small mean effects. The high end of the distribution — Leonardo, Michelangelo, Caravaggio — does not disappear because the population average is near zero.

The same logic applies to shoe size in a different way. If you study average adult shoe size with mean and standard deviation, you describe the central tendency well, but you do not detect the question “do some people have size-22 feet?” That question has a definite yes answer (Robert Wadlow had size 37; Shaquille O’Neal wears size 23) that the mean-and-SD description does not capture. The tail of a distribution is not summarized by its center.

If psi is a trait-distributed phenomenon — and the parapsychology literature has internal evidence that it is — random-population designs estimate the population mean, which for trait-distributed phenomena is small and close to chance. Random-population designs underestimate the existence of the phenomenon in the same systematic way that the creativity study underestimates Leonardo.

2. The four-predictor framework from Honorton’s autoganzfeld

The strongest internal evidence for trait distribution in the parapsychology literature comes from Bem and Honorton’s 1994 Psychological Bulletin autoganzfeld replication. They identified four pre-experimental subject characteristics that consistently predict ganzfeld performance across the autoganzfeld database (n = 322 subjects across 11 series):

1. Extraversion — measured by standard personality inventories. Bem & Honorton (1994)^[1] report r = .18, t(216) = 2.67, p = .004 (scores available for 218 of 240 autoganzfeld subjects) for the correlation between extraversion and direct-hit performance. Extraversion is one of the Big Five personality traits and the correlation is small but mainstream-statistically robust.
2. Creative or artistic background — subjects with formal training in performing arts, music, visual arts, or drama outperformed unselected subjects. Honorton’s autoganzfeld participant pool was drawn substantially from creative-arts and music-conservatory populations for this reason.
3. Prior psi experience — subjects reporting prior spontaneous psi-type experiences (precognitive dreams, telepathic episodes, etc.) outperformed subjects who did not.
4. Belief in psi — subjects scoring high on belief inventories outperformed low-belief subjects. This is the formal expression of the sheep-goat effect (Section 3 below).

These are not post hoc subgroups discovered after the data was collected. They were pre-specified moderators based on the prior ESP-personality literature dating to Schmeidler (1958). They were the basis for participant selection in Honorton’s autoganzfeld program at Psychophysical Research Laboratories (PRL). The high effect sizes in the autoganzfeld are not accidental — they reflect deliberate selection on documented predictors.

Direct empirical confirmation from a later mainstream-journal meta-analysis: Storm, Tressoldi & Di Risio (2010), in their Psychological Bulletin meta-analysis of free-response studies 1992-2008, found explicitly that “selected participants (believers in the paranormal, meditators, etc.) had a performance advantage over unselected participants, but only if they were in the ganzfeld condition.” This is the trait-distribution + state-dependence interaction not as theoretical prediction but as empirical finding from a published meta-analysis: subject selection on the documented predictors matters, and it matters specifically in the state-eliciting protocol. The interaction is exactly what the framework predicts.

3. The sheep-goat effect: belief moderates performance

The most-replicated personality moderator in parapsychology is the sheep-goat effect. Schmeidler and McConnell’s 1958 book^[2] ESP and Personality Patterns documented it first: subjects who believed psi was possible (“sheep”) consistently outperformed subjects who did not (“goats”) on the same experimental protocols, with both groups drawn from the same population and run by the same experimenters.

Lawrence (1993)^[3] meta-analyzed 73 sheep-goat studies conducted between 1947 and 1993, with a combined n of approximately 4,500 subjects: combined r = .029, Stouffer Z = 8.17, p = 1.33 × 10⁻¹⁶. The effect is small, but it is one of the most robustly replicated psychological-moderator findings in parapsychology — and small effect sizes across thousands of subjects in 73 studies cannot easily be explained as artifact-only without a specific theory of how belief produces the artifact.

The skeptical reading: belief produces expectancy effects via experimenter cuing, scoring decisions, or psychological set; the effect reflects expectancy artifact, not psi. The proponent reading: belief modulates the trait-distributed sensitivity, the way performance anxiety modulates athletic or musical performance. Both readings agree on one operational consequence: random-population designs that don’t measure or stratify by belief are averaging across a moderator that produces real performance differences. This is a measurement-design issue independent of which interpretation is correct.

4. Tellegen Absorption and other personality predictors

Beyond the four Bem-Honorton predictors and sheep-goat, the parapsychology literature has identified additional personality correlates of ESP performance:

• Tellegen Absorption Scale (Tellegen & Atkinson 1974^[4], Journal of Abnormal Psychology) — measures capacity for becoming absorbed in experiences. High absorbers show vivid imagery, deep involvement in fiction or music, and capacity for altered-state experiences. The Absorption Scale was developed as part of hypnotic-susceptibility research; it correlates substantially with hypnotizability. Multiple ganzfeld and remote-viewing studies report positive correlations between absorption scores and psi-task performance (see, e.g., Stanford & Angelini 1984; Palmer 1997; Schmeidler 1988 for representative findings — Tellegen-Atkinson 1974 itself only establishes the absorption ↔ hypnotic-susceptibility link). Absorption is psychometrically more defensible than personality-type classifications like Myers-Briggs, which has well-documented test-retest reliability problems.
• Hypnotic susceptibility — independent of absorption, hypnotizability correlates with ESP performance in several studies (Stanford 1984^[5]). The mechanism is not assumed to be hypnotic-state-specific; the inference is that the same personality characteristics that enable hypnotic experience enable the relaxed, attention-flexible state ganzfeld protocols are designed to elicit.
• Meditation experience — practiced meditators outperform non-meditators on multiple parapsychology paradigms (Roney-Dougal & Solfvin 2011^[6], among others).

None of these correlations is dramatic in effect size. The pattern across them is what matters: multiple independently-measured personality traits, each well-grounded in mainstream behavioral science, predict ESP performance in directions consistent with each other. A trait-distributed sensitivity to ganzfeld-type tasks — modulated by belief, openness, absorption, hypnotic susceptibility, meditation experience — is exactly the multi-predictor profile mainstream individual-differences psychology would describe.

5. State dependence: the phenomenon may require specific conditions

The trait-distributed argument explains one dimension of variance: who tends to show effects. State-dependence explains another: under what conditions. Many cognitive and perceptual phenomena are state-dependent — they appear reliably in some states and rarely or never in others. REM-only phenomena (dream content, sleep paralysis) are obvious examples. State-dependent learning and state-dependent retrieval are documented across mainstream cognitive psychology. The phenomenon doesn’t disappear when the relevant state is absent; it is simply not measurable.

Honorton’s altered-states meta-analysis^[7] (1977, in Wolman’s Handbook of Parapsychology) reviewed comparisons between studies conducted in altered states — ganzfeld sensory restriction, hypnosis, meditation, dream states — and standard waking-state conditions. The pattern across approximately 90 studies: altered-state designs produced effect sizes 2 to 5 times larger than waking-state designs run with comparable subject pools and protocols. Cardeña’s (2018)^[8] American Psychologist review confirms the persistence of this pattern in the post-1990 literature.

The implication for replication interpretation: protocols that don’t elicit the relevant psychological states are not testing the phenomenon under conditions where it has been documented to appear. A standard cognition-task design with no relaxation induction, no perceptual restriction, no participant pre-screening — the kind of design that produces clean Bayesian analysis but doesn’t activate the moderators the literature has identified — is not a fair test of phenomenon-existence. It is a fair test of “does psi appear unmoderated and unconditional in random subjects under non-eliciting conditions,” which the parapsychology literature itself doesn’t claim.

6. What personality correlates rule out (and what they do not)

Sections 2 through 5 above documented multiple personality and trait correlates of psi-task performance: extraversion (Bem & Honorton 1994, r = .181, p < .001), sheep-goat belief (Lawrence 1993 meta-analysis, r = .029, k = 73, Stouffer Z = 6.4), Tellegen Absorption, hypnotic susceptibility, meditation experience, and creative-arts background. These correlates have been replicated across decades, multiple paradigms, and many laboratories.

This pattern has a non-trivial implication for theories of what psi is. If psi were purely an artifact — sensory leakage, equipment bias, scoring bias, file-drawer effects — there is no principled reason the artifact should correlate with personality. Artifacts have method-correlates: the equipment used, the experimenter running the session, the protocol details, the scoring procedure. Pure artifacts do not have predictable subject-personality correlates. Yet the published parapsychology record has predictable subject-personality correlates that replicate across method changes. That is a constraint on what kind of theory can explain the data.

The remaining artifact-compatible move is to claim the personality correlates themselves reflect artifact-producing behaviors. The argument: believers in psi are more motivated, pay more sustained attention, are more compliant with implicit experimenter cues, give more confident scoring decisions, are more cooperative with experimental protocols. That produces apparent effects that correlate with belief and personality — not psi, but artifact in personality clothing. This is not an unreasonable hypothesis; it is the strongest artifact-compatible explanation of the personality-correlate data.

But this hypothesis has its own empirical burden. It requires that the same pattern of belief-mediated, attention-mediated, motivation-mediated artifact-producing behaviors operates consistently:

• Across decades of methodological change (early non-blind designs through modern automated double-blind protocols);
• Across multiple paradigms with very different procedural structures (ganzfeld vs. forced-choice precognition vs. RNG/PK);
• Across many laboratories with different experimenters, equipment, and protocols;
• And specifically under automated protocols where the experimenter cannot influence scoring after the trial because the scoring is computer-generated against pre-specified target sets.

The skeptical literature has not, to the editorial author’s knowledge, produced a worked-out behavioral theory of how sheep-goat-style and absorption-correlated artifacts could be generated under post-1986 Joint Communique protocols where experimenter scoring discretion is procedurally eliminated. The hypothesis remains a candidate explanation; the supporting evidence for it as a comprehensive account of the personality-correlate pattern is incomplete.

The skeptical literature’s recent prominent engagement helps clarify what the published case for and against psi actually is. Reber & Alcock (2020)^[9] in American Psychologist — the most-recent major skeptical paper in a mainstream-journal venue — explicitly shifts the skeptical case away from methodological critique and toward a prior-probability argument: psi phenomena, if real, would violate established physical, biological, and information-theoretic principles, so the prior probability is so low that no behavioral-science evidence can overturn it. Their conclusion: positive psi results must be artifact, fraud, or chance, regardless of methodology. A peer-reviewed response from Williams (2019)^[10] in Journal of Scientific Exploration argues that the Reber-Alcock argument is structurally circular — the prior-probability premise itself requires empirical justification, and selectively-strict priors for one domain (psi) but not others (mainstream subtle-effects research with comparable effect sizes) is a methodological choice that needs defending. French (2018)^[11], in the MIT Press Pseudoscience volume, takes a different position: he implicitly accepts the personality-correlate empirical pattern and reorients the research program from “does psi exist?” to “why do people believe in psi?” — treating the correlates as belief-explanatory data rather than psi-evidential data. Alcock’s earlier book (1981)^[12] remains a substantive methodological critique, but predates the modern automated double-blind protocols.

What is striking about this body of recent prominent skeptical literature: it does not produce a worked-out behavioral theory of how belief-mediated artifact-producing behaviors generate the personality-correlate pattern under post-1986 automated protocols. Reber-Alcock skip the question by appeal to prior probability. French concedes the data and reframes the question. Alcock’s 1981 critique predates the relevant protocols. The skeptical-compatible explanation of the personality-correlate pattern remains underspecified in the published literature, not because skeptics haven’t engaged the field, but because the recent skeptical engagement has been philosophical (prior probability) or interpretive (study belief, not psi) rather than mechanistic (specify how the artifact is generated). This is the gap. It does not prove psi is real. It does mean that any reader trying to determine whether the personality-correlate pattern reflects phenomenon or artifact does not find a complete account on either side.

The complementary research direction — experimenter psi as a documented phenomenon needing its own theoretical account — has been engaged in the recent parapsychology methods literature. Kruth (2022)^[13] synthesizes the experimenter-psi tradition in a Journal of Parapsychology editorial; Graff (2023)^[14] and Drucker (2023)^[15] offer follow-on engagement. Braud’s 1975 “Psi-Conducive States”^[16] is the foundational reference. The field has not converged on whether experimenter effects represent a methodological problem (artifact-mediated) or a real moderator of a real phenomenon (skill-mediated), but the documentation that experimenter is a non-trivial source of variance is consistent across decades. The interpretation question is open; the empirical observation is not.

What this means for editorial reading: the personality-correlate data is one of the strongest constraints the published parapsychology record places on artifact theories. Pure-method-artifact accounts have to explain why method changes do not eliminate the personality correlates. The available skeptical-compatible explanation requires a specific theory of belief-mediated artifact generation under blind and double-blind protocols, and that theory remains underspecified. This is not a proof that psi is real. It is an empirical constraint that any complete account — proponent or skeptic — has to honor.

7. Implications for effect-size interpretation

The trait-distributed, state-dependent framing changes how to read the small mean effect sizes the hub reports. Three specific implications:

First, small mean effect sizes in unselected-sample meta-analyses are floor estimates of a trait-distributed phenomenon, not phenomenon-absence proofs. The autoganzfeld effect size (Cohen’s h = 0.20) is computed across subjects deliberately selected on the four-predictor framework, in a state-eliciting protocol (ganzfeld sensory restriction), with experimenters who scored positively on belief and rapport measures. Unselected-subject, non-state-eliciting designs would, on the trait-distribution model, produce smaller effects than the autoganzfeld — and they do (post-1990 broader ganzfeld replication: Milton & Wiseman 1999^[17], Stouffer Z = 0.70, mean d = 0.013). This pattern is what the trait-distribution model predicts. The post-Joint-Communique “broader independent replication” the field requested in 1986 was, in practice, both broader-investigator AND less-selected-subjects. The 30 studies in Milton & Wiseman’s meta-analysis varied in selection profile, but many used standard undergraduate volunteer pools without the four-predictor screening Honorton’s PRL used. The Ritchie/Wiseman/French (2012) preregistered Bem replications similarly used university student samples without explicit selection on Bem’s original predictors. The model’s prediction was confirmed empirically; whether that empirical confirmation supports the model or undermines the phenomenon claim is the active interpretive disagreement.

Second, the experimenter-effect pattern (Wiseman & Schlitz 1997^[18] and 2006 follow-ups) is exactly what the trait-and-state-dependence model predicts. If experimenter rapport, belief, and skill in eliciting relaxed-attentive participant states matters for trait-activated performance, then the same protocol run by different experimenters will produce different effect sizes — and it does. The skeptical reading (experimenter expectancy via subtle cues) and the proponent reading (skill-modulated phenomenon) both require what the data shows: experimenter is not a nuisance variable to be statistically controlled away. It is part of the experimental design that needs careful documentation, ideally via adversarial collaboration in the Hyman-Honorton tradition.

This has a non-trivial consequence for evaluating specific replication-failure results. Consider the well-cited Ritchie, Wiseman & French (2012) preregistered replication of Bem (2011): three independent labs, high statistical power (99.92% combined for Bem’s original effect size), preregistered protocols, open data — structural-credibility features that are genuine and not in dispute. The replications returned combined p = 0.83 (null). What the framework on this page predicts about this result: given the page’s own moderators — documented experimenter belief-and-rapport effects (Wiseman & Schlitz 1997), trait distribution requiring subject selection, and the use of unselected university student samples by Ritchie/Wiseman/French — the null result is what the framework forecasts, not what falsifies it. If experimenter belief profile and population selection are real moderators (which the page has argued at length they are), then a skeptic-led replication with unselected samples should produce null results regardless of whether the underlying phenomenon is real. The Ritchie-Wiseman-French (2012) data is consistent with the framework being correct; it would also be consistent with the framework being incorrect and Bem (2011) being an artifact. The data does not distinguish between these. That is the methodologically honest interpretation, and it is independent of any view about any individual researcher’s integrity.

Third, the standards-of-evidence question is fairly framed by the integrated argument, not by the statistical argument alone. A skeptic who reads only the statistical part of the case can respond: “I accept your vote-counting critique, but the small effect sizes still don’t move me toward psi.” That response is incomplete because it doesn’t engage the trait-distribution / state-dependence framework that explains why the effect sizes are small. The integrated response: “the effect sizes are small because the standard designs are floor-estimating a trait-distributed phenomenon. The literature reports much larger effects in selected-subject, state-eliciting protocols. If you want to test phenomenon-existence rigorously, you need to test it in the conditions under which it has been documented to appear.”

8. Why this is not special pleading

A reader might worry that the argument above is “psi research wants special rules.” It is not — and the distinction matters.

The behavioral-science principles invoked here — trait distribution, state dependence, moderator effects, individual-differences psychology — are foundational to behavioral science and are applied routinely to mainstream phenomena. Creativity researchers select for creative-trait subjects. Hypnotic-susceptibility researchers stratify by hypnotic capacity. Meditation researchers select for practice. Memory researchers manipulate encoding state to demonstrate state-dependent retrieval. Sport psychologists study elite athletes, not random adults. These are not special accommodations; they are standard methodology for studying trait-distributed and state-dependent phenomena.

What this page argues is that psi research is a behavioral-science research domain and should be evaluated by behavioral-science standards. Insisting that random subjects under non-eliciting conditions must show robust effects to validate phenomenon-existence is not “high methodological standards.” It is a category error — applying the methodology of rock-physics (which doesn’t have trait-distributed responders or state-dependent moderation) to behavioral science (which does).

The legitimate skeptical demand — and it is legitimate — is for independent replication of the trait-selected, state-eliciting protocols by skeptic-led laboratories. That demand is what the post-1986 Joint Communique standards specify and what the Milton-Wiseman 1999 broader-replication meta-analysis attempted. The fact that broader replication has not consistently confirmed autoganzfeld magnitudes is the genuine unresolved evidence on the question of psi existence. That unresolved evidence is the right place for skeptic-proponent disagreement to live — not in misapplied methodology that asks the wrong question.

9. What this page establishes and what it does not

This page establishes:

1. Psi research is behavioral science and is properly evaluated by behavioral-science methodological principles, including trait distribution, state dependence, and individual-differences psychology.
2. Multiple personality and trait predictors of ESP performance have been identified in the published parapsychology literature (Bem & Honorton 1994 four-predictor framework; sheep-goat meta-analysis Lawrence 1993; Tellegen Absorption; hypnotic susceptibility; meditation experience). These are well-grounded in mainstream individual-differences psychology.
3. Standard “unselected subjects under non-eliciting conditions” designs estimate floor effects for trait-distributed, state-dependent phenomena. The small effect sizes reported in such designs are consistent with phenomenon-existence under trait-and-state moderation; they are not consistent with phenomenon-absence.
4. The experimenter-effect pattern (Wiseman & Schlitz 1997, 2006) is what trait-and-state moderation predicts when experimenter rapport and belief modulate participant psychological states.

This page does NOT establish:

1. That psi exists. The behavioral-science framing strengthens the case that the evidential question is open and that standard skeptical dismissals are incomplete; it does not resolve the question.
2. That trait-selected and state-eliciting protocols reliably replicate at the autoganzfeld effect-size magnitudes when conducted by skeptic-led independent laboratories. Milton & Wiseman (1999) and the post-Bem preregistered failures (Ritchie, Wiseman & French 2012^[19]) are the principal counter-evidence; the post-1999 replication record remains contested.
3. That every parapsychology research program has rigorously applied the four-predictor framework or its analogs. Many older studies did not stratify or select on these predictors and produced correspondingly noisier data.

The integrated argument — statistical + behavioral-science — does not prove psi. It does establish that the question is open under a methodology fair to the kind of phenomenon being studied. If psi is real, dismissing it via methodology that doesn’t match the phenomenon’s distribution and state-dependence profile is the failure mode this page exists to document.

Related topics on ESP-Nexus

Replication in Psi Research | Measurement and Methodology | Ganzfeld | Autoganzfeld | Dean I. Radin | Marilyn J. Schlitz