Honorton: Experimenter effects and subject selection

Deeper dives, Honorton:

Back to hub Ganzfeld Methodology Autoganzfeld System Altered States Dream Telepathy Bem-Honorton Replication Hyman-Honorton Communiqué

Key findings

Honorton studied the experimenter as an explicit independent variable in ESP work, rather than treating the experimenter as a neutral fixture.¹
His forced-choice precognition meta-analysis identified subject selection on prior performance as one of four moderators significantly covarying with study outcome.²
A meta-analysis of extraversion and ESP reported a reliable association between trait extraversion and free-response ESP scoring, with a fresh confirmation dataset.³
PRL first-timer studies asked whether inexperienced, unscreened participants could replicate ganzfeld success without extensive selection.⁴
The autoganzfeld system was built to remove experimenter handling of targets and judging, reducing the experimenter’s role to a target-blind monitor.⁵

Overview

Experimenter effects and subject selection are among the field’s most cited Type-II vulnerabilities: ganzfeld and free-response ESP effects are typically small, state-dependent, and sensitive to who is tested and who runs the session, so a single null run by one experimenter with unscreened participants does not by itself refute the paradigm, nor does a strong run by a charismatic experimenter with hand-picked subjects confirm it. Honorton approached both factors directly, measuring the experimenter as a variable and treating selection criteria as moderators whose effects could be estimated rather than assumed.¹ His later automation work was, in part, an effort to hold the experimenter’s behavior constant so that subject-level variation could be read more cleanly.⁵

Why Selection and Experimenter Variables Matter for Replication

Honorton framed replicability itself as bound up with experimenter influence and the conditions reliably associated with scoring, arguing that the study of replicability in parapsychology was inseparable from identifying the moderating conditions, including who conducts the test, under which effects appear.⁶ His methodological essay on improving replicability cast the problem as one of specifying and stabilizing the conditions of success rather than expecting uniform results across heterogeneous experimenters and samples.⁷

Experimenter effects as an explicit variable

Honorton, Ramsay and Cabibbo set out to measure whether the experimenter’s manner with the subject changed ESP scoring, treating the experimenter’s interpersonal stance as a manipulated condition rather than an uncontrolled nuisance.¹ The interest was not merely whether different experimenters obtained different rates, but whether something about the experimenter–subject relationship was itself a determinant of scoring direction and magnitude.¹

This concern carried forward into the design philosophy of the automated ganzfeld, where the explicit goal was to reduce opportunities for human error and to let a single target-blind experimenter conduct sessions without handling targets or judging, collapsing several experimenter-mediated pathways at once.⁵

From Manipulated Experimenter to Removed Experimenter

The autoganzfeld put target selection, target presentation, the receiver’s blind judging, subject feedback, and data recording under computer control, leaving the experimenter as a monitor rather than a handler.⁵ The receiver sat in an IAC sound-isolation room while the sender occupied an RF-shielded, sound-attenuated room about 15 feet away across the experimenter’s monitoring room, with two physically separate, electrically isolated audio systems.⁵ A later description of the Edinburgh implementation emphasized that the computer-based system provided automatic data recording and shielding against sensory cues with resistance to both subject bias and intentional experimenter bias.⁸

Subject selection and pre-screening

A recurring strategic question in Honorton’s program was whether ganzfeld success required talented, pre-selected participants or could be obtained with ordinary, unscreened volunteers, a question with direct bearing on whether the paradigm was portable to other laboratories.⁴ His PRL “first-timer” series tested inexperienced novices contributing a single session each, precisely to see how much screening replicators would need.⁴

The data spoke against the necessity of heavy pre-screening: the report noted that studies with inexperienced participants had nonetheless yielded both significant overall psi performance and a significant replication rate, which it framed as encouraging for potential replicators since a reasonable likelihood of success could be expected without extensive screening.⁴ The proposed interpretation was that the paradigm’s effect did not depend on rare “star” subjects; the data themselves are separable from that reading, and the question of how selection interacts with effect size remained open within Honorton’s own framework.⁹

First-Timer Series Outcomes and the Liking Effect

In each first-timer series 50 novice participants contributed a single session, with direct hits and sum-of-ranks specified in advance and alpha set at 5% one-tailed.⁴ The first series (FT1) was nonsignificant on both indices (p[direct hits] = .38, p[sum of ranks] = .26), while in the second series, with 23 of 50 sessions completed, both indices approached significance (p[direct hits] = .04, p[sum of ranks] = .046).⁴ A companion selected-group study found psi-hitting subjects liked their targets (mean rank 1.73) more than psi-missing subjects (2.60), a difference reported as significant (Z = 1.80, U = 42.5, p < .036, one-tailed) and described as the most consistent finding in that line of research over the prior decade.⁴

Combined Methods for High- and Low-Scoring Selection

Honorton’s earlier work on selection sought to identify and separate high- and low-scoring subjects using combinations of techniques and predictor measures, an explicit attempt to operationalize “who scores” rather than relying on reputation.¹⁰ Related selection studies combined methods across attitude, state, and demographic indices to predict scoring direction.¹¹ The autoganzfeld retained subject-level psychometric data collection (the Participant Information Form, Myers–Briggs Type Indicator, and Tellegen Absorption Scale) so that scoring could be correlated with individual-difference measures rather than guessed at.⁴

Personality moderators: extraversion and the novice

If selection on prior performance is one route, selection on stable personality traits is another, and Honorton’s most quantitative contribution here was on extraversion. With Ferrari and Bem he reported a meta-analysis finding a reliable relationship between trait extraversion and ESP scoring, accompanied by a new confirmatory dataset drawn from the autoganzfeld participants.³ The proposed reading was that extraversion functions as a subject-level moderator; alternative readings, that the correlation reflects response style or differential engagement with free-response tasks, were part of the contested interpretation rather than settled.¹²

The Extraversion Meta-Analysis and Its Confirmation Sample

The extraversion–ESP analysis distinguished the accumulated literature from a fresh confirmation: the meta-analytic association was followed by an independent test on a new sample, an attempt to guard against the possibility that the historical correlation was an artifact of selective reporting or shifting measurement of the trait across decades.³ The companion conference report frames the same result as meta-analysis plus new confirmation, underscoring that the confirmation step, not the retrospective correlation alone, carried the evidential weight against a publication-bias explanation.¹²

Predicting the Ganzfeld Novice’s First Session

Honorton’s work on the ganzfeld novice examined predictors of initial ESP performance, that is, what about a first-time participant forecasts scoring before any track record exists, a problem of pre-screening rather than post-hoc selection.¹³ A related model of initial ganzfeld success used target-retrieval performance with the automated system to characterize who succeeds on a first attempt.⁹ A separate study tested ganzfeld performance within an artistically gifted population, probing whether a non-psi selection criterion (artistic giftedness) predicted scoring.¹⁴

Moderators in the meta-analytic record

The clearest statement of selection-as-moderator comes from Honorton and Ferrari’s forced-choice precognition meta-analysis, which pooled a half-century of published studies and then asked which study-level features predicted larger effects.² Subject selection on the basis of prior testing performance emerged as one of four moderators significantly covarying with outcome, alongside individual (versus group) testing, trial-by-trial feedback, and short response-to-target intervals.² Notably, the experimenter-contact dimension, subjects tested individually by an experimenter, showed significantly larger effects than group testing, a result that bears directly on experimenter involvement as a moderator.²

Four Moderators and a Selective-Reporting Test

The precognition meta-analysis retrieved 309 studies by 62 investigators, with nearly two million individual trials from more than 50,000 subjects, yielding a small but reliable overall effect (z = 11.41).² The file-drawer assessment estimated that 46 unreported null studies per reported study would be required to reduce the result to nonsignificance, directly addressing selective reporting as the leading non-psi explanation.² No systematic relationship was found between outcomes and eight indices of research quality, and the combined impact of the four moderators was large: independently significant outcomes appeared in seven of the eight studies that used selected subjects tested individually with trial-by-trial feedback.¹⁵

Eliminating the File-Drawer in the Autoganzfeld

For its own database the autoganzfeld program addressed selective reporting structurally rather than statistically: every session completed since the system’s 1983 inauguration, including pilot and ongoing studies, was included, which the authors stated left the database with no file-drawer problem.¹⁶ The sample was 241 volunteers (100 men, 141 women), ages 17–74 (mean 37.3), with strong stated psi belief, a self-selected but not performance-screened pool, allowing the moderator question to be posed on participants who were enthusiastic but largely unselected for prior success.¹⁶ Two secondary hypotheses were specified in advance, that dynamic targets and sender–receiver acquaintance would raise hit rates, both selection-of-condition variables rather than selection-of-subject.¹⁶

The open question Honorton’s framework leaves is adversarial and concrete: a preregistered multi-laboratory autoganzfeld series in which experimenter identity and subject-selection criteria are crossed as factors, varying screened versus unscreened samples and high- versus low-contact experimenters under a common automated protocol, would settle whether the moderators are genuine signal-carriers or correlated artifacts of how earlier studies were assembled.⁷

Skeptical Critiques and Discussion

Critique 1: Selection and experimenter-contact moderators could be artifacts of flexible analysis rather than real signal-carriers

In their critical appraisal of the ganzfeld database, Hyman, Honorton and Saunders identified departures from ideal standards on multiple testing, randomization, and the control of sensory leakage as concerns that could inflate apparent effects, including effects attributed to subject and procedural moderators.¹⁷ The skeptical position held that post-hoc moderator relationships in a heterogeneous database are vulnerable to multiple-comparison inflation.¹⁸

Honorton’s response and the precognition meta-analysis addressed this by pre-specifying primary and secondary hypotheses, computing file-drawer tolerances (46 null studies per reported study for the precognition base), and testing whether eight quality indices tracked outcome, finding no systematic quality–outcome relationship.² The autoganzfeld’s all-sessions-included rule was an explicit structural answer to selective reporting at the moderator level.¹⁶

Response by Honorton: Honorton argued that the precognition moderator pattern survived a file-drawer test of the required magnitude and showed no covariation with research-quality indices, which he read as evidence the moderators were not mere quality artifacts.¹⁹

Analysis. Hyman, Honorton and Saunders agree that the early manual ganzfeld corpus departed from ideal standards on randomization and sensory-leakage control, and the Joint Communiqué records continued disagreement over how much of the residual effect counts as psi. The precognition meta-analysis pre-specified moderators and reported a large file-drawer tolerance with no quality–outcome covariation; the autoganzfeld included all sessions to remove the file-drawer at source. Whether subject-selection and experimenter-contact moderators reflect a real process or correlated features of how studies were assembled is not resolved by these analyses alone and turns on independent crossed-factor replication.

Critique 2: Experimenter handling of targets and judging could introduce cueing that masquerades as a subject effect

Critics of the manual ganzfeld pointed to opportunities for sensory leakage and experimenter involvement in target handling and judging as routes by which apparent subject performance could be contaminated, a concern recorded among the agreed methodological weaknesses of the early database.¹⁸

The autoganzfeld eliminated experimenter target-handling and judging by placing target selection, presentation, and the receiver’s judging under computer control, with the receiver in an IAC sound-isolation room and the sender in an RF-shielded, sound-attenuated room served by physically separate, electrically isolated audio systems, physically isolating the cueing pathway rather than merely cautioning against it.⁵ The Edinburgh implementation was further documented as providing automatic data recording and resistance to both subject bias and intentional experimenter bias.⁸

Analysis. The Joint Communiqué documents agreement that target handling and judging in the manual protocol were potential leakage routes. Berger and Honorton’s automated system removed experimenter control of targets and judging and physically separated sender and receiver with isolated audio systems, eliminating direct sensory cueing within a session; it did not by construction address sender–experimenter interaction during target loading or broader questions of whether automation altered which subjects scored. The security review of the Edinburgh system extends the automated controls to a second laboratory.

References

Honorton, C., Ramsay, M., & Cabibbo, C. (1975). Experimenter effects in extrasensory perception. Journal of the American Society for Psychical Research, 69(2), 135–139. R001 [Honorton 1975] ↩︎
Honorton, C., & Ferrari, D. C. (1989). “Future telling”: A meta-analysis of forced-choice precognition experiments, 1935–1987. Journal of Parapsychology, 53(4), 281–308. R002 [Honorton 1989] ↩︎
Honorton, C., Ferrari, D. C., & Bem, D. J. (1998). Extraversion and ESP performance: A meta-analysis and a new confirmation. Journal of Parapsychology, 62(3), 255–276. R003 [Honorton 1998] ↩︎
Honorton, C., Barker, D. R., Varvoglis, M., Berger, R. E., & Schechter, E. I. (1986). Ganzfeld First Timers. Research in Parapsychology 1985, 1–3. R004 [Honorton 1986] ↩︎
Berger, R. E., & Honorton, C. (1986). Automated Psi Ganzfeld. Research in Parapsychology 1985, 85–88. R005 [Berger 1986] ↩︎
Honorton, C. (1978). Replicability, experimenter influence, and parapsychology: An empirical context for the study of mind. Paper presented at the annual meeting of the American Association for the Advancement of Science, Washington, DC. R006 [Honorton 1978] ↩︎
Honorton, C. (1985). How to evaluate and improve the replicability of parapsychological effects. In The Repeatability Problem in Parapsychology (pp. 238–255). Parapsychology Foundation. R007 [Honorton 1985] ↩︎
Dalton, K., Morris, R. L., Delanoy, D. L., Radin, D. I., & Honorton, C. (1996). Security measures in an automated ganzfeld system. Journal of Parapsychology, 60(2), 129–147. https://koestlerunit.wordpress.com/wp-content/uploads/2015/06/dalton-et-al-1996.pdf R008 [Dalton 1996] ↩︎
Honorton, C., & Schechter, E. I. (1987). Ganzfeld target retrieval with an automated testing system: A model for initial ganzfeld success. Scarecrow. R009 [Honorton 1987] ↩︎
Honorton, C. H. (1969). A combination of techniques for the separation of high- and low-scoring ESP subjects. Journal of the American Society for Psychical Research, 63, 69–82. R010 [Honorton 1969] ↩︎
Honorton, C. H., Carlson, T., & Tietze, T. (1968). Combined methods in subject selection. Citadel Press. R011 [Honorton 1968] ↩︎
Honorton, C., Ferrari D. C., & Bem, D. J. (1992). Extraversion and ESP performance: Meta-analysis and a new confirmation. Research in Parapsychology 1990, 35–38. R012 [Honorton 1992] ↩︎
Honorton, C. (1997). The ganzfeld novice: Four predictors of initial ESP performance. Journal of Parapsychology, 61, 143–158. R013 [Honorton 1997] ↩︎
Schlitz, M., & Honorton, C. (1992). Ganzfeld psi performance within an artistically gifted population. Journal of the American Society for Psychical Research, 86(2), 83–98. R014 [Schlitz 1992] ↩︎
Honorton, C., & Ferrari, D. C. (1989). “Future telling”: A meta-analysis of forced-choice precognition experiments, 1935–1987. Journal of Parapsychology, 53(4), 281–308. R015 [Honorton 1989] ↩︎
Honorton, C., Berger, R. E., Varvoglis, M. P., Quant, M., Derr, P., Schechter, E. I., & Ferrari, D. C. (1990). Psi communication in the ganzfeld: Experiments with an automated testing system and a comparison with a meta-analysis of earlier studies. Journal of Parapsychology, 54(2), 99–139. R016 [Honorton 1990] ↩︎
Hyman, R., Honorton, C., & Saunders, D. (1985). The Ganzfeld Psi experiment: a critical appraisal. Journal of Parapsychology, 49(1), 3–91. R017 [Hyman 1985] ↩︎
Hyman, R., & Honorton, C. (2019). A Joint Communiqué: The Psi Ganzfeld Controversy. Journal of Parapsychology, 82(3), 108–117. https://doi.org/10.30891/jopar.2018s.01.09 R018 [Hyman 2019] ↩︎
Honorton, C. (1985). Meta-analysis of psi ganzfeld research: A response to Hyman. Journal of Parapsychology, 49(1), 51–91. R019 [Honorton 1985] ↩︎