Notes on Random Target Selection: The PRL Autoganzfeld Target and Target Set Distributions Revisited

Bierman, D. J., Broughton, R. S., & Berger, R. E. (1998). Notes on random target selection: The PRL autoganzfeld target and target set distributions revisited. The Journal of Parapsychology, 62(4), 339–350.

AI Assessment

A post-hoc methodological re-analysis, co-authored by parapsychologists including Rick E. Berger, that scrutinizes the target randomization behind the influential PRL autoganzfeld database (Bem & Honorton, 1994). Applying a deliberately conservative response-bias correction, the authors find the overall hit rate barely above chance (.2598 versus a theoretical .25) and, more pointedly, they weaken one of the original study’s firm claims: the reported roughly 10% scoring advantage of dynamic over static targets shrinks to a non-significant 6.8%. They also document a genuinely odd target-set frequency distribution (one set used about three times as often as expected) but conclude it cannot explain the reported excess of hits. It is a notable example of a field auditing its own flagship data and downgrading a secondary claim in the process. This audit reports what the paper argues and how; it takes no position on whether the autoganzfeld results reflect psi.

Provenance

Publication. The Journal of Parapsychology, 1998, volume 62, pages 339 to 350. This is a methodological “notes” paper rather than a new experiment.

Study type. A post-hoc, exploratory re-analysis of an existing dataset, the target and target-set distributions of the Psychophysical Research Laboratories (PRL) autoganzfeld studies.

Authors. Dick J. Bierman, Richard S. Broughton, and Rick E. Berger, all established parapsychology researchers. That the re-analysis is conducted by researchers within the field, on the field’s own high-profile data, is part of what makes it notable.

Data basis. The published PRL autoganzfeld data (Bem & Honorton, 1994), re-examined at the level of individual target sets, positions, series, and experimenters (reported in the paper’s Tables 1 to 4 and Figure 1).

Source basis. Every figure below is taken from the paper’s own Abstract, text, and tables. (A cataloguing note: the study database had stored the lead author’s surname with a typo, “Berman”; the correct byline, used here, is Bierman.)

What the paper reports

The paper draws a careful distinction between a random target-selection procedure, which in principle excludes sequential dependencies, and the resulting random target sequence, which can still contain peculiarities, such as some targets appearing more often than others, that may line up with subjects’ response biases and inflate apparent hit rates.¹ Applying this distinction to the PRL autoganzfeld data, the authors ask two questions: does correcting for response bias change the headline results, and is the target sequence itself anomalous? Their answers are that a conservative correction modestly reduces the overall effect and substantially weakens the claimed dynamic-versus-static target difference, and that although the target-set distribution is genuinely peculiar, it cannot account for the reported excess of hits.

The reported deviations from a well balanced target frequency distribution can not explain the excess of hits reported for the PRL autoganzfeld study.

How it was run

• The randomization under review. In the PRL autoganzfeld experiment one of 160 available targets was selected each session by a hardware random number generator (based on thresholding an electronic-noise signal), passed through a software shift register that assembled a full byte from eight separate samples to avoid sequential dependence; after September 1985 the procedure was simplified to a single-byte read. Targets were grouped into sets of four (one selected target plus three decoys), with only the target chosen at random.
• The correction. Following the approach Bem used, the authors computed a corrected hit probability per set by multiplying, for each clip, the relative frequency it appeared as a target by the relative frequency subjects selected it, summing across the four clips, weighting by how often each set was used, and dividing by total trials.
• Two correction strengths. A conservative correction treats all responses as potential bias; a “miss-based” correction uses only the non-hit responses. Position preferences and the single-set Series 302 were analyzed separately (Tables 2 and 3).
• Distribution check. The frequency with which each target and set actually occurred was examined for anomalies (Figure 1 and Table 4).

Results, as reported

Values are reproduced from the paper’s text and Tables 1 to 3. The overall effect survives the conservative correction (z = 2.47), while the specific claim that dynamic targets outperform static ones is downgraded to “suggestive at best,” and a real distributional oddity is documented and then argued to be immaterial to the hit excess.

Eleven-dimension audit

Pre-registration

This is an explicitly post-hoc re-analysis of an existing dataset, so preregistration does not apply. The authors are transparent that these are exploratory corrections applied after the fact, and they present more than one correction strength rather than selecting a single favourable one.

Randomization

Randomization is the paper’s subject rather than a feature of a new experiment. The authors document the PRL selection procedure in technical detail and draw the key distinction between a sound selection procedure and a structured resulting sequence. Their finding is nuanced: the procedure itself appears proper, but the realized target sequence is peculiar (set 20 heavily over-used), which they treat as a real anomaly to be explained rather than dismissed.

Sensory leakage

The analogous concern here is bias-driven inflation of hits: if certain targets recur and subjects favour certain responses, apparent hits can exceed chance without psi. The whole method is a correction for exactly this, and it is applied in two strengths, which is the appropriate way to bound the concern.

Blinding

Not applicable to a re-analysis of already-collected data; there are no new participants, raters, or conditions to blind.

Optional stopping

Not applicable: the analysis works with a fixed, previously published dataset rather than accumulating new trials.

Outcome measure

Corrected hit probabilities (conservative and miss-based), the static-versus-dynamic contrast, and the overall significance. The measures are clearly defined and follow the method Bem introduced, which makes the re-analysis directly comparable to the original.

Effect size

Modest and, in one respect, reduced. The overall corrected hit rate is just above chance (.2598) and remains significant (z = 2.47), but the headline secondary effect, the dynamic-over-static advantage, drops from a significant roughly 10% to a non-significant 6.8% (or 7.4% under the miss-based correction). The paper’s own conclusion, that the dynamic advantage is “suggestive at best,” is well matched to these numbers.

Multiple comparisons

Several analyses are run (two correction strengths, position preferences, per-series and per-experimenter breakdowns). Because the paper’s purpose is to probe robustness rather than to hunt for a significant result, this multiplicity is a feature of a thorough audit rather than a fishing concern, and the authors report the analyses that cut against their field’s prior claims as readily as those that support them.

Internal replication

The corrections are applied consistently across sets, positions, and series, and the two independent correction strengths converge on the same qualitative conclusion (overall effect survives, dynamic-static difference does not). That convergence is a form of internal robustness.

External replication

This is a single-dataset re-analysis and does not constitute independent replication of the autoganzfeld effect. It extends and stress-tests Bem’s own earlier bias analysis of the same data rather than testing new data.

Transparency

High, and it is the paper’s defining quality. The authors apply a deliberately conservative correction that works against their own field’s interest, openly weaken a previously firm conclusion, document an inconvenient distributional anomaly, and lay out the full per-set numbers in their tables. This is self-scrutiny of the field’s flagship data by researchers within the field.

The adversarial record

• The corrections are post-hoc and their strength is a choice. The conservative correction treats every response as potential bias, which the authors acknowledge is extreme and would understate any genuine effect, while a less conservative correction gives a more favourable picture. A critic can note that where the true effect sits depends on which correction one prefers.
• The distributional anomaly cuts both ways. A target set used roughly three times its expected frequency is a real oddity in a study whose validity rests on proper randomization. The authors judge it immaterial to the hit excess, but a skeptic can reasonably treat such a sequence anomaly as a caution about the dataset rather than a settled non-issue.
• Single-dataset scope. Everything here concerns the PRL autoganzfeld data specifically; the analysis inherits that dataset’s limits and does not speak to the broader ganzfeld literature.
• What is genuinely admirable. This is proponents auditing their own most-cited data and reporting that a specific published claim (dynamic beats static) does not survive a conservative correction. The overall effect remains significant (z = 2.47) even under that correction, so the core result is not overturned, but a secondary conclusion is honestly downgraded. That willingness to weaken one’s own prior claim is exactly the transparency that critics often say the field lacks.