Screening seven measures: the paired design fixes specificity, nothing yet fixes randomness

Rendered from gate/RESULT-MEASURES.md
Contents
  1. Question or issue resolved
  2. What the paired design makes possible
  3. C1 and C2, at g = 1.05
  4. One implementation finding worth keeping
  5. Status and limitations

Run 2026-09-20. Code: gate/artificial_measures.py. Output: gate/artificial_measures.json. Substrate and controls as in gate/RESULT-ARTIFICIAL.md.

Question or issue resolved

gate/RESULT-ARTIFICIAL.md established that PCIst does not port to a recurrent network. Establishing that one measure fails is half a result. This asks the other half: is there a measure that does behave on this substrate, and if not, what exactly is in the way?

The simulation is now a testbed with known ground truth — coupling sets how integrated the network is, an adaptation current implements the bistability that collapses PCI in cortex — and a control battery that has already caught six artifacts. Candidates can be screened against it cheaply.

Four pass criteria, fixed before the run. A measure passes if all four hold.

C1 shuffle phase-shuffling the response must not raise the measure
C2 specificity a perturbation 1% of real amplitude must score far below the real one
C3 bistability the anesthesia analogue must lower the measure at matched coupling
C4 non-trivial the measure must not be monotone in coupling alone

C3 is the only criterion with an external ground truth. C1 is the one PCIst failed.

What the paired design makes possible

The noise draws in gate/artificial.py do not depend on the stimulus, so a perturbed run and an unperturbed run from the same seed share an identical noise realization. Verified: their difference is exactly zero at every sample before the stimulus, and 1.50 after it.

That difference is the causal effect of the perturbation with the ongoing dynamics removed exactly — not estimated, removed. No brain can be re-run with the same noise, which is why no biological measure is built this way. Four of the seven candidates use it.

C1 and C2, at g = 1.05

Measure Real Shuffled C1 Sham 1% C2
PCIst 18.99 38.59 fail 10.75 (57%) fail
LZ of response 8.57 7.50 ok 5.80 (68%) fail
Participation ratio 2.12 6.41 fail 3.89 (183%) fail
δ magnitude 147.83 62.23 ok 1.97 (1.3%) ok
δ Lempel-Ziv 7.00 19.60 fail 0.10 (1.4%) ok
δ dimensionality 2.14 7.03 fail 2.62 (122%) fail
δ spread × duration 99.67 100.00 fail 0.00 (0%) ok

Two findings, and they point in opposite directions.

The paired design solves specificity outright

Every measure built on the paired difference scores the 1% sham at 0 to 1.4% of the real perturbation. Every measure built on the response itself scores it at 57 to 183% — that is, the unpaired measures cannot tell a real perturbation from one a hundred times weaker, because what they are mostly reading is the network's ongoing activity, which is identical in both.

This is the concrete form of the failure in RESULT-ARTIFICIAL.md. It is not that PCIst is badly designed; it is that any measure computed on the response alone is reading the substrate as much as the perturbation, and subtracting a matched unperturbed run removes that term exactly.

Nothing solves the randomness problem, and the reason is structural

C1 is failed by PCIst, by δ Lempel-Ziv, by participation ratio and by δ dimensionality. The only measure that passes it cleanly is δ magnitude, which is not a complexity measure at all — it is effect size.

That is not seven coincidences. Compression-based complexity ranks randomness as maximal. A phase-shuffled response is closer to noise than the real one, and noise is incompressible, so any measure whose currency is compressibility must score the shuffle higher. The same holds for dimensionality: randomizing phases decorrelates units and raises effective dimension by construction.

This reframes the PCIst result. Its failure is not a quirk of this simulation. It is an instance of the known and contested weakness of compression as a proxy for complexity, and this substrate merely makes it visible — in cortex it is masked, because real evoked responses are never phase-random and because the measure is compared between two states that are both structured.

So the honest position after this screen: the paired design is a real advance in specificity, and no candidate here distinguishes structured complexity from randomness. A measure that passed C2 and failed C1 is a measure that would call a seizure or a noise burst maximally conscious.

One implementation finding worth keeping

The first run of this screen mis-implemented four of the seven measures, in the same way and for an interesting reason. Each thresholded the paired difference against its own baseline — but the paired difference is exactly zero before the stimulus, so the threshold was 10⁻¹², every sample cleared it, and the binarized matrix was all ones. Lempel-Ziv of an all-ones string is 0.57.

The paired design removes the noise floor along with the noise. Any measure needing a floor must borrow one from the unperturbed run, which is also the meaningful comparison: the causal effect counts where it exceeds what the network was doing anyway.

δ spread × duration still saturates at ~100 with the borrowed floor and has no dynamic range at this coupling. It is reported as failing C1, but the correct reading is that its threshold is uncalibrated, not that the quantity is wrong.

Status and limitations