Conformance vectors

Rendered from spec/vectors/README.md
Contents
  1. What this is
  2. Why it is diagnostic rather than pass-or-fail
  3. Completeness is enforced, not assumed
  4. The vectors
  5. Conformance

What this is

Eight test vectors that decide whether an implementation of spec/PCIST-1.0.md is conformant, and — where it is not — name the clause it got wrong.

python spec/vectors/verify.py yourmodule:your_function

The function is called as f(signal, times, baseline_window, response_window, k=, min_snr=, max_var=, n_steps=, resample=) and must return a float. The script exits non-zero on any failure.

Why it is diagnostic rather than pass-or-fail

A conventional vector set answers one question: does this implementation agree? That is enough when a specification is tight and the likely error is a typo. It is not enough here, and the register knows why: an implementation written from the standard before PCIST-1.0 existed got four separate clauses wrong at once and returned a plausible number (gate/RESULT-DUAL.md). No pass-or-fail signal would have told anyone which four.

So each vector also records what eight single-clause mis-implementations return on it. A failing candidate is matched against those profiles across all eight vectors, and the mutation that explains the most vectors is reported with its clause number. Four of the eight are errors that were actually made in practice rather than invented for the exercise.

Matching is done on the whole profile, not vector by vector. Several broken clauses return zero on the noise vector, so a per-vector match reports two or three confident diagnoses for one error; the profile separates them on the vectors where they do not collide. Demonstrated: an implementation with only clause 4.2 wrong is identified at 8/8, with the next-best explanation at 2/8.

Completeness is enforced, not assumed

build.py refuses to publish a set in which any modeled mutation is undetected by every vector. That gate did real work on its first run: it could not distinguish two of the ten mutations originally modeled, because both were provably immaterial — decomposing the response rather than its transpose gives the same channel-space basis, and reversing the recurrence inequality cannot survive the absolute differencing that follows it. PCIST-1.0 had warned about both. The annex was corrected and the two are now carried as invariance assertions checked on every vector, so the claim is tested rather than argued.

The vectors

id construction purpose
V1 pure noise, 20 ch the instrument's floor; catches any implementation returning zero where the reference does not
V2 damped multi-frequency, 20 ch the standard positive case
V3 damped multi-frequency, 8 ch the max_var stopping rule where few components exist
V4 rank-one response, 24 ch one dominant component
V5 undamped single tone, 16 ch highly recurrent; the transition definition and threshold range both bite
V6 step change, 16 ch almost no transitions; separates counting transitions from counting recurrences
V7 damped multi-frequency, 20 ch, human set resampling and the shorter windows
V8 damped multi-frequency, 32 ch at 2500 Hz the heaviest case

Vectors ship as generators — a seed and a few shape parameters — not as arrays. The input is reproducible in any language with a standard normal generator and an FFT, with nothing to download and nothing to drift. V1's expected value is 87.28, which is not zero, because the reference computation does not return zero on noise; see METHOD §2.3.

Conformance

An implementation conforms where all eight expected values are reproduced to a relative tolerance of 1 × 10⁻⁶. That statement is what METHOD §12 condition 1 has been inviting a third party to test since 0.3.6, and until now there was no means to test it.