Cross-tabulate classification backends at every fingerprint level

Classifies the eval corpus once per voter and cross-tabulates the resulting IRs at all three [ir_fingerprint()] levels (`abstention`, `layer_chain`, `full`).

What this measures - and what it does not. Agreement is computed pre-execution over voters that all read the SAME spec text: it measures translation consistency and prompt variance, not correctness, and it is NOT independent double programming (every voter shares one spec, one layer registry, and one IR schema). The corpus it runs on is a self-scoring regression corpus - its expected values were generated by the `rules` backend itself - not an oracle, so voter agreement is agreement between translation strategies, never evidence that any translation is right. For LLM voters to say anything meaningful they should differ in model family AND in prompt variant: two voters sharing one convention-asserting prompt measure prompt compliance, not agreement.

Fails loudly, naming every credential environment variable it looked for, when an LLM voter cannot actually be run - an unmeasured result must never be readable as a measured one.

Usage

run_agreement(voters, dataset = "ADSL")

Arguments

voters: Named list of voters. Each is either a backend name (`"rules"`, `"llm"`) or a list with `backend` and optionally `chat`, `prompt_variant`, `samples`, `consensus`. LLM voters should differ in model family AND prompt variant (see description).
dataset: Dataset tag of the corpus cases to run (default `"ADSL"`).

Value

An `aa_agreement` list with `dataset`, `voters`, `per_case` (one row per case per level with each voter's fingerprint and an `agree` flag), `rates` (agreement rate per level) and `crosstab` (per level, a contingency table of the first two voters' human-readable labels). Printed via [print.aa_agreement()].

Examples

# Two rules voters agree trivially (same deterministic translator):
res <- run_agreement(list(a = "rules", b = "rules"), dataset = "ADSL")
print(res)
#> admiralagent agreement report
#>   dataset: ADSL
#>   voters:  a, b
#>   agreement @ abstention:   1.000
#>   agreement @ layer_chain:  1.000
#>   agreement @ full:         1.000
#>   note: pre-execution agreement over a self-scoring regression corpus -
#>         translation consistency, not correctness or double programming.