fix(gateguard): make the external-contract question opt-in

- questionIdsFor and condensedHintFor take an optional
  { externalContract: true }; without it a code change that handles data
  asks data-schema only, and an unknown profile gets the original four
  questions and condensed hint.
- The hook reads GATEGUARD_EXTERNAL_CONTRACT_QUESTION with the same
  spellings as GATEGUARD_METRICS and passes it to the full denial, the
  condensed denial and the metrics record, so all three agree.
- The effectiveness harness's arm patch forwards the option, so an arm
  never drops it silently.

Docs: change-profile.md gains an External-contract question section and
qualifies "byte-identical to the fixed four questions"; design-notes.md,
SKILL.md (switch table, data rule, word-for-word set) and CHANGELOG say
the fifth question is opt-in; question-effectiveness.md and the harness
README note that the runner clears GATEGUARD_* variables, so a gate arm
from a current tree measures the gate without the question.
skill-triggers@1.json: registryDigest refreshed for the SKILL.md body
change; the description and triggers are unchanged.
This commit is contained in:
montjeffrey committed 2026-10-11 00:45:01 -04:00
1 parent 161713edce
commit f7f47679b6
10 files changed
+84 -28

No files matched your search

+17 -1
View File
@@ -116,6 +116,21 @@ or a PowerShell file or web cmdlet (`Out-File`, `Get-Content`,
`*-Clixml`; `Import-Csv` matches `csv`). Redirection is found without
reading quotes, so a `>` inside a string also counts.
## External-contract question
`GATEGUARD_EXTERNAL_CONTRACT_QUESTION` (`1`, `true`, `on`, `enabled`,
`enable` or `yes`) adds `external-contract` right after every `data-schema`
in the table above: "Name what outside this repository decides the format,
units, timezone or protocol semantics here (the consumer, the producer, or a
stated convention), or state that the choice is unconstrained". The condensed
hint and the decision metrics name it too. It is off by default because it
has no evidence behind it: it was designed after inspecting the
hidden-intent corpus, and in the rerun that added it the gated arm did not
improve (see
[question-effectiveness.md](question-effectiveness.md#adding-an-external-contract-question-did-not-help)).
`questionIdsFor(class, isWrite, profile, { externalContract: true })` returns
the ids with it.
## Trivial edits
An Edit is trivial when, for every entry, old and new have the same code once
@@ -293,7 +308,8 @@ questions were asked without storing text.
| config | any | `config-reader`, `config-effect`, `no-plaintext-secrets`, `quote-instruction` |
`[data-schema]` is asked only when `touchesData`. With an unknown profile the
code text is byte-identical to the fixed four questions. Sensitive and
code text is byte-identical to the fixed four questions, unless the opt-in
[external-contract question](#external-contract-question) is on. Sensitive and
hard-linked targets and NotebookEdit calls are never profiled, so they always
get the full code questions. Instruction and
config questions carry no change-dependent item, and the test and prose items
+3 -1
View File
@@ -207,7 +207,9 @@ their absolute path. A Windows-style path has no real location on a POSIX host
The first matching class wins: instruction, test, prose, config, code (the
fallback). Cursor `.mdc` rule files are instructions wherever they live. A
basename starting with `.env` is config unless it has a code extension
(`.env.example.ts` is source). Code targets ask the original four questions,
(`.env.example.ts` is source). Code targets ask the original four questions
(five with the opt-in
[external-contract question](change-profile.md#external-contract-question)),
narrowed by the change profile (see
[Questions from the change profile](change-profile.md#questions-from-the-change-profile)); other
classes get class-specific questions and a class-specific condensed hint.
+8
View File
@@ -529,6 +529,14 @@ and the corpus suggests that kind of answer matters. What is not claimed is
that adding it improved anything measurable here. A fair test needs scenarios
written by independent authors who have not seen that question or these results.
Because nothing here shows it helps, the question is now opt-in: it is asked
only with `GATEGUARD_EXTERNAL_CONTRACT_QUESTION` on (see
[change-profile.md](change-profile.md#external-contract-question)). The
harness clears every `GATEGUARD_*` variable from the agent's environment, so a
`gate` arm built from a candidate tree that includes this change measures the
gate without it, the "before the question" condition above. The runs reported
here used candidate trees from before the change, where it was always asked.
**Standing conclusion.** Across 336 billed trials the gate's benefit is not
established, its direction has been positive in the unflagged runs, and its cost
is a consistent 35 to 45% in turns and spend. "Not confirmed" is not "no