mirror of
https://github.com/affaan-m/ECC.git
synced 2026-10-26 18:35:05 +00:00
fix(gateguard): make the external-contract question opt-in
- questionIdsFor and condensedHintFor take an optional
{ externalContract: true }; without it a code change that handles data
asks data-schema only, and an unknown profile gets the original four
questions and condensed hint.
- The hook reads GATEGUARD_EXTERNAL_CONTRACT_QUESTION with the same
spellings as GATEGUARD_METRICS and passes it to the full denial, the
condensed denial and the metrics record, so all three agree.
- The effectiveness harness's arm patch forwards the option, so an arm
never drops it silently.
Docs: change-profile.md gains an External-contract question section and
qualifies "byte-identical to the fixed four questions"; design-notes.md,
SKILL.md (switch table, data rule, word-for-word set) and CHANGELOG say
the fifth question is opt-in; question-effectiveness.md and the harness
README note that the runner clears GATEGUARD_* variables, so a gate arm
from a current tree measures the gate without the question.
skill-triggers@1.json: registryDigest refreshed for the SKILL.md body
change; the description and triggers are unchanged.
This commit is contained in:
10 files changed
+84
-28
No files matched your search
@@ -116,6 +116,21 @@ or a PowerShell file or web cmdlet (`Out-File`, `Get-Content`,
|
||||
`*-Clixml`; `Import-Csv` matches `csv`). Redirection is found without
|
||||
reading quotes, so a `>` inside a string also counts.
|
||||
|
||||
## External-contract question
|
||||
|
||||
`GATEGUARD_EXTERNAL_CONTRACT_QUESTION` (`1`, `true`, `on`, `enabled`,
|
||||
`enable` or `yes`) adds `external-contract` right after every `data-schema`
|
||||
in the table above: "Name what outside this repository decides the format,
|
||||
units, timezone or protocol semantics here (the consumer, the producer, or a
|
||||
stated convention), or state that the choice is unconstrained". The condensed
|
||||
hint and the decision metrics name it too. It is off by default because it
|
||||
has no evidence behind it: it was designed after inspecting the
|
||||
hidden-intent corpus, and in the rerun that added it the gated arm did not
|
||||
improve (see
|
||||
[question-effectiveness.md](question-effectiveness.md#adding-an-external-contract-question-did-not-help)).
|
||||
`questionIdsFor(class, isWrite, profile, { externalContract: true })` returns
|
||||
the ids with it.
|
||||
|
||||
## Trivial edits
|
||||
|
||||
An Edit is trivial when, for every entry, old and new have the same code once
|
||||
@@ -293,7 +308,8 @@ questions were asked without storing text.
|
||||
| config | any | `config-reader`, `config-effect`, `no-plaintext-secrets`, `quote-instruction` |
|
||||
|
||||
`[data-schema]` is asked only when `touchesData`. With an unknown profile the
|
||||
code text is byte-identical to the fixed four questions. Sensitive and
|
||||
code text is byte-identical to the fixed four questions, unless the opt-in
|
||||
[external-contract question](#external-contract-question) is on. Sensitive and
|
||||
hard-linked targets and NotebookEdit calls are never profiled, so they always
|
||||
get the full code questions. Instruction and
|
||||
config questions carry no change-dependent item, and the test and prose items
|
||||
|
||||
@@ -207,7 +207,9 @@ their absolute path. A Windows-style path has no real location on a POSIX host
|
||||
The first matching class wins: instruction, test, prose, config, code (the
|
||||
fallback). Cursor `.mdc` rule files are instructions wherever they live. A
|
||||
basename starting with `.env` is config unless it has a code extension
|
||||
(`.env.example.ts` is source). Code targets ask the original four questions,
|
||||
(`.env.example.ts` is source). Code targets ask the original four questions
|
||||
(five with the opt-in
|
||||
[external-contract question](change-profile.md#external-contract-question)),
|
||||
narrowed by the change profile (see
|
||||
[Questions from the change profile](change-profile.md#questions-from-the-change-profile)); other
|
||||
classes get class-specific questions and a class-specific condensed hint.
|
||||
|
||||
@@ -529,6 +529,14 @@ and the corpus suggests that kind of answer matters. What is not claimed is
|
||||
that adding it improved anything measurable here. A fair test needs scenarios
|
||||
written by independent authors who have not seen that question or these results.
|
||||
|
||||
Because nothing here shows it helps, the question is now opt-in: it is asked
|
||||
only with `GATEGUARD_EXTERNAL_CONTRACT_QUESTION` on (see
|
||||
[change-profile.md](change-profile.md#external-contract-question)). The
|
||||
harness clears every `GATEGUARD_*` variable from the agent's environment, so a
|
||||
`gate` arm built from a candidate tree that includes this change measures the
|
||||
gate without it, the "before the question" condition above. The runs reported
|
||||
here used candidate trees from before the change, where it was always asked.
|
||||
|
||||
**Standing conclusion.** Across 336 billed trials the gate's benefit is not
|
||||
established, its direction has been positive in the unflagged runs, and its cost
|
||||
is a consistent 35 to 45% in turns and spend. "Not confirmed" is not "no
|
||||
|
||||
Reference in new issue
Block a user