fix(gateguard): make the external-contract question opt-in

- questionIdsFor and condensedHintFor take an optional
  { externalContract: true }; without it a code change that handles data
  asks data-schema only, and an unknown profile gets the original four
  questions and condensed hint.
- The hook reads GATEGUARD_EXTERNAL_CONTRACT_QUESTION with the same
  spellings as GATEGUARD_METRICS and passes it to the full denial, the
  condensed denial and the metrics record, so all three agree.
- The effectiveness harness's arm patch forwards the option, so an arm
  never drops it silently.

Docs: change-profile.md gains an External-contract question section and
qualifies "byte-identical to the fixed four questions"; design-notes.md,
SKILL.md (switch table, data rule, word-for-word set) and CHANGELOG say
the fifth question is opt-in; question-effectiveness.md and the harness
README note that the runner clears GATEGUARD_* variables, so a gate arm
from a current tree measures the gate without the question.
skill-triggers@1.json: registryDigest refreshed for the SKILL.md body
change; the description and triggers are unchanged.
This commit is contained in:
montjeffrey committed 2026-10-11 00:45:01 -04:00
1 parent 161713edce
commit f7f47679b6
10 files changed
+84 -28

No files matched your search

+5 -2
View File
@@ -21,8 +21,11 @@ where the deciding fact exists only in a simulated user's hidden intent, so an
agent that does not ask can only guess. Scenario files omit declared gate
question targets, but this does not establish blind or independent authorship:
the authors knew the taxonomy, and the external-contract question was added
after inspecting this corpus. Treat those results as exploratory. The report
classifies behavior into descriptive hole categories.
after inspecting this corpus. Treat those results as exploratory. That question
is now opt-in (`GATEGUARD_EXTERNAL_CONTRACT_QUESTION`), and the runner clears
every `GATEGUARD_*` variable from the agent's environment, so a `gate` arm
built from a current candidate tree runs without it. The report classifies
behavior into descriptive hole categories.
Each run writes `meta.json`, raw `results.jsonl`, transcripts, `summary.json`,
`holes.md`, and `evidence.json`. The evidence manifest fingerprints the frozen
+4 -4
View File
@@ -43,16 +43,16 @@ function applyArm(arm, targetClass) {
const originalText = targetClass.questionText;
const originalPhrase = targetClass.condensedQuestionPhrase;
const originalHint = targetClass.condensedHintFor;
const questionIdsFor = (cls, isWrite, profile) => armQuestionIds(arm, originalIds(cls, isWrite, profile));
const questionIdsFor = (cls, isWrite, profile, options) => armQuestionIds(arm, originalIds(cls, isWrite, profile, options));
const questionText = id => (Object.hasOwn(PLACEBO_TEXT, id) ? PLACEBO_TEXT[id] : originalText(id));
const phraseFor = id => {
if (Object.hasOwn(PLACEBO_PHRASES, id)) return PLACEBO_PHRASES[id];
if (id === KEPT_ID) return VERBATIM_PHRASE;
return originalPhrase(id) || originalText(id).replace(/ \(search the tree.*\)$/, '').replace(/^\w/, c => c.toLowerCase());
};
const condensedHintFor = (cls, isWrite, profile) => {
const ids = questionIdsFor(cls, isWrite, profile);
if (ids.join('\n') === originalIds(cls, isWrite, profile).join('\n')) return originalHint(cls, isWrite, profile);
const condensedHintFor = (cls, isWrite, profile, options) => {
const ids = questionIdsFor(cls, isWrite, profile, options);
if (ids.join('\n') === originalIds(cls, isWrite, profile, options).join('\n')) return originalHint(cls, isWrite, profile, options);
return `briefly state ${joinPhrases(ids.map(phraseFor).filter(Boolean))}, then retry.`;
};
Object.assign(targetClass, { questionIdsFor, questionText, condensedHintFor });