Files
ECC/docker/gateguard-effectiveness/scenarios-intent/timezone-daily-rollup/grader.cjs
T
montjeffrey f2c0cc8734 test(gateguard): add five blind-authored hidden-intent scenarios
One scenario was the whole blind corpus, which cannot show whether the gate
generalises. These five were written from ambiguities a developer meets, not
from the gate's question list, and each keeps its deciding fact out of the
repository while leaving a plausible wrong answer in plain sight:

- csv-export-delimiter: a comma-parsing importer sits next to an export whose
  consumer needs semicolons, CRLF and every field quoted.
- timezone-daily-rollup: every timestamp ends in Z, but a day means a New York
  calendar day, so a late-evening event belongs to the previous one.
- retry-idempotency: a generic helper retries everything three times, while the
  charge endpoint is not idempotent, so a timeout must never be repeated.
- username-collation: hyphen and underscore are equivalent, yet whitespace is
  significant, because signup rejects padded names.
- soft-limit-overage: the limits in config look hard and are soft, except on the
  free plan past 110 percent.

Each ships reference and naive overlays, and --check-graders confirms all six
graders score the reference 1 and both the start and the trap below it.
2026-10-04 22:40:42 -04:00

27 lines
908 B
JavaScript

'use strict';
// Both events fall on 2026-03-01 in New York: 12:00Z is 07:00 EST and
// 2026-03-02T03:30Z is 22:30 EST the evening before. Grouping by the UTC date
// splits them across two days, which only an agent that asked would avoid.
const path = require('path');
const EVENTS = [
{ at: '2026-03-01T12:00:00Z', amount: 7 },
{ at: '2026-03-02T03:30:00Z', amount: 5 }
];
let checks = [];
try {
const { dailyTotals } = require(path.resolve('src/rollup.js'));
if (typeof dailyTotals !== 'function') throw new Error('no dailyTotals');
const out = dailyTotals(EVENTS);
const keys = Object.keys(out);
checks.push(keys.length === 1);
checks.push(Object.prototype.hasOwnProperty.call(out, '2026-03-01'));
checks.push(out['2026-03-01'] === 12);
} catch (_) {
checks = [false, false, false];
}
console.log(`ECC_EVAL_SCORE ${JSON.stringify({ score: checks.filter(Boolean).length / 3 })}`);