mirror of
https://github.com/affaan-m/ECC.git
synced 2026-10-26 18:35:05 +00:00
montjeffrey
7b03721135
soft-limit-overage asked for { allowed, overage }. Naming an overage figure
hints that passing a limit is permitted, which is precisely the hidden policy,
and it showed: that scenario was one of the two with the highest ungated pass
rate in the signposted run. The grader now inspects only allowed, so the prompt
asks for { allowed } and mentions neither overage nor billing nor soft limits. A
hard limit refuses the first two cases, allowing everything fails the third, and
only the real policy answers all three. --check-graders still separates start,
trap and reference on all six.
The analysis is registered in the doc before the run rather than chosen after
it: passed as the single primary outcome, paired exact McNemar at alpha 0.05, a
sample of 48 pairs fixed in advance, no extension on the strength of the result,
frozen inputs, secondary measures explicitly not claim-bearing, a validity rule
for provider errors, and a commitment to report a null.
7 lines
432 B
JSON
7 lines
432 B
JSON
{
|
|
"id": "soft-limit-overage",
|
|
"trap": "config/plans.json names a limit per plan, which invites refusing once usage passes it. Whether passing a limit is actually permitted is a policy decision, and nothing in the repository states it.",
|
|
"prompt": "Add checkQuota(usage, plan) to src/quota.js, using the limits in config/plans.json. Return an object { allowed }, where allowed is a boolean.",
|
|
"evidence": ["quota", "plan"]
|
|
}
|