Commit Graph
11 Commits
Author SHA1 Message Date
morluto a4a8734452 test(mcp): check whole replies against canonical output contracts at the remaining held listed definitions (#11182)
* test(mcp): check whole replies against canonical output contracts at the remaining held listed definitions

Retiring the full output profile (#10693) left listed tool definitions without output schemas, so tests that handed one to the SDK as toolDefinition stopped validating complete replies. Attach the canonical output schema with withCanonicalOutputSchema at the 41 remaining sites in 28 files that hold a definition taken directly from tools/list.

* test(mcp): check continuation pages against the canonical output contract
2026-11-16 04:00:00 +08:00
morluto 5c53bcaed8 feat(mcp)!: retire the full schema profiles and declare schema facts as typed annotations (#10693)
* test(mcp): check advertised schemas against a portable dialect in every profile

* fix(mcp): re-emit advertised patterns in a portable regex grammar

* fix(mcp): present discover_tools input through the selected profile

The hot-cold catalog registered discover_tools directly with the SDK, so
the default compact input profile still advertised its canonical schema,
including page-size defaults clients apply on their own (#5322). Route
every listed input schema, transport discovery included, through one
server-owned presentation, and pass the selected presentation to the
deferred server, which serves recorded tool schemas but not discovery.

* fix(mcp): keep the Evidence union on described borrowed inputs

borrowEvidenceInput installed its advertised projection on one schema
instance. A call site's .describe() rebuilds the schema from its
definition, so match_native_functions advertised left and right as
unconstrained values in every input profile. Install the projection in
the borrowed parser's constructor so derived schemas keep it, for both
borrowed inputs and borrowed arrays.

* fix(mcp): validate advertised dialect by selected profile

* fix(mcp): preserve schema widening through references

* feat(mcp): project compact input schemas into the portable advertised dialect

One projection (advertisedDialectProjection.ts) owns the compact input
advertisement: one type per node, items on every array, string literals
as enums, oneOf widened to anyOf, allOf merged, tuples as element unions,
exclusive bounds made inclusive, and every keyword outside the dialect
dropped. Each rewrite is exact or widening; server validation is
unchanged. The root description and sentences for dropped cross-field
constraints move to the tool description, which Codex and Zed keep.

The compact baseline falls to nullable values only (29), which the
dialect cannot express without narrowing an input. The gateway shape
repairs (typeArrayItemUnions, hasGatewayIncompatibleShape), the second
dependentRequired-to-prose copy and the strict-gateway test denylist are
removed; the dialect checker gates the inline presentation instead.

* feat(mcp): omit output schemas from tools/list by default

No surveyed model API reads an output schema, MCP client validators
repeat REA's own result validation, and complete output schemas were
about 92% of the default tools/list. REA_MCP_OUTPUT_SCHEMA_PROFILE now
resolves to off through one owner (resolveMcpOutputSchemaProfile);
compact and full remain selectable. structuredContent is still
delivered, and the deferred server serves both off and full from the
recorded catalog with the matching recorded identity.

Tests and artifact verifiers that validate results against complete
advertised output contracts select full explicitly.

* docs: describe omitted output schemas as the default in README translations

* fix(mcp): select schemas in verification workflows

* fix(mcp): refuse zero-padded repeat bounds and record host-specific path patterns

* fix(mcp): keep projected schemas permissive

* fix(mcp): keep references valid when presentation widens their targets

* fix(mcp): keep a single union branch's closure scoped to its properties

* fix(mcp): select complete output schemas in the Electron verifier

* fix(mcp): select complete output schemas in standalone verifier probes

* fix(mcp): keep an allOf member's tuple tail with its own positions

* fix(mcp): relax references into negations and conditionals presentation may remove

* refactor(contracts): declare root discriminators as typed schema annotations

* test(mcp): select the borrowed Evidence contract from its family

* fix(mcp): keep astral literals and dynamic references widening in every mode

* fix(mcp): omit astral literals and relax untracked references when presentation widens

* fix(mcp): select complete output schemas in every verifier transport

* fix(mcp): read reference pointers by schema position, not segment spelling

* fix(mcp): protect references into collapsing boolean unions and keep referencing branches out of the collapse

* fix(mcp): keep reduced schemas open when their root admits names by pattern

* test(mcp): keep the reduced-schema pattern case outside the size-limited block

* fix(mcp): omit patterns whose classes name astral characters

* fix(mcp): decide union collapse and pointer protection from canonical branches

* docs(mcp): document non-u astral matching and pin BMP agreement across V8 modes

* fix(mcp): keep prototype-named properties as own properties in the dialect projection

* test(mcp): select full output schemas in packet and signal schema checks

* test(mcp): expect the advertised base64 schema to omit the unbounded pad-bit probe

* fix(mcp): protect references into unevaluated keywords lost evaluation removes

* fix(mcp): validate the doctor probe reply against the canonical output contract (#10655)

* fix(mcp): validate the doctor probe reply against the canonical output contract

* test(mcp): replay doctor probe validation under every output profile

* test(mcp): accept typed-branch nullable unions in the advertised dialect (#10649)

* test(mcp): select full output schemas in HTTP session workflows

* fix(mcp): emit stack-safe classes with non-ASCII members and protect draft-07 reference siblings

* fix(mcp): protect references into literal siblings of relaxed draft-07 references

* feat(mcp): omit output schemas from tools/list by default (#10646)

* feat(mcp): omit output schemas from tools/list by default

No surveyed model API reads an output schema, MCP client validators
repeat REA's own result validation, and complete output schemas were
about 92% of the default tools/list. REA_MCP_OUTPUT_SCHEMA_PROFILE now
resolves to off through one owner (resolveMcpOutputSchemaProfile);
compact and full remain selectable. structuredContent is still
delivered, and the deferred server serves both off and full from the
recorded catalog with the matching recorded identity.

Tests and artifact verifiers that validate results against complete
advertised output contracts select full explicitly.

* fix(mcp): select schemas in verification workflows

* fix(mcp): select complete output schemas in the Electron verifier

* fix(mcp): select complete output schemas in standalone verifier probes

* fix(mcp): select complete output schemas in every verifier transport

* test(mcp): select full output schemas in packet and signal schema checks

* fix(mcp): validate the doctor probe reply against the canonical output contract (#10655)

* fix(mcp): validate the doctor probe reply against the canonical output contract

* test(mcp): replay doctor probe validation under every output profile

* test(mcp): accept typed-branch nullable unions in the advertised dialect (#10649)

* test(mcp): select full output schemas in HTTP session workflows

* fix(mcp): keep the Evidence union on described borrowed inputs (#10597)

* fix(mcp): keep the Evidence union on described borrowed inputs

borrowEvidenceInput installed its advertised projection on one schema
instance. A call site's .describe() rebuilds the schema from its
definition, so match_native_functions advertised left and right as
unconstrained values in every input profile. Install the projection in
the borrowed parser's constructor so derived schemas keep it, for both
borrowed inputs and borrowed arrays.

* test(mcp): select the borrowed Evidence contract from its family

* docs(mcp): admit baseline growth only for restored contract declarations

* fix(mcp): state reduced multi-type properties as typed branches, not type lists

The reduced fallback derived a property's types and emitted them as a type
list, which Gemini, Moonshot and Ark reject. A property whose canonical
schema is anyOf string and number therefore traded a typed union for a
rejected construct when the inline projection fell back. Emit one type, or
anyOf typed branches, so [T, null] becomes the dialect's nullable union.

* feat(mcp)!: retire the full schema profiles and declare complete observations as a typed annotation (#10718)

* refactor(mcp): declare complete observations as a typed annotation applied only on the wire

* test(mcp): compare complete-observation declarations with their owning text

* refactor(schema): couple Zod checks with their JSON Schema constraint keywords (#10725)

* refactor(schema): couple Zod checks with their JSON Schema constraint keywords

* refactor(schema): declare pattern approximations and object output roots as typed annotations (#10754)

* refactor(schema): declare pattern approximations and object output roots as typed annotations

* fix(schema): refuse pattern declarations JSON Schema cannot state

* test(schema): select guided_workflows from its contract family

* feat(mcp)!: retire the full input and output schema profiles (#10841)

* feat(mcp)!: retire the full input and output schema profiles

The compact input presentation is now the only advertised input schema, and
output schemas are either omitted (`off`, the default) or bounded
(`compact`). Catalog identity, the build-recorded digests and the recorded
startup catalog collapse to those presentations: two digest profiles instead
of six, one recorded catalog instead of two, and no input dialect baseline.

Tests and verifiers that validated replies against listed full output
schemas now validate against the canonical output contract, which REA
already enforces server-side; tests that asserted canonical input
restrictions read the canonical input schema instead of a full listing.

BREAKING CHANGE: `REA_MCP_INPUT_SCHEMA_PROFILE=full` and
`REA_MCP_OUTPUT_SCHEMA_PROFILE=full` are refused at startup and by `rea setup`
with a configuration error naming the setting. Remove the setting from the
client registration (or rerun setup with `REA_MCP_INPUT_SCHEMA_PROFILE=compact`
/ `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact`). `REA_MCP_INPUT_SCHEMA_PROFILE`
still accepts `compact`. Server-side validation is unchanged.

* test(mcp): validate replies against canonical output contracts at every listed-schema reader

tools/list omits output schemas by default, so fixtures, verifiers and the
Android analysis runners that read a listed outputSchema now bind the
canonical contract instead. connectLocalToolsMcp and advertisedMcpToolSchemas
hand the SDK a definition carrying the canonical output schema, so the SDK
client and caller-supplied validators check every successful reply.

* test(mcp): resolve Android runners' output contracts from the selected repository

The Android efficiency runners imported the canonical output-schema helper from the harness checkout while --repository selects the revision under test. Import it from the selected repository, as the runners already do for the SDK and Ajv.

* test(mcp): let Android runners compare revisions before the canonical output helper

A selected revision without scripts/lib/canonical-output-schema.mjs could not start the Android runners. Choose the output contracts in one harness helper: the selected revision's canonical contracts when it has them, otherwise its listed schemas under the full output profile it still supports.

* fix(scripts): keep caller launch settings when the real Hopper verifier builds its server environment

The verifier deleted REA_VERIFY_SERVER_COMMAND and REA_VERIFY_SERVER_ARGS_JSON from process.env itself before reading them, so a caller selecting an installed REA command silently ran the checkout's dist/main.js. Copy the environment, as main does, so only the child's environment changes.

* test(mcp): validate whole replies against canonical output contracts wherever a listed definition is held

tools/list omits output schemas by default and the full output profile is gone, so tests that handed the SDK a listed definition no longer checked complete replies. Attach the canonical output schema with withCanonicalOutputSchema at every held REA tool definition. Environment-served tools (not REA contracts), and the deferred-runtime and doctor tests that exercise per-profile listings, keep their listed definitions.

* test(mcp): record the advertised catalog summary without the retired full profiles

Retiring the full input and output profiles leaves only the compact input and compact output listings, so the summary drops the full-profile digests and every pattern only the full listings carried. Regenerated with npm run verify:advertised-schemas -- --update-summary.
2026-11-15 21:51:18 +08:00
morluto bbe2bcc979 refactor: remove legacy seams and consolidate resource ownership (#10349) 2026-11-13 04:45:16 -05:00
morluto 60149282fc test: strengthen integration oracles and verification (#8295)
* test: strengthen integration oracles and verification

* test: bound lifecycle gates and strengthen workflow checks

* test: verify complete paged and extracted results

* test: preserve bytecode and cancellation evidence

* test: strengthen retained evidence and fixture cleanup

* test: require cross-projection bridge identities

* test: retain fixture owners through cleanup failures

* test: verify graph examples and project selectors

* test: tighten provider and lifecycle regressions

* test: consolidate hex coverage and bound cutter cleanup

* test: bound subprocess output waits and strengthen graph checks

* test: check mcp recovery and preserve cleanup owners

* test: verify structured mcp argument failures

* test: preserve artifact cleanup and gdb limit evidence

* test: preserve evidence identities and bound provider gates

* test: retain browser and comparison cleanup owners

* test: bound startup and strengthen fixture oracles

* test: bound lifecycle gates and verify recovered evidence

* test: verify public outcomes and lifecycle ownership

* test: assert precise evidence and release lifecycle gates

* test: verify selected facts and bound cleanup waits

* test: check retained evidence and decoded records

* test: verify final reports and retain cleanup ownership

* test: bound reviewed startup gates and retain cleanup

* test: verify PE errors and preserve MCP cleanup

* test: preserve MCP fixture cleanup ownership

* test: release provider gates before late startup
2026-11-01 00:15:28 +08:00
morlutoandCopilot Autofix powered by AI 675d4076de test: strengthen regression oracles and cleanup ownership (#8152)
* test: strengthen suite oracles and cleanup ownership

* test: reuse setup across independent scenarios (#8153)

* Potential fix for pull request finding 'CodeQL / Improper code sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-10-31 00:22:16 +08:00
gensokyo-dev e2463372ea feat(python): query selected pyc code-object facts (#6928)
* feat(python): query retained pyc code-object facts

* test(python): bind complete pyc selection benchmark tasks

* test(python): include server instructions in token accounting

* docs(perf): record pyc selected-facts task measurements

* test(python): keep oracle dependencies separate from selected targets

* docs(perf): record guarded oracle requalification

* test(python): narrow complete Evidence through its owning contract
2026-10-24 10:23:06 +08:00
morluto d0a8e3c60d fix(python): read legacy TYPE_INT64 marshal values
Fixes #6609
2026-10-23 19:07:45 +08:00
morluto c6ab965062 fix(python): preserve historical signed code fields
Keep signed co_stacksize and co_flags for historical magic entries whose constructors accepted them. Preserve magic 3410 with a constructor-revision limitation; reject negative values from 3411 onward.

Fixes #6487
2026-10-23 18:31:02 +08:00
morluto 81de82a1a4 fix(python): classify historical headers and signed flags
Fixes #6568, #6487
2026-10-23 17:43:07 +08:00
morluto b58aa63796 fix(python): preserve signed code-object line numbers
Fixes #6490
2026-10-23 16:31:49 +08:00
2c3dd946d5 feat(python): inspect CPython pyc headers and marshal code objects (#4826)
* feat(python): inspect CPython pyc headers and marshal code objects

Add inspect_python_pyc / inspect-python-pyc to read one explicit local
CPython 3.7–3.13 .pyc without executing bytecode or recovering source.

* chore(python): drop unused PythonPycHeader export

Knip reported the type as unused after inspect_python_pyc landed.

* fix(python): bound pyc expansion and decode marshal like CPython

- Count a nested code object reached through TYPE_REF once per copy the
  report carries, against max_objects and max_file_bytes, before counting
  or validating the tree
- Charge long digits to max_string_bytes, reject out-of-range and
  unnormalized digits, build the value in one pass, and report integers
  above 256 bits by bit length only
- Decode TYPE_UNICODE with surrogatepass semantics and reject other
  ill-formed UTF-8
- Map only 3.11 magics 3494-3495 to the py311 layout
- Report non-finite float and complex parts as inf, -inf or nan
- Allow an empty co_filename
- Require bytes for co_linetable and co_exceptiontable
- Bind cleanup-failure errors to their retained descriptor leases

* fix(python): list pyc code objects once instead of a recursive tree

Report each distinct code object once in code_objects, top-level first,
with nested_code as indexes. A child shared through TYPE_REF is no longer
copied per reference, and the MCP output schema is no longer recursive.
Names repeated through TYPE_REF still count against max_file_bytes for
each copy the report carries.

* fix(python): bound shared pyc constants and reject malformed tables

- Cache byte-string digests so a buffer shared through TYPE_REF is
  hashed once instead of on every reference.
- Charge every reported co_consts entry, including tables shared
  through TYPE_REF, against max_objects before summaries are built.
- Resolve the inspect-python-pyc path with absoluteCallerPath so
  link/../file.pyc reads the file the filesystem selects.
- Reject TYPE_NULL inside marshal sequences, as CPython does.
- Reject a co_localspluskinds length that does not match
  co_localsplusnames in the parser and the report schema.

* fix(python): bound shared pyc names and read CPython marshal graphs

Charge each reported code-constant name, keep integer bit lengths on the
decoded value, and list code objects nested inside constant containers.
Accept self-referential lists and dicts, marshal version 0 text floats,
empty code names, the original string byte count, and Python set and dict
lengths. Pass the CLI cancellation signal into command cleanup.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(python): bound pyc key expansion and shared nested code

Charge set and dictionary keys before a shared tuple doubles them, list each
nested code object once per owner, and accept code objects as CPython keys.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(python): correct pyc equality, counts, and nested walks

Summarize code objects only after the marshal graph is complete, and
collect nested code with an iterative walk that shares wrapper results
instead of copying every descendant list. Keep tuple boundaries and the
original string bytes in equality keys, and reject negative CPython counts.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(python): bound shared pyc names and keep the first dict key

Charge every copied name-table entry against the object budget before
building the report array, including empty strings that add no bytes.
When dictionary keys compare equal, keep the first key object and
replace only its value, matching CPython.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(python): walk each pyc dictionary child list once

Keep a container's child array on the nested-code frame so a wide
dictionary is not copied on every step of the synchronous marshal walk.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(python): reuse stored pyc dictionary children when combining

The nested walk keeps each container's child list on its frame.
Combining nested code rebuilt that list, and the 8,000-key regression
still accepted the old per-key copy. Use the stored list, and require
that dictionary to finish well under the copy's runtime while still
reporting a nested code object.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(python): keep distinct pyc code objects and reject corrupt ones

Encode each name table's length and entry boundaries in the equality
key so a different name list cannot share a memo key. Walk tuple and
frozenset keys on an explicit stack so a long TYPE_REF chain does not
overflow. Treat code objects CPython rejects for an odd co_code or an
undersized local table as malformed instead of complete coverage.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-22 19:00:08 +08:00