* feat(v8): inspect V8 code cache containers and their retained byte patterns
Bytenode-compiled `.jsc` files — the V8 code cache container that `vm.Script`,
`vm.compileFunction` and bytenode 2.x write, and that Node single-executable applications
and Electron apps ship — had no format-native reader. `inspect_binary_layout` shows the
surrounding ELF/PE and `inspect_artifact` inventories the file, but neither decodes the V8
structures, so an agent could not tell which V8 line produced the file, could not learn the
container is metadata rather than source, and had nothing to point at a version-pinned
decompiler. Issue #8543.
`inspect_v8_code_cache` joins the format-native family with `inspect_python_pyc`,
`inspect_erlang_beam` and `inspect_kernel_module`: one explicit local file, bounded parsing,
no execution, no installation, no child process.
**The container is version-dependent, so the reader reports roles with confidence instead
of assuming an ABI.** All eight header words are reported with byte offset, observed value,
the role this reader believes the word has, and whether that role is `confirmed`,
`inferred`, or `unconfirmed`. Only three facts gate admission: the header size this reader
pins, the V8 code cache magic high half `0xc0de` (the low half is
`ExternalReferenceTable::kSize` and varies per build, so it is a build fingerprint rather
than a constant), and a declared payload length that reaches exactly the end of the file.
I derived the field semantics from the producer rather than assuming them, and two findings
changed the design:
- **The word V8 calls the source hash holds the source length on the measured producers.**
A 54-code-unit source stored `54`, and sources with the same code-unit length but
different content stored the same value, including a source whose UTF-8 length was 154.
The reader reports it as `source_length` and the docs say that this name is what the
measurement supports, not a claim about what another build stores there.
- **`vm.compileFunction` sets a high bit in that word.** The reader masks it out and reports
the producer family as `script` or `compile_function`, so the source length stays correct
for both producers.
Embedded strings are reported as **byte-pattern observations with absolute byte ranges**,
labelled as such in the field name and in the limitations. A reported sequence is not proof
V8 retained it as a string, and a missing literal is not proof it is absent.
**Fail-closed boundaries, all covered by tests:** an unknown magic is a typed
`unsupported_target` failure because the file is not a V8 code cache; a short header or a
payload-length disagreement is a typed `invalid_request` failure. Neither yields a partial
parse and neither records Evidence.
Non-goals are explicit in the tool description and the docs: the payload is serialized V8
heap data, not source; recovering logic needs a version-pinned View8-style decompiler that
this tool neither provides nor depends on.
Fixtures are generated from the running V8 instead of committing a version-pinned binary, so
the tests exercise the real producer and stay valid across Node upgrades.
- Semantic owner and earliest changed stage: format-native V8 code cache container decoding.
- CLI and MCP/tool-catalog contract: one new `artifact-provider` tool and one new
`inspect-v8-code-cache` CLI command, registered in `TOOL_CONTRACTS`, the `artifact`
language family, `TOOL_EFFECTS`, the capability declarations and services, and the
capability inventory. `docs:check` passes, so the generated catalog agrees.
- Provider, bridge, target-format, or platform compatibility: no provider runs and no
external tool is required, so the tool is available on every supported Node host.
- Evidence, artifact, provenance, or reconstruction contract: complete Evidence with the
payload byte range as its location; no artifact format, reader, or reconstruction change.
- Generated metadata, package, or installation impact: catalog and document facts
regenerated by the existing checks; no new dependency.
Expected effects:
- [x] Target content is returned as bounded structured facts.
- `src/v8/V8CodeCacheParser.test.ts` (13 cases): real-producer header facts, per-field role
and confidence, source length for both producers, an observation's absolute offset
verified against the file bytes, the limitation text, foreign magic, payload-length
mismatch, short header, missing payload bytes, a different build constant still admitted,
and the observation budget.
- `tests/boundary/mcp/v8CodeCacheMcp.test.ts` (4 cases): advertisement and annotations,
header facts over MCP, relative-path rejection, unsupported magic, malformed container.
- `tests/acceptance/v8CodeCacheCli.test.ts` (4 cases): the public CLI, relative-path
resolution, byte-range observations, and the typed failure.
- `npx vitest run src/contracts src/v8 src/application/v8CodeCache src/domain/v8` — 242 passed
- `npx vitest run tests/boundary/mcp/v8CodeCacheMcp.test.ts tests/acceptance/v8CodeCacheCli.test.ts tests/boundary/filesystem/productCatalog.test.ts tests/acceptance/tools/localToolContracts.test.ts` — 20 passed, 4 skipped
- `npx tsc --noEmit -p .` — exit 0
- `npm run build` — exit 0
- `npm run lint` — 0 errors
- `npm run knip` — exit 0
- `npm run docs:check` — exit 0
- `npx oxfmt --check src tests docs` — exit 0
Gap: the header table is measured on node v24.22.1 / V8 13.6.233.17-node.58 only. No other
V8 build is available here, so a different build's field order is untested; that is exactly
why only three facts gate admission and the remaining roles carry a confidence level rather
than being asserted.
Additive. No existing tool, schema, or behaviour changes. The reader never executes the
selected file, never launches a process, and never accesses the network; its audited effects
are `mutatesSession: true` only, matching the other static format readers.
Human owner: @XiaoBinGan
- [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
- [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
- [x] I declare this PR ready for initial maintainer review.
- [x] This PR addresses a concrete problem or an agreed enhancement.
- [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
- [x] The PR has one focused outcome and the title follows `type(scope): outcome`.
- [x] Related issue: #8543.
- [x] Tests cover changed observable behavior and meaningful failure paths.
- [x] Owning docs, contracts, and generated metadata are updated where needed.
- [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims.
* chore: retrigger title and readiness checks after the title was corrected
* fix(v8): bound the payload scan and use the repository path conventions
Review feedback from @greptile-apps on four points.
**Bound the synchronous payload scan.** The scan is one synchronous pass over the
payload, so the 512 MiB ceiling allowed a single request to hold the process for
several seconds. The ceiling is now 64 MiB, matching the pinned CPython pyc reader,
and the default is 32 MiB. The scan stays synchronous, like the BEAM and ELF note
readers in this family, but the budget that bounds it is no longer large enough to
matter. The new `max_file_bytes` ceiling and default are documented.
**Use the absolute-path schema.** `path` now uses `absoluteLocalPathStringSchema`
instead of `localPathStringSchema`, matching `inspect_erlang_beam`. A relative name
previously passed validation and was resolved against the server's working
directory, so `module.jsc` could select a different file than the caller meant. The
rejection is now at the schema boundary.
**State the extraction step.** The handoff in `docs/v8-code-cache.md` now says a
cache embedded in a single-executable application or an Electron app must be
extracted first, because this reader requires the magic at byte zero and returns
`unsupported_target` for the surrounding ELF/PE executable.
**Load only the owning contract group in tests.** Both tests now import
`V8_CODE_CACHE_TOOL_CONTRACTS` directly instead of resolving the tool through the
full catalog, so a focused test no longer constructs every other tool's schema.
21 cases pass across the parser, MCP boundary and CLI acceptance files; `tsc` 0;
`build` 0; `knip` 0; `docs:check` 0; `oxfmt --check` 0.
* fix(v8): select the tool contract by name in the focused tests
Reverts the catalog-position selection I added for a review suggestion. This
repository has a lint rule (`rea(no-positional-tool-contract)`) that forbids
selecting a tool contract by catalog order, and `toolContract("inspect_v8_code_cache")`
is how every neighbouring focused test selects one. The positional form I tried
introduced two lint errors, so the named form stays.
The review concern was that resolving through the full catalog constructs every
other tool's schema. That cost is real but it is the repository's chosen trade:
catalog ordering must never select an operation or schema, and `toolContract` is
the named accessor that enforces it.
* test(v8): cover the new tool in the absolute-path boundary test
Main added `tests/boundary/mcp/absoluteLocalPathTools.test.ts` after this branch
was opened. It enumerates every tool whose MCP `tools/list` schema must reject a
relative path, and `inspect_v8_code_cache` uses `absoluteLocalPathStringSchema`,
so the tool belongs in that list. Adding it keeps the enumerated field set complete
instead of leaving one absolute-path tool untested.
`tests/boundary/mcp` + v8 + contracts: 256 passing. `tsc` 0; `build` 0; `knip` 0;
`docs:check` 0; `npm run lint` 0 errors.
* perf(v8): read the payload by index instead of allocating per byte
`for (const [index, byte] of payload.entries())` allocates a two-element array for
every byte of the cache payload. On a 1 MiB payload that allocation dominated the
scan.
Measured with a real cache built from this parser, alternated so the two forms run
against the same input:
| payload | entries() | indexed reads |
| --- | --- | --- |
| 1 MiB | 15.5ms | 4.4ms |
That is 3.49x on the scan, and the gap widens on smaller payloads where the
allocation cost is a larger share (7.3x at 64 KiB in a synthetic run of the same
shape). The scan is the whole cost of `inspect_v8_code_cache` once the file is read,
so this is the tool's own hot loop.
Output is unchanged: the parser returns one retained string for the 64 KiB fixture,
and `src/v8` plus the CLI and MCP boundary files stay at 21 passing. `tsc` 0;
`build` 0; `npm run lint` 0 errors.
* fix(v8): admit only decimal tokens on the numeric CLI options
Main added `tests/boundary/cli/cliDecimalArguments.test.ts` after this branch was
opened. It enumerates every numeric CLI option and requires `0x10` and `1e3` to be
rejected, because plain `Number` coercion would read them as 16 and 1000. This
command's three options forwarded the domain fields unchanged, so the test failed on
them:
inspect-v8-code-cache options.maxFileBytes: Parser.parse accepts 1e3
inspect-v8-code-cache options.maxStrings: Parser.parse accepts 0x10
inspect-v8-code-cache options.maxStrings: Parser.parse accepts 1e3
inspect-v8-code-cache options.minStringBytes: Parser.parse accepts 0x10
`cliDecimalField` publishes the input view that parses the decimal token first and
then lets the domain field own bounds, defaults and help, which is what the other
numeric commands use.
`tests/boundary/cli/cliDecimalArguments.test.ts` now passes. `tsc` 0; `build` 0;
`npm run lint` 0 errors; `knip` 0; `docs:check` 0.
* fix(v8): recognize module caches, and advise from remaining headroom
Two review findings, each measured on Node 24.22.1 before changing anything.
**A module cache was reported as a script.** `vm.SourceTextModule.createCachedData()`
sets bit 30 of the source-length word, which the source-length mask `0x1fffffff` leaves
out, and the producer check only tested the function bit at bit 29:
| producer | source word |
| --- | --- |
| `vm.Script` | `0x00000013` for a 19-unit source |
| `vm.compileFunction` | `0x20000009` for a 9-unit source |
| `vm.SourceTextModule` | `0x40000013` for a 19-unit source |
So a real module cache was labelled `script`, and `vm.Script` rejects that container. The
producer enum gains `module` and the module bit is tested first; the source length is
unchanged because both flags sit outside the mask.
**The budget advice recommended a retry that could not work.** Both limits have
ceilings — `max_strings` at 65536 and `min_string_bytes` at 256 — so an admitted file
with more than 65536 runs of 256 bytes reached the branch while every suggested retry
was a rejected input. `observationBudgetRemediation` now reads the remaining headroom
and only recommends a change that is still admissible, falling back to selecting a
smaller artifact when neither can move. The parser message states the fact and leaves
recovery to the provider, which is the only place that can see both budgets.
`MAX_INSPECT_V8_CODE_CACHE_MIN_STRING_BYTES` names the `min_string_bytes` ceiling that
the schema previously spelled inline, so the advice and the schema cannot drift.
`docs/v8-code-cache.md` describes both flag bits. `src/v8` plus the CLI and MCP boundary
files: 28 passing; 7 new cases fail on the base commit. `tsc` 0; `build` 0;
`npm run lint` 0 errors; `knip` 0; `docs:check` 0.
* refactor(v8): decide the budget recovery as data, and test the decision
The remediation tests matched fragments of the advice sentence and banned old phrases.
`docs/testing.md` asks for error codes, recorded limits and recovery behaviour instead of
frozen prose, so a harmless reword could fail those tests while wrong advice passed them.
`observationBudgetRecovery` now returns the decision — which retries remain within the
schema's admitted range, and whether a smaller artifact is the only option — and
`observationBudgetRemediation` renders one sentence from it. The tests assert the
decision, so the wording is free to change and an incorrect choice cannot pass.
This also fixes a bug in the previous shape: it took the `min_string_bytes` branch to mean
"lower it", but that branch is reachable with the value already at the schema minimum,
where lowering is not a legal retry. The decision now carries `canLowerMinStringBytes`
and a `requiresSmallerArtifact` flag for that case.
5 cases, all failing on the base commit, covering both headroom states, either ceiling
alone, and the run-length floor. `src/v8` plus the CLI and MCP boundary files: 28
passing. `tsc` 0; `build` 0; `npm run lint` 0 errors; `knip` 0; `docs:check` 0.
* ci: rerun after the wakaru release download recovered
The only failing job on this branch was `javascript_recovery / real-javascript-recovery`,
which failed on `curl: (22) The requested URL returned error: 504` while downloading
`wakaru-linux-x64.tar.gz`. The release is reachable again (HTTP 200), so this empty
commit re-triggers the lane.
* docs(v8): place the guide in the tree and record the feature
The guide sat at `docs/v8-code-cache.md`, outside the `docs/guides/` tree every other
guide uses, and neither index linked it, so it could not be reached from
`docs/README.md` or the VitePress sidebar.
Moved to `docs/guides/javascript/`, linked from both indexes, and recorded the feature
under `docs/specs/v8-code-cache-inspection/` with `spec.md`, `plan.md` and `tasks.md`.
The spec keeps only the premises that were measured on Node 24.22.1 — the three producer
words and the retained-string layout — rather than restating the implementation.
`npm run docs:check` 0; `npm run format:check` 0; `tsc` 0; `build` 0; `lint` 0 errors;
`knip` 0. V8 reader, CLI and MCP files: 28 passing.
* docs(v8): use the documented spec path and test the advice against its struct
Two review findings.
**The records were at `docs/specs/v8-code-cache-inspection/`.** `AGENTS.md:73` asks for
`docs/specs/<domain>/<feature-slug>/`, so they move to
`docs/specs/v8/code-cache-inspection/`.
**The verification section claimed more than the tests do.** It said the producer cases
were built from real `vm.Script`, `vm.compileFunction` and `vm.SourceTextModule` output,
and that the budget advice was compared with its owning formatter. In fact only
`vm.Script` and `vm.compileFunction` are real producers — the module case sets the
measured bit on a `vm.Script` cache — and only `observationBudgetRecovery` was asserted.
Corrected the claim, and closed the second gap rather than only describing it:
`observationBudgetRemediation` now has four cases holding the sentence to the struct it
renders from. The sentence must not advise a knob the struct closed, and must mention a
smaller artifact exactly when the struct requires one. Verified by making the sentence
advise `max_strings` after the ceiling was reached — the test fails. Wording stays free;
wrong advice does not.
`npm run docs:check` 0; `npm run format:check` 0; `tsc` 0; `build` 0; `lint` 0 errors;
`knip` 0. V8 reader, CLI, MCP, absolute-path and decimal-argument files: 34 passing.
* fix(v8): restore the absolute-path contract entry after the rebase
The rebase onto `main` conflicted in `tests/boundary/mcp/absoluteLocalPathTools.test.ts`:
`main` gained `inspect_api_contract`, `inspect_devicetree`, `inspect_playwright_trace` and
`inspect_python_pyc` in both the contract list and the path-field map while this branch
added `inspect_v8_code_cache`. Both sides are additive, so both are kept.
Without the resolution the branch carried conflict markers, which made `mergeable_state`
`dirty` and every check on it meaningless. Re-verified after the rebase: `tsc` 0, `build` 0,
`format:check` 0, `lint` 0 errors, `knip` 0, `docs:check` 0, and the V8 reader plus its CLI,
MCP, absolute-path and decimal-argument boundary files at 34 passing.
* test(v8): pin the producer from the runtime's own header word
An independent reviewer ran these files on Node 22.23.4, 24.22.1 and 26.12.1. Only 22.23.4
failed, on `separates the compileFunction producer bit from the source length`, expecting
`compile_function` and reading `script`.
The cause is the fixture, not the parser: `vm.compileFunction(...).cachedData` carries
`0x20000009` in the source-length word on Node 24.22.1, with bit 29 set, while Node 22.23.4
does not set that bit at all. Whether a given V8 build marks the word is a build detail, so
asserting one build's spelling made the check version-dependent.
The case now reads the word the runtime produced and asserts the parser maps it correctly,
and keeps the part that must hold everywhere: the source length is the low bits, so it
equals both `word & 0x1fffffff` and the source's length. The flag meanings themselves stay
pinned by the synthetic `withHeaderWord` cases above, which do not depend on any build.
Reverting the parser's `compile_function` arm fails this case and
`keeps the function and module flags distinct`. `src/v8` plus the CLI, MCP and absolute-path
boundary files: 24 and 34 passing. `tsc` 0; `build` 0; `npm run format:check` 0;
`npm run lint` 0 errors; `knip` 0.
* fix(v8): report an unflagged producer word as unknown, not as a script
`vm.compileFunction` marks bit 29 of the source-length word on Node 24.22.1 (`0x20000004`
for a 4-unit source) and does not mark it on Node 22.23.4 (`0x00000004` for the same call).
Whether a V8 build marks a function cache is therefore a build detail, and an unflagged
word is either a `vm.Script` cache or an unmarked function cache. The container carries
nothing that separates the two.
The reader reported `script` for that case, which is a confident claim the bytes do not
support. My previous commit made the test accept whichever value the runtime produced,
which recorded the ambiguity instead of resolving it in the contract. `unknown` is now a
producer value, the unflagged word maps to it, and the observed word stays in
`header.fields` for a caller that trusts a specific build fingerprint. A limitation line
says the same in the report itself.
`header.producer` widens to `["script", "compile_function", "module", "unknown"]`; `script`
is no longer produced, because no unflagged word can be shown to be one.
The spec's producer rule and its verification notes are updated to match. Reverting the
mapping to `script` fails 2 of the 24 `src/v8` cases. `src/v8` with the CLI, MCP and
absolute-path boundary files: 33 passing. `tsc` 0; `build` 0; `docs:check` 0;
`npm run format:check` 0; `npm run lint` 0 errors. `knip` reports one unused export
(`builtinDataEffectMethod`) on `main` as well, so it is not from this change.
* fix(artifacts): retain completed identity across native parser failures
* docs(v8): preserve unknown unflagged cache provenance
---------
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
* feat(python): qualify a recognized 3.14/3.15 pyc header
Python 3.15 is the default `python3` on current Homebrew, but its pyc was rejected with
"Pyc magic 3666 is outside the pinned CPython 3.7–3.13 table" — a message that says
nothing about the file and reads as if the version were unknown. Issue #8206.
Two facts were wrong.
**3.15 was not in the identity table at all,** so `pythonBytecodeIdentity` reported it as
unknown even though the table already pins 3.14 (3627). 3666 is now pinned as 3.15.
**A known identity without a pinned code layout shared one refusal with an unknown
magic.** The two are different outcomes: every 3.7+ pyc carries the same packed 16-byte
header, so a recognized newer magic can still be qualified even when no marshal layout is
pinned for it. The parser now separates them:
magic 3666 (3.15) -> CPython 3.15 header is recognized (magic 3666, 16-byte header,
flags 0x0, timestamp-based), but no marshal code layout is
pinned for that version, so its code objects cannot be decoded.
Standalone marshal decoding is pinned for CPython 3.7–3.13.
magic 1 -> Pyc magic 1 is not a pinned CPython identity, so its header
cannot be qualified; ...
The recognized case reports the observed version, the header shape, the PEP 552 flags and
whether the pyc is hash-based or timestamp-based, which is what an agent needs to decide
that a `.jsc`-style text-recovery attempt is pointless and that the file is a 3.15 pyc
rather than a corrupt one. It stays a typed `unsupported` failure: no pinned layout means
no decoded code object, so there is no partial success.
I did not guess a 3.14/3.15 marshal layout. The 3.11+ `TYPE_CODE` order is pinned through
3.13; asserting the same order holds for 3.14 or 3.15 without a real interpreter to
verify against would be an unsupported claim, which is why this change qualifies the
header instead of decoding it.
Docs and the tool description now say marshal decoding is pinned for 3.7–3.13 while the
identity table also recognizes 3.14 and 3.15.
Three regressions fail on the base commit.
* chore: retrigger title and readiness checks after the title was corrected
* test(python): pin the qualified header facts to the packed fields
Review finding. Every new case used `pythonPycHeader(magic)` with its default zero flags
and never asserted the reported flags or header mode, so a refusal that hardcoded
`flags 0x0, timestamp-based` would have passed all of them.
The cases now build the header from explicit fields and assert the facts that field
produces: `flags 0x0/timestamp-based` and `flags 0x1` or `0x3` with a PEP 552 source hash
for hash-based. A separate case parses the same magic in both modes and requires each
refusal to name its own mode and not the other.
Verified by hardcoding `flags 0x0` in the refusal: the two hash-based cases fail.
Bind typed service creation, observed availability and retryable cleanup in one composition owner. Reuse selected CLI/MCP factories and canonical contract groups without copying provider operation inventories.
Closes#8429. Refs #740.
Validated paired real application workflows, actual local Codex CLI Terra/high use, complete Evidence and Unknown checkpoint parity, compiler-backed import boundaries, SDK schemas and final real mixed routing. Automated review findings were fixed at their producing owners. Existing exact-base CodeQL alerts are documented on the PR under the owner-authorized unrelated-check exception; no global alerts or security rules were changed.
Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
Centralize completed file observations and cleanup error projection across stable-file providers while preserving each provider’s failure classification and exact cleanup leases. Cover BEAM finalization cancellation and devicetree cleanup retries.
Validated 223 focused lifecycle and MCP tests and check:fast.
Share CPython magic framing, version and header facts while keeping standalone marshal coverage and PyInstaller reconstruction eligibility distinct. Preserve legacy headers, opaque unknowns and source bytes.
Validated 272 focused tests across source, CLI and MCP workflows, a 38-test reader recheck, and check:fast.
Normalize completed stable-file path, digest and length in one shared observation owner. Migrate thirteen identical provider projections while preserving their selected paths, schema checks and cleanup composition.
Keep specialized metadata and worker reply observations at their existing owners. Validation: shared real-descriptor coverage and provider lifecycle tests passed; build, typecheck, lint and formatting passed.
Retain exact completed file path, digest and length in Python pyc and Erlang BEAM failure observations. Translate Python close failures at the provider boundary and bind them to the retained descriptor cleanup leases so later successful retries resolve prior errors.
Exercise actual services and file handles with completed reads, cancellation during finalization and physical read failure followed by cancellation. Assert no fabricated snapshot on early failure and resolution of every prior cleanup report.
Validation: six focused descriptor regressions plus existing filesystem, parser and MCP coverage passed; build and static checks passed. Authored format fixtures establish portable reader behavior, not CPython/Erlang runtime execution.
Carry descriptor outcomes, cancellation, retention and physical origin as named facts so wrappers cannot drop ownership context. Replace raw primary failure unwrapping with provenance facts across artifact consumers, and use Result for Kuna span reads including undefined rejections.
Exercise actual descriptors, rollback, cancellation and cleanup retry; update brittle snapshot and completion fixtures to use their producing boundaries.
Preserve source code-point prefix endpoints in the shared marshal UTF-8 decoder, including separately encoded surrogate values. Keep native TextDecoder spans, full validation, byte identity and the existing 80-unit budget. Update the PyInstaller consumer without changing its text contract.
Validation: 207 relevant source tests;24 affected compiled CLI/MCP/filesystem CI tests; independent CPython3.11.2 compiler/marshal oracle and real argparse;18 equivalent performance tasks; build/check passed. Codex reviewed7c0f7abc6b without major findings.
CI bypass authorized for demonstrated baseline-only failures:101 source and179 built cases reproduced on actual tested main ba9c5bc68. Cancelled shard remainder is unverified. See PR body and discussion for scope and evidence.
Fixes#6476
Use one marshal surrogatepass decoder for PYZ directories and Python bytecode.
Validate ordinary UTF-8 spans with the native fatal decoder, preserving BOMs,
lone surrogates, raw bytes and marshal reference slots.
Distinguish legacy full-pyc CArchive modules from modern marshal-only modules
before reconstructing headers. Share typed source/codec failures between stream
output and completion while retaining native-read drain and cleanup ownership.
Validated focused producer, reader, lifecycle, CLI and MCP regressions, plus
public inventory/extraction of CPython 3.14-generated marshal tables and code.
No packaged application was executed. Document original versus derived bytes.
Fixes#6800Fixes#6789
Keep signed co_stacksize and co_flags for historical magic entries whose constructors accepted them. Preserve magic 3410 with a constructor-revision limitation; reject negative values from 3411 onward.
Fixes#6487
* fix(python): reject code keys and set members with unhashable consts
* fix(python): refuse to hash a tuple that is still being read
* fix(python): hash a code object's co_consts tuple, not its item list
---------
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
* feat(python): inspect CPython pyc headers and marshal code objects
Add inspect_python_pyc / inspect-python-pyc to read one explicit local
CPython 3.7–3.13 .pyc without executing bytecode or recovering source.
* chore(python): drop unused PythonPycHeader export
Knip reported the type as unused after inspect_python_pyc landed.
* fix(python): bound pyc expansion and decode marshal like CPython
- Count a nested code object reached through TYPE_REF once per copy the
report carries, against max_objects and max_file_bytes, before counting
or validating the tree
- Charge long digits to max_string_bytes, reject out-of-range and
unnormalized digits, build the value in one pass, and report integers
above 256 bits by bit length only
- Decode TYPE_UNICODE with surrogatepass semantics and reject other
ill-formed UTF-8
- Map only 3.11 magics 3494-3495 to the py311 layout
- Report non-finite float and complex parts as inf, -inf or nan
- Allow an empty co_filename
- Require bytes for co_linetable and co_exceptiontable
- Bind cleanup-failure errors to their retained descriptor leases
* fix(python): list pyc code objects once instead of a recursive tree
Report each distinct code object once in code_objects, top-level first,
with nested_code as indexes. A child shared through TYPE_REF is no longer
copied per reference, and the MCP output schema is no longer recursive.
Names repeated through TYPE_REF still count against max_file_bytes for
each copy the report carries.
* fix(python): bound shared pyc constants and reject malformed tables
- Cache byte-string digests so a buffer shared through TYPE_REF is
hashed once instead of on every reference.
- Charge every reported co_consts entry, including tables shared
through TYPE_REF, against max_objects before summaries are built.
- Resolve the inspect-python-pyc path with absoluteCallerPath so
link/../file.pyc reads the file the filesystem selects.
- Reject TYPE_NULL inside marshal sequences, as CPython does.
- Reject a co_localspluskinds length that does not match
co_localsplusnames in the parser and the report schema.
* fix(python): bound shared pyc names and read CPython marshal graphs
Charge each reported code-constant name, keep integer bit lengths on the
decoded value, and list code objects nested inside constant containers.
Accept self-referential lists and dicts, marshal version 0 text floats,
empty code names, the original string byte count, and Python set and dict
lengths. Pass the CLI cancellation signal into command cleanup.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(python): bound pyc key expansion and shared nested code
Charge set and dictionary keys before a shared tuple doubles them, list each
nested code object once per owner, and accept code objects as CPython keys.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(python): correct pyc equality, counts, and nested walks
Summarize code objects only after the marshal graph is complete, and
collect nested code with an iterative walk that shares wrapper results
instead of copying every descendant list. Keep tuple boundaries and the
original string bytes in equality keys, and reject negative CPython counts.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(python): bound shared pyc names and keep the first dict key
Charge every copied name-table entry against the object budget before
building the report array, including empty strings that add no bytes.
When dictionary keys compare equal, keep the first key object and
replace only its value, matching CPython.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(python): walk each pyc dictionary child list once
Keep a container's child array on the nested-code frame so a wide
dictionary is not copied on every step of the synchronous marshal walk.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(python): reuse stored pyc dictionary children when combining
The nested walk keeps each container's child list on its frame.
Combining nested code rebuilt that list, and the 8,000-key regression
still accepted the old per-key copy. Use the stored list, and require
that dictionary to finish well under the copy's runtime while still
reporting a nested code object.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(python): keep distinct pyc code objects and reject corrupt ones
Encode each name table's length and entry boundaries in the equality
key so a different name list cannot share a memo key. Walk tuple and
frozenset keys on an explicit stack so a long TYPE_REF chain does not
overflow. Treat code objects CPython rejects for an odd co_code or an
undersized local table as malformed instead of complete coverage.
Co-authored-by: morluto <morluto@users.noreply.github.com>
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
