mirror of
https://github.com/morluto/rea.git
synced 2026-11-08 07:46:28 +00:00
feat(python): qualify a recognized 3.14/3.15 pyc header (#8927)
* feat(python): qualify a recognized 3.14/3.15 pyc header Python 3.15 is the default `python3` on current Homebrew, but its pyc was rejected with "Pyc magic 3666 is outside the pinned CPython 3.7–3.13 table" — a message that says nothing about the file and reads as if the version were unknown. Issue #8206. Two facts were wrong. **3.15 was not in the identity table at all,** so `pythonBytecodeIdentity` reported it as unknown even though the table already pins 3.14 (3627). 3666 is now pinned as 3.15. **A known identity without a pinned code layout shared one refusal with an unknown magic.** The two are different outcomes: every 3.7+ pyc carries the same packed 16-byte header, so a recognized newer magic can still be qualified even when no marshal layout is pinned for it. The parser now separates them: magic 3666 (3.15) -> CPython 3.15 header is recognized (magic 3666, 16-byte header, flags 0x0, timestamp-based), but no marshal code layout is pinned for that version, so its code objects cannot be decoded. Standalone marshal decoding is pinned for CPython 3.7–3.13. magic 1 -> Pyc magic 1 is not a pinned CPython identity, so its header cannot be qualified; ... The recognized case reports the observed version, the header shape, the PEP 552 flags and whether the pyc is hash-based or timestamp-based, which is what an agent needs to decide that a `.jsc`-style text-recovery attempt is pointless and that the file is a 3.15 pyc rather than a corrupt one. It stays a typed `unsupported` failure: no pinned layout means no decoded code object, so there is no partial success. I did not guess a 3.14/3.15 marshal layout. The 3.11+ `TYPE_CODE` order is pinned through 3.13; asserting the same order holds for 3.14 or 3.15 without a real interpreter to verify against would be an unsupported claim, which is why this change qualifies the header instead of decoding it. Docs and the tool description now say marshal decoding is pinned for 3.7–3.13 while the identity table also recognizes 3.14 and 3.15. Three regressions fail on the base commit. * chore: retrigger title and readiness checks after the title was corrected * test(python): pin the qualified header facts to the packed fields Review finding. Every new case used `pythonPycHeader(magic)` with its default zero flags and never asserted the reported flags or header mode, so a refusal that hardcoded `flags 0x0, timestamp-based` would have passed all of them. The cases now build the header from explicit fields and assert the facts that field produces: `flags 0x0/timestamp-based` and `flags 0x1` or `0x3` with a PEP 552 source hash for hash-based. A separate case parses the same magic in both modes and requires each refusal to name its own mode and not the other. Verified by hardcoding `flags 0x0` in the refusal: the two hash-based cases fail.
This commit is contained in:
1 parent
99bcd3e4e0
commit
fe9797b487
7 files changed
+123
-12
No files matched your search
@@ -157,12 +157,97 @@ it.each([
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps a known newer CPython identity outside standalone decoder coverage", () => {
|
||||
expect(() => parse(pythonPycHeader(3627))).toThrow(
|
||||
it.each([
|
||||
{ magic: 3627, version: "3.14", flags: 0x0, mode: "timestamp-based" },
|
||||
{ magic: 3666, version: "3.15", flags: 0x0, mode: "timestamp-based" },
|
||||
// A non-zero flag word and a PEP 552 hash-based header, so a refusal that hardcoded
|
||||
// `flags 0x0, timestamp-based` could not pass.
|
||||
{ magic: 3666, version: "3.15", flags: 0x3, mode: "hash-based" },
|
||||
{ magic: 3627, version: "3.14", flags: 0x1, mode: "hash-based" },
|
||||
] as const)(
|
||||
"qualifies the recognized CPython $version header without a pinned code layout",
|
||||
({ magic, version, flags, mode }) => {
|
||||
// The header is still a packed 16-byte 3.7+ header, so its identity and flags are
|
||||
// reported even though no marshal layout is pinned for this version.
|
||||
const header = pythonPycHeader(magic, {
|
||||
flags,
|
||||
...(flags === 0 ? {} : { sourceHash: Buffer.alloc(8, 0xab) }),
|
||||
});
|
||||
expect(() => parse(header)).toThrow(
|
||||
expect.objectContaining({
|
||||
kind: "unsupported",
|
||||
message: expect.stringContaining(
|
||||
`CPython ${version} header is recognized`,
|
||||
),
|
||||
}),
|
||||
);
|
||||
expect(() => parse(header)).toThrow(
|
||||
expect.objectContaining({
|
||||
message: expect.stringContaining(`magic ${magic}, 16-byte header`),
|
||||
}),
|
||||
);
|
||||
// The reported facts are the ones packed into this header.
|
||||
expect(() => parse(header)).toThrow(
|
||||
expect.objectContaining({
|
||||
message: expect.stringContaining(`flags 0x${flags.toString(16)}`),
|
||||
}),
|
||||
);
|
||||
expect(() => parse(header)).toThrow(
|
||||
expect.objectContaining({
|
||||
message: expect.stringContaining(mode),
|
||||
}),
|
||||
);
|
||||
expect(() => parse(header)).toThrow(
|
||||
expect.objectContaining({
|
||||
message: expect.stringContaining(
|
||||
"no marshal code layout is pinned for that version",
|
||||
),
|
||||
}),
|
||||
);
|
||||
expect(() => parse(header)).toThrow(
|
||||
expect.objectContaining({
|
||||
message: expect.stringContaining(
|
||||
"marshal decoding is pinned for CPython 3.7",
|
||||
),
|
||||
}),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("reports timestamp-based for a pyc without the hash flag and hash-based with it", () => {
|
||||
// Same magic, opposite header mode: the two refusals must differ only in that fact.
|
||||
const timestamp = () =>
|
||||
parse(pythonPycHeader(3666, { mtime: 7, sourceSize: 9 }));
|
||||
const hash = () =>
|
||||
parse(
|
||||
pythonPycHeader(3666, { flags: 0x1, sourceHash: Buffer.alloc(8, 1) }),
|
||||
);
|
||||
expect(timestamp).toThrow(
|
||||
expect.objectContaining({
|
||||
message: expect.stringContaining("timestamp-based"),
|
||||
}),
|
||||
);
|
||||
expect(timestamp).toThrow(
|
||||
expect.not.objectContaining({
|
||||
message: expect.stringContaining("hash-based"),
|
||||
}),
|
||||
);
|
||||
expect(hash).toThrow(
|
||||
expect.objectContaining({ message: expect.stringContaining("hash-based") }),
|
||||
);
|
||||
expect(hash).toThrow(
|
||||
expect.not.objectContaining({
|
||||
message: expect.stringContaining("timestamp-based"),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("does not qualify a magic outside the pinned identity table", () => {
|
||||
expect(() => parse(pythonPycHeader(1))).toThrow(
|
||||
expect.objectContaining({
|
||||
kind: "unsupported",
|
||||
message: expect.stringContaining(
|
||||
"outside the pinned CPython 3.7–3.13 table",
|
||||
"is not a pinned CPython identity, so its header cannot be qualified",
|
||||
),
|
||||
}),
|
||||
);
|
||||
|
||||
Reference in new issue
Block a user