mirror of
https://github.com/morluto/rea.git
synced 2026-11-06 23:06:18 +00:00
e69f27fba0
* fix(web-capture): keep mitmproxy request targets and status codes in previews Capture previews selected HAR/SAZ headline names (url, status) only, so a native mitmproxy record lost its request target (scheme, host, port, authority, path) and its status_code while keeping the reason phrase, and the matching numeric sidecars were filtered out. Select those mitmproxy headline fields too; headers and payloads stay omitted. * fix(comparison): recheck historical calls-only changes (#2223) * fix(comparison): preserve incomplete function call evidence Keep missing, unknown, partial and unresolved call relationships unknown in function comparisons and reconstruction claims. Aggregate IDA caller and callee coverage without overwriting uncertainty. Validate with 144 focused tests, typecheck/lint, docs checks and test discovery. Real providers were not verified; the full headless IDA session suite requires unavailable Windows native controls. * fix(comparison): recheck historical calls-only changes --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> * fix(ci): isolate Linux Hopper process fixture inventory * fix(android): make DEX descriptor pattern portable * style: format comparison and IDA sources * fix(web-capture): preview mitmproxy targets only as safe scalars Select the mitmproxy target and status fields only for mitmproxy records, only when they are scalars, and the path only in origin form. An absolute, network-path or otherwise ambiguous path spelling can carry userinfo that the native bridge's redaction does not recognise, and a nested value could copy an arbitrary record subtree into the preview; both stay in complete record pages. Other formats keep their existing headline fields. * fix(web-capture): keep the asterisk-form OPTIONS target in mitmproxy previews * fix(web-capture): exclude ambiguous and oversized mitmproxy targets from previews Origin-form checks alone accepted slash-backslash and slash-control spellings such as /\user:password@host/a, which WHATWG resolves as an authority with credentials and the recorded transport-URL interpretation already treats as ambiguous credentials. Preview a mitmproxy path only when parseRecordedTransportUrl finds no credentials, and omit mitmproxy headline strings longer than 8 KiB so one record cannot exceed the reply budget; both stay in complete record pages. * fix(web-capture): keep userinfo-bearing mitmproxy target fields out of previews Native redaction covers only request path and authority, so a malformed state such as host "user:password@example.test" passed through the preview. Omit every non-path mitmproxy headline string that contains a userinfo marker (@); valid schemes, hosts and ports never do, and the value stays in complete record pages. * fix(web-capture): admit mitmproxy headline values only in their well-formed shapes A per-string length cap still let a malformed capture fill every target slot with escape-heavy text, so one default preview page could expand to about 18 MiB of JSON. Preview each mitmproxy field only in its own shape: a short scheme token, host/authority without controls, whitespace or userinfo markers, integer port and status code, and a path of at most 2 KiB without control characters. That bounds every record's headline to about 4 KiB serialized, even for an escape-heavy path. * fix(web-capture): bind mitmproxy headlines to their message side and integer type Select mitmproxy target fields only from the request and status_code only from the response, so an unknown scalar extension such as response.path or request.status_code is not promoted to a headline. Admit port and status_code only when the record's numeric sidecar reports the producer's integer type; a native float such as 443.0 is otherwise indistinguishable from 443 after JSON parsing. * fix(web-capture): require one canonical integer sidecar for mitmproxy numeric headlines An inline parent could pair request.port 443 with an integer sidecar whose literal is unbounded or does not represent the value, or with duplicate sidecars, and the selected-pointer filter would copy them into the preview. Admit port and status_code only with exactly one integer sidecar whose literal is the value's canonical decimal, which also bounds the copied literal to a few characters. * fix(web-capture): add mitmproxy headlines only for HTTP flows The mitmproxy container format does not make every retained flow HTTP; the bridge keeps non-HTTP and unknown flow states with their extensions. Add the mitmproxy target and status candidates only when the record's own type is "http". * fix(web-capture): require consistent numeric sidecars for every mitmproxy headline Selecting a field also selects its numeric sidecars, so a string target such as request.path with an unrelated multi-megabyte sidecar copied that literal into the preview. Admit integer slots only with exactly one canonical integer sidecar and string slots only with no sidecar. * fix(web-capture): fail closed on any userinfo marker in previewed mitmproxy paths URL parsing can fail before recognising userinfo: /\user:password@example.test:65536/a has an invalid port, so the parser reported no credentials and the path was previewed with them. Every userinfo needs "@", so preview a path only when it contains none and does not start with // or /\; such paths stay on complete pages. The parser check is no longer needed. * fix(web-capture): never promote a redacted pointer to a mitmproxy headline Inline or imported Evidence can carry a value at a pointer that its own redactions mark as sensitive. Skip any mitmproxy headline whose pointer has a redaction entry, so a preview never re-emits a value the record says was redacted. * fix(web-capture): check integer headline types before comparing sidecar literals The sidecar comparison called String(value) before the numeric shape check, so an inline object such as {"toString": null} at request.port threw a TypeError out of the view instead of being omitted. Require a number before canonicalizing it. * fix(web-capture): honor ancestor value redactions before selecting mitmproxy headlines A value redaction at /request or the record root excludes its whole subtree, but only exact-pointer matches were honored, so contradictory inline Evidence could still preview /request/host or /request/path. Treat a value redaction as covering every descendant pointer; a property-name redaction still only identifies the parent of an omitted key and hides no sibling values. * fix(web-capture): treat redacted byte descriptors as exclusions for mitmproxy headlines Producers mark excluded bytes with a redacted binary descriptor as well as a redaction entry, and the record schema accepts either facet alone. Skip a mitmproxy headline when a redacted descriptor covers its pointer, so contradictory Evidence that keeps the value cannot preview it. * fix(web-capture): refuse a redacted flow type as the mitmproxy HTTP discriminator Contradictory Evidence can keep type "http" while marking /type redacted. Apply the same redaction check to /type before treating the flow as HTTP, so no mitmproxy headline is added when the flow kind is unavailable. * fix(web-capture): refuse mitmproxy headlines under contradictory numeric sidecars A numeric sidecar records that the producer held a number at that pointer. Treat the flow as HTTP only when /type has no sidecar of its own, and promote a mitmproxy headline only when no ancestor (the record root, /request or /response) carries one. * fix(web-capture): require a textual flow type before adding mitmproxy headlines mitmproxy's flow state carries type as a string. When the native value was bytes, the bridge shows the UTF-8 display text "http" next to a retained binary descriptor, so the producer did not supply the discriminator mitmproxy defines. Treat the flow as HTTP only when /type has no numeric sidecar and no binary descriptor at or above it. * fix(web-capture): refuse mitmproxy headlines under byte-valued ancestors The mitmproxy bridge emits either a byte descriptor for a value or children for a dictionary, never both. Treat a byte descriptor on an ancestor like a numeric sidecar, and refuse a byte descriptor on an integer headline, while keeping the exact descriptor of byte-backed string fields such as the request path. * fix(web-capture): check child facets and byte provenance of mitmproxy headlines A facet beneath a scalar coordinate says the producer held a container there, so refuse the flow type or a headline field with a numeric sidecar, byte descriptor or redaction below it. A string headline may carry at most one byte descriptor, and only retained mitmproxy producer bytes whose content is the value's UTF-8; HAR content bytes or mismatched content keep it out. * fix(web-capture): verify the byte count and digest of byte-backed headlines The record schema checks a descriptor's length and digest shapes, not their relation to its content. Require a byte-backed string headline's descriptor to match the value's UTF-8 in content, byte count and SHA-256 before promoting it. * fix(web-capture): reject byte-backed headlines without a UTF-8 form Buffer.from replaces an unpaired surrogate with U+FFFD bytes, so a descriptor for those bytes matched a string no producer could have decoded. Require the value to round-trip through UTF-8 before comparing it with its descriptor, as CaptureRedaction already does. * fix(web-capture): index mitmproxy facets once and refuse non-UTF-8 targets Reject host, authority and path text containing an unpaired surrogate whether or not a byte descriptor backs it, since no producer could have reported such text. Build a per-record index of facet pointers (sidecars and descriptors by pointer, scalar pointers, container ancestors, withheld values and omitted names) once, only for mitmproxy HTTP records, so each headline check is a few set lookups instead of several passes over every facet array. * fix(web-capture): bound the mitmproxy facet index to headline coordinates Facet pointers have no length or depth bound, and recording every ancestor kept one set entry per prefix (a million for a 1 MB pointer). Record only the fixed headline coordinates (/type and the request/response target and status fields) that have a facet beneath them, examining prefixes no longer than those coordinates. * fix(web-capture): index only facets at headline coordinates and their ancestors Grouping every numeric sidecar and byte descriptor allocated an entry per facet even though only /type, the request/response headline fields and their ancestors are queried. Keep facets only at those pointers (at most two per pointer, which already decides every check), and let other facets merely mark the headline coordinates they lie beneath, using prefix comparisons that derive no strings. * test(web-capture): pin headline omissions and controls beside many unrelated facets The bounded facet index must not change what a mitmproxy preview withholds or keeps. Beside 20,000 unrelated sidecars, a redacted host and a path with a facet beneath it stay withheld while the scheme, port and status remain, with only their own sidecars retained. The test passes against both the current main projection and this change. * test(web-capture): count long set additions in the million-character pointer tests The two million-character pointer tests had no time or size bound, so on Node 22 and 24 they passed against the old ancestor index too, and only Node 26 caught a regression by running out of memory. Count the strings longer than 1,000 characters the preview adds to sets instead: only the pointer and strings cut from it are that long, the old index added 999,000 and 499,507 of them, and the new index adds none. --------- Co-authored-by: Lucien <164857476+dakjdakd@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>