Commit Graph
11 Commits
Author SHA1 Message Date
morluto 5f718ce30e refactor(kernel): give error-message, byte-hex and SHA-256 hex helpers one owner in a new src/kernel layer (#11542)
* refactor(kernel): give error-to-message text one owner in a new src/kernel layer

* fix(kernel): keep the regex worker's own error text inside its isolated source

* refactor(kernel): exclude copies that run in another realm from the error-message guard by name

The regex worker's source string and the page.evaluate fetch in PlaywrightScenarioRequest cannot import errorMessage. The guard now names them, so the counted baseline does not invite a later slice to rewrite them.

* refactor(kernel): migrate the long tail of inline error-message copies to errorMessage (#11552)

Every provider and adapter folder with six or fewer copies now calls errorMessage from src/kernel. Local helpers whose whole body was the copy (failureMessage, gdbErrorMessage, message, describe, reasonOf, recoveryFailureMessage and four local errorMessage definitions) are removed and their call sites call the owner.

Refs #11548

* refactor(kernel): give byte-to-hex formatting one owner in src/kernel (#11560)

byteToHex returns the two lowercase hexadecimal digits of one byte. All eight inline copies (native, contracts, domain firmware and desktop, pcap) call it, keeping their own 0x, % or \x notation, and a counted guard with an empty baseline keeps new copies out.

Refs #11548

* refactor(kernel): give one-shot SHA-256 hex digests one owner in src/kernel (#11562)

sha256Hex digests one in-memory value. A counted guard records the one-shot createHash("sha256").update(x).digest("hex") chains (115 in 101 files) and the first slice migrates domain, contracts and process (35 copies), removing local wrappers (sha256, sha256Text, hash) and the exported sha256Text. Incremental and streamed hashing is not counted.

Refs #11548

* refactor(kernel): migrate the application layer's inline error-message copies to errorMessage (#11563)

All 69 copies in src/application call errorMessage from src/kernel. Local helpers whose whole body was the copy (errorDetail, causeReason and five local errorMessage definitions) are removed and their call sites call the owner.

Refs #11548

* refactor(kernel): migrate the native provider's inline error-message copies to errorMessage (#11567)

All 49 copies in src/native call errorMessage from src/kernel. The local errorReason helpers in LldbCallTracer and LldbRetainedObservations, whose whole body was the copy, are removed and their call sites call the owner.

Refs #11548

* refactor(kernel): migrate the remaining inline error-message copies to errorMessage (#11569)
2026-11-16 07:05:30 +08:00
morluto d87be930c4 refactor(providers)!: key Ghidra and Hopper analysis profiles on the producer identity instead of hand-bumped markers (#11320)
* refactor(ghidra)!: key analysis profiles on the producer identity instead of hand-bumped markers

Ghidra snapshot validity depended on 53 hand-maintained "-vN" semantic
markers in the analysis profile; a behaviour fix that forgot to bump its
marker left old snapshots admissible (#3664). #10562 added a digest of the
bridge sources, but TypeScript projection changes still needed a bump and
the REA release was not committed.

- src/filesystem/ProducerIdentity.ts owns the producer identity: the REA
  release plus a framed SHA-256 of a producer's packaged scripts, read once
  per process.
- Ghidra profiles commit `producer` in place of `bridge_sources_sha256` and
  the constant revision markers (shared semantics block, load-image, entry
  seed, PDB, jump-table, DOL import, MIPS ISA-mode markers) are removed.
- Runtime-selected values keep their meaning without a revision suffix
  (language_selection, annotation_policy, relocation/no-return/launch
  policy, MIPS/PowerPC/PSP lanes, MIPS ABI basis). Bridge wire policy ids
  shared with the Java scripts (ilp32-prototype-v1,
  o32-soft-float-prototype-v1) are unchanged.

BREAKING CHANGE: Ghidra Evidence `analysis_profile.parameters` replace the
semantic marker keys and `bridge_sources_sha256` with `producer`
({rea_version, sources_sha256}), and selection values lose their "-vN"
suffix (for example `language_selection: "provider-pinned"`). Ghidra
Evidence IDs and profile digests change, and analysis snapshots saved by
earlier builds are refused at open. Migration: open the target without the
old snapshot and save a fresh one; consumers reading the removed marker keys
should read `producer` instead.

* fix(hopper)!: commit the REA release and bridge script in Hopper analysis profiles (#11324)

Hopper analysis profiles committed the launcher digest and one hand-bumped
marker (function_dossier_coverage "...-v1"), but neither the REA release
nor bridge/hopper_bridge.py. A snapshot saved before a bridge or REA change
replayed results the current bridge would not produce.

HopperBridgeSources owns the packaged bridge script URL (used to launch it)
and hopperProducerIdentity() on the shared ProducerIdentity owner; the
profile commits `producer` in place of the marker.

BREAKING CHANGE: Hopper Evidence `analysis_profile.parameters` replace
`function_dossier_coverage` with `producer` ({rea_version, sources_sha256}).
Hopper profile digests and Evidence IDs change, and analysis snapshots saved
by earlier builds are refused at open. Migration: open the target without
the old snapshot and save a fresh one.

* fix(ghidra): build the legacy reference snapshot from another producer and use the shared temp directory

* refactor(providers): key Kuna exports and the Native AOT profile on the producer identity (#11353)

* fix(ghidra): keep the bridge source table import-free for the Node-loaded verifier

scripts/verify-ghidra-bridge-sources.mjs loads GhidraBridgeSources.ts with
Node type stripping, which does not resolve .js specifiers to .ts sources.
The producer identity import made every verifier run fail with
ERR_MODULE_NOT_FOUND; the identity now lives in GhidraProducerIdentity.ts.

* fix(artifacts): import the moved file source and abort check from their resources owner after merging main
2026-11-16 05:37:57 +08:00
morluto cef87c498a refactor(architecture): add the ports and resources layers, move ports and shared resource owners there, and ratchet the layer baseline (#11208)
* test(architecture): fail on a stale layer baseline and document the declared layers

verify:module-boundaries reported a layer baseline entry whose imports were
gone but still passed, so main kept process -> artifacts at 20 after it fell
to 17 and a later change could add three imports back unnoticed. A stale
entry now fails like growth; --shrink-layer-baseline lowers it in the change
that removes the imports. The architecture map and CONTRIBUTING name
scripts/lib/module-layers.mjs as the per-directory layer declaration.

* test(architecture): refuse to lower the layer baseline after an inspection failure

A file that cannot be parsed contributes no imports, so a shrink run would record counts that fail as growth once the file is fixed. Whole-command tests cover stale rejection, shrinking, refused growth and the inspection failure.

* refactor(ports): move the progress, work-scope and inspection-scope ports to src/ports (#11214)

* refactor(ports): move the progress, work-scope and inspection-scope ports to src/ports

Pure move with git mv and import specifier updates.

* test(architecture): declare the ports layer and lower the layer baseline

src/ports sits between the foundation and the providers. Moving the three
ports removes 11 provider -> application imports.

* refactor(ports): move AnalysisProvider to src/ports (#11217)

* refactor(ports): move AnalysisProvider to src/ports

Pure move with git mv and import specifier updates in its importers, the
built-fixture path, the CI scope glob and the lint-rule test spellings.

* test(architecture): lower the layer baseline after moving AnalysisProvider

* refactor(ports): move the per-feature provider ports to src/ports (#11226)

* refactor(ports): move the per-feature provider ports to src/ports

Pure move with git mv and import specifier updates: the 47 *Port.ts and
*Ports.ts files that providers implement, plus OptionalObservationProviders,
which only composes the observation ports.

* test(architecture): lower the layer baseline after moving the feature ports

* refactor(providers): move closeAnalysisClient and the Frida session input errors to their owners (#11233)

* refactor(providers): move closeAnalysisClient to src/providers and the Frida session input errors to src/frida

Pure move with git mv and import specifier updates.

* test(architecture): drop the application-providers and application-frida cycles from the layer baseline

* fix(ci): keep provider lanes selecting the ports moved to src/ports

Each lane glob that matched a moved port under src/application gains its src/ports counterpart; the change-scope test pins one moved port per lane.

* refactor(domain): declare the localPath host exception and read HOST_PLATFORM there (#11256)

absoluteLocalPathStringSchema decides which paths are absolute on the running host, so it reads HOST_PLATFORM instead of process.platform. Domain may not import outside domain/contracts; module-layers now declares LAYER_EXCEPTIONS, exact file-to-file imports with a reason, and both the layer count and the pure-layer boundary skip only that import. Any other domain -> platform import still fails.

* refactor(ports): move the doctor check and provider inspection types to src/ports (#11246)

* refactor(ports): move the doctor check and provider inspection types to src/ports

Declaration move from application/Doctor.ts (unchanged text) and import updates; no re-export from Doctor.ts.

* test(architecture): lower the layer baseline after moving the doctor provider types

* refactor(ports): move the packet capture and signal recording ports to src/ports (#11249)

* refactor(ports): move the packet capture and signal recording ports to src/ports

Declaration move (unchanged text) out of the two application services, with import updates.

* test(architecture): lower the layer baseline after moving the capture and signal ports

* refactor(domain): move the analysis error projection schema from contracts to domain (#11251)

* refactor(domain): move the analysis error projection schema from contracts to domain

Declaration move (unchanged text) of the projection schemas that domain modules embed; contracts/errorSchemas.ts keeps the CLI envelope and the published JSON Schema. Import updates only.

* test(architecture): drop the contracts-domain cycle from the layer baseline

* refactor(ports): drop the imports the moved capture, signal and doctor types left behind

Also give the packet capture and signal recording lanes the new port paths, with change-scope cases.

* refactor(resources): add src/resources and move AnalysisCleanupLease and ArtifactReader there (#11259)

* refactor(resources): move AnalysisCleanupLease to src/resources

Pure move: git mv plus import specifier rewrites (source, tests, and the dist paths the verifier scripts load).

* refactor(resources): declare src/resources as a foundation unit

src/resources owns the shared analysis resource modules that process, windows and providers hold; AnalysisCleanupLease is the first. Its tests join the adapters project, the apk signature lane selects it beside src/artifacts, and the layer baseline drops the process and windows imports of the lease.

* test(ci): pin the apk signature lane on the moved cleanup lease

* refactor(resources): move ArtifactReader to src/resources

Pure move: git mv plus import specifier rewrites (source, tests, and the dist path a verifier script loads).

* test(architecture): lower the layer baseline after moving ArtifactReader

windows no longer imports src/artifacts; process -> artifacts 11 -> 9. The CI lanes that selected src/artifacts/ArtifactReader.ts select its new path, pinned in the change-scope test.

* refactor(resources): move ArtifactResourceScope to src/resources (#11265)

* refactor(artifacts): name the output tree and private root capabilities the resource scope releases

ArtifactResourceScope typed its output-tree and private-runtime-root owners with SafeOutputTree (src/artifacts) and PrivateRuntimeRoot (src/process). It only calls rollback() and close(), so it now declares those capabilities itself and names neither unit, which lets it move to src/resources without a resources <-> process cycle. The withheld-export test narrows to SafeOutputTree where it reads the published flag.

* refactor(resources): move ArtifactResourceScope to src/resources

Pure move: git mv plus import specifier rewrites (source, tests, the dist paths verifier scripts load, and the FIFO probe's module-relative URL).

* test(architecture): lower the layer baseline after moving ArtifactResourceScope

process -> artifacts 9 -> 5, and the artifacts <-> dotnet mutual pair is gone. The sqlite and agentenv lanes select the scope at its new path, pinned in the change-scope test.

* refactor(resources): move ArtifactFileOperation to src/resources (#11269)

* refactor(resources): move ArtifactFileOperation to src/resources

Pure move: git mv plus import specifier rewrites (source, tests, and the dist paths verifier scripts load).

* test(architecture): lower the layer baseline after moving ArtifactFileOperation

process -> artifacts 5 -> 1.

* refactor(resources): move readStableArtifact to src/resources (#11271)

* refactor(resources): move readStableArtifact to src/resources

Pure move: git mv plus import specifier rewrites.

* test(architecture): lower the layer baseline after moving readStableArtifact

process no longer imports src/artifacts: process -> artifacts 1 -> 0 and the artifacts <-> process mutual pair is gone. The seven lanes that selected readStableArtifact select its new path, pinned in the change-scope test; the architecture map lists what src/resources owns.

* refactor(hopper): take Linux launcher verification from composition through a port (#11278)

The Hopper launcher imported inspectLinuxHopperLauncher and retryLinuxHopperPublisherCleanup from src/application (the two hopper -> application imports). src/ports/LinuxHopperLauncherVerification.ts now names that capability; src/application/LinuxHopper.ts provides LINUX_HOPPER_LAUNCHER_VERIFICATION, binary composition passes it to HopperProvider, and the provider hands it to the launcher. A launcher built without one refuses a Linux demo launch with reason verification_unavailable before running the launcher. Layer baseline: hopper -> application 2 -> 0.

* refactor(config): move Hopper launcher discovery to a src/hopperPaths foundation unit (#11292)

* refactor(hopper): move the launcher discovery and macOS Hopper paths to src/hopperPaths

Pure move: git mv plus import specifier rewrites.

* refactor(hopper): declare src/hopperPaths as a foundation unit with the Linux launcher candidates

Hopper install paths and launcher discovery are pure path facts that the
Hopper provider, doctor and setup all read. Declare src/hopperPaths in the
foundation layer, add it to the adapters test project, the architecture
map and the package-baseline CI lane, and move the Linux launcher
candidates out of src/application/LinuxHopper.ts into
src/hopperPaths/LinuxHopperPaths.ts so discovery no longer imports
application code.

* refactor(config): leave Hopper launcher discovery to the Hopper provider

parseConfig resolved an unset HOPPER_LAUNCHER_PATH by probing the
filesystem through application-owned discovery, so config imported
application (config -> application 2, application <-> config cycle) and
read process.platform ambiently.

AppConfig.hopperLauncherPath is now the caller's HOPPER_LAUNCHER_PATH
unchanged (string | undefined). src/hopperPaths/HopperLauncherDiscovery.ts
owns the selection (selectedHopperLauncherPath: the configured spelling,
else discovery from the caller environment's home on the provider's
platform); HopperProvider and the two real-Hopper verifiers read it.
Discovery tests move from config.test.ts to src/hopperPaths; a provider
test covers discovery from the provider's own platform and home.

Module layers: config -> application 2 -> 0; application <-> config removed.
Architecture guards: ambient platform reads in parseConfig.ts 2 -> 0.

* refactor(providers): move owned-process prerequisite availability out of application (#11294)

* refactor(providers): move the owned-process prerequisite availability mapping to src/providers

Pure move: git mv plus import specifier rewrites.

* test(architecture): lower the layer baseline after moving owned-process availability

android -> application 1 -> 0; apktool -> application 1 -> 0.

* refactor(process): move the regular-file read owner out of application (#11299)

* refactor(process): move the regular-file read owner from src/application to src/process

Pure move: git mv plus import specifier rewrites.

* test(architecture): lower the layer baseline after moving the regular-file read owner

desktop -> application 5 -> 4; dotnet -> application 2 -> 1.

The agentenv lane listed src/application/RegularFileRead.ts. At its new
path it is a src/process/ shared boundary: fast qualification selects the
code and runtime checks (and the single owning lane, web_source_map, via
src/process/**), and full qualification selects every lane, as for every
other src/process/ module. A second owner entry would only suppress the
web_source_map selection, so the agentenv entry is removed, not renamed.

* refactor(resources): give atomic text writes their own owner (#11302)

* refactor(resources): give atomic text writes their own owner in src/resources

src/application/JsonFiles.ts owned both JSON input reads and the atomic
text write (writeTextFile / writeTextParts and their staged-output
publication). The write side depends only on domain, filesystem and
resource owners, but il2cpp's Ghidra seed writer had to import
application to reach it (il2cpp -> application).

Move the write side verbatim to src/resources/TextFileWrite.ts
(TextWriteCancellation, writeTextFile, writeTextParts and their private
staging, publication and failure classification helpers). JsonFiles keeps
the JSON read owner and reads the shared errno-to-reason classifier
(fileAccessFailureReason) from the write owner instead of a copy.
Importers (il2cpp seed writer, evidence bundle files, desktop build
solves, analysis snapshot files and four boundary tests) read the new
path; no re-export is left in JsonFiles.

* test(architecture): lower the layer baseline after moving the text write owner

il2cpp -> application 1 -> 0.

* refactor(ports): move the desktop build host contract to src/ports (#11303)

* refactor(domain): move the setup cancellation check from src/application to src/domain

Pure move: git mv plus import specifier rewrites.

* refactor(ports): move the desktop build host contract to src/ports

The desktop provider implements doctor's desktop build facts reader and
setup's desktop build host, but the contract it implements lived in
application (DesktopBuildDoctor, SetupDesktopBuild, Doctor's
clientRegistrations signature, ClientRegistrationEnvironment's setting
types), so desktop imported application four times and the two units
were a recorded mutual pair.

src/ports/DesktopBuildHost.ts now holds that contract, moved verbatim:
DesktopBuildCommand, DesktopBuildFacts, DesktopBuildFactsInput,
DesktopBuildSetupFacts, DesktopBuildSetupHost, RecordedProviderSettings
and OwnedProviderSetting. Two names are new: DesktopBuildRegistration,
the registration facts the host reads (client, command and the
recorded-settings names; client registration status satisfies it), and
DesktopBuildRegistrationReader, which replaces
DoctorHost["clientRegistrations"] in the setup host. The facts input's
registration reader returns DesktopBuildRegistration instead of the full
application status; the setup facts' settings are spelled
Readonly<Record<string, string>> (SetupProviderEnvironment's definition).
Application imports the types from the port; no re-export is left.

* test(architecture): lower the layer baseline after the desktop build host port

desktop -> application 4 -> 0; the application <-> desktop mutual pair is gone.

* refactor(domain): define the resolved app bundle resource root beside its schema (#11304)

* refactor(domain): define the resolved app bundle resource root beside its provenance schema

ResolvedAppBundleResourceRoot (an app directory's resource-only identity)
is a pure type over BundleResourceRoot's layout, but it lived in
src/application/AppBundleExecutable.ts, so the native and artifact
providers that receive it imported application.

Move the interface verbatim to src/domain/apple/bundleResourceRoot.ts,
which owns BundleResourceRoot, and point its eight readers there (native
macOS provider and plist source, artifact provider, binary composition,
and the binary session, status, port and bundle-resource session). The
resolver that produces it stays in application.

* test(architecture): lower the layer baseline after moving the resource root type

native -> application 2 -> 0; artifacts -> application 2 -> 1 (the
artifact provider still constructs BundleResourceSession).

* refactor(config): move the pwntools input bound parser to src/config (#11306)

* refactor(config): move the pwntools input bound parser from src/native to src/config

Pure move: git mv plus import specifier rewrites, and the module path in docs/design/rea-limit-inventory.md (table realigned by oxfmt).

* test(architecture): lower the layer baseline after moving the pwntools input bound

config -> native 1 -> 0. The binary_layout and recorded_crash lanes, which
select src/native/pwntools/**, also select src/config/pwntoolsInputLimit*.ts;
ciChangeScope.test.ts pins both.

* refactor(filesystem): move REA cache directory helpers out of config (#11309)

* refactor(filesystem): move the REA cache directory and lexical directory helpers from src/config to src/filesystem

Pure move: git mv plus import specifier rewrites.

* test(architecture): drop the config and filesystem cycle from the layer baseline

config no longer imports filesystem; filesystem still reads the Windows
ChangeTime settle budget from config.

* refactor(windows): break the process and windows import cycle (#11317)

* refactor(domain): move the runtime-control and private-runtime-root refusals from src/process to src/domain

Pure move: git mv plus import specifier rewrites (including the dist path scripts/verify-windows-native.mjs loads).

* refactor(windows): type Job Object cleanup by what a Windows job reports

WindowsOwnedProcess.cleanup() returned process-ownership's
ProcessCleanupResult, so src/windows imported src/process for a type while
process imports windows to supervise Windows trees. A Job Object stop only
ever reports settled-and-signalled or a reason it did not settle; name that
subset WindowsJobCleanupResult in windows. It stays assignable to
ProcessCleanupResult, so the process supervisor reads it unchanged.

* test(architecture): drop the process and windows cycle from the layer baseline

* test(architecture): ratchet cross-provider imports and imports past a provider entry (#11386)

* test(architecture): ratchet cross-provider imports and imports past a provider entry

A provider folder may import the domain, contracts, foundation and ports layers, the shared provider core (src/providers) and its own files; an import of another provider folder is a recorded exception. Application, composition and the CLI/MCP adapters reach a provider only through src/<provider>/index.ts. verify:module-boundaries counts both kinds of import per unit pair in the layer baseline (crossProvider, providerEntry); like the upward counts they may only shrink, and a pair missing from the baseline fails.

* refactor(resources): move StableArtifactFailure and StableArtifactObservation into src/resources (#11387)

* refactor(resources): move StableArtifactFailure and StableArtifactObservation into src/resources

Providers project stable-read failures through StableArtifactFailure next to the other shared provider toolkit owners (ArtifactReader, ArtifactResourceScope, ArtifactFileOperation, AnalysisCleanupLease, readStableArtifact) that already live in src/resources. Under src/artifacts it made every provider that reads a stable artifact import the artifacts provider. Pure move: git mv plus import specifier and baseline updates; the layer baseline drops the provider-boundary counts the move removes.

* refactor(resources): reach the shared provider toolkit through src/resources/index.ts (#11390)

* refactor(resources): import the shared provider toolkit through src/resources/index.ts

Adds src/resources/index.ts, which exports the toolkit names that production code outside src/resources uses, and rewrites those importers (420 files) to import from it, merging each importer's toolkit imports into one declaration. Mechanical: import declarations only.

* test(architecture): reach src/resources only through its entry file

verify:module-boundaries now counts any production import of a src/resources file other than src/resources/index.ts, from any unit, with the imports past a provider entry. The count is renamed from providerEntry to entry ("import past the entry file"); it may only shrink, and src/resources starts at zero.

* test(architecture): cover provider boundary counts through the verifier command

* refactor(resources): move the safe output tree and artifact path owners into src/resources (#11398)

* refactor(resources): move the safe output tree and artifact path owners into src/resources

SafeOutputTree (with its admission, creation failure, readback, cleanup and identity files), StreamBytes, ArtifactPaths and readArtifactFileChunks depend only on domain, filesystem, platform and the toolkit, and browser, firmware, ghidra, javascript, android and apktool use them to publish outputs. Under src/artifacts every such provider imported the artifacts provider. Pure move: git mv, importers moved onto src/resources/index.ts (names added to the entry), the moved files import toolkit modules directly, path keys in the CI scopes and guard baseline follow the move, and the layer baseline drops the removed imports.

* test(architecture): give the shared provider core its own layer below the providers (#11401)

src/providers (deferred client owners and failure attribution) now imports only domain, ports, process, filesystem and the resources toolkit. Declaring it as the providerCore layer, between ports and providers, makes any import from it into a provider folder an upward import that verify:module-boundaries rejects, and lets every unit in the providers layer be a provider folder.

* test(architecture): declare each provider's entry files instead of a provider barrel (#11407)

* chore: ignore the source map parser move (#11348) in blame

* test(architecture): declare inspector's use of the browser CDP client as a provider dependency (#11420)
2026-11-16 03:56:10 +08:00
morluto 271e375924 refactor(application): read the host from HostPlatform in the remaining application modules (#11124)
Scenario planning and plan Evidence take a required platform. Host file operations (target resolution, output trees, Windows spelling lookup, executable checks, native snapshot admission), the Linux distribution read and the process CLI adapter read HOST_PLATFORM explicitly. ByteEntropyService takes an optional platform seam, so its Windows casing test injects win32 instead of stubbing process.platform. src/application no longer reads process.platform or process.arch.
2026-11-15 22:03:14 +08:00
5e1d504664 fix(environment): admit the output root before creating its parents
environment-build/export ran an unclassified recursive mkdir of the
destination's parent before SafeOutputTree validated anything. A
trailing `..` therefore created directories before it was refused, and
a regular file in the parent path surfaced as unreadable_output instead
of refusing the selection.

SafeOutputTree.create gains a createParents option that admits the
root's own syntax first, then creates the parents, mapping EEXIST and
ENOTDIR to a destination refusal. The environment bundle writer uses it
instead of its own mkdir.

Fixes #7688
Fixes #7571

The non-directory mapping comes from #7734 by HeadmasterEggy, and the
syntax-first check from #7588 by harshitethic; their tests are kept.

Co-Authored-By: Avocado <headmastereggy@gmail.com>
Co-Authored-By: Harshit Sharma <66710144+harshitethic@users.noreply.github.com>
Co-Authored-By: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-11-14 14:59:49 +01:00
morluto f46e9a8479 fix(environment): retain committed bundle receipts after finalization failure (#9602)
* fix(environment): retain published bundle receipts

* fix(environment): retain receipt after tarball lookup failures
2026-11-09 19:34:48 +08:00
02e049744f feat(environment): build agent environments from a prompt (#6946)
* feat(environment): report what an environment serves with an environment-coverage view

A new inspect_analysis_view kind over capture Evidence. It projects the
state an environment derives with the given template rules and reports
each tool as answered from state, replayed with its view reason, a served
write, or a withheld write with its reason, the records behind it and a
hint about what to record next. The hints cover every write reason, so a
new reason without one fails to compile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(environment): build an environment bundle in one command

rea environment-build <capture> [directory] inspects a HAR (or reads
capture Evidence), projects the environment manifest from the in-memory
Evidence, authenticates the pair, prepares the state and writes the bundle
environment-export writes. Every bundle file is byte-identical to the
three-step path for the same capture and options.

The report adds the capture and manifest identities, the coverage summary,
warnings (a loopback capture named after its file, credential-named body
members by pointer) and the next command. --dry-run writes nothing. The
default template rules are the four the published environments use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(environment): cover environment-build and the coverage view

Domain tests check the hint table against the state view's reasons, the
summary, and the view against the environment-state view it summarizes.
Service tests build the fixture capture and compare every file with the
three-step path, a dry run, loopback naming and the refusals. A CLI test
compares a HAR built in one step with the three CLI steps byte for byte,
and an MCP test projects the view from every source kind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(environment): build the real-agentenv lane's bundle with environment-build

The lane now runs the command users run: it builds the Grafana-shaped HAR
with environment-build and the packed checkout, checks the report, then
runs the bundle with agent-env as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): document environment-build and the coverage view

The agent environments guide opens with the one-step build: the report,
the hint for each withheld write reason, and what a bundle ships. The CLI
guide, the capture guide and the skill reference name the command and the
environment-coverage view.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): harden environment-build after review

- The credential scan walks with its own stack, so a deeply nested body
  cannot crash the build, and it also reports credential-named header
  names, query names, form body members and string arrays.
- The build checks SIGINT between stages: cancelled before the bundle
  stage it writes nothing, during the write it names the written bundle.
- A dry run opens the package tarball, so it refuses what a build refuses.
- Repeated template rules are refused, a HAR saved as .json gets a hint,
  file-derived names follow the bundle name rules, `localhost.` counts as
  a loopback origin, and the next command is shell-quoted.
- --capture-timeout-ms passes through to the HAR inspection, and a
  missing directory is refused before inspection.
- The no-observation hint covers updates and deletes of entities the state
  never held; the coverage view states that it does not apply the
  serve-time admission checks; the inspect_analysis_view description
  names the view.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): state the build's conditions and record its verification

The guides now state when the one-step bundle equals the three-step one,
which headers inspection excludes, what the credential warning scans, how
cancellation behaves, and that the coverage view does not apply the
serve-time admission checks. The spec folder records the census, the end
to end runs, the gates and the review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): dispatch a pending interrupt before each build stage check

The manifest, state and coverage stages run synchronously, so a SIGINT
that arrived during them was only dispatched once the bundle write had
begun. Each check now yields first, so an interrupt before the bundle
stage cancels the build with nothing written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(browser): send declared requests from a scenario page with a request step

A capture_browser_scenario step `request` sends one declared HTTP request
through the page's own fetch, with its cookies and session, the way the
application's frontend would, and reads the whole response so the capture
records the transaction like any other. The path must resolve to the start
URL's origin, and the page must be on that origin when the step runs.
Query values and header values may be literals or declared secrets, and a
header may read its value from a cookie or a local storage entry, which is
then redacted from later records like a declared secret. `expected_status`
fails the step on any other status. Scenarios without a request step are
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(browser): verify request steps in a real browser through to a built environment

verify:browser:request records a write session against a fixture item API
that refuses writes without its CSRF cookie header and stored bearer
token. It checks that every request lands in its own step, that the page
values authenticated each write and appear in no record, header or body,
that an unexpected status fails the step and cancels the rest, and that
environment-build serves the recorded create, update and delete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(environment): export environment bundles over MCP

export_environment_bundle writes the bundle environment-build writes, from
capture Evidence retained in the session or one capture Evidence file,
into an absolute output directory that must not exist and whose parent
must exist, as extract_firmware and export_web_scripts require; it never
creates the parent or reuses a directory. It returns the same report,
whose schema now lives in the domain so the CLI and the tool share it, and
dry_run reports without writing. A loopback capture without a file needs a
name. The build's refusal messages no longer name CLI flags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): scan scenario credentials, own the dry-run tarball and serialize Evidence once

A browser scenario's retained header values and the query parameters of its recorded URLs are scanned for credential
names as a HAR's are, through a review projection of its records. A dry run admits and releases the package tarball
through the export's regular-file scope, so a close it cannot confirm keeps its owner, and checks for cancellation
after that last await. Evidence is written into one exactly sized buffer instead of chunks joined by a copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record the review round's checks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skill): build agent environments from a prompt with request steps and the bundle tool

Add the environments reference to the skill and install it with setup: record
against a disposable instance, sign in through the UI, write with request
steps and read each write back, check coverage, re-record by withheld reason
and export the bundle. Route it from SKILL.md, name agent environments in the
skill description, and document the request step, the export tool and
recording writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(browser): split the request lane and header resolution into smaller functions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* style(environment): format the build spec

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): say when an untimed capture learns no write clock

Coverage of a capture whose records are not all timed, which includes every
browser scenario capture, now states that the state learns no clock, so an
update stamped with a server-set modification time stays unexplained-change
however often it is recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skill): give agents a whole scenario shape with step timeouts

Agents guessed the scenario schema field by field and two hung on steps
without timeout_ms. The reference now starts from a complete scenario with a
timeout on every step, and says a scenario capture cannot serve updates the
application stamps with a modification time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(browser): keep request steps on their origin and their page values redacted

- Follow no redirects: a redirect fails the step and nothing reaches the
  location it names (checked in real Chrome), and check the page's origin
  again in the page as the request is sent.
- Send nothing once the step's time has run out or it was cancelled while
  header values were read; the in-page deadline is what remains of the step's.
- Keep every page value redacted under a repeated or colliding label, keep a
  value's first marker when it is read again, and leave values shorter than
  8 characters unredacted rather than rewriting ids, ports and hosts.
- Refuse a header name given twice, User-Agent (Chrome drops it) and line
  breaks in literal values and prefixes; fail before sending when a secret or
  page value holds them.
- Append query entries after the path's literal query without re-encoding it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): document request step limits and word bundle refusals for both surfaces

Document redirects, timeouts, page value redaction, HttpOnly cookie sources
and header refusals for request steps. Refusals shared by environment-build
and export_environment_bundle no longer name CLI flags, and the retained
loopback refusal no longer prints an empty file name. The skill reference
asks for read-backs after each write, shows the coverage view's source, names
package_tarball for unreleased builds and the browser path a scenario ships.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): scan every form occurrence and MIME-less scenario bodies

The build's credential scan collapsed form bodies with Object.fromEntries, so a
repeated field kept only its last occurrence: `api_key=secret&api_key=` warned
about nothing although the Evidence ships the value, and repeated values were
undercounted. Form fields are now scanned as ordered name/value pairs, as
headers and query strings already are.

The review projection of a capture_browser_scenario dropped a request body
recorded without a media type, so the scan never saw it. Review keeps such a
body's text with an empty media type; the manifest projection still reads only
bodies it can classify.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(browser): close request step edge cases from the second review

- Append query entries correctly after a path ending in "?", "&" or "#".
- Refuse 3xx expected statuses, which an unfollowed redirect can never
  report, and say a redirected write may already have been applied.
- Redact a page value also as fetch sends it, without surrounding spaces
  and tabs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): document page value body costs, redirects and MCP dry runs

Document that reading a page value shrinks the bodies a capture keeps under
the default allocation limit, that redirected writes may have landed, that
the MCP dry_run still checks its output directory and which refusals name
the tool, and keep the .har hint in the HAR-as-JSON refusal for both
surfaces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record the prompt-to-environment verification

Gates, the real-browser lanes, five environments built from prompts by an
agent on disposable VMs and evaluated with two models, and the two reviews.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(skill): count rollback failures from the managed skill files

The environments reference made the skill eight files, so the recovery tests now derive the counts they expect from
MANAGED_SKILL_FILES instead of fixing them at six and seven.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(adb): restore the portable Java float pattern a merge rewrote

A lint autofix during a merge unescaped the hyphens in javaFloatValuePattern, which is invalid in a v-mode character
class, so the advertised start_adb_activity schema failed to compile in every JS regex mode. The file now matches main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(browser): show the request step inside a whole scenario

Every JSON example in the browser scenario guide is a capture_browser_scenario input the catalog test parses, so the
request step example now sits in a launch scenario instead of standing alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): scan structured HAR form params and the request side of unattributed scenario transactions

A HAR request recorded with postData.params and no text is scanned by those parameters, each occurrence counting, and
the review projection keeps the request side of a scenario transaction whose response cannot be attributed, so the
credential warning covers what evidence.json ships in both cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): validate build paths and read dry-run tarballs

* fix(environment): report bodies too large for the credential scan

Text bodies over the scan's 4 MiB limit, including a response decoded only as a prefix, are reported as a
credential-scan-incomplete warning by record and side, never by content, so an absent credential-named-members
warning no longer reads as a complete check. The spec and agent-environments guide also state the path validation and
dry-run tarball read that e52ae1cd added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(verification): release cold source transform services

* fix(javascript): retain capture context along covered mutation paths

* fix(environment): guard request dispatch and report export limits

* fix(environment): admit serialized exports and scan HAR form fallbacks

* fix(environment): admit in-memory build inputs against heap headroom and scan SaveHar null-text forms

The inspected HAR's Evidence and the projected manifest are checked against live heap headroom before they are
serialized, with the same check the file route uses, so a capture too large to parse and seal is refused as
resource_constraint before anything is written. A request whose postData text is SaveHar's null is scanned by its
structured params.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): measure the credential scan limit in UTF-8 bytes

A body counts against the 4 MiB limit by its UTF-8 length, not its UTF-16 code units, so multibyte text over the
limit is reported as unscanned instead of parsed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(browser): send request steps only before their deadline and keep page reads on the start origin

The page receives the step's absolute deadline and a one-second start window, so a busy page that runs the request
late, or after the step is cancelled, sends nothing; local storage is read only on the start origin; the response is
drained without keeping its body; an unparseable path is a field issue; retained Evidence is refused for heap headroom
before it is serialized; and the skill says how to reach server-assigned ids.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* style: format integrated export admission

* fix(javascript): erase inline type imports before binding values

* fix(browser): run the request lane under the shared verifier command owner

The browser request lane runs each CLI call through createPackageVerifierCommands on a private runtime root, so a
timeout, output overflow or failure stops the CLI and the browser it launched before the root is removed, and a failed
stop is reported with the root retained. The verification record states the lane's current counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): scan scenario content by event and read bodies by their media type

A browser scenario's credential scan reads every retained network-content event, including a response attached
mid-flight and a transaction no record represents, with event pointers, and each transaction URL's query once; the
review record projection it replaces is removed. A body is parsed as a form or as JSON by its recorded media type,
falling back to its shape, so valid forms such as `password=x&` are scanned and sentences are not read as forms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-24 21:51:41 +08:00
Muhammad Hamza b5dba08dba fix(web): refuse an existing script-export destination as invalid input (#6313)
* fix(web): refuse an existing script-export destination as invalid input

export_web_scripts refused an existing output directory or link correctly,
but reported it as artifact_operation_failed / execution_failure with no
field issue or recovery. SafeOutputTree.create raises that refusal as a
path failure before any output tree exists, and the export mapper wrapped
it as an artifact failure.

Attribute pre-admission path refusals to output_directory as an
AnalysisInputError, as environment export already does for its directory.
Both now share one helper owned beside SafeOutputTree; failures after the
tree is admitted keep their artifact semantics.

Fixes #6029

* fix(artifacts): type SafeOutputTree destination refusals

Admission conversion treated any cleanup-free path failure raised before
admission as a refusal of the selected directory, so an acquired-tree setup
failure, such as being unable to restrict a created root to 0700, blamed
valid output_directory or directory input.

SafeOutputTree.create now raises SafeOutputTreeDestinationRefusal for the
refusals of the selection itself: a relative or reserved root, an existing
root, and an accepted existing root that is not an empty directory. The shared
conversion accepts only that type, directly or wrapped by
SafeOutputTreeCreationFailure, so script and environment export no longer
track admission separately.
2026-10-23 23:42:33 +09:00
Kaoru0822andClaude Opus 5.5 bd8d73ab1d fix(environment): report later bundle write failures as host failures (#5581)
* fix(environment): report later bundle write failures as host failures

Only destination admission now refuses the selected directory as invalid
input. A path failure on a nested bundle file after admission, such as a
host pathname-length limit, is reported as an output failure that names the
host cause and the absolute failed path, with recovery guidance for the
partial folder it leaves in place.

Fixes #5522

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): name the host-refused path in export permission failures

A permission failure after earlier bundle files were written was reported
as access denied for the bundle root, so the documented failed path was
lost. The refusal now names the absolute path the host reported, falling
back to the root, and keeps the root as `output_directory`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): attribute pathless bundle write failures to their file

Descriptor operations such as `write` and `fsync` fail without a path, so
a disk-full failure after earlier files named only the bundle root. The
bundle writer now attributes such failures to the file it was writing.
Host-write recovery is applied only to actual host failures; an integrity
refusal keeps its own message without disk-space advice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): attribute package tarball read failures to the tarball

The destination-write attribution also caught failures from reading the
admitted package tarball during the copy, so a source `EIO` named the
destination file and advised freeing disk space. Host failures are now
attributed by direction: tarball reads name the tarball with read
recovery, bundle writes name the file being written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): state which export failures name a path

The export documentation and host-write message promised a failed path
for every later write failure, but a directory sync at commit time fails
without one. They now promise the failed bundle file or package tarball
when the failure is on one, and only the cause for a folder sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 22:55:05 -07:00
morluto ab58edcdf5 fix(environment): retain tarball descriptor ownership during export
Use the shared positional reader for both tarball passes so early output refusal cannot close the descriptor ahead of its owner.

Closes #5684
2026-10-21 05:33:35 +08:00
7a0c47bfa4 feat(environment): export an environment manifest as an agent environment bundle (#4999)
Export an authenticated capture and environment manifest as a pinned agentenv bundle with provenance, a replay verifier and smoke task.

Followups preserve immutable input ownership, bounded discovery, caller paths and numeric values; bind bundle writes and package copying to retained filesystem owners; verify POSIX modes; retain cleanup authority; and verify framework/npm executable selection. Default MCP remains stdio.

Validation: pinned Linux toolchain, typecheck/build/lint/format/docs/knip, 452 focused tests, ten product-catalog cases, real GitHub response replay in the generated non-root Docker image, and the real pinned agentenv framework smoke workflow. Platform and resource-measurement limits are recorded in the feature verification document.

Co-authored-by: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-20 05:29:43 +08:00