docs(evm): record compatible wiring and real task costs

Record six exact SDK catalog/fact pairs, 59 real CLI/MCP cases, six cross-domain artifact workflows and six actual Codex CLI trials. Preserve failed setup attempts privately and qualify unchanged complete delivery, host-state variance and unknown final resource counters.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
N0zoM1z0andfactory-droid[bot] committed 2026-10-12 17:13:31 +00:00
1 parent 152984da77
commit d999d9874b
2 files changed
+89 -5

No files matched your search

+3 -2
View File
@@ -44,8 +44,9 @@ acceptance conditions for every optimization.
experiment control; it does not itself establish reduced CPU consumption.
Migrate domain by domain with focused real-artifact checks and relevant shared
contract checks. The domain matrix is a growing roadmap; native PE and DOS, for
example, remain unverified in this study. A passed lane does not imply all
contract checks. The native follow-up verifies selected PE and DOS MZ entry facts;
DOS COM and complete native application behavior remain unverified. The domain
matrix is a growing roadmap. A passed lane does not imply all
advertised platforms/providers have passed. Newly discovered REA bugs remain
in the separate local handoff document for later team repair.
@@ -21,7 +21,7 @@ real EVMole/WABT claims. Nine service and 26 provider-boundary cases also pass;
two unsupported-host scenarios are skipped on this Linux host. The existing seven
WASM MCP scenarios pass. Full `npm run check` passes; its approximately 3 GiB
sampled peak and `memory.high` events are excluded from performance claims.
Fresh candidate real-artifact/catalog and model comparisons are still pending.
Candidate real-artifact/catalog and model observations are recorded below.
## Public inputs and independent checks
@@ -94,5 +94,88 @@ and `--output` selecting a new private receipt. It acquires no dependencies and
does not overwrite an existing receipt.
The [template model question](questions/evm-uniswap.txt) requires the same complete
inferred interface facts and explicit unknowns. Real model trials and a candidate
recommendation are pending; an ABI-count match is not used as a model grade.
inferred interface facts and explicit unknowns. An ABI-count match is not used as
a model grade.
## Candidate compatibility and real-provider verification
Candidate: `152984da771450048ee9d60db8a410994a125878`, frozen runtime file-table
SHA-256 `725563640c27eb50ddb6f977d2dfbe1bdaddf7e05c02897318bceaa36fa602d7`.
Two repetitions for each EVM input produce six passing paired fact/settings
comparisons with the same driver and explicit limits. Each report preserves both
original Evidence reference maps. The complete actual SDK catalog is byte-identical
on every pair, including order, names, descriptions, schemas and annotations;
catalog SHA-256 is
`8664e42fd5225645cfd910f440f731c67ac198d3d53aa85b5c485374c74cf603`.
Result byte counts remain unchanged. Candidate Factory/Pool selectors also match
the independent published ABI oracle.
| Question | Control seconds, repetitions 1 / 2 | Candidate seconds, repetitions 1 / 2 |
| -------- | ---------------------------------: | -----------------------------------: |
| Factory | 3.628 / 3.650 | 3.708 / 3.605 |
| Pool | 11.211 / 3.591 | 3.749 / 3.581 |
| WETH | 11.432 / 3.694 | 3.572 / 3.677 |
First controls preceded wiring; each second pair ran candidate then control.
Every measurement starts a new MCP process; host cache/page state is not fully
controlled. Slower initial controls do not establish a descriptor speedup.
Second-pair latency and last-observed CPU are similar. Sampled EVM SDK family peaks
range from 510,255,104 to 527,622,144 bytes. No scope abort occurred; final counters
remain unknown, so these measurements are descriptive.
The candidate passes all 59 existing real CLI/MCP verifier cases with solc 0.8.30
and unchanged EVMole 0.9.3. Actual syscall traces report target execution and network
sockets absent for those scenarios, and the verifier reports no remaining
descendants. The scope took 177.456 seconds, sampled peak 1,237,020,672 bytes and
last-observed CPU 144.913 seconds. Different totals from the earlier control are
not attributed to unchanged provider algorithms.
Post-descriptor public MCP regression uses the original pinned shipped corpus:
| Domain / selected question | Actual provider / scope | Calls | Seconds |
| ------------------------------------------- | -------------------------------------------------- | ----: | ------: |
| SQLite WASM counts / exports | WABT 1.0.42; selected summary and retained read | 2 | 5.516 |
| ShareX managed metadata / native boundaries | Static PE/CLI, no assembly execution | 2 | 3.710 |
| Obsidian manifest / inventory | Complete inventory retained, no AST claim | 4 | 5.376 |
| NewPipe package / MainActivity methods | Real headless JADX 0.7.1 | 3 | 21.669 |
| OpenWrt regions | Real firmware signature scan, no device execution | 1 | 3.581 |
| jq native entry | Ghidra 12.1.4, default analysis, ephemeral Program | 4 | 270.986 |
All six workflows pass named SDK output validation and original digest checks;
input bytes remain unchanged. Maximum sampled family peak is 1,284,648,960 bytes.
They establish those available Linux workflows, not paired performance or
Windows/macOS, IDA/Hopper, Android-device, DOS COM, browser or JavaScript AST
behavior. Browser/AST were not repeated for this narrow migration.
## Real model complete-output calibration
Six serial Codex CLI 0.161.0 trials request `gpt-5.6-terra`, high reasoning, with
isolated temporary profiles pointing to the frozen candidate. Resolved server-side
model identity is not inferred. Only the EVM inspection tool is enabled, matching
the earlier WETH calibration; this is not full-catalog tool selection verification.
Every trial makes one successful call and no failed call.
| Trial | Total input | Uncached input | Output | Seconds |
| --------- | ----------: | -------------: | -----: | ------: |
| Factory 1 | 66,158 | 36,974 | 1,329 | 43.483 |
| Pool 1 | 65,600 | 36,416 | 2,307 | 69.546 |
| WETH 1 | 54,700 | 25,516 | 1,144 | 56.372 |
| Factory 2 | 65,964 | 36,780 | 1,453 | 26.412 |
| Pool 2 | 65,753 | 36,569 | 1,939 | 74.706 |
| WETH 2 | 54,502 | 25,318 | 1,208 | 35.137 |
Every trial reports 29,184 cached input tokens. Selected fact/reference/lifecycle
grades all pass: original/decoded identities, every inferred selector, offset,
dispatch, argument/mutability string including null, explicit unknowns, no runtime
execution, observed Evidence references, unchanged bytes and removed profiles.
Raw transcripts and grades remain private. Sampled peaks are
594,853,888–776,335,360 bytes; last-observed CPU is 6.181–7.034 seconds. Strict
overall grades remain unverified because final resource counters are unknown;
they are not relabeled as wholly passing resource-qualified runs.
This calibrates the current complete path after compatible wiring, not a paired
model performance experiment. The next token experiment should answer the same
interface question through an explicitly selected precise view, retain full
bytecode/original Evidence under the existing lifetime, and preserve complete
defaults. Discover support before sending new options to an older server. Include
discovery, follow-up, failures and retries in the complete task cost.