mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 17:35:11 +00:00
docs(evm): record compatible wiring and real task costs
Record six exact SDK catalog/fact pairs, 59 real CLI/MCP cases, six cross-domain artifact workflows and six actual Codex CLI trials. Preserve failed setup attempts privately and qualify unchanged complete delivery, host-state variance and unknown final resource counters. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
152984da77
commit
d999d9874b
2 files changed
+89
-5
No files matched your search
@@ -44,8 +44,9 @@ acceptance conditions for every optimization.
|
||||
experiment control; it does not itself establish reduced CPU consumption.
|
||||
|
||||
Migrate domain by domain with focused real-artifact checks and relevant shared
|
||||
contract checks. The domain matrix is a growing roadmap; native PE and DOS, for
|
||||
example, remain unverified in this study. A passed lane does not imply all
|
||||
contract checks. The native follow-up verifies selected PE and DOS MZ entry facts;
|
||||
DOS COM and complete native application behavior remain unverified. The domain
|
||||
matrix is a growing roadmap. A passed lane does not imply all
|
||||
advertised platforms/providers have passed. Newly discovered REA bugs remain
|
||||
in the separate local handoff document for later team repair.
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ real EVMole/WABT claims. Nine service and 26 provider-boundary cases also pass;
|
||||
two unsupported-host scenarios are skipped on this Linux host. The existing seven
|
||||
WASM MCP scenarios pass. Full `npm run check` passes; its approximately 3 GiB
|
||||
sampled peak and `memory.high` events are excluded from performance claims.
|
||||
Fresh candidate real-artifact/catalog and model comparisons are still pending.
|
||||
Candidate real-artifact/catalog and model observations are recorded below.
|
||||
|
||||
## Public inputs and independent checks
|
||||
|
||||
@@ -94,5 +94,88 @@ and `--output` selecting a new private receipt. It acquires no dependencies and
|
||||
does not overwrite an existing receipt.
|
||||
|
||||
The [template model question](questions/evm-uniswap.txt) requires the same complete
|
||||
inferred interface facts and explicit unknowns. Real model trials and a candidate
|
||||
recommendation are pending; an ABI-count match is not used as a model grade.
|
||||
inferred interface facts and explicit unknowns. An ABI-count match is not used as
|
||||
a model grade.
|
||||
|
||||
## Candidate compatibility and real-provider verification
|
||||
|
||||
Candidate: `152984da771450048ee9d60db8a410994a125878`, frozen runtime file-table
|
||||
SHA-256 `725563640c27eb50ddb6f977d2dfbe1bdaddf7e05c02897318bceaa36fa602d7`.
|
||||
Two repetitions for each EVM input produce six passing paired fact/settings
|
||||
comparisons with the same driver and explicit limits. Each report preserves both
|
||||
original Evidence reference maps. The complete actual SDK catalog is byte-identical
|
||||
on every pair, including order, names, descriptions, schemas and annotations;
|
||||
catalog SHA-256 is
|
||||
`8664e42fd5225645cfd910f440f731c67ac198d3d53aa85b5c485374c74cf603`.
|
||||
Result byte counts remain unchanged. Candidate Factory/Pool selectors also match
|
||||
the independent published ABI oracle.
|
||||
|
||||
| Question | Control seconds, repetitions 1 / 2 | Candidate seconds, repetitions 1 / 2 |
|
||||
| -------- | ---------------------------------: | -----------------------------------: |
|
||||
| Factory | 3.628 / 3.650 | 3.708 / 3.605 |
|
||||
| Pool | 11.211 / 3.591 | 3.749 / 3.581 |
|
||||
| WETH | 11.432 / 3.694 | 3.572 / 3.677 |
|
||||
|
||||
First controls preceded wiring; each second pair ran candidate then control.
|
||||
Every measurement starts a new MCP process; host cache/page state is not fully
|
||||
controlled. Slower initial controls do not establish a descriptor speedup.
|
||||
Second-pair latency and last-observed CPU are similar. Sampled EVM SDK family peaks
|
||||
range from 510,255,104 to 527,622,144 bytes. No scope abort occurred; final counters
|
||||
remain unknown, so these measurements are descriptive.
|
||||
|
||||
The candidate passes all 59 existing real CLI/MCP verifier cases with solc 0.8.30
|
||||
and unchanged EVMole 0.9.3. Actual syscall traces report target execution and network
|
||||
sockets absent for those scenarios, and the verifier reports no remaining
|
||||
descendants. The scope took 177.456 seconds, sampled peak 1,237,020,672 bytes and
|
||||
last-observed CPU 144.913 seconds. Different totals from the earlier control are
|
||||
not attributed to unchanged provider algorithms.
|
||||
|
||||
Post-descriptor public MCP regression uses the original pinned shipped corpus:
|
||||
|
||||
| Domain / selected question | Actual provider / scope | Calls | Seconds |
|
||||
| ------------------------------------------- | -------------------------------------------------- | ----: | ------: |
|
||||
| SQLite WASM counts / exports | WABT 1.0.42; selected summary and retained read | 2 | 5.516 |
|
||||
| ShareX managed metadata / native boundaries | Static PE/CLI, no assembly execution | 2 | 3.710 |
|
||||
| Obsidian manifest / inventory | Complete inventory retained, no AST claim | 4 | 5.376 |
|
||||
| NewPipe package / MainActivity methods | Real headless JADX 0.7.1 | 3 | 21.669 |
|
||||
| OpenWrt regions | Real firmware signature scan, no device execution | 1 | 3.581 |
|
||||
| jq native entry | Ghidra 12.1.4, default analysis, ephemeral Program | 4 | 270.986 |
|
||||
|
||||
All six workflows pass named SDK output validation and original digest checks;
|
||||
input bytes remain unchanged. Maximum sampled family peak is 1,284,648,960 bytes.
|
||||
They establish those available Linux workflows, not paired performance or
|
||||
Windows/macOS, IDA/Hopper, Android-device, DOS COM, browser or JavaScript AST
|
||||
behavior. Browser/AST were not repeated for this narrow migration.
|
||||
|
||||
## Real model complete-output calibration
|
||||
|
||||
Six serial Codex CLI 0.161.0 trials request `gpt-5.6-terra`, high reasoning, with
|
||||
isolated temporary profiles pointing to the frozen candidate. Resolved server-side
|
||||
model identity is not inferred. Only the EVM inspection tool is enabled, matching
|
||||
the earlier WETH calibration; this is not full-catalog tool selection verification.
|
||||
Every trial makes one successful call and no failed call.
|
||||
|
||||
| Trial | Total input | Uncached input | Output | Seconds |
|
||||
| --------- | ----------: | -------------: | -----: | ------: |
|
||||
| Factory 1 | 66,158 | 36,974 | 1,329 | 43.483 |
|
||||
| Pool 1 | 65,600 | 36,416 | 2,307 | 69.546 |
|
||||
| WETH 1 | 54,700 | 25,516 | 1,144 | 56.372 |
|
||||
| Factory 2 | 65,964 | 36,780 | 1,453 | 26.412 |
|
||||
| Pool 2 | 65,753 | 36,569 | 1,939 | 74.706 |
|
||||
| WETH 2 | 54,502 | 25,318 | 1,208 | 35.137 |
|
||||
|
||||
Every trial reports 29,184 cached input tokens. Selected fact/reference/lifecycle
|
||||
grades all pass: original/decoded identities, every inferred selector, offset,
|
||||
dispatch, argument/mutability string including null, explicit unknowns, no runtime
|
||||
execution, observed Evidence references, unchanged bytes and removed profiles.
|
||||
Raw transcripts and grades remain private. Sampled peaks are
|
||||
594,853,888–776,335,360 bytes; last-observed CPU is 6.181–7.034 seconds. Strict
|
||||
overall grades remain unverified because final resource counters are unknown;
|
||||
they are not relabeled as wholly passing resource-qualified runs.
|
||||
|
||||
This calibrates the current complete path after compatible wiring, not a paired
|
||||
model performance experiment. The next token experiment should answer the same
|
||||
interface question through an explicitly selected precise view, retain full
|
||||
bytecode/original Evidence under the existing lifetime, and preserve complete
|
||||
defaults. Discover support before sending new options to an older server. Include
|
||||
discovery, follow-up, failures and retries in the complete task cost.
|
||||
Reference in new issue
Block a user