docs(experiment): measure bounded native definition projections

Preserve a diagnostic-only patch and reproducible lossless encoding/projection probe. The real VM graph exceeds several public capacities; selected SSA neighborhoods are much smaller, with explicit frontiers. Record actual resource costs and keep public engine/API limits unchanged.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

experiments/atomic-codemode/README.md

@@ -191,3 +191,9 @@ retry. Both public MCP adapters preserve that error and retry successfully.
 These use an injected tagged failure, not an observed engine cleanup incident;
 native reads and subsequent cleanup use the real provider. See the recorded
 scope and resource receipts in [native-engine.md](native-engine.md).
+
+A [subsequent diagnostic probe](projection-experiment.md) measures the VM's full
+17 MB recovered graph and much smaller selected definition neighborhoods. It
+supports trying a bounded operation/value facet, while preserving unresolved
+definitions and unavailable CFG. This is offline feasibility evidence; selected
+definition bodies remain an unimplemented public API gap.

experiments/atomic-codemode/kuna-graph-scale-diagnostic.patch

@@ -0,0 +1,28 @@
+--- a/decompiler/crates/kuna-decomp/src/substrate/atomic_facts.rs
++++ b/decompiler/crates/kuna-decomp/src/substrate/atomic_facts.rs
+@@ -64,10 +64,10 @@
+ pub fn collect(arch: &Architecture, fd: &Funcdata) -> Result<FunctionFacts, String> {
+     // Bounds cover allocations before JSON serialization, independently of the
+     // transport byte budget. Overflow is an unavailable facet, not clipped IR.
+-    const MAX_OPERATIONS: usize = 12_000;
+-    const MAX_VALUES: usize = 36_000;
+-    const MAX_INPUTS: usize = 50_000;
+-    const MAX_DISPLAY_BYTES: usize = 2 * 1024 * 1024;
++    const MAX_OPERATIONS: usize = 32_000;
++    const MAX_VALUES: usize = 72_000;
++    const MAX_INPUTS: usize = 128_000;
++    const MAX_DISPLAY_BYTES: usize = 16 * 1024 * 1024;
+     if fd.bblocks_get_size() <= 0 {
+         return Err("recovered basic-block graph is unavailable".into());
+     }
+--- a/decompiler/crates/kuna-decomp/src/substrate/atomic_values.rs
++++ b/decompiler/crates/kuna-decomp/src/substrate/atomic_values.rs
+@@ -7,7 +7,7 @@
+ use slotmap::Key;
+ use std::collections::BTreeSet;
+
+-pub const MAX_TEXT_BYTES: usize = 2 * 1024 * 1024;
++pub const MAX_TEXT_BYTES: usize = 16 * 1024 * 1024;
+ #[derive(Debug, Clone, Serialize)]
+ pub struct ValueFact {
+     pub id: String,

experiments/atomic-codemode/measure-native-projection.py

@@ -0,0 +1,123 @@
+"""Size diagnostic SQLite SSA projections; does not implement a provider API."""
+
+import argparse
+import hashlib
+import json
+from pathlib import Path
+
+
+COLUMNS = {
+    'blocks': ['id', 'start', 'operations', 'predecessors', 'successors'],
+    'operations': ['id', 'address', 'block', 'opcode', 'statement', 'is_dead', 'inputs', 'output'],
+    'inputs': ['input_index', 'value_id', 'phi_predecessor', 'phi_predecessor_edge'],
+    'successors': ['id', 'target', 'index', 'polarity', 'condition_value', 'condition_test'],
+    'values': ['id', 'space', 'offset', 'size_bytes', 'is_constant', 'is_input',
+               'is_annotation', 'definition', 'type_name'],
+}
+
+
+def encoded(value):
+    return json.dumps(value, ensure_ascii=True, separators=(',', ':')).encode()
+
+
+def pack(table, rows):
+    result = []
+    for row in rows:
+        assert set(row) == set(COLUMNS[table]), 'Do not silently omit unknown fields'
+        result.append([pack(key, row[key]) if key in ('inputs', 'successors') else row[key]
+                       for key in COLUMNS[table]])
+    return result
+
+
+def unpack(table, rows):
+    return [{key: unpack(key, value) if key in ('inputs', 'successors') else value
+             for key, value in zip(COLUMNS[table], row, strict=True)} for row in rows]
+
+
+def projection(operations, values, roots, depth, cap):
+    retained, wave = set(roots), set(roots)
+    exhausted = False
+    for _ in range(depth):
+        following = {values[i['value_id']]['definition'] for key in wave
+                     for i in operations[key]['inputs'] if i['value_id'] is not None
+                     and values[i['value_id']]['definition'] in operations} - retained
+        if len(retained | following) > cap:
+            exhausted = True
+            break
+        retained |= following
+        wave = following
+    ids = {operations[key]['output'] for key in retained
+           if operations[key]['output'] is not None} | {
+               i['value_id'] for key in retained for i in operations[key]['inputs']
+               if i['value_id'] is not None}
+    frontier = {values[i['value_id']]['definition'] for key in retained
+                for i in operations[key]['inputs'] if i['value_id'] is not None
+                and values[i['value_id']]['definition'] is not None} - retained
+    payload = {'operations': [operations[key] for key in sorted(retained)],
+               'values': [values[key] for key in sorted(ids)]}
+    return {'requested_definition_depth': depth, 'operation_budget': cap,
+            'operation_budget_exhausted': exhausted, 'operations': len(retained),
+            'values': len(ids), 'encoded_bytes': len(encoded(payload)),
+            'unexpanded_definition_frontier': len(frontier)}
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('native_json', type=Path)
+    parser.add_argument('report_json', type=Path)
+    args = parser.parse_args()
+    budget = 32 * 1024 * 1024
+    with args.native_json.open('rb') as source:
+        raw = source.read(budget + 1)
+    assert len(raw) <= budget, 'Use a bounded diagnostic capture'
+    document = json.loads(raw)
+    assert len(document['functions']) == 1
+    function = document['functions'][0]
+    assert function['address_hex'] == '0xea6d0', 'This is the seeded SQLite VM experiment'
+    graph = function['atomic_facts']
+    assert graph['schema_version'] == 3
+    assert len(graph['operations']) <= 32000 and len(graph['values']) <= 72000
+    tables = ['blocks', 'operations', 'values']
+    assert set(graph) == {'schema_version', 'coverage', *tables}
+    packed = {'encoding': 'diagnostic-tuples-proposal-not-public',
+              'schema_version': graph['schema_version'], 'columns': COLUMNS,
+              'coverage': graph['coverage'],
+              **{key: pack(key, graph[key]) for key in tables}}
+    restored = {'schema_version': packed['schema_version'], 'coverage': packed['coverage'],
+                **{key: unpack(key, packed[key]) for key in tables}}
+    assert restored == graph
+    operations = {row['id']: row for row in graph['operations']}
+    values = {row['id']: row for row in graph['values']}
+    assert len(operations) == len(graph['operations'])
+    assert len(values) == len(graph['values'])
+    calls = function['atomic_calls']['items']
+    selected = [row['id'] for row in calls
+                if row['origin_address'] in ('0xf115c', '0xf238b')]
+    assert len(selected) == 2
+    report = {
+        'scope': 'Offline diagnostic sizes, not public-provider or model results. '
+                 'Tuple roundtrip is lossless, but no native codec is implemented. '
+                 'Projections follow retained SSA operands only; no CFG, aliases, '
+                 'source equivalence or runtime proof. Bytes are not token/time savings.',
+        'source': str(args.native_json), 'source_sha256': hashlib.sha256(raw).hexdigest(),
+        'whole_native_document_bytes': len(raw), 'full_graph_bytes': len(encoded(graph)),
+        'lossless_tuple_graph_bytes': len(encoded(packed)), 'tuple_roundtrip_equal': True,
+        'estimated_whole_tuple_document_bytes': len(raw) - len(encoded(graph)) + len(encoded(packed)),
+        'operations': len(operations), 'values': len(values), 'blocks': len(graph['blocks']),
+        'inputs': sum(len(row['inputs']) for row in operations.values()),
+        'cfg_edges': sum(len(row['successors']) for row in graph['blocks']),
+        'statement_utf8_bytes': sum(len(row['statement'].encode()) for row in operations.values()),
+        'value_type_utf8_bytes': sum(len(row['type_name'].encode()) for row in values.values()),
+        'selected_call_projections': [projection(operations, values, selected, depth, 512)
+                                      for depth in range(1, 6)],
+        'all_call_projections': [projection(operations, values, [r['id'] for r in calls], depth, 4096)
+                                 for depth in range(1, 5)],
+    }
+    with args.report_json.open('x') as output:
+        json.dump(report, output, indent=2)
+        output.write('\n')
+    print(json.dumps(report, indent=2))
+
+
+if __name__ == '__main__':
+    main()

experiments/atomic-codemode/projection-experiment.md

@@ -0,0 +1,131 @@
+# Selected definitions after whole-graph capacity failure
+
+This is a diagnostic feasibility probe following the completed boundary model
+pairs, not a new public provider capability. The analyst question is whether
+selected call-argument definitions can remain readable without materializing an
+entire recovered graph in the public transport. The alternatives are a smaller
+whole-graph encoding and a bounded selection of operations/values. Neither
+fewer bytes nor an offline graph traversal establishes model-token or time gains.
+
+## Actual graph and capacity boundaries
+
+The artifact remains Debian SQLite `3.40.1-2+deb12u2`, SHA-256
+`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
+The selected implementation is `sqlite3VdbeExec`, entry `0xea6d0`.
+The disposable diagnostic patch temporarily admits 32,000 operations, 72,000
+values, 128,000 inputs and 16 MiB text. It is applied after
+`kuna-native-ir.patch`, against the same pinned Kuna upstream revision. Only
+this diagnostic capture uses a 32 MiB encoded/stdout budget. The committed
+public patch, preserved public engine and 8 MiB public budget are unchanged.
+
+| Observed quantity | Value |
+| ----------------- | ----: |
+| Recovered operations | 21,349 |
+| Recovered values | 35,755 |
+| Recovered blocks | 1,229 |
+| Operand inputs | 51,951 |
+| CFG successor edges | 2,061 |
+| Operation statement UTF-8 bytes | 2,255,616 |
+| Value type-name UTF-8 bytes | 90,013 |
+| Compact ordinary graph JSON bytes | 17,008,223 |
+| Entire native document bytes | 17,801,972 |
+
+Raising only the current 12,000-operation bound would not suffice: inputs also
+exceed 50,000, display text exceeds 2 MiB, and the entire document exceeds 8 MiB.
+Values happen to remain below 36,000, with little headroom. Native coverage
+reports no producer issues or omitted defining-operation IDs. This is the
+retained recovered graph, not proof of complete/correct machine or runtime
+semantics. The ordinary function record, including C and mappings, and the
+independent call facet deeply equal the preserved public-engine capture.
+
+`measure-native-projection.py` constructs a lossless tuple encoding of every
+graph field and verifies decoding equals the original graph. Its graph occupies
+8,225,047 bytes; estimated whole-document size is 9,018,796 bytes after retaining
+C/calls/metadata. Thus this simple encoding still does not fit the unchanged
+8,388,608-byte public budget. This is an exact synthetic serialization size and
+roundtrip check, not an implemented native codec or measured codec speed.
+
+## Focused definition facts
+
+Using the two existing call identities at `0xf115c` and `0xf238b` as roots, the
+script follows native operand values to their defining operations. It retains
+exact IDs and complete selected operation/value records. A 512-operation budget
+is not reached in these depth samples; unresolved definitions remain explicit.
+
+| Requested backward depth | Operations | Values | Encoded payload bytes | Unexpanded definition frontier |
+| ------------------------ | ---------: | -----: | --------------------: | -----------------------------: |
+| 1 | 5 | 12 | 4,335 | 6 |
+| 2 | 11 | 24 | 8,589 | 3 |
+| 3 | 14 | 29 | 10,447 | 1 |
+| 4 | 15 | 32 | 11,457 | 1 |
+| 5 | 16 | 33 | 11,934 | 1 |
+
+The same experiment over all 416 recovered call roots gives:
+
+| Requested backward depth | Operations | Values | Encoded payload bytes |
+| ------------------------ | ---------: | -----: | --------------------: |
+| 1 | 965 | 2,269 | 830,233 |
+| 2 | 1,494 | 3,076 | 1,188,208 |
+| 3 | 2,007 | 3,920 | 1,541,735 |
+| 4 | 2,401 | 4,670 | 1,849,015 |
+
+The 4,096-operation probe budget is not reached in these samples. Depth three
+still has 394 unexpanded definitions. These payload sizes cover selected
+operations/values alone, before source/provenance/transport framing. Depths are
+probe parameters, not proposed product limits. This is backward SSA dependence,
+without CFG, control dependence, complete use lists, memory aliases or feasible
+runtime paths. No model receives these offline projections in the completed
+trials. They do not become accessible through an existing snapshot by this probe.
+
+The signal favors trying a bounded call-rooted operation/value facet when the
+full graph is unavailable. The native producer already has the live function at
+the capture boundary, so this could reuse that analysis rather than reconstruct
+definitions from C or force repeated decompilation. A focused initial capture
+is another option; later requests would need a new analysis or a separately
+owned live native model. That lifecycle/time tradeoff remains unresolved.
+
+Any prototype must share native identity/value/operand interpretation, retain
+frontier reasons and source Evidence, expose partial definition/use scope, and
+keep CFG-dependent reads unavailable. A native block/origin ID does not establish
+a machine instruction extent or branch target. Both direct and Code Mode must
+receive the same facts through the shared API before another model comparison.
+
+## Reproduction and measured scope
+
+Apply `kuna-native-ir.patch`, then `kuna-graph-scale-diagnostic.patch`, to pinned
+upstream `fe01d6294eb0d1053ce343c8943646df88e143fc`. Reuse its stock SLA assets
+and the one-job embedded-spec build profile in [native-engine.md](native-engine.md).
+The preserved diagnostic executable has SHA-256
+`6d0d1cef9de13180e19af77a6f62075c417178831740bccc7174bbac0af764cc`.
+It is copied outside Cargo outputs, separately from the public `e7d3f0cc...` engine.
+The initial build launch failed before Cargo because its PATH omitted the existing
+Cargo installation; that setup failure remains recorded, followed by a corrected
+build. No toolchain was installed or upgraded.
+
+The capture reuses `KunaTargetCommands.run`, a fresh private installation
+`configuration.cacheRoot`, owned target staging and normal supervision. Its argv
+is `decompile INPUT 0xea6d0 --addr --json --atomic-facts
+--atomic-output-bytes 33554432 --max-fn-memory 1024`, with matching 32 MiB
+supervised stdout capacity. It does not execute the target. The temporary native
+source changes are restored after capture; diagnostic patch reverse/forward and
+public patch reverse checks pass.
+
+The diagnostic build takes 316.764 family wall / 316.136 CPU seconds, 2,113.5 MiB
+kernel family peak. Native capture takes 37.078 / 36.869 seconds, 551.4 MiB.
+Offline sizing/roundtrip takes 1.428 / 1.242 seconds, 170.0 MiB. All use one CPU,
+4 GiB hard memory, 3 GiB early stop and verify cleanup/OOM 0; peaks include charged
+file cache. OS caches are uncontrolled. There is no equivalent completed-task
+speed comparison, public-provider success claim or model-token measurement here.
+
+Run the bounded sizing script against the retained diagnostic native JSON:
+
+```sh
+python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
+  python3 experiments/atomic-codemode/measure-native-projection.py \
+  /absolute/native.json /absolute/new-sizing-report.json
+```
+
+Raw receipts are `graph-scale-diagnostic-engine.json`,
+`graph-scale-diagnostic-receipt.json`, `graph-scale-repro-sizing.json` and their
+resource logs under `.cache/atomic-codemode/`. Binaries and the 17.8 MB native
+capture remain local; they are not committed.