Files
reverse-engineering-anything/docs/guides/native/go-binaries.md
T
2026-11-14 12:13:56 +01:00

274 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Go binary build metadata
`inspect_go_binary` / `inspect-go-binary` answers which Go compiler version,
modules, replacements and build settings are embedded in one selected local
executable or library. It reads the file without running it and requires no
Go installation, active binary target, Hopper, Ghidra or IDA.
If a stable read completes but descriptor finalization fails or receives
cancellation, the error retains the selected path, SHA-256 and byte length as
a `file-snapshot-hash` partial observation. It does not claim that Go metadata
was decoded. Failed cleanup stays incomplete until its exact owner confirms
release; cancellation keeps the caller's reason.
```bash
rea inspect-go-binary /absolute/path/to/application --json
```
The CLI resolves relative paths against the caller's working directory. MCP
requires an absolute filesystem path and preserves that exact selection:
```json
{
"name": "inspect_go_binary",
"arguments": { "path": "/artifacts/application" }
}
```
For build identity and counts, select `detail: "summary"`. MCP retains the same
complete Evidence and returns `normalized_result.parent_evidence_id`. Read
selected dependency versions without reopening the file:
```json
{
"name": "inspect_analysis_view",
"arguments": {
"source": {
"kind": "retained-evidence",
"evidence_id": "ev_<parent digest>"
},
"view": {
"kind": "go",
"collection": "dependencies",
"paths": ["golang.org/x/crypto"],
"fields": ["path", "version", "replacement"],
"offset": 0,
"limit": 10
}
}
}
```
Go views support `dependencies` (exact `paths`), `settings` (exact `keys`) and,
when collected with `include_string_literals: true`, `string_literals` (exact
`values` and substring `contains`), plus optional original `indices`, field
selection, and pagination. Filters intersect; empty filter arrays select no rows
and `fields: []` returns only row identity. Each row keeps its original `index`,
scoped to the parent Evidence and collection; duplicate module paths and setting
keys keep distinct indices. Page offsets count matching records. Exhausting
retained rows does not establish complete runtime dependencies or string
coverage. Original bytes, unknown lines and all complete metadata remain in the
parent and can be exported as an Evidence bundle.
One-shot CLI summaries save complete Evidence before returning the projection:
```bash
rea inspect-go-binary ./application --detail summary --evidence_output ./go-evidence.json --json
```
`--evidence_output` is required for CLI summary delivery and never overwrites an
existing file. The default CLI/MCP reply remains complete.
[Measured workflows](https://github.com/morluto/rea/blob/42e1a17d6e6c0487f17be0bae248553c4caba7f7/experiments/analysis-efficiency/go-build-views.md) include
catalog repetition and extra calls. Identity/count questions benefit from a
single summary. In the tested Codex full-catalog configuration, a compound
question about identity and one dependency uses fewer tokens in captured request JSON with
one complete inspection than with a summary followed by a lookup. The complete
reply remains useful for that initial question; retained views support later
selected reads. Token figures are serialization estimates, not model usage.
## Observations
The inline Evidence result contains the artifact path, SHA-256, byte length,
container format, architecture, word size and byte order. A recognized
`build_info` record contains:
- `go_version`, preserved as embedded UTF-8 text rather than forced into a
semantic version, or `null` when the embedded bytes are not UTF-8;
- `go_version_bytes_base64`, preserving the exact compiler-string bytes;
- exact file offsets and byte lengths for the header, compiler string and module
string;
- `module_text`, the decoded module-information body, or `null` when the complete
body cannot be represented as UTF-8;
- `module_bytes_base64`, the original module-string bytes including Go framing;
- `module.path`, the embedded package path;
- `module.main`, the main module's path, version, checksum and optional
replacement;
- `module.dependencies`, with the same fields and their original order;
- `module.settings`, retaining ordered key/value entries and duplicate keys;
- `module.unparsed_lines`, `module.unparsed_line_bytes_base64` and
`module.complete`, which says whether every present module-text line was
parsed. It does not establish that the dependency inventory is complete.
Missing checksums are `null`; empty fields and development/local replacement
versions remain unchanged. Build settings such as `GOOS`, `GOARCH` or `vcs.revision`
are reported when present. Their absence does not establish a build choice.
Go strings may contain arbitrary bytes. Non-UTF-8 records remain available as
base64 while other decodable module records are still reported; the reader does
not insert replacement characters or reject the entire artifact for those
strings.
When no main-module (`mod`) record was decoded, `module.main` is `null` and
dependency coverage is unknown. Successfully decoded dependency records remain
visible. GOPATH-mode builds may contain only a package `path` and build settings;
their empty dependency array does not establish the absence of dependencies.
The result's limitations state this explicitly. A module-mode build with a main
module record and no dependency records has an observed empty dependency list.
When `include_string_literals: true` (CLI `--include-string-literals`, optional
`max_string_literals` / `--max-string-literals`, default `2000`)
is selected, `string_literals` reports `evidence_kind: "inferred"` alongside
deduplicated printable UTF-8 literals referenced by word-aligned two-word
`(ptr, len)` string headers in non-executable initialized data. Because Go packs
read-only string data contiguously without NUL terminators, slicing by `(ptr, len)`
recovers exact literal boundaries instead of concatenated C-string runs. Each
target slice (default `1`–`4096` bytes) must stay within a single read-only mapped
section or segment, decode as printable UTF-8 (`\t`, `\n`, `\r`, joiners
`U+200C` / `U+200D` and printable Unicode scalar values, excluding control and
other format characters), and fit within
a default 64 MiB header-scan budget and 512 KiB aggregate string budget
(`truncated: true` when capped). Select `max_string_literal_bytes`,
`max_string_bytes`, `max_string_header_scan_bytes`, or `max_string_candidates`
to change these independent budgets. Their CLI flags use hyphens, for example
`--max-string-literal-bytes 16384`. `limits_reached` names exhausted controls;
increase those controls for a new inspection of the same complete artifact. Compiler-inlined or
stack-constructed strings without static headers, unapplied relocation addends
and Mach-O `LC_DYLD_CHAINED_FIXUPS` pointer encodings are noted in `limitations`.
## Supported boundary
The built-in TypeScript reader supports ELF, PE and thin Mach-O images, and one
selected slice of a universal Mach-O, with Go's inline or historical pointer
build-info encoding. Source ranges refer to the original stable file snapshot,
including pointer-backed strings. The shared native container owns address
mapping, including file-backed PE headers. A descriptor or string can cross
adjacent mappings when their virtual addresses map to physically contiguous
file bytes with the same translation. Gaps, conflicting aliases, zero-fill
aliases and noncontiguous string source bytes cannot establish a single source
range and are refused. Native export C strings use the same mapping checks but
can join multiple source chunks because their result has no single-span source
contract. Legacy pointer-backed fields are decoded at
the width declared by Go's build-info record, independently of the image's ELF
class. It does not scan arbitrary byte strings as compiler metadata.
Legacy string descriptors and contents may use the declared file-backed PE
headers. A descriptor or string may also cross adjacent loader mappings when
their virtual ranges cover every requested byte and identify one contiguous
file range. Gaps, conflicting file aliases and nonempty zero-fill overlap remain
invalid. Discovery policy and the aggregate 1 MiB string-content limit are
unchanged. These are static reads over REA's native mapping facts; they do not
establish Windows/Linux loading or parity with Go's standard reader, whose
[Go 1.24.0 ELF/PE readers](https://github.com/golang/go/blob/go1.24.0/src/debug/buildinfo/buildinfo.go#L445)
read within individual segments or sections. The historical two-word string
descriptor follows the
[Go 1.17.13 reader](https://github.com/golang/go/blob/go1.17.13/src/cmd/go/internal/version/version.go#L212).
A universal (FAT) Mach-O needs `architecture` (`x86`, `x86_64`, `arm`, `arm64`
or `arm64e`; CLI `--architecture`). REA reads its own FAT table, without `lipo`,
rejects a table whose slices overlap, and validates the selected slice's range,
alignment and Mach-O header CPU against the table. The result keeps the
universal file as `artifact` (its SHA-256 and size), adds `selected_slice` with
the slice offset, length, raw CPU type/subtype and the FAT table digest, and
reports every source range as a file offset inside that slice. Without
`architecture`, or when the file has no slice of that CPU family, the request
fails on `architecture` and lists the slices present. A CPU family with one
slice is selected by its family name, so `arm64` also reads a lone `arm64e`
slice; `arm64e` reads only an `arm64e` slice. When several slices share the
family, the value must name the slice's subtype, as `lipo -thin` does: `arm64`
reads the `arm64` (or `arm64v8`) slice, `arm64e` the `arm64e` (or `arm64e.x1`)
slice, and `x86_64` the canonical `x86_64` slice. Capability bits are ignored
when subtypes are compared. REA does not choose a slice by preference. When no
slice or several slices carry the named subtype, the request fails on
`architecture` and names any other value that selects exactly one verified
slice; `arm64e` beside `arm64e.x1`, for example, is refused. Other subtypes,
such as `x86_64h` and the `x86` (i386) and 32-bit `arm` subtypes, have no value
of their own; they are read only as the only slice of their family.
`limitations` lists the same-family slices that were not read and the value, if
any, that selects each. A slice whose Mach-O header does not match its FAT
declaration, such as a different CPU subtype, is listed as unverified, and no
value is offered for it. `architecture` on any other image is rejected rather
than ignored.
For ELF images without a named build-info section, discovery checks each
nonempty writable, non-executable load segment in program-header order. This
includes sectionless PIE images with metadata in a later data segment. Named
sections retain priority, and alignment, file mapping and record validation
remain required. When ELF section names are unreadable, limitations report that
coverage gap; the fallback search policy is unchanged.
A named ELF or Mach-O build-info section selects the record's discovery range.
For inline encoding, the following version and module strings may continue past
that section into its file-backed load mapping; their bytes are still checked
against the mapping and the existing aggregate string limit.
Historical moduleless Go binaries can store their empty module-string header in
declared zero-initialized virtual memory rather than file-backed bytes. The reader
recognizes this only when the complete optional module header lies in that memory
without a conflicting file mapping. Its module contents are empty and
`module_location` has zero bytes, anchored at the build-info field containing the
module-header pointer. Compiler strings and other unmapped pointers remain strict
file-backed reads.
`build_info: null` means no recognized record was found in the inspected image.
It does not prove that the binary is not Go. Stripping, old builds, deliberate
removal and obfuscation can limit metadata. A malformed supported container or
recognized build-info record returns an input-format failure. Unsupported
containers receive a separate unsupported-target failure.
Symlinks and nonregular files are rejected. The shared stable reader detects
replacement and observed in-place changes during acquisition. Missing files,
host permission denial, changed artifacts and cancellation retain distinct
errors. The selected file is never modified; this inspection creates no
temporary files, starts no subprocesses and accesses no network endpoint.
The existing default resource budgets are now explicit per-call controls:
| Parameter | Default | What it admits |
| ------------------------------ | ----------- | ------------------------------------------------------------- |
| `max_input_bytes` | 256 MiB | Complete stable input file |
| `max_report_bytes` | 8 MiB | Complete normalized metadata representation |
| `max_build_info_bytes` | 1 MiB | Aggregate embedded string contents, excluding length prefixes |
| `max_header_search_bytes` | 256 MiB | Cumulative build-info search work |
| `max_string_literals` | 2,000 | Returned inferred string rows |
| `max_string_literal_bytes` | 4,096 bytes | Each inferred string candidate |
| `max_string_bytes` | 512 KiB | Aggregate returned inferred text |
| `max_string_header_scan_bytes` | 64 MiB | String-header and candidate-byte examination |
| `max_string_candidates` | 100,000 | Distinct inferred candidates retained before projection |
Every value is a positive safe integer. Input and string byte maxima follow the
runtime Buffer/string representation, with base64 expansion accounted for in
build-info admission; schemas advertise their exact maxima. CLI options use the same names with
hyphens, such as `--max-build-info-bytes 2097152`. Inferred-string controls
require `include_string_literals: true`. Omitted controls keep the defaults
and stay omitted in request Evidence. Error recovery names the exhausted
parameter and effective bytes; increase that parameter on the next call,
without restarting REA. The original complete artifact and its identity are
preserved. An unchanged retry cannot increase a deterministic allowance.
The shared native container reader separately bounds structural table decoding
to 16 MiB. None of these Go controls changes that boundary. Inline strings must
also fit their actual mapped source range. Identical search spans with the same
virtual alignment are searched once; partially overlapping spans consume the
cumulative search budget. Oversized complete reports fail without partial
success and without allocating the full JSON serialization. MCP's separate
[response transport budget](../../reference/mcp-contracts.md) can still require
export or paging of successfully retained Evidence. These controls do not
establish operating-system memory or CPU isolation.
Embedded metadata is evidence of file content. It does not authenticate a
compiler, dependency, commit or supply chain, and it does not establish runtime
behavior. Function names, source mappings, types and decompiled code are outside
this tool's coverage. Use the [native workflow](native-investigation.md) when
those questions require a disassembler.
## Real producer verification
`npm run verify:go:binary` exercises compiler-built fixtures through the public
CLI and MCP, comparing their metadata with Go's standard `debug/buildinfo`
reader. The verification lane requires Go; normal inspection does not.
Reader architecture names remain verbatim in the report. Evidence subjects use
the recognized canonical family: `386` maps to `x86`, `amd64` to `x86_64`, and
`ppc` to `powerpc`. Unknown families such as `ppc64` remain unknown rather than
being inferred from a related spelling.