diff --git a/docs/.vitepress/documentation.ts b/docs/.vitepress/documentation.ts index 992e6c79ba..ea13010ef8 100644 --- a/docs/.vitepress/documentation.ts +++ b/docs/.vitepress/documentation.ts @@ -68,6 +68,11 @@ export const documentationSections = [ source: "guides/native/windows-shortcut.md", route: "/windows-shortcut", }, + { + source: "guides/native/ntfs-metadata.md", + route: "/ntfs-metadata", + text: "Acquired NTFS metadata", + }, ], }, { diff --git a/docs/README.md b/docs/README.md index b14c188077..b4af0cbbb9 100644 --- a/docs/README.md +++ b/docs/README.md @@ -21,6 +21,7 @@ product pages. The VitePress documentation source is this `docs/` tree. - [Linux kernel build config](guides/native/kernel-build-config.md) - [Recorded crash evidence](guides/native/recorded-crashes.md) - [Windows Shell Link shortcuts](guides/native/windows-shortcut.md) +- [Acquired NTFS metadata and change journals](guides/native/ntfs-metadata.md) ## JavaScript and Electron diff --git a/docs/guides/native/ntfs-metadata.md b/docs/guides/native/ntfs-metadata.md new file mode 100644 index 0000000000..181f8f5788 --- /dev/null +++ b/docs/guides/native/ntfs-metadata.md @@ -0,0 +1,71 @@ +# Acquired NTFS metadata and change journals + +Use offline acquisitions of the logical `$MFT` stream and `$UsnJrnl:$J` +stream from the same volume. REA reads the selected files without mounting a +volume, following data runs, querying Windows, or replaying journal operations. +Pairing the acquisitions is the caller's responsibility. + +```bash +rea inspect-artifact ./MFT --json +rea inspect-ntfs-journal ./MFT ./UsnJrnl-J --maxRows=50 --json +``` + +MCP accepts absolute local paths: + +```json +{ + "name": "inspect_ntfs_journal", + "arguments": { + "mft_path": "/artifacts/MFT", + "journal_path": "/artifacts/UsnJrnl-J", + "max_rows": 50 + } +} +``` + +The paired workflow admits at most 64 MiB per acquisition. Its Evidence records +both paths, lengths and SHA-256 digests. Keep the acquisitions and compare both +digests between pages. Offsets refer to those original files, whose bytes remain +unchanged; sector fixups are restored only in a scratch record before attribute +interpretation. + +`mftMetadata` reports aggregate inventory and fixup coverage, and at most 32 +record details. Names retain their individual parent references, namespaces, +FILETIMEs, sizes and flags; several names are not collapsed into a path. Standard +information timestamps stay separate from filename timestamps. All 64-bit times +and sizes use decimal strings. Resident attributes retain value spans; +nonresident attributes retain sizes, VCN ranges and validated packed mapping +pairs, including sparse and fragmented runs. Attribute-list entries retain full +references and IDs. Extension references are not followed, so filename coverage +remains incomplete when a list may contribute additional names. + +Journal rows follow acquisition order. V2 records use full 64-bit MFT references, +including sequence numbers. Reused, inconsistent and unverifiable identities +remain unresolved. V3 and V4 contain opaque 128-bit IDs, represented losslessly +as unsigned little-endian decimal values with `referenceBytes: 16`; they are not +joined to 64-bit MFT identities. V4 records carry no timestamp or name, so those +fields are `null`. Their inline extent sample contains at most 16 entries, with +source offsets, exact signed values, total and omitted counts, and +`remainingExtents` for the recorded continuation state. + +Follow `nextFileOffset` with MCP `start_file_offset` or CLI `--startFileOffset`. +Decoding starts exactly there; preceding bytes are not examined. For another +MFT detail page, pass `mftMetadata.next_record_header_index` as MCP +`mft_record_start` or CLI `--mftRecordStart`. MFT pages and journal pages are +independent. Use the journal byte length as its starting offset when only MFT +details are needed. A `null` continuation means that view has no further page. + +Padding, unsupported versions, malformed records and truncation remain coverage +rows. A bad record with a valid declared length can be skipped; a length that +cannot locate the next record stops journal decoding. `historyCompleteness` +remains `unknown`: supplied bytes alone cannot establish missing captures, +wrap, absent rename partners or a complete activity history. Multiple reasons, +rename rows and parent changes remain distinct source facts. + +Current verification uses synthesized acquisitions through CLI and MCP. Real +Windows acquisition verification remains pending. Nonresident payload recovery +and automatic extension-list assembly are outside the implemented coverage. + +Format references: [Microsoft USN V3](https://learn.microsoft.com/en-us/windows/win32/api/winioctl/ns-winioctl-usn_record_v3), +[USN V4](https://learn.microsoft.com/en-us/windows/win32/api/winioctl/ns-winioctl-usn_record_v4), +and [Linux NTFS on-disk definitions](https://github.com/torvalds/linux/blob/master/fs/ntfs3/ntfs.h). diff --git a/docs/specs/native/ntfs-mft/plan.md b/docs/specs/native/ntfs-mft/plan.md index 83f03c27ab..60f3eebaa0 100644 --- a/docs/specs/native/ntfs-mft/plan.md +++ b/docs/specs/native/ntfs-mft/plan.md @@ -4,7 +4,9 @@ Follow the existing artifact readers. `src/domain/windows/ntfsMft.ts` owns FILE record bytes and coverage; `src/domain/artifactFormatMetadata.ts` owns the -`ntfs-mft` result schema. `NtfsMftArtifactReader` reads the admitted file +`ntfs-mft` result schema. `ntfsMftAttributes.ts` owns verified attribute +values and descriptor contracts. Journal joins retain only record headers +outside the selected 32-record detail page. `NtfsMftArtifactReader` reads the admitted file through `ArtifactFileSource` and exposes its facts through `formatMetadata()`. A record stream is its own content, so the root occurrence carries every fact and no members are enumerated. @@ -26,7 +28,8 @@ Recognition lives in `hasMftFileRecordSignature` and is applied in ## Effects and compatibility -The workflow reads one caller-selected local file. It adds no target execution, +The inventory reads one caller-selected MFT file; the journal workflow reads +the explicitly paired MFT and journal files. It adds no target execution, injection, mutation, protection recovery, network access, volume mount or live filesystem access, so no maintainer scope decision is required. `inspect_artifact` routing for every existing format is unchanged, and the new @@ -37,3 +40,6 @@ signature. - https://learn.microsoft.com/en-us/windows/win32/fileio/master-file-table - https://github.com/libyal/libfsntfs/blob/main/documentation/New%20Technologies%20File%20System%20(NTFS).asciidoc + +Current analyst behavior and limitations live in the +[NTFS guide](../../../guides/native/ntfs-metadata.md). diff --git a/docs/specs/native/ntfs-mft/spec.md b/docs/specs/native/ntfs-mft/spec.md index 1f88f42886..1ab0a709dd 100644 --- a/docs/specs/native/ntfs-mft/spec.md +++ b/docs/specs/native/ntfs-mft/spec.md @@ -1,8 +1,8 @@ # NTFS $MFT and USN Journal inspection This specification records an implementation increment and its original problem -statement. For current behavior and limitations, see the artifact inventory -workflow below. +statement. For current behavior and limitations, see the +[NTFS workflow guide](../../../guides/native/ntfs-metadata.md). ## Problem and outcome @@ -59,12 +59,11 @@ file has a 64 MiB working-set admission limit. Follow `nextFileOffset` as `start_file_offset` on MCP or `--startFileOffset` on CLI and compare the two input digests between pages. -USN V2 results preserve the full record reference, sequence number, -source offset, FILETIME, name and raw flags. Older and newer record layouts -are reported as unsupported. Padding, malformed records and truncation -appear as coverage rows. Record order follows the acquisition, and the -results do not establish which journal history is absent due to wrap or a -partial acquisition. +USN V2, V3 and V4 records are supported. V3/V4 opaque IDs remain unresolved +against MFT references; V4 delivers bounded extent samples with omission counts. +Both the journal events and the MFT metadata have independent continuation +positions. Padding, malformed records and truncation appear as coverage rows. +Record order follows the acquisition, and history completeness remains unknown. The CLI and MCP workflow tests use synthesized acquisitions. Verification with real `$MFT` and journal files from a Windows host remains pending. diff --git a/docs/specs/native/ntfs-mft/tasks.md b/docs/specs/native/ntfs-mft/tasks.md index fc83ce435b..158e827cff 100644 --- a/docs/specs/native/ntfs-mft/tasks.md +++ b/docs/specs/native/ntfs-mft/tasks.md @@ -4,10 +4,13 @@ - [x] Parse FILE record headers and update-sequence fixups with synthesized fixtures and explicit limitations. - [x] Add the reader, format descriptor and byte classification. -- [ ] Decode attribute records, then `$FILE_NAME` attributes with parent +- [x] Decode attribute records, then `$FILE_NAME` attributes with parent references, names, hard links and timestamps. - [x] Parse `$UsnJrnl:$J` V2 records and correlate them by full file reference. - [x] Return a bounded source-order page through CLI and MCP, preserving both input digests and explicit coverage rows. -- [ ] Decode supported non-V2 journal layouts and surface acquisition gaps. +- [x] Decode V3/V4 layouts, bound extent samples, and report unknown history coverage. +- [x] Page MFT metadata independently of journal events. +- [x] Decode resident standard information, attribute-list references and nonresident mapping metadata. +- [ ] Assemble extension-list payloads and recover nonresident values when required. - [ ] Verify against real `$MFT` and journal captures from a Windows host.