OpenSource/reverse-engineering-anything
1
0 Fork 0
mirror of https://github.com/morluto/rea.git synced 2026-10-26 17:35:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(analysis-view): share templated positions across manifest tools (#5664)

Browse Source 
* docs(specs): plan a template rule that shares templated positions

Record the outcome, requirements, non-goals, plan and tasks for
prefix-variance, an opt-in environment-manifest rule that templates a
literal segment with the parameter another tool already uses at that
position, so a session's writes to one entity share the read's template.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(analysis-view): share templated positions across manifest tools

Add the opt-in prefix-variance template rule. It runs after the other
opt-in rules and templates a literal at a position that another tool,
of any method or segment count, templates by one of those rules after
the same preceding parts, when the literal is one of that parameter's
observed values or carries a digit and is spelled like one. The
parameter takes the shared parameter's name, tools that now match merge,
and a merged position keeps a tool's own rule when one templated it.

A write to the one entity a session edited, such as
PATCH /api/v1/collections/0RZ576QZQA5KG, becomes
patch_collections_by_collection_id on the read's template instead of a
literal single-entity tool.

Tool naming moves to its own module, and the three comparing passes
share the 32-segment split. Manifests that do not list the rule are
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(analysis-view): drop the unused ManifestRule alias

Parameters now type their rule as a format rule, a shared opt-in rule
or prefix-variance, so nothing reads the union alias.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(specs): record the prefix-variance checks and corpus results

Record the gates, and the 50 showcase captures with and without the
rule: identical output without it, 25 write and 34 read tools templated
with it, and the write tools the write-plan branch serves on top.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(analysis-view): share positions with digit-free generated identifiers

prefix-variance accepted an unobserved literal only when it carried a
digit, so a generated identifier that happened to hold none stayed a
literal tool: an n8n workflow id such as FkMwxmXNGaGVsXFG, whose reads
then never joined the item view. A literal now also qualifies when every
value observed at the position has one length of eight or more characters
and the literal has that length and uses only their character classes.

Across the 50 showcase captures this templates 10 more read tools in
Memos and n8n and no words.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(specs): record five environments rebuilt end to end with the rule

Five randomly drawn environments rebuilt on VMs with the rule and the
write-plan branch: smoke, retrieval and write tasks for two models, and
the same calls against the live applications. Record the results, the
misses and their causes, and the corpus numbers after the fixed-length
clause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(analysis-view): share a position with the parameter that observed the literal

A shared position folded every donor's values into the earliest donor's
parameter. With an opaque-token read before sibling-variance reads at
the same position, a PATCH of /api/items/b2 qualified on the sibling's
evidence but became /api/items/{uid} instead of joining the item read's
/api/items/{item_id}. Each position now keeps one donor per naming rule
with its own values, and a literal takes the donor that observed it,
else the earliest one it is spelled like.

Each donor's spelling (length and character classes of its values, and
its fixed-length family) is summarised once per position, so a literal
is checked in time independent of how many values were observed, rather
than copying and rescanning them for every candidate.

A family now forms from one observed value, as the requirement says, so
a digit-free id of the same length shares a position that only one
opaque-token or after-uid-literal tool supplied. The 50 showcase
captures give the same tools as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(specs): record the prefix-variance review round

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(analysis-view): bound the spelling summary of shared positions

spellingSummary classified every observed value with three full-size
replaceAll copies and a set per distinct character, so a donor holding
a multi-megabyte segment cost unaccounted memory and time in finish().
Characters are now classified in one pass, and a value longer than 128
characters takes no part in spelling comparisons: it shares a position
only as an exactly observed value. The 50 showcase captures give the
same tools.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Edgar ArakelyanandClaude Opus 5.5 authored and GitHub committed 2026-10-21 18:11:51 +08:00
1 parent 19f2969c56
commit ea3534759f
13 files changed
+962 -196
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/specs/agent-environments/prefix-variance/plan.md
+18
View File
@@ -0,0 +1,18 @@
# Plan
1. Move tool naming (`parameterNames`, `pathTemplate`, `toolName`,
`compareTools`) from `environmentManifestRules.ts` to a new
`environmentManifestNames.ts`, so the rules module has room for the rule.
2. Add `prefix-variance` to `TEMPLATE_RULES`. A `prefix-variance` parameter
records the rule it shares (`sharedRule`), which names it and keys it.
3. Add the pass after `sibling-variance`: walk positions in order, index the
opt-in parameters at each position by their preceding parts, template
qualifying literals, then rebucket by naming key, keeping a tool's own rule
for each merged position.
4. Share the 32-segment split between the three comparing passes and list the
segment-limited rules once for the limitation.
5. Document the rule in the view schema description and in
`docs/web-network-captures.md`.
6. Tests: shared positions across methods and depths, the spelling condition,
merges that keep a tool's own rule, pairs, format-rule positions and the
segment limit.
docs/specs/agent-environments/prefix-variance/spec.md
+59
View File
@@ -0,0 +1,59 @@
# Share templated positions across tools
Issue: https://github.com/morluto/rea/issues/4933
## Outcome
A session usually edits one scratch entity: it creates it, then updates and
deletes it at a fixed path such as `/api/v1/collections/0RZ576QZQA5KG`. The
manifest templates a segment only when sibling routes of the same method vary
there, so those writes stay literal tools (`patch_collections_0rz576qzqa5kg`)
even though the reads of the same collection were templated as
`/api/v1/collections/{collection_id}`. A literal write tool addresses one
entity that may not exist in a later state, and nothing can model its effect.
A new opt-in template rule, `prefix-variance`, templates such a literal with
the parameter another tool already uses at that position, so the write becomes
`patch_collections_by_collection_id` on the same template as the read.
## Requirements
- `prefix-variance` applies only when listed, after every other opt-in rule, so
it can share the parameters they produce.
- A literal at position `p` is templated when another tool on the same origin,
of any method and any segment count, has a parameter at `p` from one of the
other opt-in rules, after the same preceding parts. Parameters in the
preceding parts compare by the rule that names them, so a position templated
by `prefix-variance` matches the parameter it shares.
- The literal qualifies when it is one of the values such a parameter
observed at that position, when it carries a digit and has the length and
character classes (digits, lower case, upper case, each other character) of
one of them, or when every value the parameter observed, one or more, has one
length of eight or more characters, as generated identifiers often do, and
the literal has that length and uses only their character classes, so an
identifier that happens to hold no digit still qualifies. An empty literal, a
stop word or a `v1`-style version never qualifies. Each parameter's spelling
is summarised once, so a literal is checked without rescanning its values,
and a value longer than 128 characters is compared only exactly, so a huge
segment costs no classification.
- Positions templated by the four format rules are never shared, so
`/api/teams/{numeric}` still absorbs no literal.
- Values stay with the parameter that observed them: when parameters of
different rules share a position, the literal shares the one that observed
it, else the earliest one it is spelled like.
- The parameter takes the name of the parameter it shares (`collection_id`,
`uid`, a pair's `_namespace` and `_name`) and is labelled `prefix-variance`.
- Tools whose routes now match, by method and by the rule that names each
parameter, merge. A merged parameter keeps the rule of a tool that templated
that position itself, when one did.
- Like the sibling rules, it compares routes of at most 32 segments and says so
in a limitation when a longer route exists.
## Non-goals
- No change to the default rules or to any other rule: manifests that do not
list `prefix-variance` are byte-for-byte unchanged.
- No templating from response bodies, such as an id a create returned; the
manifest reads request lines only.
- No change to which groups are excluded: a write whose response has no body
or content type outside `/api/` is still excluded as `non-json`.
docs/specs/agent-environments/prefix-variance/tasks.md
+10
View File
@@ -0,0 +1,10 @@
# Tasks
- [x] Record the outcome, requirements and non-goals.
- [x] Move tool naming into its own module.
- [x] Add the `prefix-variance` rule, its names, its merge and its limitation.
- [x] Document the rule in the schema and the capture guide.
- [x] Add rule tests.
- [x] Run static, formatting, duplication, documentation and test gates.
- [x] Project the 51 corpus captures with and without the rule and record the results.
- [x] Rebuild five randomly drawn environments end to end and compare them with their published versions and the live applications.
docs/specs/agent-environments/prefix-variance/verification.md
+149
View File
@@ -0,0 +1,149 @@
# Verification
Verified base: 33b0643e on main 1629266e (the rule, the alias removal `knip`
asked for, and the fixed-length clause the end-to-end runs below led to). Executed on macOS with Node v22.23.1 (the repository pins
24.18.0 in `.nvmrc`, which was not installed).
## Gates
- `npm run build:cached`, `npm run check:fast` (typecheck and lint),
`npm run format:check`, `npx knip`, `npm run scan:todos`,
`npm run docs:check`, `npm run verify:test-discovery` and
`npm run mcp-catalog:check`: passed.
- `npm run jscpd`: 3.49% duplicated lines over 3561 files with 3067 clones;
main measures 3.49% with 3068, since the shared segment-limit split removes
one clone.
- oxlint reports no new finding on the changed files. The manifest accumulator
keeps main's three warnings (file length, and the complexity and parameter
count of its record append), one line longer.
- `npx vitest run --project domain --project services`: 7119 passed, 5
pre-existing skips.
- `npm run test:prepare`, then `npm run test:focused --` the analysis-view MCP
and CLI boundaries, the compact input schema and tool schema validity
boundaries, and the environment serve and export boundaries:
7 files, 76 passed.
New tests in `environmentManifestView.test.ts` cover:
- a literal shared across methods and depths, and the same capture without the rule;
- the spelling condition: same classes and length, case, missing digit, a colon word, a version;
- a digit-free member of a fixed-length identifier family, against a shorter word, a hyphenated
value and a family shorter than eight characters;
- merges that keep a tool's own rule, including a sibling-variance parameter under a shared one;
- both halves of a templated pair;
- format-rule positions and the segment limit.
## The showcase captures
The 50 showcase captures (45 HAR inspections and 5 `capture_browser_scenario`
captures) were projected with main's build and this branch's.
- Without the rule, both builds give identical results for every capture: the
manifest with no rules, the manifest with `after-uid-literal`,
`opaque-token`, `sibling-variance` and `sibling-pair-variance`, and the
environment-state view. 150 of 150 outputs hash equal.
- With `prefix-variance` added to `after-uid-literal`, `opaque-token` and
`sibling-variance`, 69 literal tools in 10 captures are templated, 25 of them
writes:
- Komga: collection and readlist `PATCH` and `DELETE`, book pages.
- n8n: workflow `PATCH`, `DELETE` and archive, and the reads of a workflow
whose id holds no digit.
- Memos: memo `PATCH` and `DELETE`, user settings reads, and the reads of
two memos whose ids hold no digit.
- Navidrome: playlist `PUT` and `DELETE`.
- Mealie: recipe `PATCH` and `DELETE`.
- Grist: document `PATCH` and `DELETE`, and table record writes.
- Redpanda Console: subject and topic writes.
- Sentry: release `PUT` and `DELETE`.
- Shlink: short URL `PATCH`.
- Mattermost: `me` reads, which `sibling-variance` already folds into the
user parameter.
Tools go from 1591 to 1578, because Redpanda Console's per-subject reads and
the per-memo and per-workflow reads merge.
- Literals the rule leaves alone, as intended, include Espo CRM `layout`,
Federal Register `facets`, Ghost `slug`, Shlink `stats`, Lychee
`Metrics::photo` and Vault's path names.
- The environment-state view succeeds on every capture with the rule. Its types
move to the templated collections (531 become 526 as literal ones merge); the
one fewer item view is the Redpanda Console merge.
With the write plans of #5600 (its branch with this commit and main merged in,
locally):
- Served write tools rise from 33 to 39: Komga collection create, update and
delete; n8n workflow delete; Sentry release update and delete.
- Write tools with no state target fall from 102 to 87.
- The newly modelled tools that stay unserved are limited by the capture:
- no read after the write: Komga readlists, Mealie, Navidrome, Redpanda Console;
- n8n's item read shows an expanded `shared` relation its create response
never returned;
- Shlink returns `tags` sorted while the update echoes the request's order.
## Five environments end to end
Five of the ten environments the rule changes were drawn at random, redrawing
until the draw held one whose new writes are served: Komga, Mealie, n8n,
Sentry and Shlink. Each was rebuilt on its own Linux VM with `npm pack` of
#5600, main and this branch merged locally, from the same scoped HAR as its
published version: manifest with `prefix-variance`, exported bundle, smoke
task, the generated retrieval tasks for `gpt-oss-120b` and `GLM-5.3-Flash`, and
a dry-run dataset. Where writes are served, hand-written write tasks ask for a
change in plain language and are graded from the environment's state through
`data/get`. The first n8n run found that a digit-free workflow id stayed
literal; the fixed-length clause fixed it, and n8n was rebuilt.
| Environment | Tools (published → rebuilt) | Served from state (reads / writes) | Retrieval gpt-oss | Retrieval GLM | Writes gpt-oss | Writes GLM |
| ----------- | --------------------------- | ---------------------------------- | ----------------- | ------------- | -------------- | ---------- |
| Komga | 42 → 46 | 26 / 3 | 12/12 (was 11/12) | 12/12 (11/12) | 3/4 | 4/4 |
| Mealie | 39 → 39 | 36 / 0 | 12/12 (12/12) | 12/12 (12/12) | – | – |
| n8n | 30 → 33 | 17 / 1 | 9/9 (9/9) | 9/9 (9/9) | 2/2 | 2/2 |
| Sentry | 34 → 34 | 26 / 3 | 12/12 (12/12) | 11/12 (12/12) | 2/4, rerun 0/4 | 4/4 |
| Shlink | 17 → 17 | 5 / 0 | 5/7, rerun 7/7 | 7/7 (7/7) | – | – |
Every smoke task passed. The misses:
- Sentry writes with gpt-oss: every failed episode passed `id: "sentry"` for
the `/api/0/` segment the tool schema describes as `observed: 0`, so its calls
addressed `/api/sentry/…`; every passing episode, and every GLM episode, passed
`0`. The served tools work; the format-rule parameter for a segment that only
ever held `0` invites the guess.
- Komga update with gpt-oss and Shlink's first gpt-oss run: the model provider
rejected tool calls the model generated (`null` and `undefined` for optional
query arguments) before the environment saw them.
- Sentry `qa-03` with GLM: the model never called the alert-rules tool.
The same calls were also replayed against the live applications, seeded by
the same scripts (ids differ, so each side maps its ids to names):
- Komga 1.29.1, 9 of 12 steps equal: create, update and delete of a
collection and the reads after them agree except for the generated id. The
collection list after a create omits the new collection, because that list
keeps its recorded membership, and the 404 for a deleted collection replays
the recorded body with its path and timestamp.
- n8n 1.123.85, 3 of 8 steps equal: n8n refuses to delete a workflow that is
not archived (400) and the environment deletes it; the workflow list replays
its recorded response, so a delete does not show in it; archiving is not
served. The read of the digit-free workflow failed in the first build (the
state held only its list fields) and agrees in the second, apart from ids the
seed generates.
## After the review round
The branch merged main at 52c84f6f, then kept each shared position's values
with the parameter that observed them (one donor per naming rule, the literal
taking the donor that observed it, else the earliest it is spelled like),
summarised each donor's spelling once, and let one observation form a
fixed-length family, as the requirement states. Two new tests, a mixed-rule
position and a single observed workflow id, fail on the previous head and pass
here. The gates above passed again (domain and services 7255 passed with 5
pre-existing skips; the focused boundaries 76; `jscpd` 3.49%). Over the 50
showcase captures the tools with and without `prefix-variance` are unchanged,
name for name: 69 literal tools templated, 25 of them writes.
A second review round bounded the spelling summary: characters are classified
in one pass, and a value longer than 128 characters is compared only exactly.
Its test fails on the previous head, the gates passed again (domain and
services 7257, focused boundaries 76, `jscpd` 3.49%), and the census over the 50
captures is again unchanged.
docs/web-network-captures.md
+18 -7
View File
@@ -449,12 +449,13 @@ template and `params` carry the four format rules with names `id`, `uuid`,
only when listed, each at most once (a repeated rule is an `invalid_request`),
in this order:
| Rule | Applies to | Parameter |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- |
| `after-uid-literal` | The literal segment right after a literal `uid`, unless it is a stop word | `uid` |
| `opaque-token` | A lowercase alphanumeric segment of twelve or more characters mixing letters and digits that is not a `v1`-style version | `uid` |
| `sibling-pair-variance` | Groups equal in method, origin and segment count that differ in two adjacent literals right after a plural literal at index 2 or later and followed by a literal, with at least three distinct pairs and both positions varying (`/repos/{owner}/{repo}/issues`); stop words and `v1`-style versions never qualify | singular plural literal plus `_namespace`, then `_name` |
| `sibling-variance` | Groups equal in method, origin and segment count that differ in one literal at index 2 or later; the siblings whose literal is identifier-like (a digit or `:`, or three or more hyphenated names under a plural literal) merge, never a stop word, and the other literals stay their own tools | singular previous literal plus `_id`, else `id` |
| Rule | Applies to | Parameter |
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
| `after-uid-literal` | The literal segment right after a literal `uid`, unless it is a stop word | `uid` |
| `opaque-token` | A lowercase alphanumeric segment of twelve or more characters mixing letters and digits that is not a `v1`-style version | `uid` |
| `sibling-pair-variance` | Groups equal in method, origin and segment count that differ in two adjacent literals right after a plural literal at index 2 or later and followed by a literal, with at least three distinct pairs and both positions varying (`/repos/{owner}/{repo}/issues`); stop words and `v1`-style versions never qualify | singular plural literal plus `_namespace`, then `_name` |
| `sibling-variance` | Groups equal in method, origin and segment count that differ in one literal at index 2 or later; the siblings whose literal is identifier-like (a digit or `:`, or three or more hyphenated names under a plural literal) merge, never a stop word, and the other literals stay their own tools | singular previous literal plus `_id`, else `id` |
| `prefix-variance` | A literal at a position that another tool, of any method or segment count, templates by one of the rules above after the same preceding segments, when the literal is one of that parameter's observed values, carries a digit and has the length and character classes (digits, lower case, upper case, each other character) of one, or has the single length of eight or more characters every observed value has and only their character classes; stop words and `v1`-style versions never qualify, and tools that now match merge | the name of the parameter it shares |
With `sibling-variance` selected, literal siblings within its comparison scope
fold into existing opt-in parameters as observed values unless they are stop words.
@@ -471,7 +472,17 @@ cut to their first label, `_by_<params>` appended when parameters exist,
sanitized to `[a-z0-9_]`, at most 60 characters and made unique with numeric
suffixes: `get_dashboards_by_uid`, `get_plugins_settings_by_plugin_id`.
`sibling-variance` and `sibling-pair-variance` compare routes of at most 32 segments; longer routes keep
`prefix-variance` runs last so it can share the parameters the other rules
produce: with `sibling-variance`, a `PATCH` or `DELETE` of the one entity a
session edited, `/api/v1/collections/0RZ576QZQA5KG`, becomes
`/api/v1/collections/{collection_id}` beside the reads that varied there, and
so do deeper routes such as `/api/v1/collections/{collection_id}/books`. Its
parameter is labelled `prefix-variance`, unless a merged tool templated the
position by its own rule. When parameters of different rules share a position,
a literal shares the one that observed it, else the earliest one it is spelled
like.
`sibling-variance`, `sibling-pair-variance` and `prefix-variance` compare routes of at most 32 segments; longer routes keep
their grouped templates and the manifest says so in a limitation. `body_shape`
is derived for `POST`, `PUT` and `PATCH` from the tool's earliest record when it
carries an `application/json` or `application/*+json` request body: top-level
src/domain/analysisView/EnvironmentManifestAccumulator.ts
+11 -10
View File
@@ -14,17 +14,19 @@ import { completeWithinViewCoverage } from "./analysisViewCoverage.js";
import type { UnsignedAnalysisView } from "./analysisView.js";
import { WebNetworkProjectionAccumulator } from "./WebNetworkProjectionAccumulator.js";
import {
applyTemplateRules,
compareTools,
parameterNames,
pathTemplate,
SIBLING_VARIANCE_SEGMENT_LIMIT,
toolName,
toolNameRegistry,
} from "./environmentManifestNames.js";
import {
applyTemplateRules,
SEGMENT_LIMITED_RULES,
SIBLING_VARIANCE_SEGMENT_LIMIT,
type BodyShape,
type JsonTypeName,
type LiteralPart,
type ManifestRule,
type TemplateRule,
type ToolGroup,
} from "./environmentManifestRules.js";
@@ -40,6 +42,7 @@ import {
scanRouteTemplate,
textBytes,
type RecordedRequestLine,
type RouteRule,
type RouteTemplate,
} from "./webNetworkRouteTemplates.js";
@@ -72,7 +75,7 @@ type MutablePart =
| LiteralPart
| {
readonly kind: "param";
readonly rule: ManifestRule;
readonly rule: RouteRule;
readonly values: Set<string>;
};
@@ -453,12 +456,10 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumula
)
)
return [];
return (["sibling-variance", "sibling-pair-variance"] as const)
.filter((rule) => this.#rules.has(rule))
.map(
(rule) =>
`Routes with more than ${String(SIBLING_VARIANCE_SEGMENT_LIMIT)} segments are not compared by ${rule}; they keep their grouped templates.`,
);
return SEGMENT_LIMITED_RULES.filter((rule) => this.#rules.has(rule)).map(
(rule) =>
`Routes with more than ${String(SIBLING_VARIANCE_SEGMENT_LIMIT)} segments are not compared by ${rule}; they keep their grouped templates.`,
);
}
/** Project the complete manifest; tools, records and exclusions are sorted deterministically. */
src/domain/analysisView/environmentManifestNames.ts
+157
View File
@@ -0,0 +1,157 @@
import {
compareUnicodeCodePoints as compare,
compositeKey,
} from "../unicodeCodePointOrder.js";
import { VERSION_TOKEN, type RoutePart } from "./environmentManifestRules.js";
import { ROUTE_RULES } from "./webNetworkRouteTemplates.js";
const DROPPED_NAME_LITERALS = new Set(["api", "apis", "namespaces", "default"]);
const TOOL_NAME_LIMIT = 60;
const BASE_RULES: ReadonlySet<string> = new Set(ROUTE_RULES);
const sanitizeToken = (text: string): string =>
text
.toLowerCase()
.replaceAll(/[^a-z0-9_]+/gu, "_")
.replaceAll(/_+/gu, "_")
.replace(/^_/u, "")
.replace(/_$/u, "");
const singular = (word: string): string => {
if (word.endsWith("ies") && word.length > 3) return `${word.slice(0, -3)}y`;
if (word.endsWith("s") && !word.endsWith("ss")) return word.slice(0, -1);
return word;
};
const paramBaseName = (parts: readonly RoutePart[], index: number): string => {
const part = parts[index];
if (part?.kind !== "param") return "";
// A prefix-variance parameter is named as the parameter it shares a position with.
switch (part.rule === "prefix-variance" ? part.sharedRule : part.rule) {
case "numeric":
return "id";
case "uuid":
return "uuid";
case "hex":
return "hash";
case "iso-date":
return "date";
case "after-uid-literal":
case "opaque-token":
return "uid";
case "sibling-variance": {
const previous = parts[index - 1];
const stem =
previous?.kind === "literal"
? sanitizeToken(singular(previous.value))
: "";
return stem.length > 0 ? `${stem}_id` : "id";
}
case "sibling-pair-variance": {
// The role was fixed when the pair formed; the plural literal precedes the pair's first half.
const role = part.pairRole ?? "namespace";
const plural = parts[role === "name" ? index - 2 : index - 1];
const stem =
plural?.kind === "literal" ? sanitizeToken(singular(plural.value)) : "";
return stem.length > 0 ? `${stem}_${role}` : role;
}
}
};
/** Parameter names in segment order; repeated names get numeric suffixes. */
export const parameterNames = (parts: readonly RoutePart[]): string[] => {
const used = new Map<string, number>();
return parts.flatMap((part, index) => {
if (part.kind !== "param") return [];
const base = paramBaseName(parts, index);
const count = used.get(base) ?? 0;
used.set(base, count + 1);
return [count === 0 ? base : `${base}_${String(count + 1)}`];
});
};
/** Format-rule placeholders keep their api-surface spelling; opt-in rules use the parameter name. */
export const pathTemplate = (
parts: readonly RoutePart[],
names: readonly string[],
): string => {
let next = 0;
return `/${parts
.map((part) => {
if (part.kind === "literal") return part.value;
const name = names[next] ?? "";
next += 1;
return BASE_RULES.has(part.rule) ? `{${part.rule}}` : `{${name}}`;
})
.join("/")}`;
};
/** Names assigned in one manifest and the next suffix to try for each colliding truncated base. */
export interface ToolNameRegistry {
readonly used: Set<string>;
readonly nextSuffix: Map<string, number>;
}
export const toolNameRegistry = (): ToolNameRegistry => ({
used: new Set(),
nextSuffix: new Map(),
});
/** Deterministic tool name from the method and route literals, unique within a manifest. */
export const toolName = (
method: string,
parts: readonly RoutePart[],
names: readonly string[],
registry: ToolNameRegistry,
): string => {
const literals = parts.flatMap((part, index) => {
if (part.kind !== "literal") return [];
const value = part.value;
if (
value.length === 0 ||
DROPPED_NAME_LITERALS.has(value) ||
VERSION_TOKEN.test(value) ||
(value === "uid" && parts[index + 1]?.kind === "param")
)
return [];
return [value.split(".")[0] ?? ""];
});
const base =
sanitizeToken(
[
method.toLowerCase(),
...literals,
...(names.length > 0 ? ["by", ...names] : []),
].join("_"),
) || "tool";
const truncate = (text: string, limit: number): string =>
text.slice(0, limit).replace(/_+$/u, "");
let candidate = truncate(base, TOOL_NAME_LIMIT);
if (registry.used.has(candidate)) {
let suffix = 2;
for (;;) {
const tail = `_${String(suffix)}`;
const prefix = truncate(base, TOOL_NAME_LIMIT - tail.length);
// Suffixes can truncate distinct full names to the same prefix. Share
// progress for that actual candidate family, separately for each width:
// a longer suffix can change the prefix and must not skip free short names.
const namespace = compositeKey([prefix, tail.length]);
const next = registry.nextSuffix.get(namespace) ?? suffix;
if (next > suffix) {
suffix = next;
continue;
}
candidate = `${prefix}${tail}`;
registry.nextSuffix.set(namespace, suffix + 1);
if (!registry.used.has(candidate)) break;
suffix += 1;
}
}
registry.used.add(candidate);
return candidate;
};
/** Sort tools by template then method with locale-independent ordering. */
export const compareTools = (
left: { readonly template: string; readonly method: string },
right: { readonly template: string; readonly method: string },
): number =>
compare(left.template, right.template) || compare(left.method, right.method);
src/domain/analysisView/environmentManifestRules.test.ts
+1 -1
View File
@@ -1,5 +1,5 @@
import { expect, it, vi } from "vitest";
import { toolName, toolNameRegistry } from "./environmentManifestRules.js";
import { toolName, toolNameRegistry } from "./environmentManifestNames.js";
it("shares suffix work when distinct names truncate to the same suffixed prefix", () => {
const registry = toolNameRegistry();
src/domain/analysisView/environmentManifestRules.ts
+182 -177
View File
@@ -1,8 +1,10 @@
import { compositeKey } from "../unicodeCodePointOrder.js";
import {
compareUnicodeCodePoints as compare,
compositeKey,
} from "../unicodeCodePointOrder.js";
import { ROUTE_RULES, type RouteRule } from "./webNetworkRouteTemplates.js";
spelledLikeObserved,
spellingSummary,
type SpellingSummary,
} from "./environmentManifestSpelling.js";
import type { RouteRule } from "./webNetworkRouteTemplates.js";
/** Opt-in template rules applied after the four exact format rules. */
export const TEMPLATE_RULES = [
@@ -10,9 +12,11 @@ export const TEMPLATE_RULES = [
"opaque-token",
"sibling-variance",
"sibling-pair-variance",
"prefix-variance",
] as const;
export type TemplateRule = (typeof TEMPLATE_RULES)[number];
export type ManifestRule = RouteRule | TemplateRule;
/** The opt-in rules whose parameters a prefix-variance parameter can share; never itself, so shared evidence does not chain. */
type SharedRule = Exclude<TemplateRule, "prefix-variance">;
export type JsonTypeName =
| "string"
@@ -32,13 +36,19 @@ export interface LiteralPart {
readonly kind: "literal";
readonly value: string;
}
export interface ParamPart {
interface ParamFields {
readonly kind: "param";
readonly rule: ManifestRule;
readonly values: ReadonlySet<string>;
/** Which half of a sibling-pair-variance pair; fixed when the pair forms so later merges cannot change it. */
readonly pairRole?: "namespace" | "name";
}
export type ParamPart =
| (ParamFields & { readonly rule: RouteRule | SharedRule })
| (ParamFields & {
readonly rule: "prefix-variance";
/** The rule of the parameter this one shares a position with, which also names it. */
readonly sharedRule: SharedRule;
});
export type RoutePart = LiteralPart | ParamPart;
/** One candidate tool: a merged request-line group on the primary origin. */
@@ -56,7 +66,7 @@ export interface ToolGroup {
}
const OPAQUE_TOKEN = /^(?=.*\d)(?=.*[a-z])[a-z0-9]{12,}$/u;
const VERSION_TOKEN = /^v\d/u;
export const VERSION_TOKEN = /^v\d/u;
const STOP_WORDS = new Set([
"search",
"settings",
@@ -71,14 +81,17 @@ const STOP_WORDS = new Set([
"dto",
"markdown",
]);
const DROPPED_NAME_LITERALS = new Set(["api", "apis", "namespaces", "default"]);
const TOOL_NAME_LIMIT = 60;
/** Longest route sibling-variance compares; every position rebuilds a key over the whole route. */
export const SIBLING_VARIANCE_SEGMENT_LIMIT = 32;
const BASE_RULES: ReadonlySet<string> = new Set(ROUTE_RULES);
/** The rules that compare routes and so skip those beyond the segment limit. */
export const SEGMENT_LIMITED_RULES = [
"sibling-variance",
"sibling-pair-variance",
"prefix-variance",
] as const;
const OPT_IN_RULES: ReadonlySet<string> = new Set(TEMPLATE_RULES);
const param = (rule: ManifestRule, values: Iterable<string>): ParamPart => ({
const param = (rule: SharedRule, values: Iterable<string>): ParamPart => ({
kind: "param",
rule,
values: new Set(values),
@@ -139,19 +152,25 @@ const mergeGroups = (
};
};
const rebucket = (groups: readonly ToolGroup[]): ToolGroup[] => {
const rebucket = (
groups: readonly ToolGroup[],
keyOf: (part: RoutePart) => unknown = partKey,
shapeOf: (bucket: readonly ToolGroup[]) => ToolGroup = earliest,
): ToolGroup[] => {
const buckets = new Map<string, ToolGroup[]>();
for (const group of groups) {
const key = compositeKey([
group.method,
group.origin,
...group.parts.map(partKey),
...group.parts.map(keyOf),
]);
const bucket = buckets.get(key);
if (bucket === undefined) buckets.set(key, [group]);
else bucket.push(group);
}
return [...buckets.values()].map((bucket) => mergeGroups(bucket));
return [...buckets.values()].map((bucket) =>
mergeGroups(bucket, shapeOf(bucket)),
);
};
const templateAfterUidLiteral = (parts: readonly RoutePart[]): RoutePart[] =>
@@ -258,12 +277,7 @@ const mergeSiblings = (
/** Merge groups that differ in exactly one identifier-like segment at index two or later. */
const mergeSiblingVariance = (groups: readonly ToolGroup[]): ToolGroup[] => {
const beyondLimit = groups.filter(
(group) => group.parts.length > SIBLING_VARIANCE_SEGMENT_LIMIT,
);
let current = groups.filter(
(group) => group.parts.length <= SIBLING_VARIANCE_SEGMENT_LIMIT,
);
let current = [...groups];
const longest = Math.max(0, ...current.map((group) => group.parts.length));
for (let position = 2; position < longest; position += 1) {
const buckets = new Map<string, ToolGroup[]>();
@@ -289,7 +303,7 @@ const mergeSiblingVariance = (groups: readonly ToolGroup[]): ToolGroup[] => {
output.push(...mergeSiblings(bucket, position));
current = output;
}
return [...current, ...beyondLimit];
return current;
};
/** Fewest distinct literal pairs that make two adjacent positions one templated pair. */
@@ -357,12 +371,7 @@ const mergePairBucket = (
/** Merge groups that differ in two adjacent literals after a plural literal, such as `/repos/{owner}/{repo}/…`. */
const mergeSiblingPairs = (groups: readonly ToolGroup[]): ToolGroup[] => {
const beyondLimit = groups.filter(
(group) => group.parts.length > SIBLING_VARIANCE_SEGMENT_LIMIT,
);
let current = groups.filter(
(group) => group.parts.length <= SIBLING_VARIANCE_SEGMENT_LIMIT,
);
let current = [...groups];
const longest = Math.max(0, ...current.map((group) => group.parts.length));
for (let position = 2; position + 1 < longest; position += 1) {
const buckets = new Map<string, ToolGroup[]>();
@@ -388,9 +397,146 @@ const mergeSiblingPairs = (groups: readonly ToolGroup[]): ToolGroup[] => {
output.push(...mergePairBucket(bucket, position));
current = output;
}
return [...current, ...beyondLimit];
return current;
};
/** A part by the rule that names it: a prefix-variance parameter keys as the parameter it shares. */
const nameKey = (part: RoutePart): unknown =>
part.kind === "literal"
? part.value
: [
"param",
part.rule === "prefix-variance" ? part.sharedRule : part.rule,
part.pairRole ?? null,
];
const prefixKey = (group: ToolGroup, position: number): string =>
compositeKey([group.origin, ...group.parts.slice(0, position).map(nameKey)]);
/** A parameter a position can share: the earliest tool's parameter of one naming rule, and every value tools named by that rule observed there. */
interface SharedDonor {
readonly part: ParamPart & { readonly rule: SharedRule };
readonly values: Set<string>;
}
const isShared = (
part: RoutePart | undefined,
): part is ParamPart & { readonly rule: SharedRule } =>
part?.kind === "param" &&
part.rule !== "prefix-variance" &&
OPT_IN_RULES.has(part.rule);
/** A position's donors, earliest first, with each one's spelling summarised once for every literal it is compared with. */
type SharedPosition = readonly (SharedDonor & {
readonly spelling: SpellingSummary;
})[];
/** Opt-in parameters at one position by their preceding parts, one donor per naming rule, so a value stays with the parameter that observed it. */
const sharedPositions = (
groups: readonly ToolGroup[],
position: number,
): Map<string, SharedPosition> => {
const donors = new Map<string, Map<string, SharedDonor>>();
const ordered = [...groups].sort(
(left, right) => (left.positions[0] ?? 0) - (right.positions[0] ?? 0),
);
for (const group of ordered) {
const part = group.parts[position];
if (!isShared(part)) continue;
const key = prefixKey(group, position);
const byRule = donors.get(key) ?? new Map<string, SharedDonor>();
donors.set(key, byRule);
const rule = compositeKey([nameKey(part)]);
const known = byRule.get(rule);
if (known === undefined)
byRule.set(rule, { part, values: new Set(part.values) });
else for (const value of part.values) known.values.add(value);
}
return new Map(
[...donors].map(([key, byRule]) => [
key,
[...byRule.values()].map((donor) => ({
...donor,
spelling: spellingSummary(donor.values),
})),
]),
);
};
/** The donor a literal shares a position with: one that observed it, else one it is spelled like, earliest first; never for an empty literal, a stop word or a version. */
const donorFor = (
value: string,
donors: SharedPosition,
): SharedDonor | undefined =>
value === "" || STOP_WORDS.has(value) || VERSION_TOKEN.test(value)
? undefined
: (donors.find((donor) => donor.values.has(value)) ??
donors.find((donor) => spelledLikeObserved(value, donor.spelling)));
/** The bucket's earliest group, with each position's parameter taken from a group that templated it by its own rule when one did. */
const sharedShape = (bucket: readonly ToolGroup[]): ToolGroup => {
const first = earliest(bucket);
return withParts(
first,
first.parts.map((part, index) =>
part.kind === "param" && part.rule === "prefix-variance"
? (bucket.map((group) => group.parts[index]).find(isShared) ?? part)
: part,
),
);
};
/** Template literals at positions another tool templates by an opt-in rule after the same preceding parts, at any method or length. */
const mergePrefixVariance = (groups: readonly ToolGroup[]): ToolGroup[] => {
let current = [...groups];
const longest = Math.max(0, ...current.map((group) => group.parts.length));
for (let position = 0; position < longest; position += 1) {
const shared = sharedPositions(current, position);
if (shared.size === 0) continue;
current = current.map((group) => {
const part = group.parts[position];
const donors =
part?.kind === "literal"
? shared.get(prefixKey(group, position))
: undefined;
const known =
part?.kind === "literal" && donors !== undefined
? donorFor(part.value, donors)
: undefined;
if (part?.kind !== "literal" || known === undefined) return group;
// Copies the shared parameter's pair role; the values are this literal only.
const templated: ParamPart = {
...known.part,
rule: "prefix-variance",
sharedRule: known.part.rule,
values: new Set([part.value]),
};
return withParts(
group,
group.parts.map((other, index) =>
index === position ? templated : other,
),
);
});
}
return rebucket(current, nameKey, sharedShape);
};
/** Run a comparing pass over routes within the segment limit; longer routes keep their groups. */
const withinSegmentLimit = (
groups: readonly ToolGroup[],
pass: (groups: readonly ToolGroup[]) => ToolGroup[],
): ToolGroup[] => [
...pass(
groups.filter(
(group) => group.parts.length <= SIBLING_VARIANCE_SEGMENT_LIMIT,
),
),
...groups.filter(
(group) => group.parts.length > SIBLING_VARIANCE_SEGMENT_LIMIT,
),
];
/** Apply the selected opt-in rules in fixed order; every merge is an inference. */
export const applyTemplateRules = (
groups: readonly ToolGroup[],
@@ -408,153 +554,12 @@ export const applyTemplateRules = (
current = rebucket(current);
// Pairs merge first: a single-position merge could otherwise template one
// half of a pair and leave too few literal pairs to recognize.
if (rules.has("sibling-pair-variance")) current = mergeSiblingPairs(current);
if (rules.has("sibling-variance")) current = mergeSiblingVariance(current);
if (rules.has("sibling-pair-variance"))
current = withinSegmentLimit(current, mergeSiblingPairs);
if (rules.has("sibling-variance"))
current = withinSegmentLimit(current, mergeSiblingVariance);
// Last, so it can share the parameters every other rule produced.
if (rules.has("prefix-variance"))
current = withinSegmentLimit(current, mergePrefixVariance);
return current;
};
const sanitizeToken = (text: string): string =>
text
.toLowerCase()
.replaceAll(/[^a-z0-9_]+/gu, "_")
.replaceAll(/_+/gu, "_")
.replace(/^_/u, "")
.replace(/_$/u, "");
const singular = (word: string): string => {
if (word.endsWith("ies") && word.length > 3) return `${word.slice(0, -3)}y`;
if (word.endsWith("s") && !word.endsWith("ss")) return word.slice(0, -1);
return word;
};
const paramBaseName = (parts: readonly RoutePart[], index: number): string => {
const part = parts[index];
if (part?.kind !== "param") return "";
switch (part.rule) {
case "numeric":
return "id";
case "uuid":
return "uuid";
case "hex":
return "hash";
case "iso-date":
return "date";
case "after-uid-literal":
case "opaque-token":
return "uid";
case "sibling-variance": {
const previous = parts[index - 1];
const stem =
previous?.kind === "literal"
? sanitizeToken(singular(previous.value))
: "";
return stem.length > 0 ? `${stem}_id` : "id";
}
case "sibling-pair-variance": {
// The role was fixed when the pair formed; the plural literal precedes the pair's first half.
const role = part.pairRole ?? "namespace";
const plural = parts[role === "name" ? index - 2 : index - 1];
const stem =
plural?.kind === "literal" ? sanitizeToken(singular(plural.value)) : "";
return stem.length > 0 ? `${stem}_${role}` : role;
}
}
};
/** Parameter names in segment order; repeated names get numeric suffixes. */
export const parameterNames = (parts: readonly RoutePart[]): string[] => {
const used = new Map<string, number>();
return parts.flatMap((part, index) => {
if (part.kind !== "param") return [];
const base = paramBaseName(parts, index);
const count = used.get(base) ?? 0;
used.set(base, count + 1);
return [count === 0 ? base : `${base}_${String(count + 1)}`];
});
};
/** Format-rule placeholders keep their api-surface spelling; opt-in rules use the parameter name. */
export const pathTemplate = (
parts: readonly RoutePart[],
names: readonly string[],
): string => {
let next = 0;
return `/${parts
.map((part) => {
if (part.kind === "literal") return part.value;
const name = names[next] ?? "";
next += 1;
return BASE_RULES.has(part.rule) ? `{${part.rule}}` : `{${name}}`;
})
.join("/")}`;
};
/** Names assigned in one manifest and the next suffix to try for each colliding truncated base. */
export interface ToolNameRegistry {
readonly used: Set<string>;
readonly nextSuffix: Map<string, number>;
}
export const toolNameRegistry = (): ToolNameRegistry => ({
used: new Set(),
nextSuffix: new Map(),
});
/** Deterministic tool name from the method and route literals, unique within a manifest. */
export const toolName = (
method: string,
parts: readonly RoutePart[],
names: readonly string[],
registry: ToolNameRegistry,
): string => {
const literals = parts.flatMap((part, index) => {
if (part.kind !== "literal") return [];
const value = part.value;
if (
value.length === 0 ||
DROPPED_NAME_LITERALS.has(value) ||
VERSION_TOKEN.test(value) ||
(value === "uid" && parts[index + 1]?.kind === "param")
)
return [];
return [value.split(".")[0] ?? ""];
});
const base =
sanitizeToken(
[
method.toLowerCase(),
...literals,
...(names.length > 0 ? ["by", ...names] : []),
].join("_"),
) || "tool";
const truncate = (text: string, limit: number): string =>
text.slice(0, limit).replace(/_+$/u, "");
let candidate = truncate(base, TOOL_NAME_LIMIT);
if (registry.used.has(candidate)) {
let suffix = 2;
for (;;) {
const tail = `_${String(suffix)}`;
const prefix = truncate(base, TOOL_NAME_LIMIT - tail.length);
// Suffixes can truncate distinct full names to the same prefix. Share
// progress for that actual candidate family, separately for each width:
// a longer suffix can change the prefix and must not skip free short names.
const namespace = compositeKey([prefix, tail.length]);
const next = registry.nextSuffix.get(namespace) ?? suffix;
if (next > suffix) {
suffix = next;
continue;
}
candidate = `${prefix}${tail}`;
registry.nextSuffix.set(namespace, suffix + 1);
if (!registry.used.has(candidate)) break;
suffix += 1;
}
}
registry.used.add(candidate);
return candidate;
};
/** Sort tools by template then method with locale-independent ordering. */
export const compareTools = (
left: { readonly template: string; readonly method: string },
right: { readonly template: string; readonly method: string },
): number =>
compare(left.template, right.template) || compare(left.method, right.method);
src/domain/analysisView/environmentManifestSpelling.test.ts
+29
View File
@@ -0,0 +1,29 @@
import { describe, expect, it } from "vitest";
import {
spelledLikeObserved,
spellingSummary,
} from "./environmentManifestSpelling.js";
describe("spelledLikeObserved", () => {
it("compares digit-bearing values by length and character classes, and digit-free ones by family", () => {
const observed = spellingSummary(["ydr9n16753tzr7zc"]);
expect(spelledLikeObserved("ab1cd23ef4gh5ij6", observed)).toBe(true);
expect(spelledLikeObserved("fkmwxmxngagvsxfg", observed)).toBe(true);
expect(spelledLikeObserved("FkMwxmXNGaGVsXFG", observed)).toBe(false);
expect(spelledLikeObserved("notifications", observed)).toBe(false);
});
it("compares no value longer than the spelling limit, observed or candidate", () => {
const long = (length: number, last: string) =>
`${"a".repeat(length - 1)}${last}`;
expect(
spelledLikeObserved(long(128, "2"), spellingSummary([long(128, "1")])),
).toBe(true);
const beyond = spellingSummary([long(129, "1")]);
expect(beyond).toEqual({ shapes: new Set(), family: undefined });
expect(spelledLikeObserved(long(129, "2"), beyond)).toBe(false);
// A multi-megabyte segment is summarised without classifying its characters.
expect(spellingSummary([`${"é".repeat(4_000_000)}1`]).shapes.size).toBe(0);
});
});
src/domain/analysisView/environmentManifestSpelling.ts
+84
View File
@@ -0,0 +1,84 @@
import {
compareUnicodeCodePoints as compare,
compositeKey,
} from "../unicodeCodePointOrder.js";
/** Fewest characters of a fixed-length identifier family whose members need not carry a digit. */
const FIXED_LENGTH_MINIMUM = 8;
/** Longest value, in UTF-16 units, compared by spelling; a longer one shares a position only as an observed value. */
const SPELLED_LENGTH_LIMIT = 128;
/** The distinct classes of a text's characters, digits, lower case, upper case and each other character, in one pass; undefined past the spelling limit. */
const characterClasses = (text: string): string | undefined => {
if (text.length > SPELLED_LENGTH_LIMIT) return undefined;
const classes = new Set<string>();
for (const character of text)
classes.add(
character >= "0" && character <= "9"
? "0"
: character >= "a" && character <= "z"
? "a"
: character >= "A" && character <= "Z"
? "A"
: character,
);
return [...classes].sort(compare).join("");
};
const shapeOf = (length: number, classes: string): string =>
compositeKey([length, classes]);
/** How a set of observed identifiers is spelled, summarised once so each candidate is checked without rescanning them. */
export interface SpellingSummary {
/** Each observed value's length and character classes. */
readonly shapes: ReadonlySet<string>;
/** The one length of eight or more every observed value has, with the classes they use together. */
readonly family:
| { readonly length: number; readonly classes: ReadonlySet<string> }
| undefined;
}
export const spellingSummary = (
observed: Iterable<string>,
): SpellingSummary => {
const shapes = new Set<string>();
const lengths = new Set<number>();
const classes = new Set<string>();
for (const value of observed) {
lengths.add(value.length);
const kinds = characterClasses(value);
if (kinds === undefined) continue;
shapes.add(shapeOf(value.length, kinds));
for (const kind of kinds) classes.add(kind);
}
const [length] = lengths;
return {
shapes,
family:
lengths.size === 1 &&
length! >= FIXED_LENGTH_MINIMUM &&
length! <= SPELLED_LENGTH_LIMIT
? { length: length!, classes }
: undefined,
};
};
/** Spelled like an observed identifier: a digit-bearing value with the length and character classes of one, or a member of their fixed-length family. */
export const spelledLikeObserved = (
value: string,
observed: SpellingSummary,
): boolean => {
const classes = characterClasses(value);
if (classes === undefined) return false;
const { family } = observed;
if (
family !== undefined &&
value.length === family.length &&
[...classes].every((kind) => family.classes.has(kind))
)
return true;
return (
/\d/u.test(value) && observed.shapes.has(shapeOf(value.length, classes))
);
};
src/domain/analysisView/environmentManifestView.test.ts
+243
View File
@@ -1174,3 +1174,246 @@ it.each([
expect(projected.records).toHaveLength(values.length);
},
);
it("templates a literal that another method's opt-in parameter observed at the same position, at any depth", () => {
const routes = syntheticCapture([
["GET", "https://k.test/api/v1/collections/0RZ51AB"],
["GET", "https://k.test/api/v1/collections/0RZ52CD"],
["GET", "https://k.test/api/v1/collections/0RZ53EF"],
["PATCH", "https://k.test/api/v1/collections/0RZ53EF", null, "{}"],
["DELETE", "https://k.test/api/v1/collections/0RZ53EF", null],
["GET", "https://k.test/api/v1/collections/0RZ52CD/books"],
["GET", "https://k.test/api/v1/collections/layout"],
["POST", "https://k.test/api/v1/collections/search", null, "{}"],
]);
const shared = manifest(["sibling-variance", "prefix-variance"], routes);
expect(shared.tools.map(line)).toEqual([
"GET /api/v1/collections/layout",
"POST /api/v1/collections/search",
"DELETE /api/v1/collections/{collection_id}",
"GET /api/v1/collections/{collection_id}",
"PATCH /api/v1/collections/{collection_id}",
"GET /api/v1/collections/{collection_id}/books",
]);
expect(shared.tools.find((tool) => tool.method === "PATCH")).toMatchObject({
name: "patch_collections_by_collection_id",
api_surface_template: "/api/v1/collections/0RZ53EF",
params: [
{
index: 3,
name: "collection_id",
rule: "prefix-variance",
basis: "inferred",
observed_values: ["0RZ53EF"],
},
],
record_ordinals: [3],
});
expect(
shared.tools.find((tool) => tool.path_template.endsWith("/books"))?.name,
).toBe("get_collections_books_by_collection_id");
expect(manifest(["sibling-variance"], routes).tools.map(line)).toEqual([
"GET /api/v1/collections/0RZ52CD/books",
"DELETE /api/v1/collections/0RZ53EF",
"PATCH /api/v1/collections/0RZ53EF",
"GET /api/v1/collections/layout",
"POST /api/v1/collections/search",
"GET /api/v1/collections/{collection_id}",
]);
});
it("templates an unobserved literal only when it carries a digit and is spelled like an observed value", () => {
const projected = manifest(
["sibling-variance", "prefix-variance"],
syntheticCapture([
["GET", "https://k.test/api/readlists/0RZ51KSZKA53J"],
["GET", "https://k.test/api/readlists/0RZ56NR37AC1T"],
["PATCH", "https://k.test/api/readlists/0RZ576R5KA57B", null, "{}"],
["PUT", "https://k.test/api/readlists/0rz576r5ka57b", null, "{}"],
["DELETE", "https://k.test/api/readlists/ABCDEFGHIJ", null],
["POST", "https://k.test/api/readlists/Metrics::12", null, "{}"],
["GET", "https://k.test/api/items/a1"],
["GET", "https://k.test/api/items/b2"],
["PUT", "https://k.test/api/items/v2", null, "{}"],
]),
);
expect(projected.tools.map(line)).toEqual([
"PUT /api/items/v2",
"GET /api/items/{item_id}",
"PUT /api/readlists/0rz576r5ka57b",
"DELETE /api/readlists/ABCDEFGHIJ",
"POST /api/readlists/Metrics::12",
"GET /api/readlists/{readlist_id}",
"PATCH /api/readlists/{readlist_id}",
]);
});
it("merges tools whose routes now match and keeps the parameter a tool templated by its own rule", () => {
const routes = syntheticCapture([
["GET", "https://b.test/api/boards/b-2/cards/c-3"],
["GET", "https://b.test/api/boards/b-1"],
["GET", "https://b.test/api/boards/b-2"],
["GET", "https://b.test/api/boards/b-1/cards/c-1"],
["GET", "https://b.test/api/boards/b-1/cards/c-2"],
["GET", "https://b.test/api/subjects/red-one"],
["GET", "https://b.test/api/subjects/blue-two"],
["GET", "https://b.test/api/subjects/green-three"],
["GET", "https://b.test/api/subjects/red-one/versions/1"],
["GET", "https://b.test/api/subjects/blue-two/versions/2"],
]);
const projected = manifest(["sibling-variance", "prefix-variance"], routes);
expect(projected.tools.map(line)).toEqual([
"GET /api/boards/{board_id}",
"GET /api/boards/{board_id}/cards/{card_id}",
"GET /api/subjects/{subject_id}",
"GET /api/subjects/{subject_id}/versions/{numeric}",
]);
const params = (template: string) =>
projected.tools
.find((tool) => tool.path_template === template)
?.params.map((param) => [param.name, param.rule, param.observed_values]);
expect(params("/api/boards/{board_id}/cards/{card_id}")).toEqual([
["board_id", "prefix-variance", ["b-1", "b-2"]],
["card_id", "sibling-variance", ["c-1", "c-2", "c-3"]],
]);
expect(params("/api/subjects/{subject_id}/versions/{numeric}")).toEqual([
["subject_id", "prefix-variance", ["blue-two", "red-one"]],
["id", "numeric", ["1", "2"]],
]);
expect(manifest(["sibling-variance"], routes).tools.map(line)).toEqual([
"GET /api/boards/b-1/cards/{card_id}",
"GET /api/boards/b-2/cards/c-3",
"GET /api/boards/{board_id}",
"GET /api/subjects/blue-two/versions/{numeric}",
"GET /api/subjects/red-one/versions/{numeric}",
"GET /api/subjects/{subject_id}",
]);
});
it("shares no format-rule position and compares no route beyond the segment limit", () => {
const tail = Array.from(
{ length: 36 },
(_, index) => `s${String(index)}`,
).join("/");
const projected = manifest(
["sibling-variance", "prefix-variance"],
syntheticCapture([
["GET", "https://t.test/api/teams/1"],
["GET", "https://t.test/api/teams/2"],
["PATCH", "https://t.test/api/teams/t3", null, "{}"],
["GET", "https://t.test/api/items/a1"],
["GET", "https://t.test/api/items/b2"],
["GET", `https://t.test/api/items/b2/${tail}`],
]),
);
expect(projected.tools.map(line)).toEqual([
`GET /api/items/b2/${tail}`,
"GET /api/items/{item_id}",
"PATCH /api/teams/t3",
"GET /api/teams/{numeric}",
]);
expect(projected.limitations).toContain(
"Routes with more than 32 segments are not compared by prefix-variance; they keep their grouped templates.",
);
});
it("shares both halves of a templated pair and keeps each half's name", () => {
const projected = manifest(
["sibling-pair-variance", "prefix-variance"],
syntheticCapture([
["GET", "https://g.test/api/repos/lighthouse-rota/lamp-log/issues"],
["GET", "https://g.test/api/repos/granite-ledger/ledger-core/issues"],
["GET", "https://g.test/api/repos/blue-heron/recipe-book/issues"],
[
"POST",
"https://g.test/api/repos/granite-ledger/ledger-core/issues",
null,
"{}",
],
["DELETE", "https://g.test/api/repos/blue-heron/recipe-book", null],
]),
);
expect(projected.tools.map(line)).toEqual([
"DELETE /api/repos/{repo_namespace}/{repo_name}",
"GET /api/repos/{repo_namespace}/{repo_name}/issues",
"POST /api/repos/{repo_namespace}/{repo_name}/issues",
]);
expect(
projected.tools[0]?.params.map((param) => [param.name, param.rule]),
).toEqual([
["repo_namespace", "prefix-variance"],
["repo_name", "prefix-variance"],
]);
});
it("templates a digit-free literal only as a member of a fixed-length identifier family", () => {
const projected = manifest(
["sibling-variance", "prefix-variance"],
syntheticCapture([
["GET", "https://n.test/api/workflows/yDr9n16753TZr7Zc"],
["GET", "https://n.test/api/workflows/5LPA94ML1gYbgAQx"],
["PATCH", "https://n.test/api/workflows/FkMwxmXNGaGVsXFG", null, "{}"],
["DELETE", "https://n.test/api/workflows/listForAccounts", null],
["PUT", "https://n.test/api/workflows/fkmw-xmxn-gagv-x", null, "{}"],
["GET", "https://n.test/api/tags/a1"],
["GET", "https://n.test/api/tags/b22"],
["DELETE", "https://n.test/api/tags/cc", null],
["GET", "https://n.test/api/labels/ab12cd34"],
["GET", "https://n.test/api/labels/ef56gh78"],
["DELETE", "https://n.test/api/labels/abcdefgh", null],
]),
);
expect(projected.tools.map(line)).toEqual([
"DELETE /api/labels/{label_id}",
"GET /api/labels/{label_id}",
"DELETE /api/tags/cc",
"GET /api/tags/{tag_id}",
"PUT /api/workflows/fkmw-xmxn-gagv-x",
"DELETE /api/workflows/listForAccounts",
"GET /api/workflows/{workflow_id}",
"PATCH /api/workflows/{workflow_id}",
]);
});
it("shares a literal with the parameter that observed it when parameters of different rules share a position", () => {
const projected = manifest(
["opaque-token", "sibling-variance", "prefix-variance"],
syntheticCapture([
["GET", "https://i.test/api/items/aaaa12bb34cc/details"],
["GET", "https://i.test/api/items/a1"],
["GET", "https://i.test/api/items/b2"],
["PATCH", "https://i.test/api/items/b2", null, "{}"],
]),
);
// The token parameter is earlier, but only the sibling parameter observed b2.
expect(projected.tools.map(line)).toEqual([
"GET /api/items/{item_id}",
"PATCH /api/items/{item_id}",
"GET /api/items/{uid}/details",
]);
expect(
projected.tools.map((tool) =>
tool.params.map((param) => [param.name, param.rule]),
),
).toEqual([
[["item_id", "sibling-variance"]],
[["item_id", "prefix-variance"]],
[["uid", "opaque-token"]],
]);
});
it("templates a digit-free literal in a fixed-length family that one opt-in parameter observed once", () => {
const projected = manifest(
["opaque-token", "prefix-variance"],
syntheticCapture([
["GET", "https://w.test/api/workflows/ydr9n16753tzr7zc"],
["PATCH", "https://w.test/api/workflows/fkmwxmxngagvsxfg", null, "{}"],
["DELETE", "https://w.test/api/workflows/notifications", null],
]),
);
expect(projected.tools.map(line)).toEqual([
"DELETE /api/workflows/notifications",
"GET /api/workflows/{uid}",
"PATCH /api/workflows/{uid}",
]);
});
src/domain/analysisView/environmentManifestView.ts
+1 -1
View File
@@ -30,7 +30,7 @@ export const templateRulesSchema = z
export const environmentManifestViewRequestSchema = z.strictObject({
kind: z.literal("environment-manifest"),
template_rules: templateRulesSchema.describe(
"Opt-in template rules applied after the four format rules, each listed at most once: after-uid-literal templates the segment following a literal uid, opaque-token templates long mixed alphanumeric segments, sibling-variance merges groups that differ in one identifier-like segment, sibling-pair-variance merges groups that differ in two adjacent literals between a plural literal and another literal when at least three distinct pairs vary in both. Literal sibling folding into existing opt-in parameters also requires sibling-variance. Merged parameters are inferences.",
"Opt-in template rules applied after the four format rules, each listed at most once: after-uid-literal templates the segment following a literal uid, opaque-token templates long mixed alphanumeric segments, sibling-variance merges groups that differ in one identifier-like segment, sibling-pair-variance merges groups that differ in two adjacent literals between a plural literal and another literal when at least three distinct pairs vary in both, prefix-variance templates a literal at a position another tool templates by one of the other opt-in rules after the same preceding segments, at any method or length, when the literal is one of that parameter's observed values, carries a digit and has the length and character classes of one, or has the single length (eight or more characters) every observed value has and only their character classes. Literal sibling folding into existing opt-in parameters also requires sibling-variance. Merged parameters are inferences.",
),
include_query_parameter_names: z
.boolean()