test: record native Grok lifecycle evidence

This commit is contained in:
MistyStar committed 2026-10-21 10:02:39 +08:00
1 parent d363c04261
commit dd2107ff65
6 files changed
+493 -7

No files matched your search

+16
View File
@@ -85,6 +85,22 @@ file remains for Claude/Codex. Additional connectors stay opt-in via
`mcp-configs/mcp-servers.json`. Canonical receipts record explicit per-server
consent before any MCP name is copied into an install plan.
## Native lifecycle verification
With a real `grok` executable on `PATH`, run the opt-in acceptance driver:
```bash
node tests/ci/grok-native-lifecycle.js --report /tmp/grok-native-lifecycle.json
```
It uses fresh temporary homes without inherited credentials and fails if Grok is
missing. It verifies ECC discovery, trust refusal, independent hook/MCP consent,
disabled and missing roots, Git update, rollback, immutable pins, and uninstall.
Native Git transitions use a disposable full ECC archive with synthetic versions;
canonical lifecycle tests also require real `grok inspect` readbacks. Temporary
installations are removed on exit. See the
[recorded run and evidence boundaries](../docs/releases/2.2.3/grok-native-lifecycle.md).
## Known limits
- 311 skills are advertised into Grok's skill catalog. Use a smaller working
@@ -0,0 +1,72 @@
# Grok native lifecycle evidence
The adjacent [JSON receipt](grok-native-lifecycle.json) records a real local run
on Linux x64 with Grok Build 1.0.62 and Node 24.21.0. Its `commit` identifies
the checked-out ECC code and acceptance driver; `sourceDirty: false` binds the
run to committed files. The receipt includes the Grok executable SHA-256,
command exit codes, compact native inspection results, and immutable fixture
commit IDs. This is local acceptance evidence for PR #2856.
## Reproduce
Install Grok separately, put its executable on `PATH`, and run from a checkout:
```bash
node tests/ci/grok-native-lifecycle.js --report /tmp/grok-native-lifecycle.json
```
The opt-in driver exits nonzero if Grok is unavailable, a command fails, or any
assertion fails. It does not silently skip native checks. Ordinary JS tests may
run without Grok; the driver sets `ECC_REQUIRE_GROK_NATIVE=1` and requires eight
successful native readbacks from the canonical install suite.
It creates disposable HOME, USERPROFILE, GROK_HOME and XDG directories and
passes only `PATH` plus those isolation and test settings. No user credentials,
proxy settings or provider configuration are inherited. The isolated project
must start with no discovered skills, hooks or MCP servers. Each command has a
deadline and bounded output; the driver's temporary root is removed on exit.
## Checked scenarios
| Scenario | Observable assertion |
| --- | --- |
| Trusted install | Real `grok plugin install --trust`, `plugin list --json` and `inspect --json` discover ECC skills; the native manifest keeps hooks/MCP disabled. Canonical trust plus explicit consent enables the requested capabilities. |
| Untrusted install | Native noninteractive install without `--trust` exits 1 and leaves no plugin. Canonical untrusted plans omit hooks/MCP even when requested. |
| Hook denial | Canonical hook denial with an independently consented MCP server yields no hooks and exactly one MCP server in native inspection. |
| MCP denial | Canonical MCP denial with consented hooks yields hooks and zero MCP servers in native inspection. |
| Disabled plugin | Native disable removes ECC from discovered skills; enable restores it. |
| Missing root | Moving the isolated installed root removes ECC from discovered skills; restoring it restores discovery. The adapter also resolves missing roots to null. |
| Upgrade | Real native Git update advances the full ECC fixture to its second commit. Canonical upgrade changes the full ECC payload and native capability inventory. |
| Uninstall | Native uninstall removes the registry entry, Git snapshot and discovered ECC components. Canonical uninstall removes its receipt and installed tree. |
| Rollback | Explicit native Git reinstall at the first SHA restores the first version and marker. Canonical rollback restores the recorded SHA, payload and consented capabilities, then native inspection confirms discovery. |
| Immutable pin | Native update of a pinned plugin reports skipping and preserves the exact SHA. Canonical installs ignore dirty/untracked working-tree content and repair retains the receipt's revision after catalog changes. |
## Fixture and evidence boundaries
Native Git transitions archive the complete ECC `commit` into a new local Git
repository. They add one marker skill and use a synthetic next-patch prerelease
version for the second commit. `baseSourceCommit`, `firstCommit` and
`secondCommit` distinguish the reviewed source from these disposable commits.
The driver verifies registry commit/ref, manifest version, marker contents,
`scripts/grok-install.js`, and canonical skill files after each transition.
These synthetic versions are not ECC releases. Git transport uses local
`file://` URLs; remote GitHub availability and marketplace publication are not
certified by this run.
The native local-path update reports “local symlink, already live.” The Git
fixture separately verifies actual snapshot updates. Native Git rollback here
means uninstall plus explicit reinstall at a prior SHA; Grok exposes no
dedicated rollback subcommand in this version. ECC rollback uses its canonical
apply path with an explicit source SHA.
Grok 1.0.62 still reports `plugin.enabled: true` in the plugin inventory after
disable even though ECC skills disappear from the loaded skill inventory.
The acceptance assertion checks the loaded components and records the plugin
inventory discrepancy; it does not treat that flag as proof of loading.
This run checks native discovery and lifecycle behavior. It does not invoke a
model, dispatch a hook, connect to an MCP server, or claim Windows/macOS native
coverage. The eight adapter tests and eleven canonical tests are separate from
the native CLI commands enumerated in the receipt. A maintainer or CI runner
can rerun the driver and retain its JSON as an artifact bound to that run's
commit; the checked-in receipt records this specific machine and version.
+1
View File
@@ -528,6 +528,7 @@
"security:ioc-scan": "node scripts/ci/scan-supply-chain-iocs.js",
"security:advisory-sources": "node scripts/ci/supply-chain-advisory-sources.js",
"test:plugin-setup-platform": "node docker/plugin-setup/run-platform-tests.js",
"test:grok:native": "node tests/ci/grok-native-lifecycle.js",
"claw": "node scripts/claw.js",
"orchestrate:status": "node scripts/orchestration-status.js",
"orchestrate:worker": "bash scripts/orchestrate-codex-worker.sh",
+205
View File
@@ -0,0 +1,205 @@
#!/usr/bin/env node
'use strict';
const assert = require('assert');
const crypto = require('crypto');
const fs = require('fs');
const os = require('os');
const path = require('path');
const { execFileSync, spawnSync } = require('child_process');
const { runNativeGitFixture } = require('./helpers/grok-native-git-fixture');
const ROOT = path.resolve(__dirname, '../..');
const reportIndex = process.argv.indexOf('--report');
const reportPath = reportIndex >= 0 ? path.resolve(process.argv[reportIndex + 1] || '') : null;
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'ecc-grok-native-lifecycle-'));
const homeDir = path.join(tempRoot, 'home');
const projectRoot = path.join(tempRoot, 'project');
const grokHome = path.join(homeDir, '.grok');
const env = {
PATH: process.env.PATH || '',
HOME: homeDir,
USERPROFILE: homeDir,
GROK_HOME: grokHome,
XDG_CONFIG_HOME: path.join(homeDir, '.config'),
XDG_DATA_HOME: path.join(homeDir, '.local', 'share'),
NO_COLOR: '1',
ECC_REQUIRE_GROK_NATIVE: '1',
};
let nativeSteps = [];
let grokExecutable;
function executableIdentity() {
const executable = env.PATH.split(path.delimiter).map(directory => path.join(directory, 'grok'))
.find(candidate => fs.existsSync(candidate) && fs.statSync(candidate).isFile());
assert.ok(executable, 'A real grok executable must be available on PATH');
grokExecutable = fs.realpathSync(executable);
return {
sha256: crypto.createHash('sha256').update(fs.readFileSync(grokExecutable)).digest('hex'),
};
}
function run(command, args, options = {}) {
const result = spawnSync(command === 'grok' ? grokExecutable : command, args, {
cwd: options.cwd || projectRoot,
env,
encoding: 'utf8',
timeout: options.timeout || 300_000,
maxBuffer: 16 * 1024 * 1024,
stdio: ['ignore', 'pipe', 'pipe'],
});
if (result.error) throw result.error;
if (result.status !== (options.expectedStatus ?? 0)) {
throw new Error(`${command} ${args.join(' ')} exited ${result.status}: ${result.stderr || result.stdout}`);
}
return result;
}
function grok(args, options = {}) {
const result = run('grok', args, options);
const redact = value => String(value || '').split(tempRoot).join('<temporary-native-root>')
.split(ROOT).join('<checkout>');
nativeSteps = [...nativeSteps, {
command: redact(`grok ${args.join(' ')}`).split(ROOT).join('<checkout>'),
exitCode: result.status,
stdout: redact((result.stdout || '').trim()).slice(0, 1200),
stderr: redact((result.stderr || '').trim()).slice(0, 800),
}];
return result;
}
function inspect() {
const data = JSON.parse(grok(['inspect', '--json']).stdout);
const stdout = JSON.stringify({
plugins: data.plugins.map(plugin => ({ name: plugin.name, enabled: plugin.enabled, provides: plugin.provides })),
eccSkills: data.skills.filter(skill => skill.source?.plugin_name === 'ecc').length,
hooks: data.hooks.length,
mcpServers: data.mcpServers.length,
});
nativeSteps = [...nativeSteps.slice(0, -1), { ...nativeSteps[nativeSteps.length - 1], stdout }];
return data;
}
function pluginList() {
return JSON.parse(grok(['plugin', 'list', '--json']).stdout);
}
function runNodeSuite(relativePath) {
const result = run(process.execPath, [path.join(ROOT, relativePath)], { cwd: ROOT });
const summary = result.stdout.match(/Results: Passed: (\d+), Failed: (\d+)/)
|| result.stdout.match(/Passed: (\d+), Failed: (\d+)/);
if (!summary || Number(summary[2]) !== 0) {
throw new Error(`${relativePath} emitted no passing summary or has failures:\n${result.stdout}\n${result.stderr}`);
}
const inspections = result.stdout.match(/Native Grok inspections: (\d+)/);
if (relativePath.includes('canonical')) {
assert.ok(inspections && Number(inspections[1]) >= 8, 'Native inspections must execute without skipping');
}
return {
file: relativePath,
passed: Number(summary[1]),
failed: Number(summary[2]),
...(inspections ? { nativeInspections: Number(inspections[1]) } : {}),
};
}
async function main() {
fs.mkdirSync(projectRoot, { recursive: true });
const head = execFileSync('git', ['rev-parse', 'HEAD'], { cwd: ROOT, encoding: 'utf8' }).trim();
const dirtyAtStart = Boolean(execFileSync('git', ['status', '--porcelain'], { cwd: ROOT, encoding: 'utf8' }).trim());
const executable = executableIdentity();
const version = grok(['--version']).stdout.trim();
grok(['plugin', 'validate', ROOT], { cwd: ROOT });
assert.deepStrictEqual(pluginList(), []);
const baseline = inspect();
assert.strictEqual(baseline.skills.length, 0);
assert.strictEqual(baseline.hooks.length, 0);
assert.strictEqual(baseline.mcpServers.length, 0);
const denied = grok(['plugin', 'install', ROOT], { expectedStatus: 1 });
assert.match(denied.stderr, /requires confirmation/);
assert.deepStrictEqual(pluginList(), []);
grok(['plugin', 'install', ROOT, '--trust']);
const installed = pluginList();
assert.strictEqual(installed.length, 1);
assert.strictEqual(installed[0].name, 'ecc');
const installedInspection = inspect();
const installedPlugin = installedInspection.plugins.find(plugin => plugin.name === 'ecc');
assert.ok(installedPlugin && installedPlugin.enabled);
assert.strictEqual(installedPlugin.provides.hooks, false);
assert.strictEqual(installedPlugin.provides.mcpServers, 0);
assert.ok(installedInspection.skills.some(skill => skill.source?.plugin_name === 'ecc'));
const nativeUpdate = grok(['plugin', 'update', 'ecc']);
assert.match(nativeUpdate.stdout, /already live|local symlink/i);
grok(['plugin', 'disable', 'ecc']);
const disabledInspection = inspect();
assert.strictEqual(disabledInspection.skills.some(skill => skill.source?.plugin_name === 'ecc'), false);
grok(['plugin', 'enable', 'ecc']);
assert.ok(inspect().skills.some(skill => skill.source?.plugin_name === 'ecc'));
const unavailableRoot = `${installed[0].path}.unavailable`;
fs.renameSync(installed[0].path, unavailableRoot);
assert.strictEqual(inspect().skills.some(skill => skill.source?.plugin_name === 'ecc'), false);
fs.renameSync(unavailableRoot, installed[0].path);
assert.ok(inspect().skills.some(skill => skill.source?.plugin_name === 'ecc'));
grok(['plugin', 'uninstall', 'ecc', '--confirm']);
assert.deepStrictEqual(pluginList(), []);
assert.strictEqual(inspect().plugins.some(plugin => plugin.name === 'ecc'), false);
const nativeGitFixture = await runNativeGitFixture({
tempRoot, runGrok: grok, env, repoRoot: ROOT, sourceCommit: head,
});
const suites = [
runNodeSuite('tests/lib/grok-harness-adapter.test.js'),
runNodeSuite('tests/lib/grok-canonical-install.test.js'),
];
assert.strictEqual(crypto.createHash('sha256').update(fs.readFileSync(grokExecutable)).digest('hex'),
executable.sha256, 'Grok binary must not change during verification');
assert.strictEqual(execFileSync('git', ['rev-parse', 'HEAD'], { cwd: ROOT, encoding: 'utf8' }).trim(), head,
'Checkout HEAD must not change during verification');
const report = {
schemaVersion: 1,
generatedAt: new Date().toISOString(),
commit: head,
sourceDirty: dirtyAtStart || Boolean(execFileSync('git', ['status', '--porcelain'], { cwd: ROOT, encoding: 'utf8' }).trim()),
runtime: { grok: version, executable, node: process.version, platform: process.platform, arch: process.arch },
isolation: { strategy: 'fresh temporary HOME/GROK_HOME', credentialsInherited: false },
nativeCli: {
status: 'passed',
steps: nativeSteps,
assertions: [
'untrusted local install was rejected and installed no plugin',
'trusted install was visible through grok plugin list and grok inspect',
'Grok discovered ECC skills while hooks and MCP servers remained disabled',
'disable removed ECC from loaded skills and enable restored them',
'missing installed root loaded no ECC skills; restoring root restored discovery',
'uninstall removed ECC from native discovery',
],
},
eccLifecycleSuites: suites,
nativeGitFixture,
coverageNotes: {
upgradeRollbackAndImmutablePin: 'Covered by the canonical Grok install suite using distinct source commits and an explicit rollback SHA.',
missingRoot: 'Covered by the Grok adapter boundary suite; unresolved roots return null.',
nativeUpdate: 'Real native Git lifecycle uses a full ECC source archive with a synthetic second version: pinned update stays on SHA1; explicit SHA2 install and rollback to SHA1 preserve exact registry commits; unpinned update advances to SHA2.',
disabledFlag: `${version} still reports plugin.enabled=true after disable; loaded ECC skills correctly drop to zero.`,
providerExecution: 'No model turn, tool execution, or external MCP server was invoked.',
},
};
if (reportPath) {
fs.mkdirSync(path.dirname(reportPath), { recursive: true });
fs.writeFileSync(reportPath, `${JSON.stringify(report, null, 2)}\n`);
}
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
}
main().catch(error => {
process.stderr.write(`${error.stack || error.message}\n`);
process.exitCode = 1;
}).finally(() => {
fs.rmSync(tempRoot, { recursive: true, force: true });
});
+164
View File
@@ -0,0 +1,164 @@
'use strict';
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { execFileSync, spawnSync } = require('node:child_process');
const { pathToFileURL } = require('node:url');
const { copyGitArchive } = require('../../../scripts/lib/grok-source-identity');
const PLUGIN_NAME = 'ecc';
const ROOT = path.resolve(__dirname, '../../..');
function runGit(sourceRoot, env, args) {
const result = spawnSync('git', args, {
cwd: sourceRoot,
env,
encoding: 'utf8',
timeout: 30000,
stdio: ['ignore', 'pipe', 'pipe']
});
assert.ifError(result.error);
assert.equal(result.status, 0, `git ${args.join(' ')}: ${result.stderr}`);
return result.stdout.trim();
}
function writeVersion(sourceRoot, version, content) {
const manifestPath = path.join(sourceRoot, '.grok-plugin', 'plugin.json');
const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'));
fs.mkdirSync(path.join(sourceRoot, 'skills', 'native-git-fixture'), { recursive: true });
fs.writeFileSync(manifestPath, JSON.stringify({ ...manifest, version }, null, 2));
fs.writeFileSync(path.join(sourceRoot, 'skills', 'native-git-fixture', 'SKILL.md'),
`---\nname: native-git-fixture\ndescription: Disposable native lifecycle fixture\n---\n${content}\n`);
}
async function installedSnapshot(env, expectedCommit, expectedVersion, expectedContent, pinned, runGrok) {
const grokHome = env.GROK_HOME || path.join(env.HOME, '.grok');
const registry = JSON.parse(fs.readFileSync(path.join(grokHome, 'installed-plugins', 'registry.json'), 'utf8'));
const records = Object.values(registry.repos).filter(record => record.plugins[PLUGIN_NAME]);
assert.equal(records.length, 1, 'Native fixture must have one registry entry');
const record = records[0];
assert.equal(record.kind.type, 'Git');
assert.equal(record.kind.commit, expectedCommit);
assert.equal(record.kind.git_ref, pinned ? expectedCommit : undefined);
assert.equal(record.plugins[PLUGIN_NAME].version, expectedVersion);
assert.equal(fs.lstatSync(record.path).isSymbolicLink(), false, 'Git install must be a snapshot, not a live symlink');
const manifest = JSON.parse(fs.readFileSync(path.join(record.path, '.grok-plugin', 'plugin.json'), 'utf8'));
assert.equal(manifest.version, expectedVersion);
assert.equal(manifest.hooks, '');
assert.equal(manifest.mcpServers, '');
assert.equal(fs.statSync(path.join(record.path, 'scripts', 'grok-install.js')).isFile(), true);
assert.equal(fs.statSync(path.join(record.path, 'skills', 'coding-standards', 'SKILL.md')).isFile(), true);
const skill = fs.readFileSync(path.join(record.path, 'skills', 'native-git-fixture', 'SKILL.md'), 'utf8');
assert.ok(skill.includes(expectedContent), 'Installed skill must match the selected commit');
const inspectionResult = await runGrok(['inspect', '--json']);
assert.equal(inspectionResult.status, 0, inspectionResult.stderr || inspectionResult.stdout);
const inspection = JSON.parse(inspectionResult.stdout);
const discoveredPlugin = inspection.plugins.find(plugin => plugin.name === PLUGIN_NAME);
assert.ok(discoveredPlugin && discoveredPlugin.enabled, 'Native inspection must discover the ECC Git snapshot');
assert.ok(inspection.skills.some(item => item.source?.plugin_name === PLUGIN_NAME
&& item.name === 'native-git-fixture'), 'Native inspection must discover the version marker skill');
return {
commit: record.kind.commit,
gitRef: record.kind.git_ref || null,
version: manifest.version,
skillContentVerified: true,
snapshotVerified: true,
eccCheckoutVerified: true,
nativeInspectVerified: true,
hooks: manifest.hooks,
mcpServers: manifest.mcpServers
};
}
async function uninstallFixture(runGrok, env) {
const grokHome = env.GROK_HOME || path.join(env.HOME, '.grok');
const registryPath = path.join(grokHome, 'installed-plugins', 'registry.json');
const before = JSON.parse(fs.readFileSync(registryPath, 'utf8'));
const record = Object.values(before.repos).find(item => item.plugins[PLUGIN_NAME]);
assert.ok(record, 'Native fixture must exist before uninstall');
const result = await runGrok(['plugin', 'uninstall', PLUGIN_NAME]);
assert.equal(result.status, 0, result.stderr || result.stdout);
const after = JSON.parse(fs.readFileSync(registryPath, 'utf8'));
assert.ok(Object.values(after.repos).every(item => !item.plugins[PLUGIN_NAME]));
assert.equal(fs.existsSync(record.path), false, 'Uninstall must remove the native Git snapshot');
const inspectionResult = await runGrok(['inspect', '--json']);
assert.equal(inspectionResult.status, 0, inspectionResult.stderr || inspectionResult.stdout);
const inspection = JSON.parse(inspectionResult.stdout);
assert.ok(!inspection.plugins.some(plugin => plugin.name === PLUGIN_NAME));
assert.ok(!inspection.skills.some(item => item.source?.plugin_name === PLUGIN_NAME));
return { registryEntryRemoved: true, snapshotRemoved: true };
}
// Real ECC files exercise native Git transport; only the disposable fixture's
// version and marker skill differ from the archived source commit.
async function runNativeGitFixture({ tempRoot, runGrok, env, repoRoot = ROOT, sourceCommit }) {
const sourceRoot = path.join(tempRoot, 'native-git-source');
assert.match(sourceCommit || '', /^[a-f0-9]{40}$/, 'A full immutable ECC source commit is required');
copyGitArchive(repoRoot, sourceCommit, sourceRoot, (file, args, options) =>
execFileSync(file, args, { ...options, env, timeout: 30000 }));
const originalManifest = JSON.parse(fs.readFileSync(path.join(sourceRoot, '.grok-plugin', 'plugin.json'), 'utf8'));
assert.equal(originalManifest.name, PLUGIN_NAME);
assert.equal(originalManifest.hooks, '');
assert.equal(originalManifest.mcpServers, '');
const firstVersion = originalManifest.version;
const [major, minor, patch] = firstVersion.split('.').map(Number);
assert.ok([major, minor, patch].every(Number.isInteger), 'Fixture source needs a stable semver version');
const secondVersion = `${major}.${minor}.${patch + 1}-native.2`;
runGit(sourceRoot, env, ['init', '-b', 'main']);
writeVersion(sourceRoot, firstVersion, 'Fixture version one.');
runGit(sourceRoot, env, ['add', '.']);
runGit(sourceRoot, env, ['-c', 'user.name=Native Fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '-m', 'fixture v1']);
const firstCommit = runGit(sourceRoot, env, ['rev-parse', 'HEAD']);
const sourceUrl = pathToFileURL(sourceRoot).href;
const install = async ref => {
const result = await runGrok(['plugin', 'install', `${sourceUrl}${ref ? `@${ref}` : ''}`, '--trust']);
assert.equal(result.status, 0, result.stderr || result.stdout);
};
await install(firstCommit);
const pinnedInstall = await installedSnapshot(env, firstCommit, firstVersion, 'Fixture version one.', true, runGrok);
writeVersion(sourceRoot, secondVersion, 'Fixture version two.');
runGit(sourceRoot, env, ['add', '.']);
runGit(sourceRoot, env, ['-c', 'user.name=Native Fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '-m', 'fixture v2']);
const secondCommit = runGit(sourceRoot, env, ['rev-parse', 'HEAD']);
assert.notEqual(firstCommit, secondCommit);
const update = await runGrok(['plugin', 'update', PLUGIN_NAME]);
assert.equal(update.status, 0, update.stderr || update.stdout);
assert.match(`${update.stdout}\n${update.stderr}`, /pinned to .*skipping/);
const immutablePin = await installedSnapshot(env, firstCommit, firstVersion, 'Fixture version one.', true, runGrok);
await uninstallFixture(runGrok, env);
await install(secondCommit);
const explicitUpgrade = await installedSnapshot(env, secondCommit, secondVersion, 'Fixture version two.', true, runGrok);
await uninstallFixture(runGrok, env);
await install(firstCommit);
const rollback = await installedSnapshot(env, firstCommit, firstVersion, 'Fixture version one.', true, runGrok);
await uninstallFixture(runGrok, env);
// Move only the disposable fixture's branch to exercise native unpinned update
// using the same two commits as the immutable install and rollback checks.
runGit(sourceRoot, env, ['reset', '--hard', firstCommit]);
await install();
const unpinnedInstall = await installedSnapshot(env, firstCommit, firstVersion, 'Fixture version one.', false, runGrok);
runGit(sourceRoot, env, ['reset', '--hard', secondCommit]);
const unpinnedUpdateResult = await runGrok(['plugin', 'update', PLUGIN_NAME]);
assert.equal(unpinnedUpdateResult.status, 0, unpinnedUpdateResult.stderr || unpinnedUpdateResult.stdout);
const unpinnedUpdate = await installedSnapshot(env, secondCommit, secondVersion, 'Fixture version two.', false, runGrok);
const uninstall = await uninstallFixture(runGrok, env);
return {
fixture: 'full ECC checkout with synthetic lifecycle versions',
baseSourceCommit: sourceCommit,
firstCommit,
secondCommit,
pinnedInstall,
immutablePin,
explicitUpgrade,
rollback,
unpinnedInstall,
unpinnedUpdate,
uninstall
};
}
module.exports = { runNativeGitFixture };
+35 -7
View File
@@ -20,6 +20,7 @@ const {
const repoRoot = path.resolve(__dirname, '..', '..');
let passed = 0;
let failed = 0;
let nativeInspections = 0;
function test(name, fn) {
try {
@@ -128,13 +129,17 @@ function inspectWithGrok(fixture) {
env: {
...process.env,
HOME: fixture.homeDir,
USERPROFILE: fixture.homeDir,
GROK_HOME: path.join(fixture.homeDir, '.grok'),
XDG_CONFIG_HOME: path.join(fixture.homeDir, '.config'),
XDG_DATA_HOME: path.join(fixture.homeDir, '.local', 'share'),
},
timeout: 30000,
});
if (result.error && result.error.code === 'ENOENT') return null;
if (result.error && result.error.code === 'ENOENT' && process.env.ECC_REQUIRE_GROK_NATIVE !== '1') return null;
if (result.error) throw result.error;
assert.strictEqual(result.status, 0, result.stderr || result.stdout);
nativeInspections += 1;
return JSON.parse(result.stdout);
}
@@ -193,21 +198,25 @@ function runTests() {
for (const options of [
{ trust: false, hooks: true, mcp: ['chrome-devtools', 'other-mcp'] },
{ trust: true, hooks: false, mcp: [] },
{ trust: true, hooks: false, mcp: ['other-mcp'] },
{ trust: true, hooks: true, mcp: [] },
]) {
fs.rmSync(path.join(fixture.homeDir, '.grok'), { recursive: true, force: true });
const plan = createPlan(fixture, options);
applyInstallPlan(plan);
const manifest = JSON.parse(fs.readFileSync(path.join(installedRoot(fixture), '.grok-plugin', 'plugin.json'), 'utf8'));
assert.strictEqual(manifest.hooks, '');
assert.strictEqual(manifest.mcpServers, '');
assert.ok(!fs.existsSync(path.join(installedRoot(fixture), '.mcp.json')));
assert.ok(!fs.existsSync(path.join(installedRoot(fixture), 'hooks', 'hooks.json')));
const hooksEnabled = options.trust && options.hooks;
const mcpEnabled = options.trust && options.mcp.length > 0;
assert.strictEqual(manifest.hooks, hooksEnabled ? 'hooks/hooks.json' : '');
assert.strictEqual(manifest.mcpServers, mcpEnabled ? '.mcp.json' : '');
assert.strictEqual(fs.existsSync(path.join(installedRoot(fixture), '.mcp.json')), mcpEnabled);
assert.strictEqual(fs.existsSync(path.join(installedRoot(fixture), 'hooks', 'hooks.json')), hooksEnabled);
const inspection = inspectWithGrok(fixture);
if (inspection) {
const plugin = inspection.plugins.find((item) => item.name === 'ecc');
assert.ok(plugin && plugin.enabled);
assert.strictEqual(plugin.provides.hooks, false);
assert.strictEqual(plugin.provides.mcpServers, 0);
assert.strictEqual(plugin.provides.hooks, hooksEnabled);
assert.strictEqual(plugin.provides.mcpServers, mcpEnabled ? 1 : 0);
}
}
} finally {
@@ -470,6 +479,13 @@ function runTests() {
fs.readFileSync(path.join(installedRoot(fixture), 'commands', 'plan.md'), 'utf8'),
'upgraded payload\n'
);
const upgradedInspection = inspectWithGrok(fixture);
if (upgradedInspection) {
const plugin = upgradedInspection.plugins.find(item => item.name === 'ecc');
assert.ok(plugin && plugin.enabled);
assert.strictEqual(plugin.provides.hooks, false);
assert.strictEqual(plugin.provides.mcpServers, 0);
}
const rollbackPlan = createPlan(
fixture,
@@ -484,6 +500,13 @@ function runTests() {
'pinned payload\n'
);
assert.strictEqual(rolledBackState.source.repoVersion, '2.1.0');
const rollbackInspection = inspectWithGrok(fixture);
if (rollbackInspection) {
const plugin = rollbackInspection.plugins.find(item => item.name === 'ecc');
assert.ok(plugin && plugin.enabled);
assert.strictEqual(plugin.provides.hooks, true);
assert.strictEqual(plugin.provides.mcpServers, 1);
}
fs.writeFileSync(path.join(installedRoot(fixture), 'commands', 'plan.md'), 'broken after rollback\n');
const rollbackRepair = repairInstalledStates({
@@ -513,12 +536,17 @@ function runTests() {
assert.strictEqual(uninstall.results[0].status, 'uninstalled');
assert.ok(!fs.existsSync(firstPlan.installStatePath));
assert.ok(!fs.existsSync(installedRoot(fixture)));
const uninstalledInspection = inspectWithGrok(fixture);
if (uninstalledInspection) {
assert.ok(!uninstalledInspection.plugins.some(item => item.name === 'ecc'));
}
} finally {
fs.rmSync(fixture.parent, { recursive: true, force: true });
}
})) passed++; else failed++;
console.log(`\nResults: Passed: ${passed}, Failed: ${failed}`);
console.log(`Native Grok inspections: ${nativeInspections}`);
process.exit(failed > 0 ? 1 : 0);
}