mirror of
https://github.com/affaan-m/ECC.git
synced 2026-11-07 08:35:54 +00:00
test: record native Grok lifecycle evidence
This commit is contained in:
1 parent
d363c04261
commit
dd2107ff65
6 files changed
+493
-7
No files matched your search
@@ -85,6 +85,22 @@ file remains for Claude/Codex. Additional connectors stay opt-in via
|
||||
`mcp-configs/mcp-servers.json`. Canonical receipts record explicit per-server
|
||||
consent before any MCP name is copied into an install plan.
|
||||
|
||||
## Native lifecycle verification
|
||||
|
||||
With a real `grok` executable on `PATH`, run the opt-in acceptance driver:
|
||||
|
||||
```bash
|
||||
node tests/ci/grok-native-lifecycle.js --report /tmp/grok-native-lifecycle.json
|
||||
```
|
||||
|
||||
It uses fresh temporary homes without inherited credentials and fails if Grok is
|
||||
missing. It verifies ECC discovery, trust refusal, independent hook/MCP consent,
|
||||
disabled and missing roots, Git update, rollback, immutable pins, and uninstall.
|
||||
Native Git transitions use a disposable full ECC archive with synthetic versions;
|
||||
canonical lifecycle tests also require real `grok inspect` readbacks. Temporary
|
||||
installations are removed on exit. See the
|
||||
[recorded run and evidence boundaries](../docs/releases/2.2.3/grok-native-lifecycle.md).
|
||||
|
||||
## Known limits
|
||||
|
||||
- 311 skills are advertised into Grok's skill catalog. Use a smaller working
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
# Grok native lifecycle evidence
|
||||
|
||||
The adjacent [JSON receipt](grok-native-lifecycle.json) records a real local run
|
||||
on Linux x64 with Grok Build 1.0.62 and Node 24.21.0. Its `commit` identifies
|
||||
the checked-out ECC code and acceptance driver; `sourceDirty: false` binds the
|
||||
run to committed files. The receipt includes the Grok executable SHA-256,
|
||||
command exit codes, compact native inspection results, and immutable fixture
|
||||
commit IDs. This is local acceptance evidence for PR #2856.
|
||||
|
||||
## Reproduce
|
||||
|
||||
Install Grok separately, put its executable on `PATH`, and run from a checkout:
|
||||
|
||||
```bash
|
||||
node tests/ci/grok-native-lifecycle.js --report /tmp/grok-native-lifecycle.json
|
||||
```
|
||||
|
||||
The opt-in driver exits nonzero if Grok is unavailable, a command fails, or any
|
||||
assertion fails. It does not silently skip native checks. Ordinary JS tests may
|
||||
run without Grok; the driver sets `ECC_REQUIRE_GROK_NATIVE=1` and requires eight
|
||||
successful native readbacks from the canonical install suite.
|
||||
|
||||
It creates disposable HOME, USERPROFILE, GROK_HOME and XDG directories and
|
||||
passes only `PATH` plus those isolation and test settings. No user credentials,
|
||||
proxy settings or provider configuration are inherited. The isolated project
|
||||
must start with no discovered skills, hooks or MCP servers. Each command has a
|
||||
deadline and bounded output; the driver's temporary root is removed on exit.
|
||||
|
||||
## Checked scenarios
|
||||
|
||||
| Scenario | Observable assertion |
|
||||
| --- | --- |
|
||||
| Trusted install | Real `grok plugin install --trust`, `plugin list --json` and `inspect --json` discover ECC skills; the native manifest keeps hooks/MCP disabled. Canonical trust plus explicit consent enables the requested capabilities. |
|
||||
| Untrusted install | Native noninteractive install without `--trust` exits 1 and leaves no plugin. Canonical untrusted plans omit hooks/MCP even when requested. |
|
||||
| Hook denial | Canonical hook denial with an independently consented MCP server yields no hooks and exactly one MCP server in native inspection. |
|
||||
| MCP denial | Canonical MCP denial with consented hooks yields hooks and zero MCP servers in native inspection. |
|
||||
| Disabled plugin | Native disable removes ECC from discovered skills; enable restores it. |
|
||||
| Missing root | Moving the isolated installed root removes ECC from discovered skills; restoring it restores discovery. The adapter also resolves missing roots to null. |
|
||||
| Upgrade | Real native Git update advances the full ECC fixture to its second commit. Canonical upgrade changes the full ECC payload and native capability inventory. |
|
||||
| Uninstall | Native uninstall removes the registry entry, Git snapshot and discovered ECC components. Canonical uninstall removes its receipt and installed tree. |
|
||||
| Rollback | Explicit native Git reinstall at the first SHA restores the first version and marker. Canonical rollback restores the recorded SHA, payload and consented capabilities, then native inspection confirms discovery. |
|
||||
| Immutable pin | Native update of a pinned plugin reports skipping and preserves the exact SHA. Canonical installs ignore dirty/untracked working-tree content and repair retains the receipt's revision after catalog changes. |
|
||||
|
||||
## Fixture and evidence boundaries
|
||||
|
||||
Native Git transitions archive the complete ECC `commit` into a new local Git
|
||||
repository. They add one marker skill and use a synthetic next-patch prerelease
|
||||
version for the second commit. `baseSourceCommit`, `firstCommit` and
|
||||
`secondCommit` distinguish the reviewed source from these disposable commits.
|
||||
The driver verifies registry commit/ref, manifest version, marker contents,
|
||||
`scripts/grok-install.js`, and canonical skill files after each transition.
|
||||
These synthetic versions are not ECC releases. Git transport uses local
|
||||
`file://` URLs; remote GitHub availability and marketplace publication are not
|
||||
certified by this run.
|
||||
|
||||
The native local-path update reports “local symlink, already live.” The Git
|
||||
fixture separately verifies actual snapshot updates. Native Git rollback here
|
||||
means uninstall plus explicit reinstall at a prior SHA; Grok exposes no
|
||||
dedicated rollback subcommand in this version. ECC rollback uses its canonical
|
||||
apply path with an explicit source SHA.
|
||||
|
||||
Grok 1.0.62 still reports `plugin.enabled: true` in the plugin inventory after
|
||||
disable even though ECC skills disappear from the loaded skill inventory.
|
||||
The acceptance assertion checks the loaded components and records the plugin
|
||||
inventory discrepancy; it does not treat that flag as proof of loading.
|
||||
|
||||
This run checks native discovery and lifecycle behavior. It does not invoke a
|
||||
model, dispatch a hook, connect to an MCP server, or claim Windows/macOS native
|
||||
coverage. The eight adapter tests and eleven canonical tests are separate from
|
||||
the native CLI commands enumerated in the receipt. A maintainer or CI runner
|
||||
can rerun the driver and retain its JSON as an artifact bound to that run's
|
||||
commit; the checked-in receipt records this specific machine and version.
|
||||
@@ -528,6 +528,7 @@
|
||||
"security:ioc-scan": "node scripts/ci/scan-supply-chain-iocs.js",
|
||||
"security:advisory-sources": "node scripts/ci/supply-chain-advisory-sources.js",
|
||||
"test:plugin-setup-platform": "node docker/plugin-setup/run-platform-tests.js",
|
||||
"test:grok:native": "node tests/ci/grok-native-lifecycle.js",
|
||||
"claw": "node scripts/claw.js",
|
||||
"orchestrate:status": "node scripts/orchestration-status.js",
|
||||
"orchestrate:worker": "bash scripts/orchestrate-codex-worker.sh",
|
||||
|
||||
@@ -0,0 +1,205 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
const assert = require('assert');
|
||||
const crypto = require('crypto');
|
||||
const fs = require('fs');
|
||||
const os = require('os');
|
||||
const path = require('path');
|
||||
const { execFileSync, spawnSync } = require('child_process');
|
||||
const { runNativeGitFixture } = require('./helpers/grok-native-git-fixture');
|
||||
|
||||
const ROOT = path.resolve(__dirname, '../..');
|
||||
const reportIndex = process.argv.indexOf('--report');
|
||||
const reportPath = reportIndex >= 0 ? path.resolve(process.argv[reportIndex + 1] || '') : null;
|
||||
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'ecc-grok-native-lifecycle-'));
|
||||
const homeDir = path.join(tempRoot, 'home');
|
||||
const projectRoot = path.join(tempRoot, 'project');
|
||||
const grokHome = path.join(homeDir, '.grok');
|
||||
const env = {
|
||||
PATH: process.env.PATH || '',
|
||||
HOME: homeDir,
|
||||
USERPROFILE: homeDir,
|
||||
GROK_HOME: grokHome,
|
||||
XDG_CONFIG_HOME: path.join(homeDir, '.config'),
|
||||
XDG_DATA_HOME: path.join(homeDir, '.local', 'share'),
|
||||
NO_COLOR: '1',
|
||||
ECC_REQUIRE_GROK_NATIVE: '1',
|
||||
};
|
||||
let nativeSteps = [];
|
||||
let grokExecutable;
|
||||
|
||||
function executableIdentity() {
|
||||
const executable = env.PATH.split(path.delimiter).map(directory => path.join(directory, 'grok'))
|
||||
.find(candidate => fs.existsSync(candidate) && fs.statSync(candidate).isFile());
|
||||
assert.ok(executable, 'A real grok executable must be available on PATH');
|
||||
grokExecutable = fs.realpathSync(executable);
|
||||
return {
|
||||
sha256: crypto.createHash('sha256').update(fs.readFileSync(grokExecutable)).digest('hex'),
|
||||
};
|
||||
}
|
||||
|
||||
function run(command, args, options = {}) {
|
||||
const result = spawnSync(command === 'grok' ? grokExecutable : command, args, {
|
||||
cwd: options.cwd || projectRoot,
|
||||
env,
|
||||
encoding: 'utf8',
|
||||
timeout: options.timeout || 300_000,
|
||||
maxBuffer: 16 * 1024 * 1024,
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
});
|
||||
if (result.error) throw result.error;
|
||||
if (result.status !== (options.expectedStatus ?? 0)) {
|
||||
throw new Error(`${command} ${args.join(' ')} exited ${result.status}: ${result.stderr || result.stdout}`);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function grok(args, options = {}) {
|
||||
const result = run('grok', args, options);
|
||||
const redact = value => String(value || '').split(tempRoot).join('<temporary-native-root>')
|
||||
.split(ROOT).join('<checkout>');
|
||||
nativeSteps = [...nativeSteps, {
|
||||
command: redact(`grok ${args.join(' ')}`).split(ROOT).join('<checkout>'),
|
||||
exitCode: result.status,
|
||||
stdout: redact((result.stdout || '').trim()).slice(0, 1200),
|
||||
stderr: redact((result.stderr || '').trim()).slice(0, 800),
|
||||
}];
|
||||
return result;
|
||||
}
|
||||
|
||||
function inspect() {
|
||||
const data = JSON.parse(grok(['inspect', '--json']).stdout);
|
||||
const stdout = JSON.stringify({
|
||||
plugins: data.plugins.map(plugin => ({ name: plugin.name, enabled: plugin.enabled, provides: plugin.provides })),
|
||||
eccSkills: data.skills.filter(skill => skill.source?.plugin_name === 'ecc').length,
|
||||
hooks: data.hooks.length,
|
||||
mcpServers: data.mcpServers.length,
|
||||
});
|
||||
nativeSteps = [...nativeSteps.slice(0, -1), { ...nativeSteps[nativeSteps.length - 1], stdout }];
|
||||
return data;
|
||||
}
|
||||
|
||||
function pluginList() {
|
||||
return JSON.parse(grok(['plugin', 'list', '--json']).stdout);
|
||||
}
|
||||
|
||||
function runNodeSuite(relativePath) {
|
||||
const result = run(process.execPath, [path.join(ROOT, relativePath)], { cwd: ROOT });
|
||||
const summary = result.stdout.match(/Results: Passed: (\d+), Failed: (\d+)/)
|
||||
|| result.stdout.match(/Passed: (\d+), Failed: (\d+)/);
|
||||
if (!summary || Number(summary[2]) !== 0) {
|
||||
throw new Error(`${relativePath} emitted no passing summary or has failures:\n${result.stdout}\n${result.stderr}`);
|
||||
}
|
||||
const inspections = result.stdout.match(/Native Grok inspections: (\d+)/);
|
||||
if (relativePath.includes('canonical')) {
|
||||
assert.ok(inspections && Number(inspections[1]) >= 8, 'Native inspections must execute without skipping');
|
||||
}
|
||||
return {
|
||||
file: relativePath,
|
||||
passed: Number(summary[1]),
|
||||
failed: Number(summary[2]),
|
||||
...(inspections ? { nativeInspections: Number(inspections[1]) } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
async function main() {
|
||||
fs.mkdirSync(projectRoot, { recursive: true });
|
||||
const head = execFileSync('git', ['rev-parse', 'HEAD'], { cwd: ROOT, encoding: 'utf8' }).trim();
|
||||
const dirtyAtStart = Boolean(execFileSync('git', ['status', '--porcelain'], { cwd: ROOT, encoding: 'utf8' }).trim());
|
||||
const executable = executableIdentity();
|
||||
const version = grok(['--version']).stdout.trim();
|
||||
grok(['plugin', 'validate', ROOT], { cwd: ROOT });
|
||||
assert.deepStrictEqual(pluginList(), []);
|
||||
const baseline = inspect();
|
||||
assert.strictEqual(baseline.skills.length, 0);
|
||||
assert.strictEqual(baseline.hooks.length, 0);
|
||||
assert.strictEqual(baseline.mcpServers.length, 0);
|
||||
|
||||
const denied = grok(['plugin', 'install', ROOT], { expectedStatus: 1 });
|
||||
assert.match(denied.stderr, /requires confirmation/);
|
||||
assert.deepStrictEqual(pluginList(), []);
|
||||
|
||||
grok(['plugin', 'install', ROOT, '--trust']);
|
||||
const installed = pluginList();
|
||||
assert.strictEqual(installed.length, 1);
|
||||
assert.strictEqual(installed[0].name, 'ecc');
|
||||
const installedInspection = inspect();
|
||||
const installedPlugin = installedInspection.plugins.find(plugin => plugin.name === 'ecc');
|
||||
assert.ok(installedPlugin && installedPlugin.enabled);
|
||||
assert.strictEqual(installedPlugin.provides.hooks, false);
|
||||
assert.strictEqual(installedPlugin.provides.mcpServers, 0);
|
||||
assert.ok(installedInspection.skills.some(skill => skill.source?.plugin_name === 'ecc'));
|
||||
const nativeUpdate = grok(['plugin', 'update', 'ecc']);
|
||||
assert.match(nativeUpdate.stdout, /already live|local symlink/i);
|
||||
|
||||
grok(['plugin', 'disable', 'ecc']);
|
||||
const disabledInspection = inspect();
|
||||
assert.strictEqual(disabledInspection.skills.some(skill => skill.source?.plugin_name === 'ecc'), false);
|
||||
grok(['plugin', 'enable', 'ecc']);
|
||||
assert.ok(inspect().skills.some(skill => skill.source?.plugin_name === 'ecc'));
|
||||
|
||||
const unavailableRoot = `${installed[0].path}.unavailable`;
|
||||
fs.renameSync(installed[0].path, unavailableRoot);
|
||||
assert.strictEqual(inspect().skills.some(skill => skill.source?.plugin_name === 'ecc'), false);
|
||||
fs.renameSync(unavailableRoot, installed[0].path);
|
||||
assert.ok(inspect().skills.some(skill => skill.source?.plugin_name === 'ecc'));
|
||||
|
||||
grok(['plugin', 'uninstall', 'ecc', '--confirm']);
|
||||
assert.deepStrictEqual(pluginList(), []);
|
||||
assert.strictEqual(inspect().plugins.some(plugin => plugin.name === 'ecc'), false);
|
||||
|
||||
const nativeGitFixture = await runNativeGitFixture({
|
||||
tempRoot, runGrok: grok, env, repoRoot: ROOT, sourceCommit: head,
|
||||
});
|
||||
|
||||
const suites = [
|
||||
runNodeSuite('tests/lib/grok-harness-adapter.test.js'),
|
||||
runNodeSuite('tests/lib/grok-canonical-install.test.js'),
|
||||
];
|
||||
assert.strictEqual(crypto.createHash('sha256').update(fs.readFileSync(grokExecutable)).digest('hex'),
|
||||
executable.sha256, 'Grok binary must not change during verification');
|
||||
assert.strictEqual(execFileSync('git', ['rev-parse', 'HEAD'], { cwd: ROOT, encoding: 'utf8' }).trim(), head,
|
||||
'Checkout HEAD must not change during verification');
|
||||
const report = {
|
||||
schemaVersion: 1,
|
||||
generatedAt: new Date().toISOString(),
|
||||
commit: head,
|
||||
sourceDirty: dirtyAtStart || Boolean(execFileSync('git', ['status', '--porcelain'], { cwd: ROOT, encoding: 'utf8' }).trim()),
|
||||
runtime: { grok: version, executable, node: process.version, platform: process.platform, arch: process.arch },
|
||||
isolation: { strategy: 'fresh temporary HOME/GROK_HOME', credentialsInherited: false },
|
||||
nativeCli: {
|
||||
status: 'passed',
|
||||
steps: nativeSteps,
|
||||
assertions: [
|
||||
'untrusted local install was rejected and installed no plugin',
|
||||
'trusted install was visible through grok plugin list and grok inspect',
|
||||
'Grok discovered ECC skills while hooks and MCP servers remained disabled',
|
||||
'disable removed ECC from loaded skills and enable restored them',
|
||||
'missing installed root loaded no ECC skills; restoring root restored discovery',
|
||||
'uninstall removed ECC from native discovery',
|
||||
],
|
||||
},
|
||||
eccLifecycleSuites: suites,
|
||||
nativeGitFixture,
|
||||
coverageNotes: {
|
||||
upgradeRollbackAndImmutablePin: 'Covered by the canonical Grok install suite using distinct source commits and an explicit rollback SHA.',
|
||||
missingRoot: 'Covered by the Grok adapter boundary suite; unresolved roots return null.',
|
||||
nativeUpdate: 'Real native Git lifecycle uses a full ECC source archive with a synthetic second version: pinned update stays on SHA1; explicit SHA2 install and rollback to SHA1 preserve exact registry commits; unpinned update advances to SHA2.',
|
||||
disabledFlag: `${version} still reports plugin.enabled=true after disable; loaded ECC skills correctly drop to zero.`,
|
||||
providerExecution: 'No model turn, tool execution, or external MCP server was invoked.',
|
||||
},
|
||||
};
|
||||
if (reportPath) {
|
||||
fs.mkdirSync(path.dirname(reportPath), { recursive: true });
|
||||
fs.writeFileSync(reportPath, `${JSON.stringify(report, null, 2)}\n`);
|
||||
}
|
||||
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
|
||||
}
|
||||
|
||||
main().catch(error => {
|
||||
process.stderr.write(`${error.stack || error.message}\n`);
|
||||
process.exitCode = 1;
|
||||
}).finally(() => {
|
||||
fs.rmSync(tempRoot, { recursive: true, force: true });
|
||||
});
|
||||
@@ -0,0 +1,164 @@
|
||||
'use strict';
|
||||
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const { execFileSync, spawnSync } = require('node:child_process');
|
||||
const { pathToFileURL } = require('node:url');
|
||||
const { copyGitArchive } = require('../../../scripts/lib/grok-source-identity');
|
||||
|
||||
const PLUGIN_NAME = 'ecc';
|
||||
const ROOT = path.resolve(__dirname, '../../..');
|
||||
|
||||
function runGit(sourceRoot, env, args) {
|
||||
const result = spawnSync('git', args, {
|
||||
cwd: sourceRoot,
|
||||
env,
|
||||
encoding: 'utf8',
|
||||
timeout: 30000,
|
||||
stdio: ['ignore', 'pipe', 'pipe']
|
||||
});
|
||||
assert.ifError(result.error);
|
||||
assert.equal(result.status, 0, `git ${args.join(' ')}: ${result.stderr}`);
|
||||
return result.stdout.trim();
|
||||
}
|
||||
|
||||
function writeVersion(sourceRoot, version, content) {
|
||||
const manifestPath = path.join(sourceRoot, '.grok-plugin', 'plugin.json');
|
||||
const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'));
|
||||
fs.mkdirSync(path.join(sourceRoot, 'skills', 'native-git-fixture'), { recursive: true });
|
||||
fs.writeFileSync(manifestPath, JSON.stringify({ ...manifest, version }, null, 2));
|
||||
fs.writeFileSync(path.join(sourceRoot, 'skills', 'native-git-fixture', 'SKILL.md'),
|
||||
`---\nname: native-git-fixture\ndescription: Disposable native lifecycle fixture\n---\n${content}\n`);
|
||||
}
|
||||
|
||||
async function installedSnapshot(env, expectedCommit, expectedVersion, expectedContent, pinned, runGrok) {
|
||||
const grokHome = env.GROK_HOME || path.join(env.HOME, '.grok');
|
||||
const registry = JSON.parse(fs.readFileSync(path.join(grokHome, 'installed-plugins', 'registry.json'), 'utf8'));
|
||||
const records = Object.values(registry.repos).filter(record => record.plugins[PLUGIN_NAME]);
|
||||
assert.equal(records.length, 1, 'Native fixture must have one registry entry');
|
||||
const record = records[0];
|
||||
assert.equal(record.kind.type, 'Git');
|
||||
assert.equal(record.kind.commit, expectedCommit);
|
||||
assert.equal(record.kind.git_ref, pinned ? expectedCommit : undefined);
|
||||
assert.equal(record.plugins[PLUGIN_NAME].version, expectedVersion);
|
||||
assert.equal(fs.lstatSync(record.path).isSymbolicLink(), false, 'Git install must be a snapshot, not a live symlink');
|
||||
const manifest = JSON.parse(fs.readFileSync(path.join(record.path, '.grok-plugin', 'plugin.json'), 'utf8'));
|
||||
assert.equal(manifest.version, expectedVersion);
|
||||
assert.equal(manifest.hooks, '');
|
||||
assert.equal(manifest.mcpServers, '');
|
||||
assert.equal(fs.statSync(path.join(record.path, 'scripts', 'grok-install.js')).isFile(), true);
|
||||
assert.equal(fs.statSync(path.join(record.path, 'skills', 'coding-standards', 'SKILL.md')).isFile(), true);
|
||||
const skill = fs.readFileSync(path.join(record.path, 'skills', 'native-git-fixture', 'SKILL.md'), 'utf8');
|
||||
assert.ok(skill.includes(expectedContent), 'Installed skill must match the selected commit');
|
||||
const inspectionResult = await runGrok(['inspect', '--json']);
|
||||
assert.equal(inspectionResult.status, 0, inspectionResult.stderr || inspectionResult.stdout);
|
||||
const inspection = JSON.parse(inspectionResult.stdout);
|
||||
const discoveredPlugin = inspection.plugins.find(plugin => plugin.name === PLUGIN_NAME);
|
||||
assert.ok(discoveredPlugin && discoveredPlugin.enabled, 'Native inspection must discover the ECC Git snapshot');
|
||||
assert.ok(inspection.skills.some(item => item.source?.plugin_name === PLUGIN_NAME
|
||||
&& item.name === 'native-git-fixture'), 'Native inspection must discover the version marker skill');
|
||||
return {
|
||||
commit: record.kind.commit,
|
||||
gitRef: record.kind.git_ref || null,
|
||||
version: manifest.version,
|
||||
skillContentVerified: true,
|
||||
snapshotVerified: true,
|
||||
eccCheckoutVerified: true,
|
||||
nativeInspectVerified: true,
|
||||
hooks: manifest.hooks,
|
||||
mcpServers: manifest.mcpServers
|
||||
};
|
||||
}
|
||||
|
||||
async function uninstallFixture(runGrok, env) {
|
||||
const grokHome = env.GROK_HOME || path.join(env.HOME, '.grok');
|
||||
const registryPath = path.join(grokHome, 'installed-plugins', 'registry.json');
|
||||
const before = JSON.parse(fs.readFileSync(registryPath, 'utf8'));
|
||||
const record = Object.values(before.repos).find(item => item.plugins[PLUGIN_NAME]);
|
||||
assert.ok(record, 'Native fixture must exist before uninstall');
|
||||
const result = await runGrok(['plugin', 'uninstall', PLUGIN_NAME]);
|
||||
assert.equal(result.status, 0, result.stderr || result.stdout);
|
||||
const after = JSON.parse(fs.readFileSync(registryPath, 'utf8'));
|
||||
assert.ok(Object.values(after.repos).every(item => !item.plugins[PLUGIN_NAME]));
|
||||
assert.equal(fs.existsSync(record.path), false, 'Uninstall must remove the native Git snapshot');
|
||||
const inspectionResult = await runGrok(['inspect', '--json']);
|
||||
assert.equal(inspectionResult.status, 0, inspectionResult.stderr || inspectionResult.stdout);
|
||||
const inspection = JSON.parse(inspectionResult.stdout);
|
||||
assert.ok(!inspection.plugins.some(plugin => plugin.name === PLUGIN_NAME));
|
||||
assert.ok(!inspection.skills.some(item => item.source?.plugin_name === PLUGIN_NAME));
|
||||
return { registryEntryRemoved: true, snapshotRemoved: true };
|
||||
}
|
||||
|
||||
// Real ECC files exercise native Git transport; only the disposable fixture's
|
||||
// version and marker skill differ from the archived source commit.
|
||||
async function runNativeGitFixture({ tempRoot, runGrok, env, repoRoot = ROOT, sourceCommit }) {
|
||||
const sourceRoot = path.join(tempRoot, 'native-git-source');
|
||||
assert.match(sourceCommit || '', /^[a-f0-9]{40}$/, 'A full immutable ECC source commit is required');
|
||||
copyGitArchive(repoRoot, sourceCommit, sourceRoot, (file, args, options) =>
|
||||
execFileSync(file, args, { ...options, env, timeout: 30000 }));
|
||||
const originalManifest = JSON.parse(fs.readFileSync(path.join(sourceRoot, '.grok-plugin', 'plugin.json'), 'utf8'));
|
||||
assert.equal(originalManifest.name, PLUGIN_NAME);
|
||||
assert.equal(originalManifest.hooks, '');
|
||||
assert.equal(originalManifest.mcpServers, '');
|
||||
const firstVersion = originalManifest.version;
|
||||
const [major, minor, patch] = firstVersion.split('.').map(Number);
|
||||
assert.ok([major, minor, patch].every(Number.isInteger), 'Fixture source needs a stable semver version');
|
||||
const secondVersion = `${major}.${minor}.${patch + 1}-native.2`;
|
||||
runGit(sourceRoot, env, ['init', '-b', 'main']);
|
||||
writeVersion(sourceRoot, firstVersion, 'Fixture version one.');
|
||||
runGit(sourceRoot, env, ['add', '.']);
|
||||
runGit(sourceRoot, env, ['-c', 'user.name=Native Fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '-m', 'fixture v1']);
|
||||
const firstCommit = runGit(sourceRoot, env, ['rev-parse', 'HEAD']);
|
||||
const sourceUrl = pathToFileURL(sourceRoot).href;
|
||||
const install = async ref => {
|
||||
const result = await runGrok(['plugin', 'install', `${sourceUrl}${ref ? `@${ref}` : ''}`, '--trust']);
|
||||
assert.equal(result.status, 0, result.stderr || result.stdout);
|
||||
};
|
||||
|
||||
await install(firstCommit);
|
||||
const pinnedInstall = await installedSnapshot(env, firstCommit, firstVersion, 'Fixture version one.', true, runGrok);
|
||||
writeVersion(sourceRoot, secondVersion, 'Fixture version two.');
|
||||
runGit(sourceRoot, env, ['add', '.']);
|
||||
runGit(sourceRoot, env, ['-c', 'user.name=Native Fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '-m', 'fixture v2']);
|
||||
const secondCommit = runGit(sourceRoot, env, ['rev-parse', 'HEAD']);
|
||||
assert.notEqual(firstCommit, secondCommit);
|
||||
const update = await runGrok(['plugin', 'update', PLUGIN_NAME]);
|
||||
assert.equal(update.status, 0, update.stderr || update.stdout);
|
||||
assert.match(`${update.stdout}\n${update.stderr}`, /pinned to .*skipping/);
|
||||
const immutablePin = await installedSnapshot(env, firstCommit, firstVersion, 'Fixture version one.', true, runGrok);
|
||||
|
||||
await uninstallFixture(runGrok, env);
|
||||
await install(secondCommit);
|
||||
const explicitUpgrade = await installedSnapshot(env, secondCommit, secondVersion, 'Fixture version two.', true, runGrok);
|
||||
await uninstallFixture(runGrok, env);
|
||||
await install(firstCommit);
|
||||
const rollback = await installedSnapshot(env, firstCommit, firstVersion, 'Fixture version one.', true, runGrok);
|
||||
await uninstallFixture(runGrok, env);
|
||||
|
||||
// Move only the disposable fixture's branch to exercise native unpinned update
|
||||
// using the same two commits as the immutable install and rollback checks.
|
||||
runGit(sourceRoot, env, ['reset', '--hard', firstCommit]);
|
||||
await install();
|
||||
const unpinnedInstall = await installedSnapshot(env, firstCommit, firstVersion, 'Fixture version one.', false, runGrok);
|
||||
runGit(sourceRoot, env, ['reset', '--hard', secondCommit]);
|
||||
const unpinnedUpdateResult = await runGrok(['plugin', 'update', PLUGIN_NAME]);
|
||||
assert.equal(unpinnedUpdateResult.status, 0, unpinnedUpdateResult.stderr || unpinnedUpdateResult.stdout);
|
||||
const unpinnedUpdate = await installedSnapshot(env, secondCommit, secondVersion, 'Fixture version two.', false, runGrok);
|
||||
const uninstall = await uninstallFixture(runGrok, env);
|
||||
return {
|
||||
fixture: 'full ECC checkout with synthetic lifecycle versions',
|
||||
baseSourceCommit: sourceCommit,
|
||||
firstCommit,
|
||||
secondCommit,
|
||||
pinnedInstall,
|
||||
immutablePin,
|
||||
explicitUpgrade,
|
||||
rollback,
|
||||
unpinnedInstall,
|
||||
unpinnedUpdate,
|
||||
uninstall
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = { runNativeGitFixture };
|
||||
@@ -20,6 +20,7 @@ const {
|
||||
const repoRoot = path.resolve(__dirname, '..', '..');
|
||||
let passed = 0;
|
||||
let failed = 0;
|
||||
let nativeInspections = 0;
|
||||
|
||||
function test(name, fn) {
|
||||
try {
|
||||
@@ -128,13 +129,17 @@ function inspectWithGrok(fixture) {
|
||||
env: {
|
||||
...process.env,
|
||||
HOME: fixture.homeDir,
|
||||
USERPROFILE: fixture.homeDir,
|
||||
GROK_HOME: path.join(fixture.homeDir, '.grok'),
|
||||
XDG_CONFIG_HOME: path.join(fixture.homeDir, '.config'),
|
||||
XDG_DATA_HOME: path.join(fixture.homeDir, '.local', 'share'),
|
||||
},
|
||||
timeout: 30000,
|
||||
});
|
||||
if (result.error && result.error.code === 'ENOENT') return null;
|
||||
if (result.error && result.error.code === 'ENOENT' && process.env.ECC_REQUIRE_GROK_NATIVE !== '1') return null;
|
||||
if (result.error) throw result.error;
|
||||
assert.strictEqual(result.status, 0, result.stderr || result.stdout);
|
||||
nativeInspections += 1;
|
||||
return JSON.parse(result.stdout);
|
||||
}
|
||||
|
||||
@@ -193,21 +198,25 @@ function runTests() {
|
||||
for (const options of [
|
||||
{ trust: false, hooks: true, mcp: ['chrome-devtools', 'other-mcp'] },
|
||||
{ trust: true, hooks: false, mcp: [] },
|
||||
{ trust: true, hooks: false, mcp: ['other-mcp'] },
|
||||
{ trust: true, hooks: true, mcp: [] },
|
||||
]) {
|
||||
fs.rmSync(path.join(fixture.homeDir, '.grok'), { recursive: true, force: true });
|
||||
const plan = createPlan(fixture, options);
|
||||
applyInstallPlan(plan);
|
||||
const manifest = JSON.parse(fs.readFileSync(path.join(installedRoot(fixture), '.grok-plugin', 'plugin.json'), 'utf8'));
|
||||
assert.strictEqual(manifest.hooks, '');
|
||||
assert.strictEqual(manifest.mcpServers, '');
|
||||
assert.ok(!fs.existsSync(path.join(installedRoot(fixture), '.mcp.json')));
|
||||
assert.ok(!fs.existsSync(path.join(installedRoot(fixture), 'hooks', 'hooks.json')));
|
||||
const hooksEnabled = options.trust && options.hooks;
|
||||
const mcpEnabled = options.trust && options.mcp.length > 0;
|
||||
assert.strictEqual(manifest.hooks, hooksEnabled ? 'hooks/hooks.json' : '');
|
||||
assert.strictEqual(manifest.mcpServers, mcpEnabled ? '.mcp.json' : '');
|
||||
assert.strictEqual(fs.existsSync(path.join(installedRoot(fixture), '.mcp.json')), mcpEnabled);
|
||||
assert.strictEqual(fs.existsSync(path.join(installedRoot(fixture), 'hooks', 'hooks.json')), hooksEnabled);
|
||||
const inspection = inspectWithGrok(fixture);
|
||||
if (inspection) {
|
||||
const plugin = inspection.plugins.find((item) => item.name === 'ecc');
|
||||
assert.ok(plugin && plugin.enabled);
|
||||
assert.strictEqual(plugin.provides.hooks, false);
|
||||
assert.strictEqual(plugin.provides.mcpServers, 0);
|
||||
assert.strictEqual(plugin.provides.hooks, hooksEnabled);
|
||||
assert.strictEqual(plugin.provides.mcpServers, mcpEnabled ? 1 : 0);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
@@ -470,6 +479,13 @@ function runTests() {
|
||||
fs.readFileSync(path.join(installedRoot(fixture), 'commands', 'plan.md'), 'utf8'),
|
||||
'upgraded payload\n'
|
||||
);
|
||||
const upgradedInspection = inspectWithGrok(fixture);
|
||||
if (upgradedInspection) {
|
||||
const plugin = upgradedInspection.plugins.find(item => item.name === 'ecc');
|
||||
assert.ok(plugin && plugin.enabled);
|
||||
assert.strictEqual(plugin.provides.hooks, false);
|
||||
assert.strictEqual(plugin.provides.mcpServers, 0);
|
||||
}
|
||||
|
||||
const rollbackPlan = createPlan(
|
||||
fixture,
|
||||
@@ -484,6 +500,13 @@ function runTests() {
|
||||
'pinned payload\n'
|
||||
);
|
||||
assert.strictEqual(rolledBackState.source.repoVersion, '2.1.0');
|
||||
const rollbackInspection = inspectWithGrok(fixture);
|
||||
if (rollbackInspection) {
|
||||
const plugin = rollbackInspection.plugins.find(item => item.name === 'ecc');
|
||||
assert.ok(plugin && plugin.enabled);
|
||||
assert.strictEqual(plugin.provides.hooks, true);
|
||||
assert.strictEqual(plugin.provides.mcpServers, 1);
|
||||
}
|
||||
|
||||
fs.writeFileSync(path.join(installedRoot(fixture), 'commands', 'plan.md'), 'broken after rollback\n');
|
||||
const rollbackRepair = repairInstalledStates({
|
||||
@@ -513,12 +536,17 @@ function runTests() {
|
||||
assert.strictEqual(uninstall.results[0].status, 'uninstalled');
|
||||
assert.ok(!fs.existsSync(firstPlan.installStatePath));
|
||||
assert.ok(!fs.existsSync(installedRoot(fixture)));
|
||||
const uninstalledInspection = inspectWithGrok(fixture);
|
||||
if (uninstalledInspection) {
|
||||
assert.ok(!uninstalledInspection.plugins.some(item => item.name === 'ecc'));
|
||||
}
|
||||
} finally {
|
||||
fs.rmSync(fixture.parent, { recursive: true, force: true });
|
||||
}
|
||||
})) passed++; else failed++;
|
||||
|
||||
console.log(`\nResults: Passed: ${passed}, Failed: ${failed}`);
|
||||
console.log(`Native Grok inspections: ${nativeInspections}`);
|
||||
process.exit(failed > 0 ? 1 : 0);
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user