From 3906ba98f3ccc2cd2caa420a7cf11c3fb60bd09f Mon Sep 17 00:00:00 2001 From: Wen Date: Wed, 14 Oct 2026 14:31:49 -0300 Subject: [PATCH] fix(opensource-pipeline): commit packaging and sanitizer fixes before publishing The forker makes the only commit before the sanitizer and packager run, and Step 7 publishes with `gh repo create --source=. --push`, which sends commits, not the working tree. Packager output (LICENSE, README, CLAUDE.md, setup.sh) never reached the public repo, and a secret the sanitizer caught and the agent fixed in the working tree was still pushed in that commit. - Fold sanitizer fixes and packager output into the single commit with `git commit --amend`, keeping FORK_REPORT.md and SANITIZATION_REPORT.md out through .git/info/exclude. - Re-run the sanitizer after packaging (Step 5b) and refuse to push with uncommitted changes or more than one commit. - Sanitizer Step 6 now runs the Step 1 secret patterns and Step 3 home path patterns over `git log -p --all`, not only a 4-keyword grep. Signed-off-by: Wen (cherry picked from commit fe767e2099ca09dcd4d8928e5d74bf913c0f28b5) --- agents/opensource-forker.md | 5 ++++ agents/opensource-sanitizer.md | 14 +++++++++-- skills/opensource-pipeline/SKILL.md | 36 ++++++++++++++++++++++++++--- 3 files changed, 50 insertions(+), 5 deletions(-) diff --git a/agents/opensource-forker.md b/agents/opensource-forker.md index 4c5d8cbb8..ede323d1a 100644 --- a/agents/opensource-forker.md +++ b/agents/opensource-forker.md @@ -150,8 +150,13 @@ git commit -m "Initial open-source release Forked from private source. All secrets stripped, internal references replaced with configurable placeholders. See .env.example for configuration." +printf '%s\n' FORK_REPORT.md SANITIZATION_REPORT.md >> .git/info/exclude ``` +This commit is provisional. The pipeline amends it after sanitizer fixes and packaging, so the +published history stays one commit (see the opensource-pipeline skill, Step 3). The reports never +go into it: `.git/info/exclude` keeps them out of `git add -A`. + ### Step 7: Generate Fork Report Create `FORK_REPORT.md` in the staging directory: diff --git a/agents/opensource-sanitizer.md b/agents/opensource-sanitizer.md index 36688a87a..c61e1de4a 100644 --- a/agents/opensource-sanitizer.md +++ b/agents/opensource-sanitizer.md @@ -135,10 +135,20 @@ cd PROJECT_DIR git log --oneline | wc -l # If > 1, history was not cleaned — FAIL -# Search history for potential secrets -git log -p | grep -iE '(password|secret|api.?key|token)' | head -20 +# Write the full history to a file outside the project, so Steps 1-3 never scan it +HISTORY="$(mktemp)" +git log -p --all --no-color > "$HISTORY" +echo "$HISTORY" + +# Broad keyword pass (WARNING — manual review) +grep -iE '(password|secret|api.?key|token)' "$HISTORY" | head -20 ``` +Steps 1-3 read the working tree, but the push sends commits. A secret fixed in a file but still in +a commit is published. Run every Step 1 pattern (CRITICAL) and the Step 3 home-directory patterns +over `$HISTORY` with the same tool you used for Steps 1-3. Any match is a FAIL, even when the +current file is clean. Delete `$HISTORY` when done. + ### Step 7: Unreadable Files (WARNING) Steps 1-3 only match text, so a credential sitting in a spreadsheet, a PDF or an embedded database diff --git a/skills/opensource-pipeline/SKILL.md b/skills/opensource-pipeline/SKILL.md index d72f15eb0..7d8e4c5a9 100644 --- a/skills/opensource-pipeline/SKILL.md +++ b/skills/opensource-pipeline/SKILL.md @@ -82,6 +82,23 @@ Follow the full forking protocol: Wait for completion. Read `{STAGING_PATH}/FORK_REPORT.md`. +The forker's single commit is only the starting point: `gh repo create --push` (Step 7) +publishes commits, not the working tree. Every later change (sanitizer fixes, packager output) +must be folded into that commit, so the public history is one commit that matches the tree the +sanitizer passed. The reports stay out of it. Fold changes in with: + +```bash +cd "{STAGING_PATH}" +grep -qxF SANITIZATION_REPORT.md .git/info/exclude || printf '%s\n' FORK_REPORT.md SANITIZATION_REPORT.md >> .git/info/exclude +git add -A +git commit --amend --no-edit +git status --porcelain # must print nothing +git rev-list --count HEAD # must print 1 +``` + +An amend leaves a secret that was removed from a file out of the new commit. A new commit on top +would publish it in history. + #### Step 4: Run Sanitizer Agent Spawn the `opensource-sanitizer` agent: @@ -113,7 +130,7 @@ Generate SANITIZATION_REPORT.md inside {STAGING_PATH}/ with PASS/FAIL verdict. Wait for completion. Read `{STAGING_PATH}/SANITIZATION_REPORT.md`. **If FAIL:** Show findings to user. Ask: "Fix these and re-scan, or abort?" -- If fix: Apply fixes, re-run sanitizer (maximum 3 retry attempts — after 3 FAILs, present all findings and ask user to fix manually) +- If fix: Apply fixes, fold them into the commit (see Step 3), re-run sanitizer (maximum 3 retry attempts — after 3 FAILs, present all findings and ask user to fix manually) - If abort: Clean up staging directory **Unreadable files.** After the fix/rescan loop settles, read the report's `## Unreadable Files` @@ -168,6 +185,13 @@ Generate: ) ``` +#### Step 5b: Commit the Packaging and Re-Verify + +The packager's files are not committed yet. Fold them into the commit (see Step 3), then run the +sanitizer again exactly as in Step 4, against what will be published. Apply the same gate as +Step 4, including the fix loop and the unreadable-files check. A FAIL here stops the pipeline: +packaging can add a personal email, a home path or a key in an example. + #### Step 6: Final Review Present to user: @@ -184,7 +208,7 @@ Files generated: - CONTRIBUTING.md - .env.example ({N} variables) -Sanitization: {sanitization_verdict} +Sanitization: {sanitization_verdict} (scanned after packaging, on the commit that will be pushed) Next steps: 1. Review: cd {STAGING_PATH} @@ -196,8 +220,13 @@ Proceed with GitHub creation? (yes/no/review first) #### Step 7: GitHub Publish (on user approval) +Push only the commit the sanitizer passed. If the tree changed since Step 5b (for example the user +edited files after "review first"), go back to Step 5b instead of pushing. + ```bash cd "{STAGING_PATH}" +test -z "$(git status --porcelain)" && test "$(git rev-list --count HEAD)" = 1 \ + || { echo "Uncommitted changes or extra commits: return to Step 5b" >&2; exit 1; } gh repo create "{github_org}/{github_repo}" --public --source=. --push --description "{description}" ``` @@ -270,13 +299,14 @@ $HOME/opensource-staging/ - **Never** push to GitHub without user approval - **Never** skip the sanitizer — it is the safety gate - **Never** proceed after a sanitizer FAIL without fixing all critical findings +- **Never** push with uncommitted changes or more than one commit: the push sends commits, so unfolded fixes are not published and the original secret is - **Never** leave `.env`, `*.pem`, or `credentials.json` in the staging directory ## Best Practices - Always run the full pipeline (fork → sanitize → package) for new releases - The staging directory persists until explicitly cleaned up — use it for review -- Re-run the sanitizer after any manual fixes before publishing +- After any manual fix, fold it into the single commit and re-run the sanitizer before publishing - Parameterize secrets rather than deleting them — preserve project functionality ## Related Skills