fix(opensource-pipeline): commit packaging and sanitizer fixes before publishing

The forker makes the only commit before the sanitizer and packager run,
and Step 7 publishes with `gh repo create --source=. --push`, which sends
commits, not the working tree. Packager output (LICENSE, README, CLAUDE.md,
setup.sh) never reached the public repo, and a secret the sanitizer caught
and the agent fixed in the working tree was still pushed in that commit.

- Fold sanitizer fixes and packager output into the single commit with
  `git commit --amend`, keeping FORK_REPORT.md and SANITIZATION_REPORT.md
  out through .git/info/exclude.
- Re-run the sanitizer after packaging (Step 5b) and refuse to push with
  uncommitted changes or more than one commit.
- Sanitizer Step 6 now runs the Step 1 secret patterns and Step 3 home
  path patterns over `git log -p --all`, not only a 4-keyword grep.

Signed-off-by: Wen <jwcastillo@gmail.com>
(cherry picked from commit fe767e2099)
This commit is contained in:
Wen authored and haelyra committed 2026-10-20 18:01:43 -04:00
1 parent f933bad11b
commit 3906ba98f3
3 files changed
+50 -5

No files matched your search

+5
View File
@@ -150,8 +150,13 @@ git commit -m "Initial open-source release
Forked from private source. All secrets stripped, internal references Forked from private source. All secrets stripped, internal references
replaced with configurable placeholders. See .env.example for configuration." replaced with configurable placeholders. See .env.example for configuration."
printf '%s\n' FORK_REPORT.md SANITIZATION_REPORT.md >> .git/info/exclude
``` ```
This commit is provisional. The pipeline amends it after sanitizer fixes and packaging, so the
published history stays one commit (see the opensource-pipeline skill, Step 3). The reports never
go into it: `.git/info/exclude` keeps them out of `git add -A`.
### Step 7: Generate Fork Report ### Step 7: Generate Fork Report
Create `FORK_REPORT.md` in the staging directory: Create `FORK_REPORT.md` in the staging directory:
+12 -2
View File
@@ -135,10 +135,20 @@ cd PROJECT_DIR
git log --oneline | wc -l git log --oneline | wc -l
# If > 1, history was not cleaned — FAIL # If > 1, history was not cleaned — FAIL
# Search history for potential secrets # Write the full history to a file outside the project, so Steps 1-3 never scan it
git log -p | grep -iE '(password|secret|api.?key|token)' | head -20 HISTORY="$(mktemp)"
git log -p --all --no-color > "$HISTORY"
echo "$HISTORY"
# Broad keyword pass (WARNING — manual review)
grep -iE '(password|secret|api.?key|token)' "$HISTORY" | head -20
``` ```
Steps 1-3 read the working tree, but the push sends commits. A secret fixed in a file but still in
a commit is published. Run every Step 1 pattern (CRITICAL) and the Step 3 home-directory patterns
over `$HISTORY` with the same tool you used for Steps 1-3. Any match is a FAIL, even when the
current file is clean. Delete `$HISTORY` when done.
### Step 7: Unreadable Files (WARNING) ### Step 7: Unreadable Files (WARNING)
Steps 1-3 only match text, so a credential sitting in a spreadsheet, a PDF or an embedded database Steps 1-3 only match text, so a credential sitting in a spreadsheet, a PDF or an embedded database
+33 -3
View File
@@ -82,6 +82,23 @@ Follow the full forking protocol:
Wait for completion. Read `{STAGING_PATH}/FORK_REPORT.md`. Wait for completion. Read `{STAGING_PATH}/FORK_REPORT.md`.
The forker's single commit is only the starting point: `gh repo create --push` (Step 7)
publishes commits, not the working tree. Every later change (sanitizer fixes, packager output)
must be folded into that commit, so the public history is one commit that matches the tree the
sanitizer passed. The reports stay out of it. Fold changes in with:
```bash
cd "{STAGING_PATH}"
grep -qxF SANITIZATION_REPORT.md .git/info/exclude || printf '%s\n' FORK_REPORT.md SANITIZATION_REPORT.md >> .git/info/exclude
git add -A
git commit --amend --no-edit
git status --porcelain # must print nothing
git rev-list --count HEAD # must print 1
```
An amend leaves a secret that was removed from a file out of the new commit. A new commit on top
would publish it in history.
#### Step 4: Run Sanitizer Agent #### Step 4: Run Sanitizer Agent
Spawn the `opensource-sanitizer` agent: Spawn the `opensource-sanitizer` agent:
@@ -113,7 +130,7 @@ Generate SANITIZATION_REPORT.md inside {STAGING_PATH}/ with PASS/FAIL verdict.
Wait for completion. Read `{STAGING_PATH}/SANITIZATION_REPORT.md`. Wait for completion. Read `{STAGING_PATH}/SANITIZATION_REPORT.md`.
**If FAIL:** Show findings to user. Ask: "Fix these and re-scan, or abort?" **If FAIL:** Show findings to user. Ask: "Fix these and re-scan, or abort?"
- If fix: Apply fixes, re-run sanitizer (maximum 3 retry attempts — after 3 FAILs, present all findings and ask user to fix manually) - If fix: Apply fixes, fold them into the commit (see Step 3), re-run sanitizer (maximum 3 retry attempts — after 3 FAILs, present all findings and ask user to fix manually)
- If abort: Clean up staging directory - If abort: Clean up staging directory
**Unreadable files.** After the fix/rescan loop settles, read the report's `## Unreadable Files` **Unreadable files.** After the fix/rescan loop settles, read the report's `## Unreadable Files`
@@ -168,6 +185,13 @@ Generate:
) )
``` ```
#### Step 5b: Commit the Packaging and Re-Verify
The packager's files are not committed yet. Fold them into the commit (see Step 3), then run the
sanitizer again exactly as in Step 4, against what will be published. Apply the same gate as
Step 4, including the fix loop and the unreadable-files check. A FAIL here stops the pipeline:
packaging can add a personal email, a home path or a key in an example.
#### Step 6: Final Review #### Step 6: Final Review
Present to user: Present to user:
@@ -184,7 +208,7 @@ Files generated:
- CONTRIBUTING.md - CONTRIBUTING.md
- .env.example ({N} variables) - .env.example ({N} variables)
Sanitization: {sanitization_verdict} Sanitization: {sanitization_verdict} (scanned after packaging, on the commit that will be pushed)
Next steps: Next steps:
1. Review: cd {STAGING_PATH} 1. Review: cd {STAGING_PATH}
@@ -196,8 +220,13 @@ Proceed with GitHub creation? (yes/no/review first)
#### Step 7: GitHub Publish (on user approval) #### Step 7: GitHub Publish (on user approval)
Push only the commit the sanitizer passed. If the tree changed since Step 5b (for example the user
edited files after "review first"), go back to Step 5b instead of pushing.
```bash ```bash
cd "{STAGING_PATH}" cd "{STAGING_PATH}"
test -z "$(git status --porcelain)" && test "$(git rev-list --count HEAD)" = 1 \
|| { echo "Uncommitted changes or extra commits: return to Step 5b" >&2; exit 1; }
gh repo create "{github_org}/{github_repo}" --public --source=. --push --description "{description}" gh repo create "{github_org}/{github_repo}" --public --source=. --push --description "{description}"
``` ```
@@ -270,13 +299,14 @@ $HOME/opensource-staging/
- **Never** push to GitHub without user approval - **Never** push to GitHub without user approval
- **Never** skip the sanitizer — it is the safety gate - **Never** skip the sanitizer — it is the safety gate
- **Never** proceed after a sanitizer FAIL without fixing all critical findings - **Never** proceed after a sanitizer FAIL without fixing all critical findings
- **Never** push with uncommitted changes or more than one commit: the push sends commits, so unfolded fixes are not published and the original secret is
- **Never** leave `.env`, `*.pem`, or `credentials.json` in the staging directory - **Never** leave `.env`, `*.pem`, or `credentials.json` in the staging directory
## Best Practices ## Best Practices
- Always run the full pipeline (fork → sanitize → package) for new releases - Always run the full pipeline (fork → sanitize → package) for new releases
- The staging directory persists until explicitly cleaned up — use it for review - The staging directory persists until explicitly cleaned up — use it for review
- Re-run the sanitizer after any manual fixes before publishing - After any manual fix, fold it into the single commit and re-run the sanitizer before publishing
- Parameterize secrets rather than deleting them — preserve project functionality - Parameterize secrets rather than deleting them — preserve project functionality
## Related Skills ## Related Skills