At least 5 PRs merged from auto-selected issues
KR: goals#37 — 2 configs
Winner: baseline — more candidate issues generated (4 vs 2)
baseline
Issue count
4
AC linter pass rate
100%
4/4
Total tokens
3,994
Candidate issues (4)
1.Ensure PRs include verifiable pick-next-issue selection provenance
### Summary -- What & Why Problem: merged PRs that originated from pick-next-issue.sh are hard to prove as auto-selected because PRs lack clear, machine-verifiable evidence. Without that evidence merged work may not be counted toward the KR. Consequence: valid agent work will be excluded from the metric and the team can't reliably measure progress. ### Acceptance Criteria (pre-merge) - [ ] When an engineer or agent opens a PR that resolves an issue, the PR body must include a clearly labeled provenance block containing the pick-next-issue run identifier and a link to the selection artifact (workflow log or session file). - [ ] Automated validation (CI or pre-merge check) must reject PRs that claim pick-next-issue provenance but do not include the provenance block. ### Acceptance Criteria (post-merge) - [ ] Nightly flow verifies that merged PRs containing the provenance block actually reference an existing pick-next-issue run log/session: (1) list merged PRs since last run that include the provenance block, (2) for each PR follow the provenance link and confirm the referenced run/log exists and contains the issue id, (3) emit count of validated merged PRs with provenance. ### Prerequisites for Autonomous Execution
2.Create a pipeline to auto-generate small, agent-friendly issues
### Summary -- What & Why Problem: many issues in the selection pool are large, underspecified, or require environment/context that prevents agents from completing mergeable PRs. Consequence: pick-next-issue selects items agents cannot finish, reducing merged-PR yield. ### Acceptance Criteria (pre-merge) - [ ] Repository contains a new set of at least 10 issues flagged as 'agent-ready' that describe a single, small observable change and include acceptance criteria that can be verified without third-party dashboards. - [ ] Each generated issue must have acceptance criteria that are measurable by the repo's CI/nightly checks (e.g., page content, API response, or unit test expectations). ### Acceptance Criteria (post-merge) - [ ] Nightly flow can enumerate the 'agent-ready' issue set and confirm at least 10 such issues exist with the required acceptance criteria format and no references to external-auth-required steps. ### Prerequisites for Autonomous Execution
3.Auto-create PR scaffolds for top-selected issues to lower friction to merge
### Summary -- What & Why Problem: even when pick-next-issue selects a suitable issue, opening an initial PR with a correct minimal change is an extra friction point that reduces merged PR rate. Consequence: valuable candidate work stalls in issue/open-PR transition and fewer auto-selected PRs reach merge. ### Acceptance Criteria (pre-merge) - [ ] When an issue is marked as selected by pick-next-issue, the system can open a new PR that references the issue and contains a minimal scaffold indicating intended change and selection provenance. - [ ] Any scaffolded PR must include acceptance criteria summary pulled from the issue and a checklist that maps observable outcomes to tests that CI/nightly can run. ### Acceptance Criteria (post-merge) - [ ] Nightly flow can locate scaffolded PRs, verify they contain the issue reference and provenance block, and confirm their CI/test status for at least one merged scaffolded PR. ### Prerequisites for Autonomous Execution
4.Add a lightweight selection-quality filter to increase mergeable picks
### Summary -- What & Why Problem: pick-next-issue sometimes chooses issues that are not mergeable by an autonomous agent (too big, flaky tests, or require unavailable credentials). Consequence: selection churn and low merge conversion from auto-selected issues. ### Acceptance Criteria (pre-merge) - [ ] There is an observable selection-quality field attached to candidate issues (e.g., high/medium/low) based on explicit, checkable criteria recorded on the issue. - [ ] pick-next-issue selection evidence shows that issues chosen for auto-pick come only from the high or medium quality subset when such a filter is enabled. ### Acceptance Criteria (post-merge) - [ ] Nightly flow can confirm that issues selected by pick-next-issue during the run reference a quality field and that selected issues in the run are from the allowed quality subset. ### Prerequisites for Autonomous Execution
variant-a
Issue count
2
AC linter pass rate
100%
2/2
Total tokens
5,633
Candidate issues (2)
1.Emit machine-readable selection metadata when pick-next runs
### Summary -- What & Why Agents and workflows currently rely on free-form console output to prove which issue was selected. That makes it hard for downstream automation to reliably attach selection provenance to PRs and to audit which PRs were opened from auto-selected issues. Without machine-readable selection evidence, we will miss or misattribute auto-selected PRs and fail this KR. ### Acceptance Criteria (pre-merge) - [ ] Running the selection command creates a machine-readable metadata file that contains an "issue_number" key and the human-readable run log contains the phrase "Handing off"; verify locally: bash ./pick-next-issue.sh > /tmp/pick.out 2>&1 || true && grep -q "Handing off" /tmp/pick.out && test -f .pick-next/selection.json && grep -q '"issue_number"' .pick-next/selection.json - [ ] The metadata file parses as JSON and the issue_number value is an integer; verify locally: node -e "const fs=require('fs'); const j=JSON.parse(fs.readFileSync('.pick-next/selection.json','utf8')); if(!Number.isInteger(j.issue_number)) {console.error('issue_number not integer'); process.exit(2);} console.log('ok')" ### Acceptance Criteria (post-merge) ### Prerequisites for Autonomous Execution2.Add lightweight PR-skeleton generator + PR-body validator for auto-picked issues
### Summary -- What & Why Many PRs created from auto-selected issues fail to get merged because they miss clear selection provenance and a minimal checklist that reviewers expect. Providing a small PR-skeleton generator and an automated PR-body validator increases the chance that agent-created PRs include the required evidence and pass initial review, improving merge rate for auto-selected issues. ### Acceptance Criteria (pre-merge) - [ ] The repository contains a generator script that, given an issue number, outputs a draft PR body containing explicit selection evidence; verify by running the generator and inspecting output: node scripts/generate-pr-skeleton.js 123 > /tmp/pr_body_123.md && test -f /tmp/pr_body_123.md && grep -q -E "Handing off|Selecting best candidate|issue #123" /tmp/pr_body_123.md || true - [ ] The repository contains a validator script that exits 0 when run against a PR body that includes selection evidence and exits non-zero otherwise; verify locally with the generated body: node scripts/validate_pr_body.js /tmp/pr_body_123.md ### Acceptance Criteria (post-merge) - [ ] Nightly verification: the nightly workflow can run the validator against any PR body saved in the workspace. Step-by-step: 1) Checkout main; 2) run node scripts/generate-pr-skeleton.js 999 > /tmp/nightly_pr_body.md; 3) run node scripts/validate_pr_body.js /tmp/nightly_pr_body.md and observe exit code 0. (This is runnable by the nightly job that can execute node scripts.) ### Prerequisites for Autonomous Execution