What happened
On PR #6525, the code agent (redhat-chai-bot) opened a PR on Jul 15 with backend fixes and pytest tests. Human contributor LiZhang19817 identified a gap on Jul 16: the CI e2e test coverage check failed because the PR changed source files but included no Playwright E2E tests. LiZhang19817 prepared a commit adding 5 E2E scenarios to api-v1-readonly-superuser.spec.ts.
The Playwright tests were incorporated into the branch (CodeRabbit reviewed them on Jul 16 06:14 with 2 nitpicks). However, between Jul 16 14:07 and 16:09, the code agent performed three force pushes. During this sequence, the human-contributed Playwright E2E tests were silently dropped — the area/web-ui label was removed, confirming the web files left the diff. The tests never returned.
The PR went through 8 total force pushes over 6 days, dismissing human approvals twice (sridipta approved 3 times before the final approval stuck). It merged on Jul 21 without the Playwright tests. The cherry-pick PR #6587 inherited the same gap.
The agents repo was discovered as fullsend-ai/agents@ca518d9353f5 from retro run 29800267408.
What could go better
The code/fix agent should never silently discard changes contributed by humans. When the agent rebases or squashes commits on a branch, it should detect commits authored by others and ensure their changes are preserved in the resulting commit.
I am fairly confident this is an agent-layer issue rather than a one-off human error, because: (1) all 8 force pushes were performed by redhat-chai-bot, not a human; (2) the pattern of squash-rebasing to a single commit is consistent with typical code/fix agent post-scripts; (3) the dropped content was an entire test file addition, not a merge conflict resolution.
I am less certain about the exact mechanism — the agent may have been resetting to a base commit and replaying only its own changes, or it may have been doing a squash that lost non-agent commits. The investigation could not determine which because the intermediate commits are no longer available after the force pushes.
Related but distinct issues: fullsend-ai/agents#144 proposes detecting unrelated content removals as rebase artifacts (review-side detection), and fullsend-ai/fullsend#3938 proposes guarding against force-push overwrites on bot-managed branches. Neither addresses the prevention of human contribution loss during the agent's own rebase operations.
Proposed change
Add a pre-push or post-rebase validation step to the code/fix agent's workflow (likely in the post-script or harness config in fullsend-ai/agents) that:
- Detects human contributions: Before performing a force-push, list commits on the branch that are NOT authored by the agent bot account. Track which files were added or modified by those commits.
- Validates preservation: After the rebase/squash, verify that all files touched by human contributors are still present in the resulting diff. If any human-contributed files are missing, abort the push and log a warning.
- Surfaces the conflict: If preservation fails, the agent should comment on the PR explaining which human-contributed changes could not be preserved, rather than silently dropping them.
An alternative lighter-weight approach: modify the agent's rebase strategy to use git rebase with commit preservation rather than squash-and-force-push when the branch contains commits from multiple authors. This would naturally preserve human contributions.
The specific files to modify depend on the code/fix agent's architecture in fullsend-ai/agents — likely the post-code or post-fix script, or the agent definition's instructions about git operations.
Validation criteria
- On the next 5 PRs where a human contributes commits to an agent-authored branch, the agent's subsequent force-pushes should preserve those contributions (no files silently dropped).
- If the agent cannot preserve a human contribution (e.g., due to a genuine conflict), it should post a comment on the PR explaining what was lost and why.
- Regression test: simulate a scenario where a human adds a file to an agent branch, then trigger a fix/rebase cycle, and verify the file survives.
Generated by retro agent from quay/quay#6587
What happened
On PR #6525, the code agent (
redhat-chai-bot) opened a PR on Jul 15 with backend fixes and pytest tests. Human contributorLiZhang19817identified a gap on Jul 16: the CIe2e test coverage checkfailed because the PR changed source files but included no Playwright E2E tests. LiZhang19817 prepared a commit adding 5 E2E scenarios toapi-v1-readonly-superuser.spec.ts.The Playwright tests were incorporated into the branch (CodeRabbit reviewed them on Jul 16 06:14 with 2 nitpicks). However, between Jul 16 14:07 and 16:09, the code agent performed three force pushes. During this sequence, the human-contributed Playwright E2E tests were silently dropped — the
area/web-uilabel was removed, confirming the web files left the diff. The tests never returned.The PR went through 8 total force pushes over 6 days, dismissing human approvals twice (sridipta approved 3 times before the final approval stuck). It merged on Jul 21 without the Playwright tests. The cherry-pick PR #6587 inherited the same gap.
The agents repo was discovered as
fullsend-ai/agents@ca518d9353f5from retro run 29800267408.What could go better
The code/fix agent should never silently discard changes contributed by humans. When the agent rebases or squashes commits on a branch, it should detect commits authored by others and ensure their changes are preserved in the resulting commit.
I am fairly confident this is an agent-layer issue rather than a one-off human error, because: (1) all 8 force pushes were performed by
redhat-chai-bot, not a human; (2) the pattern of squash-rebasing to a single commit is consistent with typical code/fix agent post-scripts; (3) the dropped content was an entire test file addition, not a merge conflict resolution.I am less certain about the exact mechanism — the agent may have been resetting to a base commit and replaying only its own changes, or it may have been doing a squash that lost non-agent commits. The investigation could not determine which because the intermediate commits are no longer available after the force pushes.
Related but distinct issues: fullsend-ai/agents#144 proposes detecting unrelated content removals as rebase artifacts (review-side detection), and fullsend-ai/fullsend#3938 proposes guarding against force-push overwrites on bot-managed branches. Neither addresses the prevention of human contribution loss during the agent's own rebase operations.
Proposed change
Add a pre-push or post-rebase validation step to the code/fix agent's workflow (likely in the post-script or harness config in
fullsend-ai/agents) that:An alternative lighter-weight approach: modify the agent's rebase strategy to use
git rebasewith commit preservation rather than squash-and-force-push when the branch contains commits from multiple authors. This would naturally preserve human contributions.The specific files to modify depend on the code/fix agent's architecture in
fullsend-ai/agents— likely the post-code or post-fix script, or the agent definition's instructions about git operations.Validation criteria
Generated by retro agent from quay/quay#6587