Add bash test suite and CI job#11
Merged
Merged
Conversation
| done | ||
|
|
||
| echo "--- agentskills.io skills ---" | ||
| python3 .github/validate_skills.py |
Owner
There was a problem hiding this comment.
It seems like validate_skills.py should be in tests/
Top-level ./test.bash dispatcher with --lint / --unit / --secrets / --all, mirroring the layout in autonomous-agent-bootstrap so "works locally" == "passes CI". The CI workflow rewrites into three jobs (lint, unit, secret-scan) each delegating to one ./test.bash flag. Test layout (tests/*.bats, bats-driven): - arg-validation.bats — each skill script rejects bad input before any gh or network call; verified for all four scripts. - git-sync.bats — fast-forward sync against synthetic upstream/fork bare repos built by make_contributor_fixture in tests/lib.bash. Also covers --prune against ff-ancestor / patch-equivalent / novel / current / synced branches. - git-create.bats, git-fork.bats, git-clone.bats — full end-to-end coverage. A stub `gh` at tests/stubs/gh handles the handful of subcommands the scripts invoke (api user, repo create / fork / edit / view) against a local bare-repo root, and a global url.<root>/.insteadOf https://github.com/ rewrite makes real `git clone` / `git push` hit that root. Each test runs the real script with no auth or network, asserts final repo/remote state, and verifies gh was called with the expected arguments via a per-run STUB_GH_LOG. Fixes brycelelbach#5.
robobryce
force-pushed
the
issue-5-tests
branch
from
88d3118 to
b45d67a
Compare
…g validation After PR#7 (--prune), brycelelbach#9 (auto-merge / update-branch defaults), and brycelelbach#10 (dependabot + CodeQL security endpoints) landed, the test suite was silently under-covering the new code paths. Fix the gaps and audit for the rest. Stub: - tests/stubs/gh now handles `gh api --silent --method <V> /path [-f k=v ...]` as a logged no-op. A new STUB_GH_API_FAIL env var makes the stub fail for any api path containing a given substring, letting tests exercise the `|| echo warning` branch in security-settings. New test cases: - git-create: asserts all three security endpoints are called with the expected paths; honours [security] overrides in ~/.agitentic; tolerates a failing endpoint without failing the run. - git-fork: asserts all three security endpoints are called against the fork slug; short-circuit path asserts no security calls either. - git-sync: already-synced no-op; refuses divergent commits without --force; --force resets and force-pushes to fork. - arg-validation: git-create rejects no-args and empty <name>; git-fork and git-clone reject no-args. Also extends the "default repo settings" assertions to cover the two PR#9 keys (enable-auto-merge, allow-update-branch), and drops the now-unnecessary --prune soft-skip since PR#7 is on main. Suite is 39/39 green.
Walked each script top-to-bottom and added a test for every remaining reachable branch. arg-validation.bats (+6): - git-fork: too-many-args, --help. - git-clone: too-many-args, --help. - git-sync: --help. git-create.bats (+3): - AGITENTIC_CONFIG env var redirects config lookup. - git not on PATH → 'git is required' exit 1. - gh not on PATH → 'gh (GitHub CLI) is required' exit 1. git-fork.bats (+7): - ssh:// URL form, http:// URL form, trailing-slash slug stripping. - [repo] overrides apply to the fork. - [security] overrides apply to the fork. - security endpoint failure is non-fatal. - git-missing / gh-missing guards. git-clone.bats (+5): - https URL form, ssh URL form. - Empty [account] defaults to gh user. - git-missing / gh-missing guards. git-sync.bats (+8): - git-missing guard. - Not inside a git repo. - Missing upstream / missing fork remote. - Uncommitted changes rejection. - Creates local branch when it didn't exist. - Syncs main while checked out on a different branch (update-ref). - --branch <name> syncs a non-default branch. Supporting change to tests/lib.bash: setup_restricted_paths builds two minimal PATH sandbox directories (coreutils only / coreutils + git) so the `command -v git` / `command -v gh` guards can be tested even on hosts where the real git/gh live on the default PATH. Suite is now 68/68 green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a small bash test harness under
tests/plus a GitHub Actionstestjob. All tests run withoutghauth or any network access — matching the constraint in #5.Included tests:
test-git-sync.sh— builds syntheticupstream+forkbare repos with a real git clone in between, runsgit-sync, asserts localmainandfork/mainboth end up atupstream/main.test-git-sync-prune.sh— same fixture plus ff-ancestor, patch-equivalent (cherry-picked onto a different base), novel, and current branches. Asserts the first two are pruned and the rest are kept. Self-skips whengit-syncdoesn't implement--prune, so this stays green onmainuntil that change lands.test-arg-validation.sh— feeds each script the error-path inputs that are rejected before anyghcall (bad slug,owner/name-style name, unknown flag, etc.) and checks both exit code and the expected stderr message.tests/run-allis a simple runner. Each test sets up its own tmpdir and cleans up on exit.CI changes:
testjob runstests/run-all.lintjob now alsoshellcheck --source-path=SCRIPTDIR -xovertests/so the source directives fortests/lib.shresolve.Fixes #5.
Test plan
tests/run-allpasses locally — 3/3 tests pass (prune test self-skips on main, which is expected).shellcheck --source-path=SCRIPTDIR -xclean on every script underplugins/andtests/.python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml'))"succeeds.