Skip to content

Backport of chore: Add worker check to bats tests setup into release/0.21.x#6610

Open
hc-github-team-secure-boundary wants to merge 1 commit into
release/0.21.xfrom
backport/moduli-bats-alias-flake/luckily-keen-ghoul
Open

Backport of chore: Add worker check to bats tests setup into release/0.21.x#6610
hc-github-team-secure-boundary wants to merge 1 commit into
release/0.21.xfrom
backport/moduli-bats-alias-flake/luckily-keen-ghoul

Conversation

@hc-github-team-secure-boundary
Copy link
Copy Markdown
Collaborator

Backport

This PR is auto-generated from #6421 to be assessed for backporting due to the inclusion of the label backport/0.21.x.

The below text is copied from the body of the original PR.


Description

This PR attempts to resolve some flaky bats tests, specifically the alias tests (those are the first to run due to it being first in the list).

alias.bats
   boundary/login: can login as admin user  1/361 ✓ boundary/login: can login as admin user
   boundary/alias: admin user cannot create alias with incorrectly ...  2/361 ✓ boundary/alias: admin user cannot create alias with incorrectly formatted destination id
   boundary/alias: admin user cannot create alias with incorrectly ...  3/361 ✓ boundary/alias: admin user cannot create alias with incorrectly formatted host id
   boundary/alias: admin user can create alias with host id  4/361 ✓ boundary/alias: admin user can create alias with host id
   boundary/alias: delete alias with host id  5/361 ✓ boundary/alias: delete alias with host id
   boundary/alias: admin user can create alias  6/361 ✓ boundary/alias: admin user can create alias
   boundary/alias: admin user can not create already created alias  7/361 ✓ boundary/alias: admin user can not create already created alias
   boundary/alias: admin user can list aliases  8/361 ✓ boundary/alias: admin user can list aliases
   boundary/alias: admin user can read created alias  9/361 ✓ boundary/alias: admin user can read created alias
   boundary/alias: admin user can read target using an alias 10/361 ✓ boundary/alias: admin user can read target using an alias
   boundary/alias: admin user can connect to target using an alias 11/361 ✗ boundary/alias: admin user can connect to target using an alias
   (in test file boundary/alias.bats, line 70)
     `[ "$status" -eq 0 ]' failed
   boundary/alias: admin user can connect passing in a host id 12/361 ✗ boundary/alias: admin user can connect passing in a host id
   (in test file boundary/alias.bats, line 75)
     `[ "$status" -eq 0 ]' failed
   boundary/alias: admin user can update alias host id 13/361 ✓ boundary/alias: admin user can update alias host id
   boundary/alias: admin user cannot update alias with incorrectly ... 14/361 ✓ boundary/alias: admin user cannot update alias with incorrectly formatted host id
   boundary/alias: admin user cannot update alias with incorrectly ... 15/361 ✓ boundary/alias: admin user cannot update alias with incorrectly formatted destination id
   boundary/alias: admin user can connect using an alias configured... 16/361 ✗ boundary/alias: admin user can connect using an alias configured with host id
   (in test file boundary/alias.bats, line 98)
     `[ "$status" -eq 0 ]' failed
   boundary/alias: admin user can connect using an alias configured... 17/361 ✗ boundary/alias: admin user can connect using an alias configured with host id passing in same host id
   (in test file boundary/alias.bats, line 103)
     `[ "$status" -eq 0 ]' failed
   boundary/host: admin can create new host 18/361 ✓ boundary/host: admin can create new host
   boundary/host: admin can associate new host to host set 19/361 ✓ boundary/host: admin can associate new host to host set
   boundary/alias: admin user cannot connect using an alias configu... 20/361 ✓ boundary/alias: admin user cannot connect using an alias configured with host id passing in different host id
   boundary/host: admin can delete new host 21/361 ✓ boundary/host: admin can delete new host
   boundary/alias: admin user can remove destination id and host id... 22/361 ✓ boundary/alias: admin user can remove destination id and host id will also be cleared out
   boundary/alias: admin user can update alias value 23/361 ✓ boundary/alias: admin user can update alias value
   boundary/alias: admin user can delete alias 24/361 ✓ boundary/alias: admin user can delete alias
   boundary/alias: can login as unpriv user 25/361 ✓ boundary/alias: can login as unpriv user
   boundary/alias: unpriv user cannot list aliases 26/361 ✓ boundary/alias: unpriv user cannot list aliases
   boundary/alias: unpriv user cannot read aliases 27/361 ✓ boundary/alias: unpriv user cannot read aliases
   boundary/alias: unpriv user cannot create aliases 28/361 ✓ boundary/alias: unpriv user cannot create aliases

The flaky tests are the ones where it tries to connect to a target via an alias. When trying to reproduce the error locally, it seems like it occurs if the worker hasn't finished initializing

# on one terminal, start
boundary dev

# on another terminal
cd boundary/internal/tests/cli
bats -p boundary/alias.bats

This PR adds a check to the script that runs these tests in CI to wait for the worker health status to be READY before proceeding. However, during testing, this wasn't sufficient. I'm not sure what else can be used to verify, so I've thrown in a manual sleep for now.

https://hashicorp.atlassian.net/browse/ICU-18167

PCI review checklist

  • I have documented a clear reason for, and description of, the change I am making.
  • If applicable, I've documented a plan to revert these changes if they require more than reverting the pull request.
  • If applicable, I've documented the impact of any changes to security controls.
    Examples of changes to security controls include using new access control methods, adding or removing logging pipelines, etc.

Overview of commits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants