Skip to content

Intermittent Instability Issues with Go Testcontainers #2

Description

@yulmwu

It seems that either the Go testcontainers package or the GitHub Actions runner environment is unstable. We have observed intermittent errors related to PostgreSQL containers.

Originally posted by @yulmwu in 7e11011


When using Go testcontainers, including in SMCTF, we have observed intermittent issues with database connections as shown below. (This can also occur during connection termination.)

=== RUN   TestIsUniqueViolation
[bun]  03:04:25.935   CREATE TABLE            384µs  CREATE TABLE IF NOT EXISTS "users" ("id" BIGSERIAL NOT NULL, "email" VARCHAR NOT NULL, "username" VARCHAR NOT NULL, "password_hash" VARCHAR NOT NULL, "role" VARCHAR NOT NULL, "blocked_reason" VARCHAR, "blocked_at" TIMESTAMPTZ, "created_at" TIMESTAMPTZ NOT NULL DEFAULT current_timestamp, "updated_at" TIMESTAMPTZ NOT NULL DEFAULT current_timestamp, PRIMARY KEY ("id"), UNIQUE ("email"), UNIQUE ("username")) 	  *net.OpError: read tcp [::1]:42624->[::1]:32769: read: connection reset by peer 
    errors_test.go:22: auto migrate: auto migrate create table *models.User: read tcp [::1]:42624->[::1]:32769: read: connection reset by peer
--- FAIL: TestIsUniqueViolation (0.00s)
=== RUN   TestNewAndAutoMigrate
[bun]  03:04:25.936   CREATE TABLE            485µs  CREATE TABLE IF NOT EXISTS "users" ("id" BIGSERIAL NOT NULL, "email" VARCHAR NOT NULL, "username" VARCHAR NOT NULL, "password_hash" VARCHAR NOT NULL, "role" VARCHAR NOT NULL, "blocked_reason" VARCHAR, "blocked_at" TIMESTAMPTZ, "created_at" TIMESTAMPTZ NOT NULL DEFAULT current_timestamp, "updated_at" TIMESTAMPTZ NOT NULL DEFAULT current_timestamp, PRIMARY KEY ("id"), UNIQUE ("email"), UNIQUE ("username")) 	  *net.OpError: read tcp [::1]:42626->[::1]:32769: read: connection reset by peer 
    testenv_test.go:117: auto migrate: auto migrate create table *models.User: read tcp [::1]:42626->[::1]:32769: read: connection reset by peer
--- FAIL: TestNewAndAutoMigrate (0.00s)
=== RUN   TestEnsureIndexes
[bun]  03:04:25.936   CREATE TABLE            654µs  CREATE TABLE IF NOT EXISTS "users" ("id" BIGSERIAL NOT NULL, "email" VARCHAR NOT NULL, "username" VARCHAR NOT NULL, "password_hash" VARCHAR NOT NULL, "role" VARCHAR NOT NULL, "blocked_reason" VARCHAR, "blocked_at" TIMESTAMPTZ, "created_at" TIMESTAMPTZ NOT NULL DEFAULT current_timestamp, "updated_at" TIMESTAMPTZ NOT NULL DEFAULT current_timestamp, PRIMARY KEY ("id"), UNIQUE ("email"), UNIQUE ("username")) 	  *net.OpError: read tcp [::1]:42636->[::1]:32769: read: connection reset by peer 
    testenv_test.go:138: auto migrate: auto migrate create table *models.User: read tcp [::1]:42636->[::1]:32769: read: connection reset by peer
--- FAIL: TestEnsureIndexes (0.00s)
=== RUN   TestCreateTables
[bun]  03:04:25.937   CREATE TABLE            368µs  CREATE TABLE IF NOT EXISTS "users" ("id" BIGSERIAL NOT NULL, "email" VARCHAR NOT NULL, "username" VARCHAR NOT NULL, "password_hash" VARCHAR NOT NULL, "role" VARCHAR NOT NULL, "blocked_reason" VARCHAR, "blocked_at" TIMESTAMPTZ, "created_at" TIMESTAMPTZ NOT NULL DEFAULT current_timestamp, "updated_at" TIMESTAMPTZ NOT NULL DEFAULT current_timestamp, PRIMARY KEY ("id"), UNIQUE ("email"), UNIQUE ("username")) 	  *net.OpError: read tcp [::1]:42650->[::1]:32769: read: connection reset by peer 
    testenv_test.go:170: auto migrate: auto migrate create table *models.User: read tcp [::1]:42650->[::1]:32769: read: connection reset by peer
--- FAIL: TestCreateTables (0.00s)
FAIL

The following potential causes can be considered:

  1. Container readiness issues
  2. Resource limitations of the GitHub Actions runner

For case 1, configuring a wait strategy such as ForListeningPort can be considered. For case 2, it is deemed less likely since tests are currently executed sequentially (-p 1).

Metadata

Metadata

Assignees

Labels

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions