feat: add comprehensive testing infrastructure with pytest

[llbbl]

Add Comprehensive Testing Infrastructure

Summary

This PR establishes a complete testing infrastructure for the aws-sso-util project using pytest and Poetry. The setup provides a foundation for writing unit tests, integration tests, and maintaining code coverage standards.

Changes Made

Testing Dependencies

• Added pytest (^7.4.0) - Core testing framework
• Added pytest-cov (^4.1.0) - Coverage reporting plugin
• Added pytest-mock (^3.11.1) - Mocking utilities

Configuration

• pytest configuration in pyproject.toml:

  • Test discovery patterns for test_*.py and *_test.py files
  • Custom markers: unit, integration, and slow
  • Coverage reporting with HTML and XML output
  • Verbose output with strict marker enforcement

• Coverage configuration:

  • Source coverage for aws_sso_util and aws_sso_lib
  • 80% coverage threshold (configurable)
  • Exclusions for test files, __init__.py, and vendored code
  • HTML reports in htmlcov/ directory

Directory Structure

cli/
├── tests/
│   ├── __init__.py
│   ├── conftest.py          # Shared fixtures for CLI tests
│   ├── test_setup_validation.py  # Validation tests
│   ├── unit/
│   │   └── __init__.py
│   └── integration/
│       └── __init__.py

lib/
├── tests/
│   ├── __init__.py
│   ├── conftest.py          # Shared fixtures for lib tests
│   ├── unit/
│   │   └── __init__.py
│   └── integration/
│       └── __init__.py

Shared Fixtures

Both conftest.py files provide common fixtures including:

• temp_dir - Temporary directory management
• mock_aws_config_dir / mock_aws_config - AWS configuration mocking
• mock_sso_cache_dir / mock_sso_cache - SSO cache mocking
• mock_boto3_client / mock_boto3_session - AWS API mocking
• mock_environment - Environment variable mocking
• sample_yaml_config / sample_sso_token - Test data fixtures
• mock_click_context - CLI testing utilities
• capture_logs / capture_stdout - Output capture fixtures

Poetry Commands

• poetry run test - Run all tests with coverage
• poetry run tests - Alternative command (both work)

Updated Files

• .gitignore - Added testing artifacts and Claude directories
• Migrated from deprecated [tool.poetry.dev-dependencies] to [tool.poetry.group.dev.dependencies]

How to Use

1. Install dependencies:

   cd cli && poetry install --with dev
   cd ../lib && poetry install --with dev

2. Run tests:

   # From cli directory
   poetry run test
   
   # Or with specific markers
   poetry run pytest -m unit
   poetry run pytest -m integration

3. View coverage report:

   # HTML report will be in htmlcov/index.html
   open htmlcov/index.html

Next Steps

With this infrastructure in place, developers can now:

1. Write unit tests for individual functions and classes
2. Create integration tests for end-to-end workflows
3. Mock AWS services effectively using provided fixtures
4. Maintain code coverage standards
5. Run tests in CI/CD pipelines using the XML coverage output

Notes

• The validation tests (test_setup_validation.py) verify the infrastructure is working correctly
• Coverage warnings for "no data collected" are expected until actual tests are written
• The 80% coverage threshold is configured but can be adjusted in pyproject.toml
• All pytest CLI options remain available when using the Poetry command

[iainelder]

@Libbl, thank you for the submission! I'm on vacation now, and I'll review this in July.

A better test infrastructure will help us to maintain the project and have confidence that everything still works as it should.

[llbbl]

Hey, you’re welcome. Let me know if you want a more unified testing approach and see if I can get it set up at the project root instead.

[iainelder]

@llbbl , I'm sorry I didn't get back to you sooner. My vacation was actually parental leave. I became a dad 1 month before you submitted this pull request and since then my time for open source work dropped to zero.

Found a moment to look at it again. I'll run your instructions and provide feedback.

[iainelder]

I ran the tests. They all pass!

But the coverage report failed: FAIL Required test coverage of 80.0% not reached. Total coverage: 0.00%

Here's the full output. Have I missed a step? Is there a missing dependency?

$ poetry run test
=================================== test session starts ===================================
platform linux -- Python 3.10.15, pytest-7.4.4, pluggy-1.6.0 -- /home/isme/.cache/pypoetry/virtualenvs/aws-sso-util-vy2FGYx7-py3.10/bin/python
cachedir: .pytest_cache
rootdir: /tmp/dir.Vm8/aws-sso-util/cli
configfile: pyproject.toml
testpaths: tests
plugins: mock-3.14.1, cov-4.1.0
collected 13 items                                                                        

tests/test_setup_validation.py::TestSetupValidation::test_pytest_installed PASSED   [  7%]
tests/test_setup_validation.py::TestSetupValidation::test_pytest_cov_installed PASSED [ 15%]
tests/test_setup_validation.py::TestSetupValidation::test_pytest_mock_installed PASSED [ 23%]
tests/test_setup_validation.py::TestSetupValidation::test_project_structure PASSED  [ 30%]
tests/test_setup_validation.py::TestSetupValidation::test_conftest_fixtures PASSED  [ 38%]
tests/test_setup_validation.py::TestSetupValidation::test_aws_config_fixture PASSED [ 46%]
tests/test_setup_validation.py::TestSetupValidation::test_yaml_config_fixture PASSED [ 53%]
tests/test_setup_validation.py::TestSetupValidation::test_mock_boto3_fixture PASSED [ 61%]
tests/test_setup_validation.py::TestSetupValidation::test_click_runner_fixture PASSED [ 69%]
tests/test_setup_validation.py::TestSetupValidation::test_log_capture_fixture PASSED [ 76%]
tests/test_setup_validation.py::TestSetupValidation::test_markers_defined PASSED    [ 84%]
tests/test_setup_validation.py::TestIntegrationSetup::test_integration_marker PASSED [ 92%]
tests/test_setup_validation.py::TestIntegrationSetup::test_slow_marker PASSED       [100%]/home/isme/.cache/pypoetry/virtualenvs/aws-sso-util-vy2FGYx7-py3.10/lib/python3.10/site-packages/coverage/inorout.py:509: CoverageWarning: Module aws_sso_util was never imported. (module-not-imported)
  self.warn(f"Module {pkg} was never imported.", slug="module-not-imported")
/home/isme/.cache/pypoetry/virtualenvs/aws-sso-util-vy2FGYx7-py3.10/lib/python3.10/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
  self._warn("No data was collected.", slug="no-data-collected")
WARNING: Failed to generate report: No data to report.

/home/isme/.cache/pypoetry/virtualenvs/aws-sso-util-vy2FGYx7-py3.10/lib/python3.10/site-packages/pytest_cov/plugin.py:312: CovReportWarning: Failed to generate report: No data to report.

  warnings.warn(CovReportWarning(message))


---------- coverage: platform linux, python 3.10.15-final-0 ----------

FAIL Required test coverage of 80.0% not reached. Total coverage: 0.00%

=================================== 13 passed in 0.64s ====================================

I also get an error when I try to open the coverage report: It's missing.

$ open htmlcov/index.html
gio: file:///tmp/dir.Vm8/aws-sso-util/cli/htmlcov/index.html: Error when getting information for file “/tmp/dir.Vm8/aws-sso-util/cli/htmlcov/index.html”: No such file or directory

A minor issue: The first run of poetry install in the cli folder failed because poetry.lock was out of date. I ran poetry lock to update it and the second run worked.

$ cd cli && poetry install --with dev
Creating virtualenv aws-sso-util-vy2FGYx7-py3.10 in /home/isme/.cache/pypoetry/virtualenvs
Installing dependencies from lock file

pyproject.toml changed significantly since poetry.lock was last generated. Run `poetry lock` to fix the lock file.

[iainelder]

@llbbl , I realize it's been a long time, and you might have moved on to something else. If you're still able to contribute, I'd appreciate your advice to solve the problem with the code coverage report.

[llbbl]

(i think) It's this part here in the poetry file:

fail_under = 80

The template that I was using to setup the poetry file had that set. It's saying fail test coverage if coverage is less than 80%. And because this pull request is just setting up unit testing, we don't have any test coverage.

simple fix is comment out for now.