Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
run: uv sync --group docs

- name: Build documentation
run: uv run mkdocs build --strict
run: uv run zensical build --strict

- name: Setup Pages
uses: actions/configure-pages@v4
Expand Down
57 changes: 56 additions & 1 deletion docs/concepts/materializers.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,15 @@ Materializers format and output calculated metrics. They transform raw metric da

## Available Materializers

CheckUp includes three built-in materializers:
CheckUp includes five built-in materializers:

| Materializer | Output | Use Case |
|-------------|--------|----------|
| `ConsoleMaterializer` | Terminal tables | Interactive viewing |
| `CSVMaterializer` | CSV file | Data analysis, spreadsheets |
| `HTMLMaterializer` | HTML report | Sharing, dashboards |
| `MarkdownMaterializer` | Markdown table on stdout | Pull request comments, CI job summaries |
| `SQLAlchemyMaterializer` | Database rows | Tracking metrics over time |

## Using Materializers

Expand Down Expand Up @@ -99,6 +101,59 @@ Features:
- Responsive design
- Color-coded values

## Markdown Materializer

Prints a GitHub-flavoured Markdown table to stdout. Redirect it into a pull request
comment or a CI job summary:

```python
from checkup import MarkdownMaterializer

result.materialize(MarkdownMaterializer())
```

The table holds one row per measurement, with the value column right-aligned:

```markdown
| Name | Description | Value | Unit | Diagnostics |
| --- | --- | ---: | --- | --- |
| file_count | Number of files | 42 | files | Found 42 files |
```

Pipes in a value become `\|` and newlines become `<br>`, so a diagnostic that spans
several lines still renders as one row.

## SQLAlchemy Materializer

Writes measurements as rows to a database table, creating the table when it does not
exist. Each run appends rows and stamps them with `measured_at`, which makes it the
option to reach for when tracking metrics over time:

```python
from checkup import SQLAlchemyMaterializer

result.materialize(
SQLAlchemyMaterializer(
connection_url="sqlite:///metrics.db",
table_name="metrics",
)
)
```

Any database SQLAlchemy supports works through the connection URL, including
PostgreSQL and MySQL. The default table holds these columns:

`name`, `value`, `unit`, `diagnostic`, `description`, `tags`, `measured_at`

| Argument | Default | Description |
|----------|---------|-------------|
| `connection_url` | required | SQLAlchemy connection URL, stored as a secret |
| `table_name` | `"metrics"` | Target table |
| `table_schema` | `None` | Database schema, such as `analytics` |
| `connect_args` | `None` | Arguments passed to the underlying driver |
| `expand_tags` | `False` | Write each tag to its own `tag_<key>` column instead of one JSON `tags` column |
| `batch_size` | `1000` | Rows per insert batch |

## Multiple Outputs

You can materialize to multiple formats:
Expand Down
56 changes: 30 additions & 26 deletions docs/concepts/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ from checkup.types import Context

class MyMetric(Metric):
# Required class attributes
name = "my_metric"
description = "Description of what this metric measures"
unit = "count"
name: str = "my_metric"
description: str = "Description of what this metric measures"
unit: str = "count"

def calculate(self, context: Context, measurements: dict) -> Measurement:
# Your calculation logic here
Expand Down Expand Up @@ -53,7 +53,7 @@ return self.measure(value=42, diagnostic="Explanation", tags={"key": "value"})

## The Calculate Method

The `calculate` method is where you implement your metric logic:
The `calculate` method holds your metric logic:

```python
def calculate(self, context: Context, measurements: dict) -> Measurement:
Expand All @@ -77,18 +77,18 @@ Metrics can depend on other metrics. The framework ensures dependencies are calc

```python
class BaseMetric(Metric):
name = "base_metric"
description = "A base metric"
unit = "count"
name: str = "base_metric"
description: str = "A base metric"
unit: str = "count"

def calculate(self, context: Context, measurements: dict) -> Measurement:
return self.measure(value=100)


class DerivedMetric(Metric):
name = "derived_metric"
description = "Depends on base metric"
unit = "percent"
name: str = "derived_metric"
description: str = "Depends on base metric"
unit: str = "percent"

@classmethod
def depends_on(cls) -> list[type[Metric]]:
Expand All @@ -113,9 +113,9 @@ class MyDataProvider(Provider):


class MyMetric(Metric):
name = "my_metric"
description = "Uses my_data provider"
unit = "items"
name: str = "my_metric"
description: str = "Uses my_data provider"
unit: str = "items"

@classmethod
def providers(cls) -> list[type[Provider]]:
Expand All @@ -132,22 +132,24 @@ class MyMetric(Metric):
Metrics can specify how they should be executed:

```python
from typing import ClassVar

from checkup import ExecutorType


class IOBoundMetric(Metric):
name = "io_metric"
executor = ExecutorType.THREAD # Default, for I/O-bound operations
name: str = "io_metric"
executor: ClassVar[ExecutorType] = ExecutorType.THREAD # Default, for I/O-bound operations


class CPUBoundMetric(Metric):
name = "cpu_metric"
executor = ExecutorType.PROCESS # For CPU-intensive calculations
name: str = "cpu_metric"
executor: ClassVar[ExecutorType] = ExecutorType.PROCESS # For CPU-intensive calculations


class AsyncMetric(Metric):
name = "async_metric"
executor = ExecutorType.ASYNCIO # For async I/O operations
name: str = "async_metric"
executor: ClassVar[ExecutorType] = ExecutorType.ASYNCIO # For async I/O operations
```

| Executor | Best For | Notes |
Expand All @@ -162,9 +164,9 @@ Tags allow grouping and filtering metrics. Tags can be set when creating the mea

```python
class TaggedMetric(Metric):
name = "tagged_metric"
description = "A metric with tags"
unit = "count"
name: str = "tagged_metric"
description: str = "A metric with tags"
unit: str = "count"

def calculate(self, context: Context, measurements: dict) -> Measurement:
return self.measure(
Expand Down Expand Up @@ -195,15 +197,17 @@ result.materialize(
## Example: Complete Metric

```python
from typing import ClassVar

from checkup import Metric, Provider, ExecutorType, Measurement
from checkup.types import Context


class CodeCoverageMetric(Metric):
name = "code_coverage"
description = "Percentage of code covered by tests"
unit = "percent"
executor = ExecutorType.THREAD
name: str = "code_coverage"
description: str = "Percentage of code covered by tests"
unit: str = "percent"
executor: ClassVar[ExecutorType] = ExecutorType.THREAD

@classmethod
def providers(cls) -> list[type[Provider]]:
Expand Down
30 changes: 17 additions & 13 deletions docs/concepts/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

CheckUp is built around three core concepts that work together to calculate and output metrics.

## The Three Pillars
## Providers, metrics, and materializers

```
┌─────────────┐ ┌─────────────┐ ┌──────────────┐
Expand Down Expand Up @@ -52,6 +52,8 @@ Available formats:
- Console (rich tables)
- CSV files
- HTML reports
- Markdown tables
- Database rows, through SQLAlchemy

## The CheckHub Orchestrator

Expand All @@ -75,46 +77,48 @@ result.materialize(ConsoleMaterializer(...))

## Execution Flow

1. **Registration**: Metrics and provider sets are registered with CheckHub
1. **Registration**: CheckHub registers the metrics and provider sets
2. **Dependency Resolution**: The framework builds a dependency graph and determines execution order
3. **Validation**: Provider requirements are validated against available providers
3. **Validation**: The framework validates provider requirements against the available providers
4. **Provider Execution**: Each provider set runs to enrich context
5. **Metric Calculation**: Metrics are calculated in dependency order
6. **Result Aggregation**: Results from all provider sets are combined
7. **Materialization**: Results are output in the desired format
5. **Metric Calculation**: The framework calculates metrics in dependency order
6. **Result Aggregation**: CheckHub combines the results from all provider sets
7. **Materialization**: The materializer writes the results in the format you chose

## Multi-Context Execution

CheckUp supports running metrics across multiple contexts (provider sets):

```python
CheckHub()
(
CheckHub()
.with_metrics([RepoMetric])
.with_providers([
[GitProvider(repo="repo-a")],
[GitProvider(repo="repo-b")],
[GitProvider(repo="repo-c")],
])
.measure()
)
```

This calculates the same metrics for each repository, enabling comparative analysis.

## Parallel Execution

Metrics are calculated in parallel using a process pool:
CheckUp calculates metrics in parallel using a process pool:

- Provider sets are executed concurrently
- CheckHub runs provider sets concurrently
- Within each context, metrics respect dependency order
- CPU-bound metrics can use the `PROCESS` executor
- I/O-bound metrics can use the `THREAD` executor
- Async metrics can use the `ASYNCIO` executor

## Error Handling

CheckUp provides robust error handling:
CheckUp reports failures instead of hiding them:

- Provider failures are captured and reported
- Individual context failures don't stop other contexts
- CheckUp captures and reports provider failures
- Each context fails independently, so the others keep running
- `MeasurementResult.errors` contains all failures
- Detailed logging is available for debugging
- Logging is available for debugging
8 changes: 6 additions & 2 deletions docs/concepts/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,21 +79,24 @@ class DatabaseProvider(Provider):
Use providers with different configurations:

```python
CheckHub()
(
CheckHub()
.with_metrics([MyMetric])
.with_providers([
[DatabaseProvider("postgres://prod/db")],
[DatabaseProvider("postgres://staging/db")],
])
.measure()
)
```

## Provider Sets

Provider sets allow running the same metrics against different contexts:

```python
CheckHub()
(
CheckHub()
.with_metrics([RepoMetrics])
.with_providers([
# Each inner list is a "provider set"
Expand All @@ -102,6 +105,7 @@ CheckHub()
[GitProvider(repo="repo-a"), EnvProvider(env="staging")],
])
.measure()
)
```

Each provider set runs independently, creating separate context for metrics.
Expand Down
Loading
Loading