Skip to content

Suggest dbt tests in print_profile_schema based on profile results #115

Description

@stumelius

Overview

print_profile_schema already generates a schema.yml skeleton with profile metrics stored in meta. It could go further and emit dbt test suggestions based on what the profile reveals — turning a profile run into a data contract bootstrap tool.

Motivation

Today print_profile_schema outputs:

columns:
  - name: id
    meta:
      is_unique: true
      not_null_proportion: 1.0
      distinct_count: 1000

A user then has to manually translate this into tests. The mapping is mechanical and well-defined, so the macro can do it automatically.

Suggested rules

Profile result Suggested test
is_unique = true - unique
not_null_proportion = 1.0 - not_null
distinct_count <= N (configurable, e.g. 20) - accepted_values: values: [...] populated from the actual values (requires value frequency feature)
min and max present - dbt_utils.expression_is_true: expression: "col between {min} and {max}"

API design

New optional parameter on print_profile_schema:

dbt_profiler.print_profile_schema(
    relation_name="my_model",
    suggest_tests=true,          -- off by default to avoid breaking existing usage
    max_values_for_accepted=20   -- threshold for emitting accepted_values test
)

Notes

  • The accepted_values suggestion depends on the value frequency feature (Question: add TOP x column values and distribution. #65 / a future issue) to know the actual distinct values
  • Suggestions should be clearly marked as suggestions (e.g. a comment in the YAML) so users know to review them before committing
  • This makes dbt-profiler a lightweight contract bootstrap tool: profile once → get a tested schema.yml ready to review

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions