Skip to content

add logic to allow separate var including students who exit on frist day#246

Open
rlittle08 wants to merge 1 commit into
mainfrom
feature/config_first_day_exits
Open

add logic to allow separate var including students who exit on frist day#246
rlittle08 wants to merge 1 commit into
mainfrom
feature/config_first_day_exits

Conversation

@rlittle08
Copy link
Copy Markdown
Collaborator

Description & motivation

New config: edu:enroll:first_day_exit_date_inclusive

Background

Two existing vars interact when filtering out students who never really attended:

  • edu:enroll:exclude_exit_before_first_day (default: True) — drops enrollments where the student exited before the first school day
  • edu:enroll:exit_withdraw_date_inclusive (default: True) — controls whether exit dates are treated as inclusive (<=) or exclusive (<) throughout enrollment logic

When an implementation sets exit_withdraw_date_inclusive: False, the first-day exit check inherits that exclusivity — meaning a student who exited on the first day of school is treated the same as one who exited the day before and gets dropped.

The problem

exit_withdraw_date first_school_day exit_withdraw_date_inclusive included?
Aug 25 (first day) Aug 25 True ✅ kept
Aug 25 (first day) Aug 25 False ❌ dropped
Aug 24 (day before) Aug 25 True ❌ dropped
Aug 24 (day before) Aug 25 False ❌ dropped
Aug 26 (day after) Aug 25 True ✅ kept
Aug 26 (day after) Aug 25 False ✅ kept

Implementations using exclusive exit dates lose first-day attendees even though that's not the intent of the filter.

The fix

A new var edu:enroll:first_day_exit_date_inclusive controls inclusivity for only the first-day exit check. It defaults to whatever exit_withdraw_date_inclusive is set to (preserving existing behavior), but can be set independently.

exit_withdraw_date_inclusive first_day_exit_date_inclusive first-day exit result
True (default) not set (inherits True) ✅ kept
False not set (inherits False) ❌ dropped
False True ✅ kept

Usage

Implementations that use exclusive exit dates but want to retain students who exited on the first day can add to dbt_project.yml:

vars:
  'edu:enroll:exit_withdraw_date_inclusive': False
  'edu:enroll:first_day_exit_date_inclusive': True

Breaking changes introduced by this PR:

None

PR Merge Priority:

  • Low
  • Medium
  • High

Changes to existing files:

New files created:

Tests and QC done:

Ran in SC, did quick check --

select * from dev_analytics.dev_rl_wh.fct_student_school_association
where k_student not in (select k_student from analytics.prod_wh.fct_student_school_association)
-- uncomment this to prove that all the student's we're adding exited on the first day of school
-- and k_student not in (select k_student from analytics.prod_stage.stg_ef3__student_school_associations
--                         join analytics.prod_build.bld_ef3__school_calendar_windows
--                           using(k_school_calendar)
--                         where exit_withdraw_date = first_school_day 
--                         )
;

Needs deeper QC on wider warehouse impacts

edu_wh PR Review Checklist:

Make sure the following have been completed before approving this PR:

  • Description of changes has been added to Unreleased section of CHANGELOG.md. Add under ## New Features for features, etc.
  • Code has been tested/checked for Databricks and Snowflake compatibility - EA engineers see Databricks checklist here
  • Reviewer confirms the grain of all tables are unchanged, OR any changes are expected, communicated, and this PR is flagged as a breaking change (not for patch release)
  • If a new configuration xwalk was added:
    • The code is written such that the xwalk is optional (preferred), and this behavior was tested, OR
    • The code is written such that the xwalk is required, and the required xwalk is added to edu_project_template, and this PR is flagged as breaking change (not for patch release)
    • A description for the new xwalk has been added to EDU documentation site here
  • If a new configuration variable was added:
    • The code is written such that the variable is optional (preferred), and this behavior was tested, OR
    • The code is written such that the variable is required, and a default value was added to edu_project_template, and this PR is flagged as breaking change (not for patch release)
    • A description for the new variable has been added to EDU documentation site here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant