Skip to content

Feature/sedm clean#244

Open
ryanaguilar wants to merge 8 commits into
mainfrom
feature/sedm-clean
Open

Feature/sedm clean#244
ryanaguilar wants to merge 8 commits into
mainfrom
feature/sedm-clean

Conversation

@ryanaguilar
Copy link
Copy Markdown
Contributor

@ryanaguilar ryanaguilar commented May 15, 2026

feature/sedm: Add SEDM warehouse facts for student IEPs, IDEA events, and IEP disability lines

Description & motivation

This PR adds warehouse dbt models for the SEDM (Special Education Data Model) extension so analysts can use SEDM IEP and IDEA event data alongside existing core special-ed facts. Staging for these resources lives in edu_edfi_source (stg_sedm__student_ieps, stg_sedm__idea_events, stg_sedm__student_iep_disability_collections); this work materializes the corresponding facts in edu_wh under models/sedm_warehouse/, with inner joins to dim_student so k_student and k_student_xyear match the warehouse student dimension (same pattern as other student-centric facts).
Motivation: to support local education agencies' ability to support students with IEPs and facilitate mandated reporting requirements.

Breaking changes introduced by this PR:

  • This is a breaking change. The following sources and corresponding raw tables must be added:
  - name: idea_events
    description: >

    enabled: true
    columns: *column_defaults
  - name: student_ieps
    description: >

    enabled: true
    columns: *column_defaults
  - name: student_iep_disability_collections
    description: >

    enabled: true
    columns: *column_defaults
  • Models are intended to be gated by edu:sedm:enabled (default False) via dbt_project.yml once those configs are committed.

PR Merge Priority:

  • Low
  • Medium
  • High

Changes to existing files:

CHANGELOG entry under ## New Features.

New files created:

  • models/sedm_warehouse/fct_student_iep.sql: One row per student IEP from stg_sedm__student_ieps, keyed by k_student_iep. Resolves k_student / k_student_xyear from dim_student for referential integrity. Surrogate extension columns come from edu_edfi_source.extract_extension for stg_sedm__student_ieps. Post-hooks enforce PK on k_student_iep and NOT NULL on k_student_iep, k_student, k_student_xyear, plus an FK from k_student to dim_student.
  • models/sedm_warehouse/fct_idea_event.sql: One row per IDEA event from stg_sedm__idea_events, keyed by k_idea_event, with the same dim_student join and extension extraction pattern. Post-hooks: PK on k_idea_event, NOT NULL on key columns, FK to dim_student.
  • models/sedm_warehouse/fct_student_iep_disability.sql: Long-format rows for disabilities on an IEP from stg_sedm__student_iep_disability_collections, keyed by staging’s k_student_iep (and related descriptors / order). Post-hook adds an FK from k_student_iep to fct_student_iep.
    Follow-ups (recommended before or right after merge, if not already on the branch):
  • Add sedm_warehouse folder config in dbt_project.yml (schema wh, enable via edu:sedm:enabled).

Tests and QC done:

  • dbt run --target data_dev successful in stadium_boston dev and stadium_south_carolina dev for all 3 new models

Submitter checklist

Make sure the following have been completed before approving this PR:

  • Description of changes has been added to Unreleased section of CHANGELOG.md. Add under ## New Features for features, etc.
  • Code has been tested/checked for Databricks and Snowflake compatibility - EA engineers see Databricks checklist here
  • Reviewer confirms the grain of all tables are unchanged, OR any changes are expected, communicated, and this PR is flagged as a breaking change (not for patch release) No changes to existing models
  • If a new configuration xwalk was added: n/a
  • The code is written such that the xwalk is optional (preferred), and this behavior was tested, OR n/a
  • The code is written such that the xwalk is required, and the required xwalk is added to edu_project_template, and this PR is flagged as breaking change (not for patch release) n/a
  • A description for the new xwalk has been added to EDU documentation site here
  • If a new configuration variable was added:
  • The code is written such that the variable is optional (preferred), and this behavior was tested, OR
  • The code is written such that the variable is required, and a default value was added to edu_project_template, and this PR is flagged as breaking change (not for patch release) Variable to enable was added to dbt_project.yml with default set to False. No other configuration variables required.

edu_wh PR Review Checklist:

Make sure the following have been completed before approving this PR:

  • Description of changes has been added to Unreleased section of CHANGELOG.md. Add under ## New Features for features, etc.
  • Code has been tested/checked for Databricks and Snowflake compatibility - EA engineers see Databricks checklist here
  • Reviewer confirms the grain of all tables are unchanged, OR any changes are expected, communicated, and this PR is flagged as a breaking change (not for patch release)
  • If a new configuration xwalk was added:
  • The code is written such that the xwalk is optional (preferred), and this behavior was tested, OR
  • The code is written such that the xwalk is required, and the required xwalk is added to edu_project_template, and this PR is flagged as breaking change (not for patch release)
  • A description for the new xwalk has been added to EDU documentation site here
  • If a new configuration variable was added:
  • The code is written such that the variable is optional (preferred), and this behavior was tested, OR
  • The code is written such that the variable is required, and a default value was added to edu_project_template, and this PR is flagged as breaking change (not for patch release)

@ryanaguilar ryanaguilar marked this pull request as ready for review May 15, 2026 17:20
@ryanaguilar ryanaguilar requested a review from rlittle08 May 15, 2026 17:20
Comment thread dbt_project.yml
+schema: wh
sedm_warehouse:
+schema: wh
+enabled: "{{ var('edu:sedm:enabled', False) }}"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rlittle08 I see that there is an existsing special education tag edu:special_ed. However, these new models are only available with the new SEDM Ed-Fi extension, so I'm not sure if I should use this new name and make a note about this distinction somewhere, or if these new models should fall under the existing variable.

Copy link
Copy Markdown
Collaborator

@rlittle08 rlittle08 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is in good shape, but left some naming suggestions, picky things, and an important q about the disability resource

stage.tenant_code,
stage.school_year,
stage.idea_event_id,
stage.ed_org_id,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

back in stage, can you add edorg_ref(), so we can get k_lea and k_school in the fct here?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because student is part of the grain, can you rename here to fct_student_idea_event? base/stg can stay the same to follow practice of those models exactly matching the ed-fi resource

tags: ['sedm']
enabled: "{{ var('edu:sedm:enabled', False) }}"

constraints:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for including these!

stage.idea_event_id,
stage.ed_org_id,
stage.ed_org_type,
stage.idea_event,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

back in base/stg, I think we should rename this to idea_event_type, to follow our typical practice. There are exceptions, but i think this is a case where type is important info - this is capturing the type/category of event, right?

- name: idea_event_id
- name: ed_org_id
- name: ed_org_type
- name: idea_event
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you fill in these idea-specific fields, using the data handbook descriptions?

stage.special_education_hours_per_week
{{ edu_edfi_source.extract_extension(model_name='stg_sedm__student_ieps', flatten=False) }}
from stage
inner join dim_student
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

picky, we generally just do join not inner join, and don't indent this way

- name: student_iep_association_id
- name: ed_org_id
- name: ed_org_type
- name: iep_finalized_date
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please add descriptions to these by copying from ed-fi handbook?

dim_student.k_student_xyear,
stage.tenant_code,
stage.school_year,
stage.student_iep_association_id,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my comment on upstream pr about this col - the ed-fi docs call it by a different name?

}}

with disabilities as (
select * from {{ ref('stg_sedm__student_iep_disability_collections') }}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my comment on upstream pr - is this resource available in 6.1 data model? or has it moved to being a collection on studentIEP?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

depending on your response, I may think about whether this should have its own fct model vs. be included in fct_student_iep

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm leaning toward a rename of fct_student_iep_association - thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants