feat(get_locs): migrate to DWH 'W0002_10_Watina' - #133
Open
droomelotdegendt wants to merge 4 commits into
Open
Conversation
Extract data warehouse specific query logic into 'build_locs_query' to prepare for the 'W0002_10_Watina' migration.
The structure of the new data warehouse is different than the previous one. Therefore we should not only link to the new table names, but some filters should also be altered as the values are changed or the columns do not exist anymore. 1. The column 'PeilpuntStatusCode' of the table 'DimPeilpunt' no longer has the value 'ENT'. 2. The columns 'PeilpuntOpenbaarheidTypeCode' and 'PeilpuntOpenbaarheidCode' no longer exist in the table 'DimPeilpunt' 3. The column 'MeetpuntStatusCode' of the table 'Meetpunt' has only value 'VLD' left, making this filter unnecessary.
droomelotdegendt
force-pushed
the
feat/dwh-migration
branch
from
July 16, 2026 12:01
ea228d7 to
2069061
Compare
Collaborator
|
PeilpuntStatusCode is always 'VLD' or 'CLD', the current filter has therefore no impact. PeilpuntStatusCode is added to the output as 'obswell_validity'.
2 tasks
Collaborator
Author
|
Thanks for the feedback. I created an issue for the 'Openbaarheid' fields (I have to validate with an end user if this is necessary, but they are currently OOO). The 'PeilpuntStatusCode' is updated in my latest commit, I'll validate with an end user if this field is required in the output. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Migrate function
get_locsto new data warehouse 'W0002_10_Watina'.The table structure of the new DWH is slightly different, therefore not only the connection to the tables should be altered, but also the used filters can change.
Key changes
build_locs_queryto separate all DWH query logicvwDim...-->Dim...)DimOpenbaarheidin the new DWH, but removal of these filters result in the same output )loc_validityas this field is now a constant (VLD)Extra check during review
MeetpuntStatusCodeof the tableDimMeetpuntcan only contain the option VLD?PeilpuntStatucCode?Migration checks
To verify the code changes, we compared the DWH migration snapshot tests before and after the migration (see the testing README for more information) and the output remains unchanged except for small rounding differences.
Note: to only check the DWH query changes, look at feat(get_locs): migrate to DWH 'W0002_10_Watina'* separately