Moving away from dbt core operation blocks#1435
Conversation
WalkthroughDBT profile generation and credential handling are centralized, Elementary installation is consolidated into new API and Celery workflows, EDR reporting and task orchestration are updated, and management commands support migration, deployment patching, secret consolidation, and cleanup. ChangesDBT profile runtime and setup
Consolidated Elementary installation
EDR reporting and orchestration
Migration, cleanup, and operational commands
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant DBTAPI
participant ElementaryTask
participant DBTProfile
participant EDRDataflow
Client->>DBTAPI: POST /elementary/install
DBTAPI->>ElementaryTask: dispatch install_elementary
ElementaryTask->>DBTProfile: create profile and run dbt
DBTProfile-->>ElementaryTask: installation result
ElementaryTask->>EDRDataflow: ensure send-report dataflow
EDRDataflow-->>Client: task id and progress
sequenceDiagram
participant ReportConsumer
participant elementary_service
participant S3
ReportConsumer->>elementary_service: request latest report
elementary_service->>S3: list reports within lookback window
S3-->>elementary_service: newest matching object
elementary_service->>S3: download report
elementary_service-->>ReportConsumer: report metadata and schedule
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
ddpui/services/org_cleanup_service.py (1)
260-290: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winMove dbt-profile secret block cleanup to the transformation-layer teardown
delete_org()clearsself.org.dbtbeforedelete_warehouse()runs, so this lookup isNonein the normal full-org delete path. Ifdelete_warehouse()runs on its own, the same Prefect block can be deleted once perOrgWarehouse. Move this cleanup todelete_transformation_layer()whileorgdbtis still available, and run it once.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ddpui/services/org_cleanup_service.py` around lines 260 - 290, The dbt-profile secret block cleanup currently occurs in the per-warehouse delete flow, after delete_org() may clear self.org.dbt and potentially repeats for each OrgWarehouse. Remove that cleanup from delete_warehouse() and move it into delete_transformation_layer(), using the available orgdbt record and executing it once while preserving the Prefect deletion, error logging, and OrgPrefectBlockv1 deletion behavior.
🧹 Nitpick comments (4)
ddpui/management/commands/sync_edr_secret_block.py (1)
112-117: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winBroad except conflates "not found" with real failures.
Catching bare
Exceptionhere treats any failure fromget_secret_block_by_name(auth errors, network issues, Prefect outage) the same as "block doesn't exist," printing a benign[skip] ... not presentinstead of surfacing the real error. Elsewhere in the codebase (e.g.org_cleanup_service.py) not-found lookups catch the specificHttpErrorrather than a blanketException.♻️ Suggested narrower catch
try: block = prefect_service.get_secret_block_by_name(legacy_name) - except Exception: # pylint: disable=broad-exception-caught + except HttpError: # Block doesn't exist — nothing to delete. self.stdout.write(f" [skip] {legacy_name}: not present") continuePlease confirm
get_secret_block_by_name/prefect_getraisesHttpError(vs. some other exception type) on a 404 so the narrower catch doesn't miss the legitimate not-found case.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ddpui/management/commands/sync_edr_secret_block.py` around lines 112 - 117, Replace the broad exception handler around get_secret_block_by_name with the specific HttpError type used by prefect_get for 404 responses, confirming that exception type from the client implementation or established org_cleanup_service.py pattern. Preserve the [skip] behavior only for not-found errors, while allowing authentication, network, and other failures to propagate.README.md (1)
146-152: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winSibling-directory assumption is implicit.
The
cd ../prefect-proxy/docker/dbt-X.Y.Z/paths assumeprefect-proxyis cloned as a sibling directory to this repo, but Step 5 (which links to the prefect-proxy repo) doesn't say where to clone it relative to this repo. Worth a one-line clarification to avoid confusion for new setups.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@README.md` around lines 146 - 152, Clarify in Step 5 that the prefect-proxy repository must be cloned as a sibling directory of the current repository, matching the relative ../prefect-proxy/docker/dbt-X.Y.Z paths used in the dbt setup instructions.ddpui/tests/services/test_elementary_service.py (1)
881-1003: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winTests correctly validate
install_elementary's step-wise progress emission and failure short-circuiting logic, but sincerun_dbt_commands.applyis mocked outright, they don't exercise the actual swallow-and-continue defect inrun_dbt_commandsflagged inddpui/celeryworkers/tasks.py(lines 313-376). Consider adding one integration-style test that calls the realrun_dbt_commands(with dbt subprocess calls mocked) to confirminstall_elementarystep 1 actually fails when dbt run fails.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ddpui/tests/services/test_elementary_service.py` around lines 881 - 1003, The current tests mock run_dbt_commands.apply entirely and do not verify its real failure propagation. Add an integration-style test for install_elementary that invokes the actual run_dbt_commands implementation while mocking only its dbt subprocess calls to fail, then assert step 1 emits failed, step 2 is not called, and the exception propagates.ddpui/core/pipelinefunctions.py (1)
108-141: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDuplicate
dbt_profile_secret_blockresolution logic across two functions.
setup_dbt_core_task_configandsetup_edr_send_report_task_configboth introduce the identical pattern of resolvingorg_task.org.dbt.dbt_profile_secret_blockand warning if absent. Consider extracting a small helper, e.g.resolve_dbt_profile_secret_block(org_task) -> str | None, to avoid future drift between the two warning messages/behaviors.♻️ Suggested helper
def _resolve_dbt_profile_secret_block(org_task: OrgTask, fail_hint: str) -> str | None: orgdbt = org_task.org.dbt if orgdbt and orgdbt.dbt_profile_secret_block: return orgdbt.dbt_profile_secret_block.block_name logger.warning( "OrgDbt for org=%s has no dbt_profile_secret_block — %s", org_task.org.slug, fail_hint, ) return NoneAlso applies to: 220-253
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ddpui/core/pipelinefunctions.py` around lines 108 - 141, Extract the shared dbt profile secret block lookup from setup_dbt_core_task_config and setup_edr_send_report_task_config into a helper such as _resolve_dbt_profile_secret_block. Have it return the block name when available, otherwise log the warning with the supplied failure hint and return None; update both task configuration functions to use this helper while preserving their existing env and warning behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ddpui/celeryworkers/tasks.py`:
- Around line 313-376: Update the outer exception handler in run_dbt_commands so
failures from profile writing or dbt clean/deps/run operations are re-raised
after recording the failure and updating progress. Ensure the task result is
marked failed, allowing install_elementary’s apply(...).maybe_throw() to stop
step 1 instead of emitting completed and scheduling reports.
In `@ddpui/management/commands/patch_edr_deployment_env.py`:
- Around line 58-88: Before constructing new_params in the deployment loop,
inspect each deployment’s existing config.tasks and detect pipelines containing
tasks beyond the regenerated EDR task. For mixed pipelines, skip the deployment
without calling update_dataflow_v1 (and report it consistently); only replace
config.tasks for deployments that contain exclusively the EDR task, preserving
dry-run and patched/skipped accounting.
---
Outside diff comments:
In `@ddpui/services/org_cleanup_service.py`:
- Around line 260-290: The dbt-profile secret block cleanup currently occurs in
the per-warehouse delete flow, after delete_org() may clear self.org.dbt and
potentially repeats for each OrgWarehouse. Remove that cleanup from
delete_warehouse() and move it into delete_transformation_layer(), using the
available orgdbt record and executing it once while preserving the Prefect
deletion, error logging, and OrgPrefectBlockv1 deletion behavior.
---
Nitpick comments:
In `@ddpui/core/pipelinefunctions.py`:
- Around line 108-141: Extract the shared dbt profile secret block lookup from
setup_dbt_core_task_config and setup_edr_send_report_task_config into a helper
such as _resolve_dbt_profile_secret_block. Have it return the block name when
available, otherwise log the warning with the supplied failure hint and return
None; update both task configuration functions to use this helper while
preserving their existing env and warning behavior.
In `@ddpui/management/commands/sync_edr_secret_block.py`:
- Around line 112-117: Replace the broad exception handler around
get_secret_block_by_name with the specific HttpError type used by prefect_get
for 404 responses, confirming that exception type from the client implementation
or established org_cleanup_service.py pattern. Preserve the [skip] behavior only
for not-found errors, while allowing authentication, network, and other failures
to propagate.
In `@ddpui/tests/services/test_elementary_service.py`:
- Around line 881-1003: The current tests mock run_dbt_commands.apply entirely
and do not verify its real failure propagation. Add an integration-style test
for install_elementary that invokes the actual run_dbt_commands implementation
while mocking only its dbt subprocess calls to fail, then assert step 1 emits
failed, step 2 is not called, and the exception propagates.
In `@README.md`:
- Around line 146-152: Clarify in Step 5 that the prefect-proxy repository must
be cloned as a sibling directory of the current repository, matching the
relative ../prefect-proxy/docker/dbt-X.Y.Z paths used in the dbt setup
instructions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ddad13ab-628c-4814-8d9b-c58283f320ac
⛔ Files ignored due to path filters (3)
dbt_deps/dbt-1.10.19/uv.lockis excluded by!**/*.lockdbt_deps/dbt-1.8.7/uv.lockis excluded by!**/*.lockdbt_deps/dbt-1.9.8/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (31)
README.mddbt_deps/Dockerfile.prefect-job-runnerdbt_deps/README.mddbt_deps/dbt-1.10.19/pyproject.tomldbt_deps/dbt-1.8.7/pyproject.tomldbt_deps/dbt-1.9.8/pyproject.tomlddpui/api/dbt_api.pyddpui/api/orgtask_api.pyddpui/celeryworkers/tasks.pyddpui/core/dbtfunctions.pyddpui/core/orchestrate/pipeline_service.pyddpui/core/pipelinefunctions.pyddpui/ddpairbyte/airbytehelpers.pyddpui/ddpdbt/dbt_service.pyddpui/ddpdbt/dbthelpers.pyddpui/ddpdbt/elementary_service.pyddpui/management/commands/backfill_warehouse_secret_blocks.pyddpui/management/commands/ensureedrsendreportdataflow.pyddpui/management/commands/patch_edr_deployment_env.pyddpui/management/commands/sync_edr_secret_block.pyddpui/management/commands/updateedrsendreportdataflowtarget.pyddpui/migrations/0169_orgdbt_dbt_profile_secret_block_and_more.pyddpui/models/org.pyddpui/models/tasks.pyddpui/services/org_cleanup_service.pyddpui/tests/api_tests/test_dbt_api.pyddpui/tests/core/test_dbtfunctions.pyddpui/tests/core/test_pipelinefunctions.pyddpui/tests/services/test_elementary_service.pyddpui/utils/constants.pyddpui/utils/s3_utils.py
💤 Files with no reviewable changes (5)
- dbt_deps/dbt-1.9.8/pyproject.toml
- dbt_deps/dbt-1.8.7/pyproject.toml
- dbt_deps/README.md
- dbt_deps/dbt-1.10.19/pyproject.toml
- dbt_deps/Dockerfile.prefect-job-runner
| @app.task(bind=True) | ||
| def install_elementary(self, org_id: int, task_id: str, hashkey: str): | ||
| """Consolidated Elementary setup — runs the three sub-steps as one task so | ||
| the frontend can render a single progress list. | ||
|
|
||
| Sub-steps (each emits {"stepIndex", "step", "status": running|completed}): | ||
| 0. Create dbt profile → create_elementary_profile(org) | ||
| 1. Install elementary package → run_dbt_commands (clean + deps + run --select elementary) | ||
| 2. Schedule reports → ensure_edr_sendreport_dataflow(org, org_task, cron) | ||
|
|
||
| On failure: emits {"stepIndex", "step", "status": "failed", "message"} for | ||
| the sub-step that blew up, then re-raises so Celery marks the task failed. | ||
| """ | ||
| org: Org = Org.objects.get(id=org_id) | ||
| taskprogress = TaskProgress(task_id, hashkey) | ||
|
|
||
| steps = [ | ||
| (0, "Creating dbt profile"), | ||
| (1, "Installing elementary package"), | ||
| (2, "Scheduling reports"), | ||
| ] | ||
| current = steps[0] | ||
|
|
||
| def _emit(step, status, message=None): | ||
| payload = {"stepIndex": step[0], "step": step[1], "status": status} | ||
| if message: | ||
| payload["message"] = message | ||
| taskprogress.add(payload) | ||
|
|
||
| try: | ||
| current = steps[0] | ||
| _emit(current, "running") | ||
| result = create_elementary_profile(org) | ||
| if isinstance(result, dict) and "error" in result: | ||
| raise Exception(result["error"]) | ||
| _emit(current, "completed") | ||
|
|
||
| current = steps[1] | ||
| _emit(current, "running") | ||
| # run_dbt_commands is a bound celery task; .apply() executes it | ||
| # synchronously in-process (returns an EagerResult). Use a separate | ||
| # inner task_id so its verbose dbt-output progress doesn't pollute | ||
| # the install stream — the frontend only polls the outer install | ||
| # task_id + hashkey. Use .maybe_throw() instead of .get() to re-raise | ||
| # inner-task exceptions without tripping celery's "no sync subtasks" | ||
| # guard (which fires for any .get() call inside a running task). | ||
| inner_task_id = str(uuid4()) | ||
| run_dbt_commands.apply( | ||
| args=[org.id, org.dbt.id, inner_task_id, {"options": {"select": "elementary"}}] | ||
| ).maybe_throw() | ||
| _emit(current, "completed") | ||
|
|
||
| current = steps[2] | ||
| _emit(current, "running") | ||
| result = ensure_edr_sendreport_dataflow(org, "0 0 * * *") | ||
| if isinstance(result, dict) and "error" in result: | ||
| raise Exception(result["error"]) | ||
| _emit(current, "completed") | ||
|
|
||
| except Exception as err: | ||
| _emit(current, "failed", message=str(err)) | ||
| raise | ||
|
|
||
|
|
There was a problem hiding this comment.
🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win
install_elementary step 1 can silently report "completed" even when dbt install actually fails.
run_dbt_commands's outer exception handler (unchanged, further down in this same function — the except Exception as e: block that logs "Job finished with a failure") catches all internal errors (including the newly-added write_dbt_profiles_yml failure at lines 162‑174, and the pre-existing dbt clean/deps/run failures) but never re-raises. .apply() therefore returns an EagerResult in SUCCESS state regardless of the actual outcome, so .maybe_throw() at line 362 never throws — contrary to the comment's stated intent ("re-raise inner-task exceptions"). As a result, install_elementary will emit "completed" for step 1 and proceed to schedule EDR reports (step 2) even when the elementary dbt package failed to install.
None of the current tests (test_install_elementary_step1_failure) catch this, since they mock run_dbt_commands.apply(...).maybe_throw() directly rather than exercising the real function.
🐛 Suggested fix (in `run_dbt_commands`)
except Exception as e:
taskprogress.add(
{
"message": "Job finished with a failure",
"status": "failed",
}
)
+ raise
finally:
for task_lock in task_locks:
task_lock.delete()Worth double-checking that other .delay() callers of run_dbt_commands don't depend on it never raising (they should be fine, since they poll TaskProgress in Redis rather than the Celery result).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ddpui/celeryworkers/tasks.py` around lines 313 - 376, Update the outer
exception handler in run_dbt_commands so failures from profile writing or dbt
clean/deps/run operations are re-raised after recording the failure and updating
progress. Ensure the task result is marked failed, allowing install_elementary’s
apply(...).maybe_throw() to stop step 1 instead of emitting completed and
scheduling reports.
| try: | ||
| dbt_project_params = DbtProjectManager.gather_dbt_project_params(org, orgdbt) | ||
| except Exception as err: # pylint: disable=broad-exception-caught | ||
| self.stdout.write(f" [skip] {org.slug}: gather_dbt_project_params failed ({err})") | ||
| skipped += 1 | ||
| continue | ||
|
|
||
| task_config = setup_edr_send_report_task_config(orgtask, dbt_project_params.project_dir) | ||
| new_params = { | ||
| "config": { | ||
| "tasks": [task_config.to_json()], | ||
| "org_slug": org.slug, | ||
| } | ||
| } | ||
|
|
||
| for dot in DataflowOrgTask.objects.filter(orgtask=orgtask).select_related("dataflow"): | ||
| dataflow = dot.dataflow | ||
| if options["dry_run"]: | ||
| self.stdout.write( | ||
| f" [dry-run] {dataflow.deployment_name}: would rebuild deployment_params" | ||
| ) | ||
| patched += 1 | ||
| continue | ||
|
|
||
| try: | ||
| update_dataflow_v1( | ||
| dataflow.deployment_id, | ||
| PrefectDataFlowUpdateSchema3( | ||
| deployment_params=new_params, cron=dataflow.cron | ||
| ), | ||
| ) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
set -euo pipefail
# Map relevant files first
ast-grep outline ddpui/management/commands/patch_edr_deployment_env.py --view expanded || true
echo '---'
fd -a 'pipeline_service.py' .
echo '---'
fd -a 'dataflow*' ddpui | head -n 50
echo '---'
rg -n "send report|send-report|deployment_params|DataflowOrgTask|tasks" ddpui -g'*.py' | head -n 250Repository: DalgoT4D/DDP_backend
Length of output: 25476
🏁 Script executed:
set -euo pipefail
sed -n '70,210p' ddpui/core/orchestrate/pipeline_service.py
echo '---'
sed -n '430,520p' ddpui/ddpdbt/elementary_service.py
echo '---'
sed -n '150,330p' ddpui/core/orgtaskfunctions.pyRepository: DalgoT4D/DDP_backend
Length of output: 16034
🏁 Script executed:
set -euo pipefail
sed -n '1,220p' ddpui/management/commands/patch_edr_deployment_env.pyRepository: DalgoT4D/DDP_backend
Length of output: 4093
Guard against mixed EDR pipelines before rewriting deployment_params. ddpui/management/commands/patch_edr_deployment_env.py:58-88 replaces config.tasks with only the regenerated EDR task for every deployment attached to the orgtask. Orchestrated pipelines can already include EDR alongside other tasks, so this will drop the rest of the pipeline config. Skip those deployments or merge the existing tasks instead.
🧰 Tools
🪛 Ruff (0.15.21)
[warning] 60-60: Do not catch blind exception: Exception
(BLE001)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ddpui/management/commands/patch_edr_deployment_env.py` around lines 58 - 88,
Before constructing new_params in the deployment loop, inspect each deployment’s
existing config.tasks and detect pipelines containing tasks beyond the
regenerated EDR task. For mixed pipelines, skip the deployment without calling
update_dataflow_v1 (and report it consistently); only replace config.tasks for
deployments that contain exclusively the EDR task, preserving dry-run and
patched/skipped accounting.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1435 +/- ##
=======================================
Coverage ? 61.63%
=======================================
Files ? 152
Lines ? 17973
Branches ? 0
=======================================
Hits ? 11077
Misses ? 6896
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ddpui/management/commands/backfill_dbt_profile_secret_blocks.py`:
- Around line 41-53: The warehouse-processing flow in the backfill command must
handle organizations with multiple OrgWarehouse rows before upserting the shared
dbt profile block. Group warehouses by organization, then either select the
established designated warehouse or skip/fail ambiguous organizations
explicitly; ensure each org’s dbt-profile-<org.slug> is written at most once.
- Around line 99-103: Update the migration flow around
_patch_deployments_for_org so deployment patch failures are returned or
aggregated instead of only logged. Increment ok and report the organization as
migrated only after patching succeeds; propagate any failure through the
command’s final status so the command fails when any DBTCORE deployment cannot
be updated.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 472c5828-a2e2-4c46-8052-0a6ac43b927f
📒 Files selected for processing (4)
ddpui/api/orgtask_api.pyddpui/management/commands/backfill_dbt_profile_secret_blocks.pyddpui/tests/helper/test_airbytehelpers.pyddpui/tests/services/dbt_service/test_generate_manifest_json_for_dbt_project.py
| warehouses = OrgWarehouse.objects.select_related("org").all() | ||
| if options["org"]: | ||
| warehouses = warehouses.filter(org__slug=options["org"]) | ||
|
|
||
| total = warehouses.count() | ||
| if total == 0: | ||
| print("No warehouses found") | ||
| return | ||
|
|
||
| print(f"Processing {total} warehouse(s)") | ||
| ok, skipped, failed = 0, 0, 0 | ||
|
|
||
| for warehouse in warehouses: |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Handle organizations with multiple warehouses explicitly.
At Line 41, each warehouse is processed independently, but every iteration writes the same dbt-profile-<org.slug> block. Since OrgWarehouse does not enforce one row per org, a multi-warehouse org ends with whichever credential payload is processed last. Select a designated warehouse or fail/skip ambiguous orgs before upserting.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ddpui/management/commands/backfill_dbt_profile_secret_blocks.py` around lines
41 - 53, The warehouse-processing flow in the backfill command must handle
organizations with multiple OrgWarehouse rows before upserting the shared dbt
profile block. Group warehouses by organization, then either select the
established designated warehouse or skip/fail ambiguous organizations
explicitly; ensure each org’s dbt-profile-<org.slug> is written at most once.
| print(f" [ok] {org.slug}: {block.block_name}") | ||
| ok += 1 | ||
|
|
||
| # Patch deployments so DBTCORE task_configs carry the runner env key. | ||
| _patch_deployments_for_org(org) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Do not report an org as migrated when deployment patching fails.
ok is incremented before _patch_deployments_for_org, while Lines 157-158 only log update failures. The command can exit successfully even though DBTCORE tasks still lack dbt-profile-secret-block and will not use the new runner secret. Return/aggregate patch failures and mark the org/command failed when any deployment cannot be updated.
Also applies to: 148-158
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ddpui/management/commands/backfill_dbt_profile_secret_blocks.py` around lines
99 - 103, Update the migration flow around _patch_deployments_for_org so
deployment patch failures are returned or aggregated instead of only logged.
Increment ok and report the organization as migrated only after patching
succeeds; propagate any failure through the command’s final status so the
command fails when any DBTCORE deployment cannot be updated.
1 participant
Commands/scripts to be run in this order
elementary_profiles/profiles.ymland run the report flow.Summary by CodeRabbit
New Features
Bug Fixes
Documentation