dlt version: 1.26.0
Source name: pg_replication
Describe the problem
After running dlt init pg_replication <destination> and installing only what the generated requirements.txt declares, the starter pipeline fails at import time with MissingDependencyException for dlt.sources.sql_database.
sources/pg_replication/helpers.py imports dlt.sources.sql_database (line 43), which is guarded by dlt's sql_database extras (SQLAlchemy + dialects). However the connector's dependency manifests are incomplete:
sources/pg_replication/requirements.txt declares only dlt>=1.0.0 and psycopg2-binary>=2.9.9.
[dependency-groups].pg_replication in pyproject.toml declares only psycopg2-binary>=2.9.9.
Neither pulls SQLAlchemy in. The starter pipeline can never get past from pg_replication import replication_resource until the user manually runs pip install "dlt[sql_database]" — which the docs do not mention.
The bug is invisible during normal contributor work on this repo because [tool.uv].default-groups includes both sql_database and pg_replication, so a plain uv sync drags SQLAlchemy in via the sibling group. As soon as anything installs the pg_replication group on its own — dlt init, downstream tools that consume requirements.txt, or CI matrices that pick one group at a time — the connector breaks.
Expected behavior
After running the documented onboarding flow (dlt init pg_replication <destination> + pip install -r requirements.txt), the user should be able to run the generated pipeline script without a MissingDependencyException from the verified source itself. The only remaining failure should be the expected "credentials are placeholder values" error, which prompts the user to fill in .dlt/secrets.toml.
Steps to reproduce
Clean Python 3.12 environment, following dlt's documented onboarding:
mkdir /tmp/dlt-pg-repro && cd /tmp/dlt-pg-repro
uv venv .venv --python python3.12
uv pip install --python .venv/bin/python "dlt>=1.0.0"
# documented onboarding
.venv/bin/dlt init pg_replication duckdb
# install per dlt init's instructions
uv pip install --python .venv/bin/python -r requirements.txt
# run the starter pipeline dlt init generated
.venv/bin/python pg_replication_pipeline.py
Result:
File ".../pg_replication/helpers.py", line 43, in <module>
from dlt.sources.sql_database import (
File ".../dlt/common/libs/sql_alchemy.py", line 15, in <module>
raise MissingDependencyException(
dlt.common.exceptions.MissingDependencyException:
You must install additional dependencies to run `dlt sql_database helpers`.
If you use pip you may do the following:
pip install "dlt[sql_database]"
Same failure reproduces via the contributor uv path:
uv sync --python python3.12 --group dltpure --group pg_replication \
--no-default-groups --inexact
PYTHONPATH=$PWD .venv/bin/python -c "import sources.pg_replication"
# → same MissingDependencyException
dlt init does rewrite dlt>=1.0.0 to dlt[<destination>]>=1.0.0 in the generated requirements.txt (so the destination's extras land), but it merges destination extras only — the source's extras still need to be declared in the source's own requirements.txt.
How you are using the source?
I'm considering using this source in my work, but bug is preventing this.
Operating system
macOS (also reproducible on Linux in CI)
Runtime environment
Local
Python version
3.12.7
dlt destination
duckdb (the same import-time failure occurs regardless of destination — destination only changes which destination extras dlt init writes into requirements.txt)
dlt version:
1.26.0Source name:
pg_replicationDescribe the problem
After running
dlt init pg_replication <destination>and installing only what the generatedrequirements.txtdeclares, the starter pipeline fails at import time withMissingDependencyExceptionfordlt.sources.sql_database.sources/pg_replication/helpers.pyimportsdlt.sources.sql_database(line 43), which is guarded by dlt'ssql_databaseextras (SQLAlchemy + dialects). However the connector's dependency manifests are incomplete:sources/pg_replication/requirements.txtdeclares onlydlt>=1.0.0andpsycopg2-binary>=2.9.9.[dependency-groups].pg_replicationinpyproject.tomldeclares onlypsycopg2-binary>=2.9.9.Neither pulls SQLAlchemy in. The starter pipeline can never get past
from pg_replication import replication_resourceuntil the user manually runspip install "dlt[sql_database]"— which the docs do not mention.The bug is invisible during normal contributor work on this repo because
[tool.uv].default-groupsincludes bothsql_databaseandpg_replication, so a plainuv syncdrags SQLAlchemy in via the sibling group. As soon as anything installs thepg_replicationgroup on its own —dlt init, downstream tools that consumerequirements.txt, or CI matrices that pick one group at a time — the connector breaks.Expected behavior
After running the documented onboarding flow (
dlt init pg_replication <destination>+pip install -r requirements.txt), the user should be able to run the generated pipeline script without aMissingDependencyExceptionfrom the verified source itself. The only remaining failure should be the expected "credentials are placeholder values" error, which prompts the user to fill in.dlt/secrets.toml.Steps to reproduce
Clean Python 3.12 environment, following dlt's documented onboarding:
Result:
Same failure reproduces via the contributor
uvpath:uv sync --python python3.12 --group dltpure --group pg_replication \ --no-default-groups --inexact PYTHONPATH=$PWD .venv/bin/python -c "import sources.pg_replication" # → same MissingDependencyExceptiondlt initdoes rewritedlt>=1.0.0todlt[<destination>]>=1.0.0in the generatedrequirements.txt(so the destination's extras land), but it merges destination extras only — the source's extras still need to be declared in the source's ownrequirements.txt.How you are using the source?
I'm considering using this source in my work, but bug is preventing this.
Operating system
macOS (also reproducible on Linux in CI)
Runtime environment
Local
Python version
3.12.7
dlt destination
duckdb (the same import-time failure occurs regardless of destination — destination only changes which destination extras
dlt initwrites intorequirements.txt)