Skip to content

feat(dataset): add row_factory to Relation fetch methods#4129

Open
richacode007-byte wants to merge 4 commits into
dlt-hub:develfrom
richacode007-byte:feat/4128-relation-row-factory
Open

feat(dataset): add row_factory to Relation fetch methods#4129
richacode007-byte wants to merge 4 commits into
dlt-hub:develfrom
richacode007-byte:feat/4128-relation-row-factory

Conversation

@richacode007-byte

Copy link
Copy Markdown

Description
This PR adds an opt-in, keyword-only row_factory parameter to dlt.Relation fetch methods, addressing the quality-of-life request in #4128.

Today, when fetching rows from a dlt.Relation via .fetchall(), .fetchmany(), .fetchone(), or .iter_fetch(), each row is returned as a tuple/list of values with no column names attached. Users who want named rows must manually zip column names with each row:

query = dataset.table("foo")
for row in query.fetchall():
data = dict(zip(query.columns, row))
This PR adds a built-in way to opt into dictionary rows:

for row in dataset.table("foo").fetchall(row_factory=dict):
print(row) # {"id": "...", "name": "...", ...}
What changed

Added row_factory as a keyword-only argument (default None) to:
fetchall()
fetchmany(chunk_size, *, row_factory=...)
fetchone()
iter_fetch(chunk_size, *, row_factory=...)
When row_factory=dict, each row is returned as {column_name: value} using the relation’s output column names.
Column names are resolved from sqlglot lineage via relation.columns, with a fallback to DB-API cursor description when needed.
Default behavior is unchanged: without row_factory, rows are still returned as tuples from the cursor.
v1 supports row_factory=dict only. Custom sqlite3-style Callable factories are intentionally deferred to a follow-up.
Files changed

dlt/dataset/relation.py — implementation and docstrings
tests/dataset/test_relation.py — 9 unit tests
docs/website/docs/general-usage/dataset-access/dataset_snippets/dataset_snippets.py — docs snippet
docs/website/docs/general-usage/dataset-access/dataset.md — docs reference
Related Issues
Closes #4128

Tests cover:

fetchall, fetchone, fetchmany, iter_fetch with row_factory=dict

Transformed relations (.select())

Query-based relations (dataset.query(...))

Default tuple behavior unchanged

Column order preserved

Invalid row_factory raises ValueError

I have read the Contributing to dlt guide.

I have run the tests locally and they have passed before submitting this PR.

@richacode007-byte

Copy link
Copy Markdown
Author

@kinghuang @chulkilee @PabloCastellano Pls review this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(qol): dlt.Relation query methods return dictionaries

1 participant