Fix support for distinct auth/billing and destination projects in BigQuery

[jqrd]

dlt version

1.27.2

Describe the problem

The documentation makes it seem like it's possible to set separate project IDs for auth (and thus billing) and destination: https://dlthub.com/docs/dlt-ecosystem/destinations/bigquery#using-different-project_id

Using different project_id

You can set the project_id in your configuration to be different from the one in your credentials, provided your account has access to it:

[destination.bigquery]
project_id = "project_id_destination"

[destination.bigquery.credentials]
project_id = "project_id_credentials"

In this scenario, project_id_credentials will be used for authentication, while project_id_destination will be used as the data destination.

However, the BigQuery implementation doesn't support it: https://github.com/dlt-hub/dlt/blob/devel/dlt/destinations/impl/bigquery/sql_client.py#L96

self.project_id = project_id or self.credentials.project_id

Expected behavior

Using the config described in the docs actually works, and different

Steps to reproduce

Set up 2 BigQuery projects, having roles/bigquery.jobUser in only one of them (e.g. project-a), while setting the other as the destination project (e.g. project-b).

Expected:

• dlt should run and the execution should be billed to project-a, while data should be inserted in the datasets residing in project-b.

Actual:

• error: 403 POST https://bigquery.googleapis.com/bigquery/v2/projects/<project-b>/queries?prettyPrint=false: Access Denied: Project <project-b>: User does not have bigquery.jobs.create permission in project <project-b>.

Operating system

Linux

Runtime environment

Other

Python version

3.13

dlt data source

API

dlt destination

Google BigQuery

Other deployment details

Running in GCP CloudRun

Additional information

No response

[AnnasMazhar]

The fix is straightforward — the sql_client.py uses for both the BigQuery Client constructor (which determines billing/auth project) and dataset references. We need to separate these: use credentials.project_id for the client, and the config project_id for dataset operations. I'll prepare a PR.