Dynamic generated Dag using JINJA templete is not appearing in the Airflow UI

[kkrishrao]

Apache Airflow version

2.10.5

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Hello Team
we generate the dag using below JINJA templet and its created a dag1.py ...but it is not appearing in Airflow UI and also not Runnings as per schedule time
when I am using a command inside worker pod
airflow dags reserialize
its appeared in UI and runs and dissapeared.In production some one cannot do this every time.How to fix
Airflow version is 2.10.x

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime, timedelta

def print_hello():
print("Hello World {{ dag_id }}")

default_args = {
'owner': 'fasadmin',
'depends_on_past': False,
'start_date': datetime(2024, 10, 15),
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(minutes=5),
}

dag = DAG(
'{{ dag_id }}',
default_args=default_args,
description='A simple DAG to print Hello World dynamically',
schedule_interval="*/2 * * * *", # Set to None for on-demand execution
is_paused_upon_creation=False,
catchup=False,
concurrency=1,
max_active_runs=1,
tags=["dynamic"],
)

t1 = PythonOperator(
task_id='print_hello',
python_callable=print_hello,
dag=dag,
)
t1
globals()["{{ dag_id }}"] = dag

What you think should happen instead?

When the DAG is generated in the DAG folder, the DAG should appear immediately

How to reproduce

Create a DAG using ninja template in the default DAG folder.
Login to the Airflow UI and the DAG is not displayed until we forcefully run "airflow dags reserialize"

Operating System

k8s

Versions of Apache Airflow Providers

Bitnami

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[kkrishrao]

why does the DAG with id "summout_two" disappear from the serialized_dag table of Airflow after some time.
this DAG is a generated DAG. It appears in the UI and serialized_dag when we manually run the command "airflow serialize dags"

[sjyangkevin]

From my understanding, you create this Python file and use “{{ dag_id }}” in DAG definition. I think the template variables like {{ dag_id }} are only available within specific templated fields of operators and tasks, not in the DAG definition itself. This is because the DAG ID must be explicitly defined when the DAG is created, and template variables are evaluated during runtime, not at DAG creation time.

[shahar1]

Can this issue be reproduced in Airflow 3.X?
If so, please do so and provide reproduction steps.
Otherwise, this issue will be auto-closed as Airflow 2.X is in end-of-life state.

[github-actions]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.