IfG data app

Development

Three branches exist:

• production: Production branch, hosting the live app (https://instituteforgovernment.streamlit.app/)
• staging: Staging branch, hosting the pre-production version of the app (https://instituteforgovernment-stage.streamlit.app/: NB: Note URL pattern, as staging can't be used in Streamlit Community Cloud app URLs)
• development: Development branch, hosting the latest development version of the app

Authentication

The app uses Microsoft Entra authentication, following Streamlit's guide to implementing this.

A separate set of secrets exists for each of these environments:

• Production environment on Streamlit Community Cloud
• Staging environment on Streamlit Community Cloud
• Local development

This is because the redirect_uri differs for each of these environments, and different secrets are needed for each redirect_uri.

The identity app registration that has been created has an expiry date: 3 April 2027.

Serving the app

Web

Hosted on Streamlit Community Cloud at https://instituteforgovernment.streamlit.app/.

Environment variables and authentication secrets are held as app secrets.

Local development

1. Add environment variables as system environment variables
2. Add a secrets.toml file in a .streamlit directory containing authentication details
3. Create a new virtual environment using python -m venv env
4. Activate the virtual environment using env\Scripts\activate
5. Run pip install -r requirements.txt to install packages
6. Run pre-commit install to install pre-commit
7. Run streamlit run streamlit_app.py

Project structure

├── .streamlit/
│   ├── config.toml
│
├── docs/
│   ├── design_notes.md
│
├── ifg_data_app/
|   ├── config/        <<< Variables used app-wide
|   │   ├── ag_grid_theme.py
|   │   ├── colours.py
|   │
|   ├── js/     <<< JS functions used app-wide
|   │   ├── <function name>.js
|   │
|   ├── pages/
|   │   ├── dashboard/
|   │       ├── web_metrics/
|   │          ├── md/        <<< Markdown text to be rendered in pages
|   │             ├── roadmap/
|   │                ├── doing.md
|   │                ├── done.md
|   │                ├── future.md
|   │             ├── <page name>.md
|   │          ├── <page name>.py
|   │          ├── chart_annotations.py        <<< Chart annotations used across multiple pages
|   │          ├── config.py        <<< Variables used across multiple pages
|   │          ├── definitions.py     <<< Definitions to be rendered in the UI
|   │          ├── elements.py     <<< Page elements used across multiple pages
|   │          ├── notes.py     <<< Notes to be rendered in the UI
|   │          ├── utils.py     <<< (Non-page element) functions and code snippets used across multiple pages
|   │
|   ├── sql/
|   │   ├── dashboard/
|   │       ├── web_metrics/
|   │          ├── <page name>.sql
│
├── .gitignore
├── .pre-commit-config.yaml
├── README.md
├── streamlit_app.py
└── requirements.txt

Design decisions

Documentation of key design decisions can be found in design_notes.md.

Environment variables

The following environment variables are used by the app:

• ODBC_DRIVER: ODBC driver name to be used in database connection
• ODBC_SERVER: Server name to be used in database connection
• ODBC_DATABASE: Database name to be used in database connection
• ODBC_AUTHENTICATION: Authentication type to be used in database connection
• AZURE_CLIENT_ID: Username to be used in database connection
• AZURE_CLIENT_SECRET: Password to be used in database connection
• DS_CONTACT_EMAIL_ADDRESS: Email address to be used for contacting the data science team
• IFG_WEB_ISSUES_FILE_LINK: Link to document where web issues are logged

Command line arguments

Command line arguments can be passed to the Streamlit app using the following syntax:

streamlit run streamlit_app.py -- --redact-data

NB: Note the use of -- before the addition of these command line arguments. This is required to differentiate them from Streamlit command-line arguments.

Available arguments

Argument	Description	Default
--redact-data	Enable redact data mode. This replaces table values, chart axis labels and chart tooltip values with a xxxxx string	false

Licences

An AG Grid licence is held in order to make use of all streamlit-aggrid features. The license is perpetual - i.e. no need for renewal.