Optimal life-cycle maintenance planning for ageing highway bridges exposed to two interacting hazards — slow, progressive corrosion deterioration and sudden, random earthquakes — formulated as a hybrid Markov Decision Process (MDP) and solved by dynamic programming.
The model answers a practical infrastructure-management question: given a bridge's current condition, what maintenance action minimizes the expected discounted life-cycle cost?
A deteriorating bridge is described by a ternary state that jointly tracks three independent damage processes:
| Symbol | Meaning | Driven by |
|---|---|---|
D_seis |
Accumulated seismic damage | Earthquake occurrences (Poisson) |
D_corr |
Accumulated corrosion damage | Gamma deterioration process |
D_prot |
Deterioration of the protective cover | Time / paint ageing |
At each decision epoch the manager chooses one of three actions:
| Action | Meaning | Effect |
|---|---|---|
| NA | No action | Only routine inspection; damage keeps accumulating |
| PM | Preventive maintenance | Restores the protective cover (D_prot → 0) |
| EM | Essential maintenance | Restores the bridge to its intact condition (rebuild / major repair) |
The two hazards evolve on different time scales, so the framework combines a discrete-time chain for gradual corrosion with a continuous-time chain for sudden earthquakes, then reduces them to a single equivalent discrete-time MDP with:
- an equivalent transition matrix
P_eqv(Δt)(obtained by integrating the two processes over an inspection interval via the trapezoidal rule), and - an equivalent discount factor
γ_eqv,
which is solved with value iteration to obtain the optimal maintenance policy and the value function (expected discounted life-cycle cost) for every state.
Hazard models → State-transition matrices → Equivalent MDP → Solve
───────────────── ────────────────────────── ──────────────── ─────
Gamma process (corr.) P_d (deterioration) P_eqv(Δt) Value
Poisson (earthquakes) P_e (seismic, fragility-based) γ_eqv iteration
Ground-motion + fragility → policy
- Corrosion is modelled as a gamma process with a time-dependent shape function.
- Earthquake ground motion (5%-damped PSA) is sampled from a magnitude distribution and an attenuation (ground-motion prediction) relation; seismic damage states follow lognormal fragility curves.
- Costs include construction, inspection, preventive-maintenance, essential-maintenance, and indirect (downtime) costs, with a compulsory rebuild once compound damage exceeds a critical level.
HMDP-Multihazard/
├── notebooks/
│ └── hmdp_bridge_maintenance.ipynb # End-to-end model: states → transitions → costs → policy
├── requirements.txt # Python dependencies
├── LICENSE
└── README.md
git clone https://github.com/SAMIRHOSEIN/HMDP_Multihazard.git
cd HMDP_Multihazard
pip install -r requirements.txtRequires Python 3.9+ with NumPy, SciPy, pandas, Matplotlib, and openpyxl.
Open the notebook and run all cells:
jupyter notebook notebooks/hmdp_bridge_maintenance.ipynbThe notebook builds the state space, computes the transition matrices for each action, assembles the equivalent MDP, runs value iteration, and produces the optimal-policy maps and convergence plots. Intermediate transition and cost matrices are also exported to Excel for inspection.
Key parameters (state-space resolution, discount rate, earthquake intensity, number of Monte-Carlo samples) are defined together in the configuration cell near the top of the notebook.
Released under the MIT License.
Amir Moayyedi — Portland State University GitHub @SAMIRHOSEIN