$ sacctgpu --help
sacctgpu: -j <jobid> | <sacct job selection args>
$ sacctgpu -j 880177
Job: 880177
Node: gh003
Successfully retrieved statistics for job: slurm880177.
+------------------------------------------------------------------------------+
| GPU ID: 0 |
+====================================+=========================================+
|----- Execution Stats ------------+-----------------------------------------|
| Start Time | Wed Apr 9 18:24:27 2025 |
| End Time | Wed Apr 9 21:02:24 2025 |
| Total Execution Time (sec) | 9477.01 |
| No. of Processes | 14 |
+----- Performance Stats ----------+-----------------------------------------+
| Energy Consumed (Joules) | 1289579 |
| Power Usage (Watts) | Avg: 146.573, Max: 324.684, Min: 90.282 |
| Max GPU Memory Used (bytes) | 63298338816 |
| SM Clock (MHz) | Avg: 1000, Max: 1980, Min: 345 |
| Memory Clock (MHz) | Avg: 2619, Max: 2619, Min: 2619 |
| SM Utilization (%) | Avg: 38, Max: 100, Min: 0 |
| Memory Utilization (%) | Avg: 2, Max: 19, Min: 0 |
| PCIe Rx Bandwidth (megabytes) | Avg: N/A, Max: N/A, Min: N/A |
| PCIe Tx Bandwidth (megabytes) | Avg: N/A, Max: N/A, Min: N/A |
+----- Event Stats ----------------+-----------------------------------------+
| Single Bit ECC Errors | 0 |
| Double Bit ECC Errors | 0 |
| PCIe Replay Warnings | 0 |
| Critical XID Errors | 0 |
+----- Slowdown Stats -------------+-----------------------------------------+
| Due to - Power (%) | 0 |
| - Thermal (%) | 0 |
| - Reliability (%) | Not Supported |
| - Board Limit (%) | Not Supported |
| - Low Utilization (%) | Not Supported |
| - Sync Boost (%) | 0 |
+-- Compute Process Utilization ---+-----------------------------------------+
| PID | 70264 |
| Avg SM Utilization (%) | 2 |
| Avg Memory Utilization (%) | 0 |
| PID | 70277 |
| Avg SM Utilization (%) | 2 |
| Avg Memory Utilization (%) | 0 |
| PID | 70584 |
| Avg SM Utilization (%) | 5 |
| Avg Memory Utilization (%) | 0 |
| PID | 70903 |
| Avg SM Utilization (%) | 4 |
| Avg Memory Utilization (%) | 0 |
| PID | 70917 |
| Avg SM Utilization (%) | 4 |
| Avg Memory Utilization (%) | 0 |
| PID | 71192 |
| Avg SM Utilization (%) | 2 |
| Avg Memory Utilization (%) | 0 |
| PID | 71231 |
| Avg SM Utilization (%) | 5 |
| Avg Memory Utilization (%) | 0 |
| PID | 71504 |
| Avg SM Utilization (%) | 17 |
| Avg Memory Utilization (%) | 0 |
| PID | 71546 |
| Avg SM Utilization (%) | 18 |
| Avg Memory Utilization (%) | 0 |
| PID | 71883 |
| Avg SM Utilization (%) | 2 |
| Avg Memory Utilization (%) | 0 |
| PID | 71897 |
| Avg SM Utilization (%) | 6 |
| Avg Memory Utilization (%) | 0 |
| PID | 71912 |
| Avg SM Utilization (%) | 2 |
| Avg Memory Utilization (%) | 0 |
| PID | 71926 |
| Avg SM Utilization (%) | 3 |
| Avg Memory Utilization (%) | 0 |
| PID | 71939 |
| Avg SM Utilization (%) | 16 |
| Avg Memory Utilization (%) | 0 |
+----- Overall Health -------------+-----------------------------------------+
| Overall Health | Healthy |
+------------------------------------+-----------------------------------------+
Add details of the
sacctgpucommand an it's purpose to the usage docs.Example output (for a host and device benchmark job, using a single GH200):