Add 9 new Green AI patterns to the catalogue

docs/architecture/system-topology/efficient-hardware-ai-workloads.md

@@ -0,0 +1,75 @@
+---
+version: 1.0
+submitted_by: Naveen Balani
+submitted_by_linkedin: https://www.linkedin.com/in/naveenbalani/
+published_date: 
+category: Architecture
+tags: 
+ - ai
+ - machine-learning
+ - compute
+ - cloud
+ - persona:infrastructure-engineer
+ - persona:devops-engineer
+ - persona:ai-ml-engineer
+ - persona:solution-architect
+ - size:medium
+personas: Infrastructure Engineer, DevOps Engineer, AI/ML Engineer, Solution Architect
+description: Match AI workloads to the most energy-efficient hardware accelerator or instance type to improve utilisation and reduce energy consumption per inference or training run.
+---
+
+# Select efficient accelerators and instance types for AI workloads
+
+**Applicable Role:** Provider and Consumer
+
+## Description
+
+AI workloads such as training, fine-tuning, and inference require significant compute resources. The type of hardware used, including CPUs, GPUs, TPUs, and specialized accelerators, has a direct impact on energy efficiency and performance.
+
+Different hardware options vary in their ability to execute AI workloads efficiently. Selecting appropriate hardware and compute resources, combined with intelligent workload orchestration across heterogeneous platforms, improves utilization, reduces execution time, and lowers overall energy consumption..
+
+## Solution
+
+- Profile workloads based on latency, throughput, and execution characteristics before selecting compute resources
+- Choose hardware optimized for the specific workload, such as CPUs, GPUs, TPUs, NPUs, or other specialized accelerators
+- Maintain a catalogue of supported accelerator types and their suitability for different workload classes
+- Use orchestration and scheduling systems to automatically dispatch workloads to the most energy-efficient available compute platform
+- Implement closed-loop monitoring and resource allocation to continuously optimize workload placement and avoid over-provisioning
+- Monitor utilization and dynamically adjust resource allocation to improve efficiency
+- Evaluate performance-per-watt benchmarks and runtime telemetry when selecting hardware and instance types
+
+## SCI Impact
+
+**SCI = (E × I) + M per R**
+
+**E (Energy):** Efficient hardware reduces compute time and energy consumption for AI workloads.
+
+**M (Embodied Carbon):** Better utilization can reduce the number of required machines and associated embodied emissions.
+
+## Cost Impact
+
+- **Hardware costs:** Instance type choice affects hourly compute rates significantly
+- **Utilization efficiency:** Better hardware-workload fit reduces per-inference cost
+- **Reserved instance savings:** Efficient hardware selection enables better RI negotiation
+- **Power and cooling costs:** Specialized accelerators may have lower operational energy costs
+- **Trade-off:** Premium or specialized accelerators (GPUs, TPUs, NPUs, neuromorphic processors, or in-memory computing systems) may cost more upfront but can deliver significantly lower cost-per-inference and energy consumption for suitable workloads
+
+## Assumptions
+
+- Suitable hardware options are available for the workload
+- Performance benchmarks reflect real-world usage
+
+## Considerations
+
+- Specialized hardware may not be available in all regions
+- Costs may vary across hardware options
+- Migration to different hardware may require changes in software or frameworks
+- Underutilized hardware can negate efficiency gains
+
+## References
+
+- [MLPerf Benchmarks — ML Hardware Performance](https://mlcommons.org/benchmarks/inference/)
+- [Google Cloud TPU — Purpose-built AI Accelerator](https://cloud.google.com/tpu)
+- [NVIDIA GPU Benchmarks for AI](https://developer.nvidia.com/deep-learning-performance-training-inference)
+- [Spec Power Benchmark — Server Energy Efficiency](https://www.spec.org/power_ssj2008/)
+- [NeuroBench — Benchmarking Neuromorphic Algorithms and Systems](https://neurobench.ai/)

docs/architecture/system-topology/on-demand-execution-ai-agent-workloads.md

@@ -0,0 +1,83 @@
+---
+version: 1.0
+submitted_by: Naveen Balani
+submitted_by_linkedin: https://www.linkedin.com/in/naveenbalani/
+published_date: 
+category: Architecture
+tags: 
+ - ai
+ - machine-learning
+ - compute
+ - serverless
+ - cloud
+ - persona:devops-engineer
+ - persona:software-engineer
+ - persona:ai-ml-engineer
+ - size:medium
+personas: DevOps Engineer, Software Engineer, AI/ML Engineer
+description: Trigger AI and agent workloads only when needed using serverless or event-driven platforms to eliminate idle compute and reduce unnecessary energy consumption.
+---
+
+# Use on-demand execution for AI and agent workloads
+
+**Applicable Role:** Consumer
+
+## Description
+
+AI systems increasingly operate as dynamic, multi-step workflows, especially in agentic architectures where models interact with tools, data sources, and other models.
+
+Keeping compute resources or agent workflows active when not required leads to unnecessary energy consumption. This includes idle infrastructure, continuously running agents, and long-lived orchestration processes.
+
+Using on-demand execution ensures that compute and workflows are triggered only when needed, reducing idle time and improving overall efficiency.
+
+## Solution
+
+- Use serverless or event-driven platforms to execute workloads only when triggered
+- Design agent workflows to run only when required and terminate after task completion
+- Avoid long-running or always-on agent processes unless continuously needed
+- Trigger model calls and tool usage conditionally rather than continuously
+- Use orchestration frameworks that support event-driven execution, workload dispatching, and efficient workflow management
+- Prioritize the use of existing available resources through orchestration before provisioning additional capacity
+- Dynamically scale resources only when workload demand exceeds available capacity
+- Monitor workload execution and adapt resource allocation continuously to optimize utilization and efficiency
+
+## SCI Impact
+
+**SCI = (E × I) + M per R**
+
+**E (Energy):** Reduces energy consumption by eliminating idle compute and unnecessary agent execution.
+
+**I (Carbon Intensity):** On-demand execution can be combined with carbon-aware scheduling (Pattern 5) to trigger workloads during low-carbon periods.
+
+**M (Embodied Carbon):** Improved utilization of shared infrastructure reduces overall hardware demand.
+
+**R (Functional Unit):** For event-driven systems, the functional unit may be expressed as events processed, workflows completed, or agent tasks executed. Event payload size, throughput, and latency requirements can significantly influence overall efficiency.
+
+## Cost Impact
+
+- **Compute costs:** Reduced by eliminating idle infrastructure and always-on processes
+- **Cold start overhead:** Serverless platforms may incur higher per-invocation costs than reserved instances
+- **Provisioned concurrency:** Can mitigate cold starts but adds baseline cost
+- **State management:** Event-driven architectures often require explicit state persistence, recovery, logging, and lifecycle management. Additional storage, messaging, and orchestration components may increase operational complexity and cost.
+- **Trade-off:** Per-invocation serverless pricing vs. reserved instance baseline; evaluate break-even point
+
+## Assumptions
+
+- Workloads and agent workflows can be structured as event-driven processes with appropriate state management, lifecycle instrumentation, and mechanisms for efficiently processing asynchronous event payloads
+- Execution environments support dynamic scaling and orchestration, and workloads can be safely interrupted and resumed without losing state or requiring expensive recomputation
+
+## Considerations
+
+- Cold start latency may impact responsiveness
+- Complex workflows may require careful orchestration design
+- Not all workloads are suitable for on-demand execution
+- Inefficient agent design can still lead to excessive compute even in serverless environments; orchestration and workload scheduling can improve placement efficiency but cannot fully compensate for poorly designed agents
+- Trade-offs between responsiveness, cost, and carbon should be evaluated
+- Synchronous and asynchronous event-processing models present different trade-offs in latency, scalability, and energy efficiency
+
+## References
+
+- [AWS Lambda for ML Inference](https://aws.amazon.com/lambda/)
+- [Google Cloud Functions for Serverless AI](https://cloud.google.com/functions)
+- [Azure Functions — Event-driven Serverless Compute](https://azure.microsoft.com/en-us/products/functions)
+- [Knative — Kubernetes-based Serverless](https://knative.dev/)

docs/architecture/system-topology/run-ai-models-edge.md

@@ -0,0 +1,80 @@
+---
+version: 1.0
+submitted_by: Naveen Balani
+submitted_by_linkedin: https://www.linkedin.com/in/naveenbalani/
+published_date:
+category: Architecture
+tags: 
+ - ai
+ - machine-learning
+ - compute
+ - networking
+ - deployment
+ - persona:infrastructure-engineer
+ - persona:solution-architect
+ - persona:ai-ml-engineer
+ - size:large
+personas: Infrastructure Engineer, Solution Architect, AI/ML Engineer
+description: Deploy AI inference on edge devices or local infrastructure to reduce data transfer, network energy use, and reliance on centralised cloud compute.
+---
+
+# Run AI models at the edge
+
+**Applicable Role:** Provider and Consumer
+
+## Description
+
+AI and ML workloads often rely on centralized cloud infrastructure for training and inference. This requires data to be transferred from source systems to the cloud, increasing network usage, latency, and energy consumption.
+
+Running AI models closer to where data is generated or consumed, such as on edge devices or local infrastructure, reduces data movement and enables more efficient processing. This is especially relevant for real-time, high-frequency, or latency-sensitive inference workloads.
+
+Providers also deploy edge inference capabilities through on-device ML SDKs and embedded models, making this pattern applicable to both roles.
+
+## Solution
+
+- Classify workloads and edge devices based on latency requirements, data volume, compute intensity, memory constraints, and power availability to determine suitable deployment targets
+- Evaluate hybrid architectures that combine edge and cloud processing based on workload requirements
+- Deploy models on edge devices or local infrastructure when doing so reduces data transfer, latency, or centralized compute requirements
+- Perform workload-specific data preprocessing tasks such as filtering, cleansing, aggregation, and feature generation locally
+- Use edge inference for real-time, high-frequency, or latency-sensitive applications
+- Limit transmission of raw data by sending only necessary, filtered, or aggregated results to the cloud
+- For applications using external AI services, consider on-device or local inference options to reduce repeated remote calls
+
+## SCI Impact
+
+**SCI = (E × I) + M per R**
+
+**E (Energy):** Reduced data transfer and localized processing lower energy consumption associated with network and centralized compute.
+
+**I (Carbon Intensity):** Edge devices run on local power grids which may have different carbon intensity than centralized data center regions. This should be measured and factored into the SCI calculation.
+
+**M (Embodied Carbon):** Edge deployments may increase device footprint, but can reduce reliance on large centralized infrastructure.
+
+## Cost Impact
+
+- **Cloud compute costs:** Reduced by moving inference to edge devices
+- **Network costs:** Lower data transfer to centralized systems
+- **Edge device costs:** Hardware choices range from conventional CPUs and GPUs to specialized low-power accelerators and emerging architectures, with different cost, performance, and energy-efficiency trade-offs
+- **Model management costs:** Higher due to complexity of distributed model updates
+- **Trade-off:** Cloud cost savings offset by edge device and management overhead
+
+## Assumptions
+
+- Edge or local devices have sufficient memory, compute capacity, and power to support both workload-specific preprocessing and model execution, potentially using optimized, compressed, or quantized models
+- Workloads can be partitioned effectively between edge and cloud
+
+## Considerations
+
+- Embodied emissions of edge devices must be accounted for
+- Edge environments may have limited compute, storage, connectivity, availability, and responsiveness
+- Model updates and lifecycle management can be more complex in distributed systems
+- Workload partitioning between edge and cloud should consider latency, energy consumption, network usage, device capabilities, operational constraints, and carbon intensity differences
+- Not all workloads are suitable for edge deployment; trade-offs between latency, cost, carbon, and operational complexity should be evaluated
+
+## References
+
+- [TensorFlow Lite — On-device ML Framework](https://www.tensorflow.org/lite)
+- [ONNX Runtime — Cross-platform Inference Engine](https://onnxruntime.ai/)
+- [NVIDIA Jetson Platform — Edge AI Computing](https://developer.nvidia.com/embedded-computing)
+- [Green AI for IIoT: Energy Efficient Intelligent Edge Computing](https://arxiv.org/abs/2205.02343)
+- [SCI-AI Specification — Green Software Foundation](https://github.com/Green-Software-Foundation/sci-ai/blob/dev/SPEC.md)