Energy-Aware Compute Architecture

Modular design for large-scale AI training with energy transparency and grid resilience

Architectural Layers

Layer 5: Grid & Battery Aware Scheduler

Inputs

Electricity price signals
Carbon intensity data
Battery state-of-charge
Cooling constraints

Outputs

Energy budgets per job
Deadline adjustments
Throttling directives
Priority signals

Layer 4: Workload Energy Forecasting

Models

Analytical FLOP estimates
ML-based predictors
Hybrid approaches
Dynamic coefficients

Outputs

Estimated joules
Peak power prediction
Elasticity profile
Deadline constraints

Layer 3: Runtime Energy Telemetry

Functions

Energy attribution
Per-block accounting
Dynamic coefficients
Forecast validation

Interfaces

Process-level API
Kernel hooks
Energy counters
Debug endpoints

Layer 2: Energy Control ABI

Control Points

Power caps (W)
Energy budgets (J)
Frequency limits
Concurrency controls

Properties

Versioned interface
Hardware stable
Model agnostic
Extensible

Layer 1: Microprocessor Hardware

Telemetry

Per-domain power
Performance counters
DVFS states
Throttle indicators

Controls

Power gating
SM scheduling
Memory bandwidth
Budget enforcement

Energy Control ABI Specification

Design Principles

Stability: Hardware interface remains unchanged even as forecasting models evolve
Minimalism: Only essential controls exposed to upper layers
Extensibility: Versioned interface with reserved fields for future expansion
Isolation: No dependency on network protocols or external schemas

ABI v1.0 Definition

Register	Type	Description	Range/Options
POWER_CAP	uint32_t	Maximum allowed power in milliwatts	0 - TDP
ENERGY_BUDGET	uint64_t	Allowed energy consumption in microjoules	0 - UINT64_MAX
FREQ_CEILING	uint16_t	Maximum allowed frequency in MHz	MinFreq - MaxFreq
SM_CONCURRENCY	uint8_t	Maximum concurrent SMs (0 = auto)	0 - TotalSMs
MEM_BW_LIMIT	uint16_t	Memory bandwidth limit as percentage	10 - 100
DEADLINE_HINT	uint64_t	Completion deadline in microseconds	0 (no deadline) - UINT64_MAX
ELASTICITY	uint8_t	Allowed scheduling flexibility (0-100)	0 (rigid) - 100 (elastic)
ABI_VERSION	uint16_t	Interface version (read-only)	0x0100 (v1.0)

Versioning Strategy

The ABI uses semantic versioning where the upper byte represents major version (breaking changes) and lower byte represents minor version (backward-compatible additions). Hardware must maintain backward compatibility with at least the previous major version.

Hardware Telemetry Model

Required Telemetry Domains

Compute Units: Per-SM power and utilization
Tensor Cores: Specialized accelerator power
Memory Hierarchy: DRAM, cache, and interconnect
Voltage Domains: Per-domain voltage/frequency
Thermal Sensors: Junction and package temps

Telemetry Interface

// Example telemetry register layout
struct energy_telemetry {
  uint32_t domain_id; // Domain identifier
  uint32_t power_mw; // Instantaneous power (mW)
  uint64_t energy_uj; // Accumulated energy (μJ)
  uint16_t freq_mhz; // Current frequency (MHz)
  uint8_t utilization; // Utilization percentage
  uint8_t reserved[3]; // Future expansion
};

Forecast-to-Hardware Mapping

Translation Process

Forecast model predicts energy requirements
Scheduler considers grid/battery constraints
Translates high-level goals to hardware parameters
Sets ABI registers with appropriate values
Hardware enforces constraints during execution

Example Mappings

Carbon-Aware Scheduling

High carbon intensity → Reduce POWER_CAP, increase ELASTICITY

Battery Optimization

Low battery → Strict ENERGY_BUDGET, adjust FREQ_CEILING

Deadline-Driven

Tight deadline → Maximize SM_CONCURRENCY, relax POWER_CAP

Simulation Strategy

Cycle-Accurate Simulator

Extend existing simulators (Gem5, SST) with energy models for each hardware domain. Implement ABI as simulated registers with appropriate timing.

Discrete-Event Simulation

Model the full stack in Python/SimPy with abstract timing. Focus on control flow between layers and constraint propagation.

Hardware Emulation

Use FPGAs to prototype the control interface with simplified compute elements. Enables validation of real-time constraints.

Simulation Components

Energy Model

Per-component power equations
DVFS-aware coefficients
Temperature-dependent leakage
Activity factor estimation

Workload Generator

AI training traces
HPC benchmark patterns
Parameterized intensity
Memory access profiles

Research Risks & Design Trade-offs

Hardware Coupling

Risk of baking forecasting assumptions into silicon. Mitigate by keeping ABI minimal and versioned.

Accuracy vs Overhead

Fine-grained telemetry increases energy costs. Need optimal sampling rates and aggregation.

Modularity Boundaries

Clear interfaces required between layers while maintaining system-wide optimization potential.

Forecasting Accuracy

Poor predictions may lead to inefficient execution. Need adaptive models and fallback modes.

Timescale Alignment

Grid signals (minutes) vs hardware control (μs) require careful temporal hierarchy design.

Extensibility

Future accelerators may need new control knobs. ABI must accommodate via versioning.

Extensibility Framework

Versioning Strategy

ABI_VERSION register allows runtime detection
Major versions indicate breaking changes
Minor versions add optional features
Reserved fields in all structures
Deprecation policy with multi-version support

Example Version Path

v1.0: Basic power/energy controls

v1.1: Add elasticity hints

v2.0: New memory hierarchy controls

Forward Compatibility

Optional Features

New capabilities can be discovered via capability registers without breaking existing implementations.

Fallback Behavior

Unsupported controls should degrade gracefully (e.g., ignore elasticity hints if not supported).

Extension Points

Reserved registers and structure padding allow for future expansion without layout changes.