Methodology

A deep dive into our energy consumption calculations

Calculation Process

How We Calculate AI Resource Usage

1. Predicting Response Length

We analyze your prompt to estimate how many tokens (word pieces) the AI will use in its response:

Task-Based Estimates

Code generation: 4x input length (medium confidence)
Creative writing: 5x input length (low confidence)
Analysis tasks: 3x input length (medium confidence)
Simple questions: 2x input length (high confidence)
Summaries: 0.5x input length (high confidence)

Specialized Tasks

Translation: 1.5x input length (high confidence)
Data analysis: 4.5x input length (medium confidence)
Customer service: 2.5x input length (medium confidence)
Research: 6x input length (low confidence)

Confidence Levels

High confidence: Predictable tasks like translation (±20% variance)
Medium confidence: Technical tasks like coding (±30-40% variance)
Low confidence: Open-ended tasks like research (±50% variance)

2. Computing Energy Usage

Energy consumption is calculated using three main components:

Base Power Usage

Active GPU power (70W - 700W)
Memory power consumption (25W - 120W)
Idle system power (15W - 150W)

Efficiency Factors

Batch processing efficiency (larger batches = better efficiency)
Hardware age impact (0-10% yearly degradation)
Memory bandwidth limitations
Data center efficiency (PUE: 1.1-1.6)

3. Estimating Water Usage

Water consumption is calculated based on cooling requirements:

Base Factors

Air cooling: 1.8L per kWh
Water cooling: 2.2L per kWh
Hybrid systems: 1.9L per kWh
Free cooling: 1.5L per kWh

Environmental Adjustments

Location: ±50% (hot vs cold climates)
Season: ±40% (summer vs winter)
Cooling efficiency: 85-95%

Understanding the Results

Our calculations provide estimates based on real-world hardware specifications and environmental factors. While exact usage may vary, these estimates help understand the environmental impact of AI operations and identify opportunities for optimization.

Environmental Factors

Our calculations incorporate both energy and water-related variables:

Energy Metrics

TDP Range: 70W - 700W
Memory Power: 25W - 120W
Idle Power: 15W - 150W
PUE Factor: 1.1 - 1.6

Water Usage

Base Usage: 1.5-2.2L/kWh
Location Impact: ±50%
Seasonal Variance: ±40%

Hardware Performance Factors

Our calculations now incorporate detailed hardware specifications and limitations:

Performance Metrics

Memory Bandwidth: 320-5300 GB/s
Age Degradation: 0-10% yearly
Batch Processing Efficiency
Memory Utilization Impact

Power Components

Core Processing (TDP)
Memory Power Draw
Idle Power Consumption

Detailed Calculations

Energy Calculation Formula

Energy = ((TDP × Utilization) + (Memory Power × Memory Utilization) + Idle Power) × GPU Count × Processing Time × PUE × Batch Factor × Model Factor

Processing Time = Tokens / (Inference Speed × Efficiency Factors)
Batch Factor = BatchSize^0.8 (empirically derived)
Memory Utilization = min(1.0, Model Memory / Bandwidth)
Uncertainty Range: ±15-30% depending on hardware

Water Usage Formula

Water = Energy × Cooling Factor × Location Factor × Seasonal Factor

Base Water Usage: 1.5-2.2L/kWh (system dependent)
Location Impact: 0.5x to 1.5x multiplier
Seasonal Variance: 0.6x to 1.4x multiplier
Uncertainty Range: ±20-40% based on conditions

Model Specifications

Model	Parameters	Context	Energy Multiplier
Claude 3 Sonnet	175B	200,000	1.2x
GPT-4	1000B	128,000	1.5x
Claude 3 Opus	1000B	200,000	1.6x
GPT-3.5 Turbo	175B	16,000	1x
Mistral 7B	7B	32,000	0.7x
GPT-O1	16B	32,000	0.8x

Hardware Specifications

NVIDIA A100

TDP:

400W

Inference Speed:

1,000 tokens/s

Memory Power:

95W

Idle Power:

100W

Memory Bandwidth:

1,555 GB/s

Release Year:

2020

Performance Factor:

0.95x

Relative Performance

NVIDIA H200

TDP:

700W

Inference Speed:

2,000 tokens/s

Memory Power:

130W

Idle Power:

150W

Memory Bandwidth:

4,800 GB/s

Release Year:

2023

Performance Factor:

Relative Performance

NVIDIA H100

TDP:

700W

Inference Speed:

1,900 tokens/s

Memory Power:

120W

Idle Power:

150W

Memory Bandwidth:

3,350 GB/s

Release Year:

2022

Performance Factor:

Relative Performance

NVIDIA A40

TDP:

300W

Inference Speed:

600 tokens/s

Memory Power:

70W

Idle Power:

60W

Memory Bandwidth:

696 GB/s

Release Year:

2020

Performance Factor:

0.95x

Relative Performance

NVIDIA A30

TDP:

165W

Inference Speed:

400 tokens/s

Memory Power:

50W

Idle Power:

40W

Memory Bandwidth:

933 GB/s

Release Year:

2021

Performance Factor:

0.97x

Relative Performance

NVIDIA T4

TDP:

70W

Inference Speed:

500 tokens/s

Memory Power:

25W

Idle Power:

15W

Memory Bandwidth:

320 GB/s

Release Year:

2018

Performance Factor:

0.9x

Relative Performance

NVIDIA L4

TDP:

72W

Inference Speed:

600 tokens/s

Memory Power:

30W

Idle Power:

20W

Memory Bandwidth:

408 GB/s

Release Year:

2023

Performance Factor:

Relative Performance

Google TPU v4

TDP:

175W

Inference Speed:

1,200 tokens/s

Memory Power:

45W

Idle Power:

40W

Memory Bandwidth:

1,200 GB/s

Release Year:

2022

Performance Factor:

0.98x

Relative Performance

AMD MI250X

TDP:

560W

Inference Speed:

900 tokens/s

Memory Power:

100W

Idle Power:

120W

Memory Bandwidth:

3,200 GB/s

Release Year:

2021

Performance Factor:

0.97x

Relative Performance

AMD MI210

TDP:

300W

Inference Speed:

700 tokens/s

Memory Power:

80W

Idle Power:

70W

Memory Bandwidth:

1,600 GB/s

Release Year:

2021

Performance Factor:

0.97x

Relative Performance

AWS Trainium2

TDP:

375W

Inference Speed:

800 tokens/s

Memory Power:

85W

Idle Power:

70W

Memory Bandwidth:

820 GB/s

Release Year:

2023

Performance Factor:

Relative Performance

AWS Inferentia2

TDP:

200W

Inference Speed:

1,100 tokens/s

Memory Power:

60W

Idle Power:

45W

Memory Bandwidth:

750 GB/s

Release Year:

2023

Performance Factor:

Relative Performance

AMD MI300X

TDP:

750W

Inference Speed:

1,800 tokens/s

Memory Power:

140W

Idle Power:

160W

Memory Bandwidth:

5,300 GB/s

Release Year:

2023

Performance Factor:

Relative Performance

Environmental Impact Factors

Cooling Systems

Air Cooled

Traditional air cooling with minimal water usage

Efficiency: 0.85

Water Usage: 1.8L/kWh

Water Cooled

Direct water cooling for better efficiency

Efficiency: 0.95

Water Usage: 2.2L/kWh

Hybrid

Combined air and water cooling

Efficiency: 0.9

Water Usage: 1.9L/kWh

Free Cooling

Uses outside air when possible

Efficiency: 0.92

Water Usage: 1.5L/kWh

Location Factors

Temperate

Moderate climate zones

Multiplier: 1x

Tropical

Hot and humid regions

Multiplier: 1.3x

Arid

Hot and dry regions

Multiplier: 1.5x

Cold

Cold climate zones

Multiplier: 0.8x

Seasonal Factors

December - February

Multiplier: 0.8x

March - May

Multiplier: 1x

June - August

Multiplier: 1.4x

September - November

Multiplier: 1x