nationalgrid/README.md

National Grid Real-Time Data Monitor

A Python application that fetches real-time data from the National Grid Carbon Intensity API, including:

• Load and demand information
• Fuel mixture (generation mix)
• Carbon intensity forecasts
• Regional and national statistics

Features

• Configurable Grid Supply Point (GSP) selection
• Real-time data collection from National Grid APIs
• JSON output format (perfect for Grafana integration)
• Regional generation mix data
• National statistics and carbon intensity

Installation

No dependencies required beyond Python 3 standard library and requests:

pip install requests

Usage

Default (Bishops Wood):

python3 nationalgrid_monitor.py

Specify a different GSP:

python3 nationalgrid_monitor.py --gsp "London"
python3 nationalgrid_monitor.py --gsp "North Scotland"
python3 nationalgrid_monitor.py --gsp "Yorkshire"

List available GSPs:

python3 nationalgrid_monitor.py --list

Available GSPs

• Bishops Wood (West Midlands)
• North Scotland
• South Scotland
• North West England
• North East England
• Yorkshire
• North Wales
• South Wales
• West Midlands
• East Midlands
• East England
• South West England
• South England
• London
• South East England

Output

Data is saved to <GSP_Name>.json in the current directory.

Example output structure:

{
  "gsp_name": "Bishops Wood",
  "gsp_id": 11,
  "timestamp": "2025-12-11T17:39:24.668415Z",
  "collection_time": "2025-12-11T17:39:24.668436",
  "regional": {
    "intensity": {...},
    "generation_mix": [...]
  },
  "national": {
    "intensity": {...}
  },
  "generation": {
    "mix": [...]
  }
}

VictoriaMetrics Integration

The tool pushes metrics to VictoriaMetrics using the Prometheus exposition format via the /api/v1/import/prometheus endpoint.

Supported Endpoints

Direct VictoriaMetrics (single-node):

• Default: http://localhost:8428/api/v1/import/prometheus

vmagent (if using vmagent as a proxy):

• Typically: http://localhost:8429/api/v1/import/prometheus

VictoriaMetrics Cluster (vminsert):

• Format: http://vminsert:8480/insert/0/prometheus/api/v1/import/prometheus

Usage

# Push to VictoriaMetrics single-node on localhost (default)
python3 nationalgrid_monitor.py --gsp "Bishops Wood"

# Push to vmagent
python3 nationalgrid_monitor.py --victoriametrics http://localhost:8429

# Push to VictoriaMetrics cluster
python3 nationalgrid_monitor.py --victoriametrics http://vminsert:8480/insert/0/prometheus

# Push to custom VictoriaMetrics instance
python3 nationalgrid_monitor.py --victoriametrics http://your-vm-host:8428

# Skip VictoriaMetrics push (JSON only)
python3 nationalgrid_monitor.py --no-push

Metrics Format

All metrics are exported as national_grid{gsp="<gsp_name>",metric="<metric_name>"} <value>

Available metrics:

• demand_mw - Actual national demand in MW (from Elexon BMRS API - no API key required)
• carbon_intensity - Actual carbon intensity (gCO2/kWh)
• carbon_intensity_forecast - Forecast carbon intensity (gCO2/kWh)
• load_indicator - Load indicator based on carbon intensity
• wind - Wind generation percentage
• gas - Gas generation percentage
• nuclear - Nuclear generation percentage
• biomass - Biomass generation percentage
• solar - Solar generation percentage
• hydro - Hydro generation percentage
• coal - Coal generation percentage
• imports - Imports percentage
• other - Other generation percentage

Continuous Monitoring

Run the monitoring loop to continuously push data:

# Default: Bishops Wood, 1-minute intervals
./monitor_loop.sh

# Custom GSP and interval (seconds)
./monitor_loop.sh "London" 60

# Custom interval (e.g., 5 minutes)
./monitor_loop.sh "Bishops Wood" 300

# Custom VictoriaMetrics URL
./monitor_loop.sh "London" 60 "http://10.0.0.5:8428"

Systemd Service (Autostart on Boot)

To run the monitor automatically as a system service:

1. Install the service:

sudo cp nationalgrid-monitor.service /etc/systemd/system/
sudo systemctl daemon-reload

2. Customize settings (optional): Edit /etc/systemd/system/nationalgrid-monitor.service and modify the ExecStart line to change:

• GSP name (default: "Bishops Wood")
• Update interval in seconds (default: 60)
• VictoriaMetrics URL (default: http://localhost:8428)

3. Enable and start the service:

sudo systemctl enable nationalgrid-monitor.service
sudo systemctl start nationalgrid-monitor.service

4. Check service status:

sudo systemctl status nationalgrid-monitor.service

5. View logs:

sudo journalctl -u nationalgrid-monitor.service -f

6. Stop or disable:

sudo systemctl stop nationalgrid-monitor.service
sudo systemctl disable nationalgrid-monitor.service

Grafana Integration

Once metrics are in VictoriaMetrics:

1. Add VictoriaMetrics as a Prometheus datasource in Grafana
2. Query metrics like: national_grid{gsp="Bishops_Wood",metric="wind"}
3. Create dashboards with panels for:
   • Generation mix (stacked graph)
   • Carbon intensity (time series)
   • Individual fuel sources (gauges)
   • Load/demand trends

Example PromQL Queries

# Wind generation for Bishops Wood
national_grid{gsp="Bishops_Wood",metric="wind"}

# Carbon intensity across all GSPs
national_grid{metric="carbon_intensity"}

# Gas generation comparison
national_grid{metric="gas"}

# Total renewable (wind + solar + hydro)
national_grid{metric="wind"} + national_grid{metric="solar"} + national_grid{metric="hydro"}

API Data Sources

This tool uses multiple free APIs (no API key required):

Carbon Intensity API

• Real-time carbon intensity data
• Generation mix by fuel type (biomass, coal, gas, nuclear, wind, solar, etc.)
• Regional and national statistics
• Half-hourly forecasts (minute resolution available)
• API: https://api.carbonintensity.org.uk/
• Resolution: minute (for most recent data)

Elexon BMRS API

• Actual national demand in MW (updated every 5 minutes)
• No API key required for demand data
• API: https://data.elexon.co.uk/bmrs/api/v1/demand/outturn/summary
• Resolution: minute (for latest data)

License

MIT