Initial Commit

2025-12-11 22:16:54 +00:00
commit 841d109689
5 changed files with 579 additions and 0 deletions
@@ -0,0 +1,205 @@
+# National Grid Real-Time Data Monitor
+
+A Python application that fetches real-time data from the National Grid Carbon Intensity API, including:
+- Load and demand information
+- Fuel mixture (generation mix)
+- Carbon intensity forecasts
+- Regional and national statistics
+
+## Features
+
+- Configurable Grid Supply Point (GSP) selection
+- Real-time data collection from National Grid APIs
+- JSON output format (perfect for Grafana integration)
+- Regional generation mix data
+- National statistics and carbon intensity
+
+## Installation
+
+No dependencies required beyond Python 3 standard library and `requests`:
+
+```bash
+pip install requests
+```
+
+## Usage
+
+### Default (Bishops Wood):
+```bash
+python3 nationalgrid_monitor.py
+```
+
+### Specify a different GSP:
+```bash
+python3 nationalgrid_monitor.py --gsp "London"
+python3 nationalgrid_monitor.py --gsp "North Scotland"
+python3 nationalgrid_monitor.py --gsp "Yorkshire"
+```
+
+### List available GSPs:
+```bash
+python3 nationalgrid_monitor.py --list
+```
+
+## Available GSPs
+
+- Bishops Wood (West Midlands)
+- North Scotland
+- South Scotland
+- North West England
+- North East England
+- Yorkshire
+- North Wales
+- South Wales
+- West Midlands
+- East Midlands
+- East England
+- South West England
+- South England
+- London
+- South East England
+
+## Output
+
+Data is saved to `<GSP_Name>.json` in the current directory.
+
+Example output structure:
+```json
+{
+  "gsp_name": "Bishops Wood",
+  "gsp_id": 11,
+  "timestamp": "2025-12-11T17:39:24.668415Z",
+  "collection_time": "2025-12-11T17:39:24.668436",
+  "regional": {
+    "intensity": {...},
+    "generation_mix": [...]
+  },
+  "national": {
+    "intensity": {...}
+  },
+  "generation": {
+    "mix": [...]
+  }
+}
+```
+
+## VictoriaMetrics Integration
+
+The tool pushes metrics to VictoriaMetrics using the Prometheus exposition format via the `/api/v1/import/prometheus` endpoint.
+
+### Supported Endpoints
+
+**Direct VictoriaMetrics** (single-node):
+- Default: `http://localhost:8428/api/v1/import/prometheus`
+
+**vmagent** (if using vmagent as a proxy):
+- Typically: `http://localhost:8429/api/v1/import/prometheus`
+
+**VictoriaMetrics Cluster** (vminsert):
+- Format: `http://vminsert:8480/insert/0/prometheus/api/v1/import/prometheus`
+
+### Usage
+
+```bash
+# Push to VictoriaMetrics single-node on localhost (default)
+python3 nationalgrid_monitor.py --gsp "Bishops Wood"
+
+# Push to vmagent
+python3 nationalgrid_monitor.py --victoriametrics http://localhost:8429
+
+# Push to VictoriaMetrics cluster
+python3 nationalgrid_monitor.py --victoriametrics http://vminsert:8480/insert/0/prometheus
+
+# Push to custom VictoriaMetrics instance
+python3 nationalgrid_monitor.py --victoriametrics http://your-vm-host:8428
+
+# Skip VictoriaMetrics push (JSON only)
+python3 nationalgrid_monitor.py --no-push
+```
+
+### Metrics Format
+
+All metrics are exported as `national_grid{gsp="<gsp_name>",metric="<metric_name>"} <value>`
+
+Available metrics:
+- `demand_mw` - **Actual national demand in MW** (from Elexon BMRS API - no API key required)
+- `carbon_intensity` - Actual carbon intensity (gCO2/kWh)
+- `carbon_intensity_forecast` - Forecast carbon intensity (gCO2/kWh)
+- `load_indicator` - Load indicator based on carbon intensity
+- `wind` - Wind generation percentage
+- `gas` - Gas generation percentage
+- `nuclear` - Nuclear generation percentage
+- `biomass` - Biomass generation percentage
+- `solar` - Solar generation percentage
+- `hydro` - Hydro generation percentage
+- `coal` - Coal generation percentage
+- `imports` - Imports percentage
+- `other` - Other generation percentage
+
+### Continuous Monitoring
+
+Run the monitoring loop to continuously push data:
+
+```bash
+# Default: Bishops Wood, 1-minute intervals
+./monitor_loop.sh
+
+# Custom GSP and interval (seconds)
+./monitor_loop.sh "London" 60
+
+# Custom interval (e.g., 5 minutes)
+./monitor_loop.sh "Bishops Wood" 300
+
+# Custom VictoriaMetrics URL
+./monitor_loop.sh "London" 60 "http://10.0.0.5:8428"
+```
+
+## Grafana Integration
+
+Once metrics are in VictoriaMetrics:
+
+1. Add VictoriaMetrics as a Prometheus datasource in Grafana
+2. Query metrics like: `national_grid{gsp="Bishops_Wood",metric="wind"}`
+3. Create dashboards with panels for:
+   - Generation mix (stacked graph)
+   - Carbon intensity (time series)
+   - Individual fuel sources (gauges)
+   - Load/demand trends
+
+### Example PromQL Queries
+
+```promql
+# Wind generation for Bishops Wood
+national_grid{gsp="Bishops_Wood",metric="wind"}
+
+# Carbon intensity across all GSPs
+national_grid{metric="carbon_intensity"}
+
+# Gas generation comparison
+national_grid{metric="gas"}
+
+# Total renewable (wind + solar + hydro)
+national_grid{metric="wind"} + national_grid{metric="solar"} + national_grid{metric="hydro"}
+```
+
+## API Data Sources
+
+This tool uses multiple free APIs (no API key required):
+
+### Carbon Intensity API
+- Real-time carbon intensity data
+- Generation mix by fuel type (biomass, coal, gas, nuclear, wind, solar, etc.)
+- Regional and national statistics
+- Half-hourly forecasts (minute resolution available)
+- API: https://api.carbonintensity.org.uk/
+- Resolution: minute (for most recent data)
+
+### Elexon BMRS API
+- **Actual national demand in MW** (updated every 5 minutes)
+- No API key required for demand data
+- API: https://data.elexon.co.uk/bmrs/api/v1/demand/outturn/summary
+- Resolution: minute (for latest data)
+
+## License
+
+MIT