Energy Information Administration

US Electricity

Introduction

The US Electricity dataset by the U.S. Energy Information Administration tracks how the US power grid actually ran, day by day. The data covers 81 US balancing authorities, starts in January 2019, and is delivered on a daily frequency. This dataset is created by collecting the Form EIA-930 filings that every balancing authority in the country submits to the EIA, covering demand, the day-ahead demand forecast, net generation, net interchange with neighbouring grids, and the generation split across sixteen fuel types.

For more information about the US Electricity dataset, including CLI commands and pricing, see the dataset listing.

About the Provider

The U.S. Energy Information Administration is the statistical and analytical agency within the U.S. Department of Energy, created by Congress in 1977 to collect, analyze and disseminate energy information independent of policy advocacy. The agency covers every energy source in the economy, from petroleum, natural gas and coal to electricity, nuclear and renewables, and publishes the estimates that policymakers, industry and researchers work from. Its responsibilities range from tracking production, consumption, stocks and prices across the country to monitoring the day to day operation of the power grid, and all of its data is a public record available free of charge.

Getting Started

The following snippet demonstrates how to request data from the US Electricity dataset:

self.dataset_symbol = self.add_data(EIAElectricity, EIA.BalancingAuthorities.PJM, Resolution.DAILY).symbol
_datasetSymbol = AddData<EIAElectricity>(EIA.BalancingAuthorities.PJM, Resolution.Daily).Symbol;

Data Summary

The following table describes the dataset properties:

PropertyValue
Start DateJanuary 2019
Data Points210,512
Asset Coverage81 US Balancing Authorities
ResolutionDaily
TimezoneNew York

Requesting Data

To add US Electricity data to your algorithm, call the AddDataadd_data method. The dataset is unlinked, so instead of a security Symbol you pass a balancing authority code from the EIA.BalancingAuthorities helper. Save a reference to the dataset Symbol so you can access the data later in your algorithm.

One subscription carries every series the authority reports: demand, the day-ahead demand forecast, net generation, net interchange with neighbouring grids and the generation split across sixteen fuel types. Grid operations are a signal, not a tradeable instrument, so add a separate tradeable security if you want to place orders.

class EIAElectricityDataAlgorithm(QCAlgorithm):
    def initialize(self) -> None:
        self.set_start_date(2020, 6, 1)
        self.set_end_date(2020, 9, 1)
        self.set_cash(100000)

        self._spy = self.add_equity("SPY", Resolution.DAILY).symbol
        self._pjm = self.add_data(EIAElectricity, EIA.BalancingAuthorities.PJM, Resolution.DAILY).symbol
public class EIAElectricityDataAlgorithm : QCAlgorithm
{
    private Symbol _spy, _pjm;

    public override void Initialize()
    {
        SetStartDate(2020, 6, 1);
        SetEndDate(2020, 9, 1);
        SetCash(100000);

        _spy = AddEquity("SPY", Resolution.Daily).Symbol;
        _pjm = AddData<EIAElectricity>(EIA.BalancingAuthorities.PJM, Resolution.Daily).Symbol;
    }
}

Accessing Data

To get the current US Electricity data, index the current Slice with the dataset Symbol. Slice objects deliver unique events to your algorithm as they happen, but the Slice may not contain data for your dataset at every time step. To avoid issues, check if the Slice contains the data you want before you index it.

A balancing authority only reports the series that apply to it, so the fields are nullable. A null means the series is absent, while a zero is a real reading of zero. Check for a missing value before you act on it.

def on_data(self, slice: Slice) -> None:
    if slice.contains_key(self._pjm):
        data_point = slice[self._pjm]
        if data_point.demand is not None and data_point.demand_forecast is not None:
            self.log(f"{self._pjm} load surprise at {slice.time}: {data_point.demand - data_point.demand_forecast}")
public override void OnData(Slice slice)
{
    if (slice.ContainsKey(_pjm))
    {
        var dataPoint = slice[_pjm];
        if (dataPoint.Demand.HasValue && dataPoint.DemandForecast.HasValue)
        {
            Log($"{_pjm} load surprise at {slice.Time}: {dataPoint.Demand - dataPoint.DemandForecast}");
        }
    }
}

To iterate through all of the dataset objects in the current Slice, call the Getget method.

def on_data(self, slice: Slice) -> None:
    for dataset_symbol, data_point in slice.get(EIAElectricity).items():
        self.log(f"{dataset_symbol} at {slice.time}: demand {data_point.demand}, net generation {data_point.net_generation}")
public override void OnData(Slice slice)
{
    foreach (var kvp in slice.Get<EIAElectricity>())
    {
        var datasetSymbol = kvp.Key;
        var dataPoint = kvp.Value;
        Log($"{datasetSymbol} at {slice.Time}: demand {dataPoint.Demand}, net generation {dataPoint.NetGeneration}");
    }
}

Historical Data

To get historical US Electricity data, call the Historyhistory method with the dataset Symbol. If there is no data in the period you request, the history result is empty. The data is daily, so a request for 100 data points covers a little over three months of grid operations.

# DataFrame
history_df = self.history(self._pjm, 100, Resolution.DAILY)

# Dataset objects
history_bars = self.history[EIAElectricity](self._pjm, 100, Resolution.DAILY)
var history = History<EIAElectricity>(_pjm, 100, Resolution.Daily);

For more information about historical data, see History Requests.

Remove Subscriptions

To remove your subscription to US Electricity data, call the RemoveSecurityremove_security method.

self.remove_security(self._pjm)
RemoveSecurity(_pjm);

Supported Balancing Authorities

Every balancing authority in the report has a readable named constant in the EIA.BalancingAuthorities helper, which resolves to the EIA-930 code you pass to AddDataadd_data. Type EIA.BalancingAuthorities. in the editor and autocomplete will list them all. The major grid operators also carry their common short name:

ConstantBalancing Authority
EIA.BalancingAuthorities.PJMPJM Interconnection, the largest grid operator in the country
EIA.BalancingAuthorities.ERCOTElectric Reliability Council of Texas
EIA.BalancingAuthorities.CAISOCalifornia Independent System Operator
EIA.BalancingAuthorities.MISOMidcontinent Independent System Operator
EIA.BalancingAuthorities.NYISONew York Independent System Operator
EIA.BalancingAuthorities.ISONEISO New England
EIA.BalancingAuthorities.SPPSouthwest Power Pool
EIA.BalancingAuthorities.BPABonneville Power Administration

The rest of the coverage is individual utilities and regional aggregates, which carry the name the EIA reports for them, such as EIA.BalancingAuthorities.SeattleCityLightEIA.BalancingAuthorities.SEATTLE_CITY_LIGHT and EIA.BalancingAuthorities.UnitedStatesLower48EIA.BalancingAuthorities.UNITED_STATES_LOWER_48.

A balancing authority only reports the series that apply to it, so a field can be empty. Those fields are nullable, and a null means the series is absent while a zero is a real reading. Check for a missing value before you use it.

Example Applications

The US Electricity dataset lets you trade the physical side of the energy market, where demand and generation are measured instead of forecast. Examples include the following strategies:

  • Trading utility equities on load surprise, going long when actual demand runs above the day-ahead forecast and the grid turns out tighter than operators expected.
  • Trading natural gas on power sector gas burn, using gas generation as a share of the fuel mix as a read on physical demand for the fuel.
  • Trading coal and gas producers on fuel substitution, following the share of generation that shifts from one fuel to the other.
  • Reading regional stress through net interchange, which shows which grids are importing power to cover their own load.
  • Tracking the renewable share of generation across balancing authorities as a fundamental signal for clean energy names.

Classic Algorithm Example

The following example algorithm trades SPY on load surprise in the PJM grid, the largest balancing authority in the country. When actual demand runs more than 2% above the day-ahead forecast, the grid turned out tighter than operators expected, so the algorithm goes long. When demand undershoots the forecast by the same margin, it steps aside. Grid operations are a signal, not a tradeable instrument, so the orders go to a separate security.

from AlgorithmImports import *

class EIAElectricityDataAlgorithm(QCAlgorithm):
    def initialize(self):
        self.set_start_date(2020, 6, 1)
        self.set_end_date(2020, 9, 1)
        self.set_cash(100000)

        self._spy = self.add_equity("SPY", Resolution.DAILY).symbol
        self._pjm = self.add_data(EIAElectricity, EIA.BalancingAuthorities.PJM, Resolution.DAILY).symbol
        history = self.history(EIAElectricity, self._pjm, 60, Resolution.DAILY)

        self.debug(f"We got {len(history)} items from our history request")

    def on_data(self, slice):
        if not slice.contains_key(self._pjm):
            return

        grid = slice[self._pjm]
        if grid.demand is None or not grid.demand_forecast:
            return

        # Load surprise as a fraction of the forecast, so it is comparable across days
        load_surprise = (grid.demand - grid.demand_forecast) / grid.demand_forecast

        # Demand running more than 2% above the day-ahead forecast means the grid
        # was tighter than operators expected, so go long
        if load_surprise > 0.02:
            self.set_holdings(self._spy, 1)

        # Demand undershooting by the same margin means slack, so step aside
        elif load_surprise < -0.02:
            self.liquidate(self._spy)
public class EIAElectricityDataAlgorithm : QCAlgorithm
{
    private Symbol _spy, _pjm;

    public override void Initialize()
    {
        SetStartDate(2020, 6, 1);
        SetEndDate(2020, 9, 1);
        SetCash(100000);

        _spy = AddEquity("SPY", Resolution.Daily).Symbol;
        _pjm = AddData<EIAElectricity>(EIA.BalancingAuthorities.PJM, Resolution.Daily).Symbol;
        var history = History<EIAElectricity>(_pjm, 60, Resolution.Daily);

        Debug($"We got {history.Count()} items from our history request");
    }

    public override void OnData(Slice slice)
    {
        if (!slice.ContainsKey(_pjm))
        {
            return;
        }

        var grid = slice[_pjm];
        if (!grid.Demand.HasValue || !grid.DemandForecast.HasValue || grid.DemandForecast == 0)
        {
            return;
        }

        // Load surprise as a fraction of the forecast, so it is comparable across days
        var loadSurprise = (grid.Demand.Value - grid.DemandForecast.Value) / grid.DemandForecast.Value;

        // Demand running more than 2% above the day-ahead forecast means the grid
        // was tighter than operators expected, so go long
        if (loadSurprise > 0.02m)
        {
            SetHoldings(_spy, 1);
        }

        // Demand undershooting by the same margin means slack, so step aside
        else if (loadSurprise < -0.02m)
        {
            Liquidate(_spy);
        }
    }
}

Framework Algorithm Example

The following example algorithm implements the same load surprise signal in the algorithm framework. It uses a manual universe of SPY, an alpha model that reads PJM grid operations and emits insights when demand misses the day-ahead forecast by more than 2%, and equal-weighting portfolio construction.

from AlgorithmImports import *

class EIAElectricityFrameworkAlgorithm(QCAlgorithm):
    def initialize(self):
        self.set_start_date(2020, 6, 1)
        self.set_end_date(2020, 9, 1)
        self.set_cash(100000)

        self.universe_settings.resolution = Resolution.DAILY
        # We trade SPY as the tradeable proxy for the grid signal
        symbols = [Symbol.create("SPY", SecurityType.EQUITY, Market.USA)]
        self.set_universe_selection(ManualUniverseSelectionModel(symbols))
        # A custom alpha model that emits insights from PJM grid operations
        self.add_alpha(EIAElectricityAlphaModel(self))
        # Equally invest to dissipate non-systematic capital concentration risk on individual stock
        self.set_portfolio_construction(EqualWeightingPortfolioConstructionModel())

class EIAElectricityAlphaModel(AlphaModel):
    def __init__(self, algorithm):
        # PJM is the largest balancing authority in the country
        self._pjm = algorithm.add_data(EIAElectricity, EIA.BalancingAuthorities.PJM, Resolution.DAILY).symbol
        self._tradable_symbols = []

        history = algorithm.history(EIAElectricity, self._pjm, 60, Resolution.DAILY)
        algorithm.log(f"We got {len(history)} items from our history request")

    def update(self, algorithm, slice):
        insights = []
        if not slice.contains_key(self._pjm):
            return insights

        grid = slice[self._pjm]
        if grid.demand is None or not grid.demand_forecast:
            return insights

        # Load surprise as a fraction of the forecast, so it is comparable across days
        load_surprise = (grid.demand - grid.demand_forecast) / grid.demand_forecast

        # A grid tighter than the day-ahead forecast is the bullish case, slack demand the bearish one
        if abs(load_surprise) > 0.02:
            direction = InsightDirection.UP if load_surprise > 0 else InsightDirection.DOWN
            insights = [Insight.price(symbol, timedelta(days=7), direction) for symbol in self._tradable_symbols]

        return insights

    def on_securities_changed(self, algorithm, changes):
        for security in changes.added_securities:
            if security.symbol != self._pjm:
                self._tradable_symbols.append(security.symbol)
        for security in changes.removed_securities:
            if security.symbol in self._tradable_symbols:
                self._tradable_symbols.remove(security.symbol)
public class EIAElectricityFrameworkAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2020, 6, 1);
        SetEndDate(2020, 9, 1);
        SetCash(100000);

        UniverseSettings.Resolution = Resolution.Daily;
        // We trade SPY as the tradeable proxy for the grid signal
        var symbols = new[] { QuantConnect.Symbol.Create("SPY", SecurityType.Equity, Market.USA) };
        SetUniverseSelection(new ManualUniverseSelectionModel(symbols));
        // A custom alpha model that emits insights from PJM grid operations
        AddAlpha(new EIAElectricityAlphaModel(this));
        // Equally invest to dissipate non-systematic capital concentration risk on individual stock
        SetPortfolioConstruction(new EqualWeightingPortfolioConstructionModel());
    }
}

public class EIAElectricityAlphaModel : AlphaModel
{
    private readonly Symbol _pjm;
    private readonly List<Symbol> _tradableSymbols = new();

    public EIAElectricityAlphaModel(QCAlgorithm algorithm)
    {
        // PJM is the largest balancing authority in the country
        _pjm = algorithm.AddData<EIAElectricity>(EIA.BalancingAuthorities.PJM, Resolution.Daily).Symbol;

        var history = algorithm.History<EIAElectricity>(_pjm, 60, Resolution.Daily);
        algorithm.Log($"We got {history.Count()} items from our history request");
    }

    public override IEnumerable<Insight> Update(QCAlgorithm algorithm, Slice slice)
    {
        var insights = new List<Insight>();
        if (!slice.ContainsKey(_pjm))
        {
            return insights;
        }

        var grid = slice.Get<EIAElectricity>(_pjm);
        if (!grid.Demand.HasValue || !grid.DemandForecast.HasValue || grid.DemandForecast == 0)
        {
            return insights;
        }

        // Load surprise as a fraction of the forecast, so it is comparable across days
        var loadSurprise = (grid.Demand.Value - grid.DemandForecast.Value) / grid.DemandForecast.Value;

        // A grid tighter than the day-ahead forecast is the bullish case, slack demand the bearish one
        if (Math.Abs(loadSurprise) > 0.02m)
        {
            var direction = loadSurprise > 0 ? InsightDirection.Up : InsightDirection.Down;
            foreach (var symbol in _tradableSymbols)
            {
                insights.Add(Insight.Price(symbol, TimeSpan.FromDays(7), direction));
            }
        }

        return insights;
    }

    public override void OnSecuritiesChanged(QCAlgorithm algorithm, SecurityChanges changes)
    {
        foreach (var security in changes.AddedSecurities)
        {
            if (security.Symbol != _pjm)
            {
                _tradableSymbols.Add(security.Symbol);
            }
        }
        foreach (var security in changes.RemovedSecurities)
        {
            _tradableSymbols.Remove(security.Symbol);
        }
    }
}

Data Point Attributes

The US Electricity dataset provides EIAElectricity objects, which have the following attributes:

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: