Energy Demand Forecasting: Predict Tomorrow’s Energy Infrastructure

Energy Demand Forecasting

For electricity suppliers, success is all about understanding what’s coming next. As long as providers understand both short term and long term energy demand, they’ll be able to adapt and ensure that they’re using the most reliable and affordable means possible to generate energy in any given moment. That, in turn, ensures that energy prices will remain relatively predictable, providers will remain profitable, and customers will continue to have all the power they need, whenever they need it.

The problems start when energy suppliers start getting unpleasant surprises. Unexpected spikes in energy demand or subtler shifts in longer-term patterns of energy usage can cause big problems. After all, an excess of demand without a matching increase in supply will soon result in angry customers sitting in the dark. A surfeit of supply, meanwhile, can cause serious infrastructure problems — or force providers to venture into negative pricing in a bid to put things right.

That’s nothing new, of course — but maintaining a predictable balance between supply and demand used to be a lot simpler. You could be forgiven for looking back wistfully on the bygone era when all companies had to do was maintain a hefty base-load supply — a few coal plants, say — and enough capacity from more responsive gas-fired plants to quickly make up the difference as demand fluctuated.

These days, we live in a more sophisticated and sustainable world, but also a more complicated one. Most obviously, solar and wind power have helped us decarbonize, but have also introduced far more volatility to both our supply and our prices. But it doesn’t end there: in Europe and around the world, we’re experiencing seismic shifts in the way we produce and use energy that go beyond simply the shift to greener energy generation systems.

According to a recent Carnegie Mellon study, volatility and unpredictability – in terms of price, production, and usage – increased sharply between 2005 and 2014. In fact, they increased at the fastest rates seen in 50 years. Researchers point to over-projections for coal, under-projections for oil and natural gas production, and the fallout from the 2008 financial collapse to explain the market dynamism. But in all cases, they blame forces that persist today and have arguably only worsened.

Such changes bring obvious challenges for energy producers and distributors. Energy usage and generation are transforming, but too often we’re still trying to balance electricity usage and generation using the same tools that saw us through the previous century of electricity generation.

What’s needed, we believe, is a transformation in energy consumption to mirror the broader transformation that’s taking place across the energy sector. Just as stakeholders acknowledge the need for modern generation and distribution technologies, so too do we need energy demand analysis and forecasting models and tools that are capable of grasping the new, more complex world we now inhabit, helping us to restore the delicate balance on which our companies, and our whole industry, necessarily rely.

The need for better energy forecasting is clear — but it isn’t always clear how to achieve that goal. In what follows, we’ll review some of the biggest recent changes in the energy-forecasting space, and show you a path forward for our industry.


The New Landscape for Energy Demand Analysis and Forecasting

We live in “disruptive” times, yet few industries face as many transformational forces as the energy sector. The EU recently announced a European Green Deal — an unprecedented plan to become a climate-neutral continent by 2050. Completing the sweeping changes necessary to accomplish that goal will make accurate energy forecasting far more challenging, while simultaneously making it increasingly important. To understand the future of forecasting electric demand, it’s vital to start with an understanding of the direction — or directions — in which our sector is now being pulled.

Shifting Patterns of Demand

Since 2000, global energy demand has collectively increased by 70%, while simultaneously falling in 18 of the 30 member countries in the International Energy Agency. In the EU, both generation and consumption have been relatively flat over the previous decade. Experts attribute this drop to improvements in energy efficiency, especially LED light bulbs. But they also suggest demand could come roaring back as the transportation and heating sectors begin to run on electric power. Even if demand returns, however, it won’t necessarily stabilize. It’s more likely that the trends we’re now seeing — demand waning in some places, and surging in others — represent a kind of new normal. Demand could continue to fall for years to come, then suddenly spike in the wake of a technological breakthrough or legislative milestone — and that new spike might itself ripple through the world economy unevenly, leaving some countries using far more electricity than others. The new reality isn’t necessarily one of increased or decreased demand, but rather one of volatility and change — and to adapt and survive, energy producers will urgently need accurate forecasts that can account for and flexibly adapt to local, regional, and global shifts in energy usage.

Challenging Sustainability Requirements

Individuals, businesses, governments, and the international community as a whole are all setting goals to source increasing amounts of power from renewables. For instance, EU member countries have committed to meeting 32% of their energy needs with renewables by 2030, which will require a drastic increase in solar and wind within a decade. There are many drivers behind the demand for renewable energy: the desire to “decarbonize” the power grid, the need to keep electricity prices low, and the uncertain future of non-renewable energy sources, among others. Whether because of market forces or regulatory requirements, it now seems certain in coming months and years, more and more of our energy will come from wind, solar, hydro, and other sustainable sources. For energy producers, the rise of renewables involves more than switching from one source of power to another. It means learning to adapt to energy supplies that fluctuate, both predictably and unpredictably, in ways that don’t always map clearly onto cycles of demand. That’s making it more important than ever to have access to hour-by-hour forecasting tools that take into account the interactions between local and regional weather patterns and shifts in both supply and demand.

Changing Consumer Landscape

Residential energy use has changed sharply in recent years: while we’re increasingly wedded to a panoply of electrical gadgets and gizmos, average home energy use has declined steadily as new energy efficiency improvements have gone mainstream. That could change sharply as electric vehicles and heat pumps become more widespread across Europe, however, creating both post-commute surges in electricity use and new opportunities for decentralized energy storage. Lifestyle changes such as the fact that 25 million Eurpoeans now work remotely could also lead to a big shakeup in energy use patterns, while smart meters will lead to consumers making more mindful and strategic decisions about when to use energy. All this points to a future of dynamic loads, with demand rising and falling quickly and in hard-to-predict ways. Without sophisticated forecasting capabilities, the fluctuating energy needs of tomorrow’s homes could become a major headache for providers.

Increased Weather Volatility

Short-term weather patterns can have a huge impact on both supply and demand, but long-term climate change could be an even bigger issue in coming years. As the planet warms, consumers and businesses alike will face not just hotter summers but potentially also an uptick in major events such as snowstorms, wildfires, and hurricanes, simultaneously straining energy infrastructure and dramatically changing energy demand. Data already shows an uptick in extreme weather across Europe, with floods and other hydrological events increasing for over 30 years. Given the unprecedented (and unpredictable) nature of these changes, it’s impossible to know how tomorrow’s weather will impact the energy grid, and that’s exactly the point. As the energy sector enters a new era, all precedents go out the window. Incrementally improving our energy analytics won’t be good enough – we’re going to need an entirely new approach to forecasting.

Complex Infrastructure Management

Building tomorrow’s energy grid will require significant infrastructure investments. Estimates suggest that decarbonizing the EU energy grid will cost $682 billion, and that’s probably conservative. But with both supply and demand likely to grow increasingly volatile, it’s hard to anticipate where, exactly, those investments will be needed. Add to that the challenge of building a decentralized energy grid, the burden of new regulatory mandates, and the uncertainty of combining old and new energy assets, and you can see how complex the next energy infrastructure looks. One thing’s for sure, though: planning infrastructure investments will be exponentially harder if we lack reliable, consistent energy forecasting capabilities. With the industry set to invest $52.3 billion on infrastructure in 2020 alone, we’ll need every insight available — especially in terms of supply and demand — to ensure that money goes where it matters.


Transforming Forecasting at the Pace of the Industry

To rise to these challenges, we’ll need a new kind of energy forecasting. But what does that look like? First, consider what came before: a process that leveraged historical data and top-down oversight to make broad assumptions about load management. Reactionary forecasting has worked pretty well for decades, but for all the reasons outlined above, it won’t be enough to steer us safely through the turbulence that lies ahead.

In response to dynamic demand changes and inconsistent power sources, energy providers need eyes on the ground. That means gaining visibility into individual homes and businesses so that providers can obtain real-time insights into exactly when and how power is being used. Think of this as bottom-up forecasting – using data that comes from the consumer level to inform decisions made at the grid level.

Fortunately, with roughly 40% of European electricity customers already collecting granular data about energy use, utilities and retailers have access to a vital new source of information. And while poring over the readings from millions of meters would be impossible for any human analyst, we now also have access to artificial intelligence (AI) tools capable of grasping and learning from those torrents of data, and turning them into empirical forecasts and actionable insights.

By using smart-meter data to train AI tools, we can forge new forms of forecasting that deliver better information and better tools for analysis. Research shows that bottom-up forecasting can lower supply costs by up to 10% while helping reduce financial and operational risks for the business. That makes sense when you consider what this approach entails: watching the transformation of the energy sector in action. With so much unknown, observing electricity consumption up close will be the only way to know how it’s changing. Data from smart meters isn’t just useful; it’s essential, and it will help transform supply, demand, and forecasting in years to come.

But it’s also a clear sign that it’s possible to create new ways of forecasting in order to rise to the challenges of tomorrow’s energy outlook – and that the new forecasting tools we’ll create will be vastly better than what came before, offering forensic insights derived from the data now pouring out of smart meters and into smart analytics platforms.

When change is the only constant, staying the course is the only strategy that’s sure to fail. Stakeholders throughout the energy sector must acknowledge the speed and scope of the changes in front of them and then act accordingly. Don’t let the opportunities of tomorrow’s grid pass you by, and don’t let the challenges overwhelm you either. Contact Innowatts, and find out how your company can embrace the future of forecasting.


Join our mailing list to to get notified of
new case studies, white papers, news and events.