Predicting the weather can be a frustratingly imprecise science. The weather app on your phone is pretty good at forecasting if it’s likely to rain at some point during a given day, but much less helpful if you want to know if there’s going to be a downpour in central London at 3 pm this Sunday. If you absolutely have to stay dry, you’re best off keeping an umbrella with you or staying inside.
For most people, not knowing what the weather is going to do in the next hour is a minor inconvenience. But when it comes to the power network, not knowing what the weather will do next isn’t just an annoyance, it’s a significant source of carbon emissions. If we could get better at predicting when and where the weather will change, we could stop huge amounts of carbon dioxide being sent up into the atmosphere simply because we don’t know what the clouds will do next.
Here’s the problem. On a sunny spring day in Great Britain, solar power can account for about 30 percent of all the electricity being produced on the island. The exact number varies a lot, but under ideal conditions—solar panels perform best on cool but sunny days—they might churn out 9 gigawatts (GW) of energy, a huge chunk of the 30 GW average energy demand. So far so good. But if a big cloud swoops down over the southwest, where many of Great Britain’s solar panels are situated, a significant portion of that renewable energy suddenly disappears from the grid—the equivalent of an entire gas power station instantly going offline. Hundreds of megawatts of energy gone, just like that.
Having an entire power station’s worth of power wiped out in minutes is obviously not ideal, so to compensate for this, power networks schedule some backup energy production to step in and smooth over any bumps caused by changes in solar production. In Great Britain, the responsibility for balancing and distributing this energy falls to the National Grid Electricity System Operator (ESO), which asks fossil-fuel power plants, usually burning natural gas, to produce extra energy in case solar production dips unexpectedly.
Fossil-fuel plants are slow-moving beasts. “We would really like to have a power plant that can ramp up in five minutes or in half an hour, because that’s how fast the wind and solar power generation may change,” says Jan Kleissl, a professor of renewable energy and environmental flows at University of California San Diego. But fossil fuel plants don’t work like this. They take a long time to turn on and are most efficient when they’re running at full power. This limitation further encourages power grids to overproduce energy just in case power from solar or wind drops off.
One way to get around this is to get better at forecasting the weather. If we knew exactly how much solar power Great Britain was likely to produce at any moment, the ESO could dial back the amount of power it holds in reserve, bringing down the total carbon footprint of the energy grid. In other words, if we knew exactly how much solar energy was going to flow onto the grid every five minutes, we could make sure we used every kilowatt of that energy rather than hedging our bets with surplus electricity generated from fossil-fuel power plants.
Jack Kelly thinks he knows a way to vastly improve these predictions. A former researcher at DeepMind, the Alphabet-owned artificial intelligence firm, in 2019 Kelly cofounded Open Climate Fix, a nonprofit focused on reducing greenhouse gas emissions using machine learning. “I’m a machine-learning researcher who’s terrified by climate change and keen to do everything possible to try and fix it,” Kelly says. He estimates that better solar forecasts in the UK could save 100,000 tonnes of carbon dioxide from being emitted each year, and will be critical if the National Grid ESO is going to meet its 2025 target of operating at zero emissions whenever there is enough renewable generation available.
Kelly’s idea is to use machine learning to improve what is known as solar “nowcasting”—predicting solar electricity generation less than a few hours in advance. Rather than working out what the weather will generally be like in a given area, to get really precise solar forecasts, Kelly needs to know precisely where each cloud will be located relative to a solar array, and how the size and shape of the clouds influence how much sunlight gets through to the panels.
“The system operators can then do a better job of scheduling those generators and hopefully end up with a smaller number of generators running near their max capacity,” Kelly says. Open Climate Fix is training a machine-learning model on a year and a half of satellite imagery from EUMETSAT and power generation data from 700 solar power systems in Great Britain. This is only a tiny slice of the available data—EUMETSAT has 11 years of backdated satellite imagery stored on hard-to-access metallic tape—but it should be enough to get started.
Open Climate Fix’s approach uses a deep-learning architecture known as a transformer, a key part of GPT-3, the text-generating model created by OpenAI. Transformers work out which bits of data are important in shaping a particular output, and how they all interact with each other. It’s not enough to simply predict whether a cloud will block a solar array, you also need to know what effect the light bouncing off other clouds will have, and how the changing wind direction might influence power generation. “This all comes down to attention. You need to be able to attend to bits of that satellite image to figure out which clouds are most informative and how they interact with each other,” says Kelly.
But when it comes to estimating the UK’s solar power output, there’s an even more basic problem: We don’t know where all the solar panels are. Information about solar power installations is spread across several databases, and often crucial bits of data are missing altogether. “In an ideal world we’d know the locations and capacities of all of the solar across the country; better still, we’d know what orientation angle they’d use, what tilt,” says Jamie Taylor, senior data scientist at Sheffield Solar, a solar power research team at the University of Sheffield. “For now the lowest-hanging fruit in terms of improving uncertainties is just improving our knowledge of how much solar power was installed and where it was installed."
Open Climate Fix is also working on that problem. The nonprofit is running a project to map solar installations through OpenStreetMap and will also use machine learning to identify solar panels in satellite and aerial imagery. Knowing exactly where solar installations are should also help improve the accuracy of solar forecasts.
Although our existing solar forecasts are pretty sophisticated—the team at Sheffield Solar uses statistical models that sit atop weather models to forecast sunlight, temperature, and wind speed at a post-code level—there’s still plenty of room for improvement. Kleissl estimates that even the best solar forecasts are only halfway to being perfectly accurate, compared with a model that assumes that today’s weather will be exactly the same as yesterday’s. In April 2021, Open Climate Fix received £500,000 in funding from Google’s charitable arm, Google.org, to help develop its solar forecasting project.
“Work like this has real impact—reducing forecasting errors and the need to keep costly fossil-fuel plants ticking over,” says Carolina Tortora, head of innovation and digital transformation at the National Grid ESO. “Open Climate Fix’s nowcasting research has potential to further improve the forecasting capabilities of electricity system operators around the world.”
We’ll also need to get better at predicting energy demand if the UK is to meet its new goal of reducing emissions by 78 percent by 2035 compared to 1990 levels. Electrifying transport and heating will mean we need more electricity than ever before, and a large proportion of it will have to come from variable power sources such as wind and solar. “Things like electric vehicles and heat pumps are going to have a huge impact on the electricity demand profile,” says Taylor. “And if you couple that with distributed storage at the same time, then it becomes a real headache for system operators. The approach to forecasting electricity demand will need to evolve considerably over the next few years.”
This story originally appeared on WIRED UK.