AI Weather Models Lag in Predicting Extreme Events, Study Finds

AI vs. Traditional Models: The Extreme Weather Prediction Gap

Artificial intelligence is widely promoted as the future of weather forecasting, offering speed and precision. However, a new study published in Science highlights a critical weakness: AI models often fail to predict extreme weather events accurately. Traditional physics-based models continue to outperform AI in these high-stakes scenarios.

Key Findings from the Study

Sebastian Engelke, a statistics professor at the University of Geneva and co-author of the study, explains the limitations of AI models like GraphCast and Pangu-Weather. The research compared these leading AI systems against a database of recent extreme weather events.

"They do perform well on a lot of tasks, but for very extreme events—that are the most important for society—they still struggle."

The study found that AI models consistently underestimate extreme temperatures, such as the record-breaking heat wave in Siberia in early 2020. This event triggered wildfires and accelerated permafrost melting. Another study determined that climate change made this heat wave 600 times more likely to occur.

AI models also lag behind traditional methods in predicting extreme wind speeds and record-breaking cold. Their training relies on decades of historical data, which limits their ability to forecast unprecedented events.

"They try to empirically understand, if I see a certain type of weather today, what is the weather tomorrow? Essentially, they are reproducing what has happened in the past. If we’re looking at extreme weather, and especially record-breaking events, then this has not been observed in the past. It’s really the lack of information in their training data that makes it almost impossible for them to forecast it."

Why Traditional Models Still Lead

Traditional physics-based forecasting uses complex mathematical models to simulate real-world atmospheric conditions. These models can adapt more readily to new and unprecedented weather patterns, giving them an edge in extreme event prediction. While they are not flawless, they still perform better than AI in these critical scenarios.

AI’s Strengths and Current Applications

For typical weather forecasting or extreme events within historical ranges, AI models often outperform traditional systems. For example, Nvidia’s AI forecasting model Atlas demonstrated strong performance in predicting Storm Dennis, a rapidly intensifying cyclone that struck the U.K. in 2020. The model accurately captured intense wind events and pressure gradients associated with the storm.

Mike Pritchard, director of climate simulation research at Nvidia, noted:

"You can see just clearly by visualizing the magnitude of the wind and the magnitude of the pressure gradient that the model was able to capture realistically intense wind events and really intense cyclones that cause damage."

AI models are already being used alongside traditional systems by weather agencies, companies like The Weather Company, and insurance firms. They excel in predicting hurricane paths and other high-impact weather phenomena.

Future Improvements and Challenges

Researchers are exploring ways to enhance AI’s ability to predict extreme weather. One proposed solution is to expand training datasets to include hypothetical scenarios of record-breaking events. However, the fundamental challenge remains: AI models are inherently limited by the data they are trained on.

"There’s ways to kind of coerce