In brief: Tropical Storm Alberto is about to move inland into Mexico, ending the threat of tropical rainfall across the Houston metro area. However, we are still likely to see at least some scattered showers and thunderstorms for several days—mostly passing downpours. After a few days with highs around 90 degrees, we’ll start to warm up this weekend into the low- to mid-90s.
How did the new AI models do?
Recently, I mentioned that Matt and I were assessing the value of relatively new weather forecast models, which are based in machine learning. These AI-based models have been ‘trained’ on historical datasets and have shown some promise in limited testing. They’re now starting to become publicly available in real-time, and we’ll be using them for the Atlantic hurricane season. (You can find some of them here).
Anyway, the evolution of Alberto offered the season’s first test. Last Friday, the models offered a distinct difference in forecasts. The majority of traditional, physics based models that crunch through complex physical equations to reproduce the atmosphere, produced a system that remained largely in the Southern Gulf of Mexico. I saved the output from the U.S.-based Global Forecast System model that was initialized on Friday morning. Here is its five-day forecast for the morning of Wednesday, June 19:
At the same time, the AI-based version of the European model, or AIFS, had for several runs been bringing Alberto much further north, toward the central Texas coast. This is what the output from the model, initialized at the same time as the GFS model shown above, looked like for Wednesday, June 19:
That is a lot different, and certainly would have been more eye-popping for the Houston area. I frequently appear on the Houston Matters radio program with Craig Cohen, and we discussed this difference in output at the time. I told him it would be a good test. Five days later, on Wednesday morning, here’s where the tropical system, which had just formed as Tropical Storm Alberto, was:
As you can see, the physics-based GFS model was spot on, whereas the AI-based model misfired. This was not universally true, but the majority of physics-based models were much closer on track accuracy, whereas the AI-based models tended to have a more northerly track for Alberto (Huawei’s Pangu model was a notable exception, nailing the position). In our forecasts at Space City Weather, we relied on the physics-based models since they seemed more in-line with the overall atmospheric setup.
That’s track. None of the models did a great job with precipitation. There were several runs where the GFS model splashed 25 inches of rainfall over Houston, and the AI models that have rainfall outputs were also considerably more bullish than what Houston ultimately experienced. In any case, it’s going to be a long hurricane season. One test case proves nothing, but nevertheless I thought it was interesting to share.
Thursday
After coastal waters rose fairly high on Wednesday, due to the winds and surge of Alberto, they’re starting to recede some this morning. They’ll become less and less of a problem today and tonight now that the tropical system is inland. We’ll still see some gusty winds from the east today, perhaps up to 25 mph, but they’ll also be less than those of Wednesday.
In terms of precipitation, showers will be considerably more scattered today after much of Houston picked up 1 to 2 inches of rainfall on Wednesday. The most likely time for showers will be this afternoon, but they could come at any time, especially for coastal areas. Skies will turn partly sunny this afternoon, with highs of about 90 degrees. Lows tonight will drop into the mid-70s.
Friday
Another partly sunny day, with perhaps a 30 to 50 percent chance of showers and thunderstorms. Most of this activity will be fleeting. Highs will be around 90 degrees, with lows again in the mid-70s.
Saturday
Skies should be mostly sunny, with highs in the low- to mid-90s. Rain chances will be much lower, perhaps on the order of 10 percent.
Sunday and beyond
Our weather for next week is a bit uncertain due to the development of another tropical disturbance in the Southern Gulf of Mexico. However, this system is likely to be smaller, with lesser impacts. As a result I don’t think we’ll see nearly the winds and high seas we experienced from Alberto. Rather, for Sunday and most of next week, I expect we’ll see partly to mostly sunny skies, highs in the low- to mid-90s, and a decent daily chance of showers as tropical moisture pushes in from the Gulf. At this time there is no clear threat of heavy rainfall. So the weather looks pretty typical for late June.
are yall going to continue to use both physical and the large language models at the same time? i can’t imagine using a large language model for weather. it just seems irresponsible when we have physics based models that have worked.
Anna, don’t want to speak for them so this is just my opinion. I imagine they will continue to look at all models and their experience to guide them to the best possible forecast at the time they issue out updates.
While AI models may not be ready for prime time yet they will like improve over time with additional refinement. AI gets better over time as new data sources continues to train the platforms.
It seems unlikely based on this small dataset that anyone is going to adopt the AI models in the short term or that the physics based models are going anywhere anytime soon.
https://spacecityweather.com/as-houstons-weather-settles-down-lets-talk-about-ai-and-forecasting-yes-its-a-thing/
https://arstechnica.com/ai/2024/06/as-a-potentially-historic-hurricane-season-looms-can-ai-forecast-models-help/
At the top of this post is a link to another post where they answer your question. The TLDR version from that page:
“So Matt and I may be referring to them from time to time as part of our forecasting process. We’ll be evaluating them alongside the traditional, physics-based models. If you want to know more about all of this, I recently wrote a feature about AI weather forecasting for Ars Technica.”
I agree with you Anna. I’m sure they’ll watch the AI models but to use them seems irresponsible until more data gets input. The article mentioned they used the physics-based models this week but I’m not sure I believe that given SCW issued a stage 2 flood alert and predicted 8-12” of rain for Houston which obvs didn’t materialize. AI was s a slippery slope right now and shouldn’t be used yet imo.
You’re exaggerating. They issued a Stage 2 alert several days out but and as soon as models consistently moved things south they revised it down. They didn’t “predict” 8-12 inches of rain, they said with plenty of caveats that some models showed it could happen, and were very clear as more information came in that it would not be nearly that bad.
In the arstechnica article, they do say that these are deep learning models and not large language models.
Large language models do not apply to data which is not language. Machine learning has been around for years and is essentially a sophisticated statistical analysis of data trends without starting from fundamental models. It is very useful in picking up trends that either are not modeled well, or that you did not even know to model.
Question, does a large tropical system like Alberto reduce ocean / Gulf surface temps in the wake of its path?
I would say that there might be a bit of cool water upwelling after Alberto but I don’t think the waters mixed enough to see a significant difference in water temperatures. The temperatures will probably be back to where they were pre-Alberto in a few weeks or so.
Put that AI crap in the garbage. Use what has worked for years! If it ain’t broke, don’t try to fix it!
AI strikes me as just the latest new thing/ponzi scheme to come out of Silicon Valley (or at least promoted by them)… Crypto, Self-driving cars, NFTs…
It’s not a “new” technology either. Machine learning and text generators have been around for years, if not decades. But call them “AI” and tell everyone they cost 10x as much = profit
Luckily, multiple models can be used at the same time. Stick with your clay tablets and chisels if you’d like
He said “use what has worked for years”. Maybe read things twice before posting.
Lol, that’s funny Starn!
AI probably shouldn’t exist at all, and probably should be banned. Really, it would probably have been better if the internet had never been invented. (and maybe the automobile too, since someone brought that up.) Not because they cant be used for good, but because our world uses them so badly.
Nevertheless, since no one is going to ban AI, it seems we might as well use it for something worthwhile that we already use computers for, such as weather modeling.
I do believe AI is going to be the scourge of the century building on the back of what has become the most destructive force known to mankind – the social media explosion. Trying to stem it now will be liking to trying to put the genie back in the bottle – or Loki as it turns out. The automobile was developed out of a desire to create a locomotive system that could benefit the common man, now look what it has done the the planet. Unintended consequences and once started impossible to control. It’s all rather like the insane finality purported in the Dr. Strangelove movie.
It’s not quite true that “none of the models did a great job with precipitation.” As far as I could see, the traditional Euro model did really well on the precip forecast.
Why use AI when we already have better models?
If you read some of the linked articles you will see that the primary reason is much reduced compute requirements. Use of machine learning for forecasting got a boost from an organization which was working on data collection, and they wanted something to check their data models but did not have the compute resources to run a full physics based simulation.
Bottom line is all the models performed poorly with this storm when it comes to rainfall for SE Texas. Widespread rainfall totals up to 2″ is a far cry from some of the totals they were forecasting just a couple of days ago. I think most of us have become spoiled by the accuracy of weather models. This stormed highlighted a glaring weakness in our dependence. The models are just not accurate enough in forecasting tropical systems before a low level center of circulation has formed. I think if there is a lesson to be learned it is putting too much faith in the models in the early stages of tropical storm formation. Issuing flood watches and forecasting 8-10 inches of rain before a storm even has a name was premature. Clearly more on the hype side than reality based.
100% correct. It’s borderline negligent and definitely hype. At the end of the day, every media outlet wants clicks; it’s how they justify the means.
SCW wasn’t being borderline negligent.
It would however, be negligent, if they had the data that they were given, and DIDN’T issue an alert.
correct, we need to stop relying so much on computers for weather, specially future radar. It’s so bad I dont even know why it is used.
Unrelated but I’m glad this summer is not hell’s front porch after the first few days so far.
There’s a disappointing number of folks saying “Why try out this newfangled automobile if my horse and buggy works just fine? Horses always have been and always will be the way to get around!”
It’s great to see you’re looking into new ways to predict weather, even if it’s not a polished product yet. The machine learning space is going to be growing in the coming years, so might as well lean into it. I’m glad you didn’t full on sound the alarms this time around based on the AI models, but I am interested to see what the rest of the season brings!
They issued a stage 2 flood alert and predicted 8-12” of rain before there was even a named storm. Sorry, but they most definitely prematurely sounded the alarm bells.
To be fair, you don’t need a named storm to get 8-12″ of rainfall. I believe that forecast was based just on waves of tropical moisture heading our way.
Right, which happened to slowly drift further south, and ended up mostly missing Houston. Both NHC and Space City Weather shifted expectations over time as they got real data to feed into models.
You’re complaining that we weren’t flooded. Again.
What is this, the third day now?
Right! Make it make sense..🙄
Maybe you should reread the article. It says the GFS model predicted 25 inches. They did not prematurely sound alarm bells. Even in the posts before it explained the Stage 2 alert and the possibility it may not come to that but because of the uncertainty they were issuing an alert. Just because you don’t agree with them looking at AI in comparison to other models doesn’t mean you get to sit here and negatively post when it’s clear they try to evaluate all options with models and other tools and make the best predications to keep us as prepared as possible.
Marlie, you keep saying this, PLEASE take time to read the article thoroughly. Most of what they said about the AI models being off was about the path of the storm, NOT the rainfall totals. At the end of Eric’s assessment, he said that ALL the models (AI and the physics-based) were wrong about the rainfall totals. I know SCW is known for being no-hype, but since you mentioned being irresponsible in their reporting – don’t you think it would have been more irresponsible if they had ignored what most of the models were saying about excessive rainfall, just so… what? A few negative complainers would have nothing to complain about? What is the harm that was done here?
poor comparison, this isn’t a newfangled automobile, more like a paper airplane. It is really bad.
AI is not ready for prime-time except for limited models, such as those for x-rays such as mammograms, where it sees things doctors miss. Imho they train them on too much garbage, forgetting that the models don’t “reason” on the input.
It was nice to get some rain yesterday and the cooler temps were great for the Dynamo game last night. The humidity was other-level though!
Well my cancelled Galveston plans this week have me leaning towards SCW taking note of AI models, but not considering them yet in the forecast presented to the public. If its still learning, then keep it in the learning environment until it is ready to graduate.
Meanwhile in Corpus, people took their folding chairs and coolers to the seawall
Wed morning, NWS forecast 0.86″ of rain for my area for the remainder of Wed; we ended up getting about twice that, 1.56″.
As for using AI, many people are uncomfortable because it outperforms most humans at many tasks. This is why AI it is so popular among many high school and university students. At the highest levels though, humans still outperform AI, and I think the best meteorologists are often those with the most experience. For example, Neil Frank was on top of hurricane season, and he seems to understand the link between seasonal weather and climate better than most people.
Thank you for the no hype forecast
Did we just have 2 storms back-to-back?
On a positive note, today is the longest day of the year. Everyday will be a growing shorter… And last year by this date we already broke 100F twice and we had a bunch of 95+ degree days…
I take joy in the solstice, living here now, in knowing that the amount of daylight hours will (incrementally) be getting less. That’s strongly buffered by knowing that the very worst of summer still lies ahead.
this isn’t positive at all though, I like the long days. HATE the short winter days.
I’ve been working with expert systems, model-based predictors, and weighted neural networks in a portion of my schooling and career for over thirty years. For the cost of implementation and reliable execution, model-based and expert systems always outperformed weighted neural networks when evolution time spans were short and scenario inputs deviated by only small amounts. Even in the broader cases, neural networks needed training on MANY possible outcomes to be effective, and expert systems would usually best them because we were applying an AI that HAD seen these outcomes (e.g. the 25 year veteran process operator).
The only thing that has changed today versus three decades ago is the processing power and storage available to render scenarios. We still have the problem of not being able to provide the model with enough scenarios for it to predict reliably. Why? Because these AI predictors have learned on a system that no longer exists. The boundary conditions of our weather today are different from the past and is changing.
As they say in the stock market, past performance is not an indicator of future results.
Eric – Be careful. The AI might be listening, and in the future might make you pay for what you said about it just now.
Be afraid. Very, very afraid.
Perhaps stick with what has been reliable and a better, more accurate prediction? Aside from weather forecasting, personally I don’t care for all this AI these days, esp when it comes to customer service. It takes jobs away from humans and wastes a lot of people’s time who just want to speak with a human being.
all of the predictions have been pretty bad when it comes to rain and low temperatures for the past 5 years. Good to see that we are trying but it’s getting worse instead of better. At least for these two important things. I’m sure they are accurate with high temperatures, sunny days etc but with lows, they have been a mess.