Space City Weather’s grand 2022 pre-season server upgrade

Howdy, folks—I’m Lee, and I do all the server admin stuff for Space City Weather. I don’t post much—the last time was back in 2020—but the site has just gone through a pretty massive architecture change, and I thought it was time for an update. If you’re at all interested in the hardware and software that makes Space City Weather work, then this post is for you!

If that sounds lame and nerdy and you’d rather hear more about this June’s debilitating heat wave, then fear not—Eric and Matt will be back tomorrow morning to tell you all about how much it sucks outside right now. (Spoiler alert: it sucks a whole lot.)

The old setup: physical hosting and complex software

For the past few years, Space City Weather has been running on a physical dedicated server at Liquid Web’s Michigan datacenter. We’ve utilized a web stack made up of three major components: HAProxy for SSL/TLS termination, Varnish for local cache, and Nginx (with php-fpm) for serving up Wordpress, which is the actual application that generates the site’s pages for you to read. (If you’d like a more detailed explanation of what these applications do and how they all fit together, this post from a couple of years ago has you covered.) Then, in between you guys and the server sits a service called Cloudflare, which soaks up most of the load from visitors by serving up cached pages to folks.

It was a resilient and bulletproof setup, and it got us through two massive weather events (Hurricane Harvey in 2017 and Hurricane Laura in 2020) without a single hiccup. But here’s the thing—Cloudflare is particularly excellent at its primary job, which is absorbing network load. In fact, it’s so good at it that during our major weather events, Cloudflare did practically all the heavy lifting.

Screenshot of the bandwidth graph from a Cloudflare dashboard
Screenshot from Space City Weather’s Cloudflare dashboard during Hurricane Laura in 2020. Cached bandwidth, in dark blue, represents the traffic handled by Cloudflare. Uncached bandwidth, in light blue, is traffic directly handled by the SCW web server. Notice how there’s almost no light blue.

With Cloudflare eating almost all of the load, our fancy server spent most of its time idling. On one hand, this was good, because it meant we had a tremendous amount of reserve capacity, and reserve capacity makes the cautious sysadmin within me very happy. On the other hand, excess reserve capacity without a plan to utilize it is just a fancy way of spending hosting dollars without realizing any return, and that’s not great.

Plus, the hard truth is that the SCW web stack, bulletproof though it may be, was probably more complex than it needed to be for our specific use case. Having both an on-box cache (Varnish) and a CDN-type cache (Cloudflare) sometimes made troubleshooting problems a huge pain in the butt, since multiple cache layers means multiple things you need to make sure are properly bypassed before you start digging in on your issue.

Between the cost and the complexity, it was time for a change. So we changed!

Leaping into the clouds, finally

As of Monday, June 6, SCW has been hosted not on a physical box in Michigan, but on AWS. More specifically, we’ve migrated to an EC2 instance, which gives us our own cloud-based virtual server. (Don’t worry if “cloud-based virtual server” sounds like geek buzzword mumbo-jumbo—you don’t have to know or care about any of this in order to get the daily weather forecasts!)

Screenshot of an AWS EC2 console
The AWS EC2 console, showing the Space City Weather virtual server. It’s listed as “SCW Web I (20.04)”, because the virtual server runs Ubuntu 20.04.

Making the change from physical to cloud-based virtual buys us a tremendous amount of flexibility, since if we ever need to, I can add more resources to the server by changing the settings rather than by having to call up Liquid Web and arrange for an outage window in which to do a hardware upgrade. More importantly, the virtual setup is considerably cheaper, cutting our yearly hosting bill by something like 80 percent. (For the curious and/or the technically minded, we’re taking advantage of EC2 reserved instance pricing to pre-buy EC2 time at a substantial discount.)

On top of controlling costs, going virtual and cloud-based gives us a much better set of options for how we can do server backups (out with rsnapshot, in with actual-for-real block-based EBS snapshots!). This should make it massively easier for SCW to get back online from backups if anything ever does go wrong.

Screenshot of an SSH window
It’s just not a SCW server unless it’s named after a famous Cardassian. We’ve had Garak and we’ve had Dukat, so our new (virtual) box is named after David Warner’s memorable “How many lights do you see?” interrogator Gul Madred.

The one potential “gotcha” with this minimalist virtual approach is that I’m not taking advantage of the tools AWS provides to do true high availability hosting—primarily because those tools are expensive and would obviate most or all of the savings we’re currently realizing over physical hosting. The only conceivable outage situation we’d need to recover from would be an AWS availability zone outage—which is rare, but definitely happens from time to time. To guard against this possibility, I’ve got a second AWS instance in a second availability zone on cold standby. If there’s a problem with the SCW server, I can spin up the cold standby box within minutes and we’ll be good to go. (This is an oversimplified explanation, but if I sit here and describe our disaster recovery plan in detail, it’ll put everyone to sleep!)

Simplifying the software stack

Along with the hosting switch, we’ve re-architected our web server’s software stack with an eye toward simplifying things while keeping the site responsive and quick. To that end, we’ve jettisoned our old trio of HAProxy, Varnish, and Nginx and settled instead on an all-in-one web server application with built-in cacheing, called OpenLiteSpeed.

OpenLiteSpeed (“OLS” to its friends) is the libre version of LiteSpeed Web Server, an application which has been getting more and more attention as a super-quick and super-friendly alternative to traditional web servers like Apache and Nginx. It’s purported to be quicker than Nginx or Varnish in many performance regimes, and it seemed like a great single-app candidate to replace our complex multi-app stack. After testing it on my personal site, SCW took the plunge.

Screenshot of the OLS console
This is the OpenLiteSpeed web console.

There were a few configuration growing pains (eagle-eyed visitors might have noticed a couple of small server hiccups over the past week or two as I’ve been tweaking settings), but so far the change is proving to be a hugely positive one. OLS has excellent integration with Wordpress via a powerful plugin that exposes a ton of advanced configuration options, which in turn lets us tune the site so that it works exactly the way we want it to work.

Screenshot of the LiteSpeed Cache settings page
This is just one tab from the cache configuration menu in the OLS Wordpress plugin’s settings. There are a lot of knobs and buttons in here!

Looking toward the future

Eric and Matt and Maria put in a lot of time and effort to make sure the forecasting they bring you is as reliable and hype-free as they can make it. In that same spirit, the SCW backend crew (which so far is me and app designer Hussain Abbasi, with Dwight Silverman acting as project manager) try to make smart, responsible tech decisions so that Eric’s and Matt’s and Maria’s words reach you as quickly and reliably as possible, come rain or shine or heatwave or hurricane.

I’ve been living here in Houston for every one of my 43 years on this Earth, and I’ve got the same visceral first-hand knowledge many of you have about what it’s like to stare down a tropical cyclone in the Gulf. When a weather event happens, much of Houston turns to Space City Weather for answers, and that level of responsibility is both frightening and humbling. It’s something we all take very seriously, and so I’m hopeful that the changes we’ve made to the hosting setup will serve visitors well as the summer rolls on into the danger months of August and September.

So cheers, everyone! I wish us all a 2022 filled with nothing but calm winds, pleasant seas, and a total lack of hurricanes. And if Mother Nature does decide to fling one at us, well, Eric and Matt and Maria will talk us all through what to do. If I’ve done my job right, no one will have to think about the servers and applications humming along behind the scenes keeping the site operational—and that’s exactly how I like things to be 🙂

Houston faces four straight days of potentially 100-degree heat before relief arrives next week

Be brave, my friends. After an exceptionally warm June—there is no question this month will smash Houston’s previous record for hottest June ever—we are approaching the crescendo of this heat wave. The next four days will all bring the possibility of 100-degree temperatures for the metro area, with baking sunshine, before a possible reprieve early next week.

Also, we’ve been asked a lot whether the record heat this June is predictive of abnormally warm conditions for the remainder of the summer. Matt has done some research on this, and will write about it tomorrow. The answer is hopeful.

Thursday

As expected, our region saw some scattered showers on Wednesday, and that likely will be the case today, although coverage should be more isolated. The main reason for less coverage is that high pressure is building again over the region and that will impair some passing atmospheric disturbances from getting too much traction. Rain chances appear best this afternoon along, and north, of Interstate 10. Otherwise expect mostly sunny skies, with highs of around 100 degrees.

Friday’s high temperatures will not be much fun. (Weather Bell)

Friday, Saturday, and Sunday

These will be three baking hot days, with high temperatures of 100 degrees, or slightly higher, for nearly all of the metro area away from the coast. Skies will be sunny, and rain chances virtually non-existent. A more southwesterly flow, instead of winds directly off the Gulf of Mexico, should help with humidity levels a bit. Even so, there are a number of outdoor activities this weekend, including Pride celebrations in downtown Houston, during the middle of the day. We really urge you to take heat and sunshine precautions during the exceptionally hot weather this weekend.

Next week

We’re not promising any miracles next week, but things should finally change. A front of sorts should push into the region on Monday, helping to open our atmosphere up to a few passing disturbances. By later Monday or Tuesday this should lead to the formation of a few more clouds, and push daily rain chances up to 30 or 40 percent. In addition, high temperatures probably will drop back into the mid-90s, if not even lower. The details on this are still fuzzy, but the end of June will probably, finally, feel like what most of June should have felt like.

Tropics awakening?

Typically, during this time of year, wind shear runs high in a central Atlantic Ocean area called the “main development region” for tropical storms and hurricanes. This is the place where low pressure systems that move off of Africa during the summer can regularly form into tropical storms during the months of August and September. I say August and September, because wind shear is usually hostile to storm formation before then. (This is also why hurricane activity peaks during August and September).

Tropical weather outlook for Thursday morning. (National Hurricane Center)

However, as we’re seeing lower wind shear now in the main development region, it is possible we may start to see these “Cape Verde” storms coming off of Africa and developing earlier this year. To that end, the National Hurricane Center is watching development of a possible tropical wave there five to seven days from now. This is something to watch, but not worry about at this time.

Top 10 reasons to be thankful for our blistering heat and emerging drought

You know the heat is getting to us when we resort to a gimmick like this. But with high pressure holding sway for our weather over the next several days, there’s just not much else to to say. So we’re publishing a list of the reasons—some tongue in cheek, some not—of why we ought to be thankful for this heat and drought. We received hundreds of great suggestions, so thank you to everyone who shared them with us. Without further ado …

10. The BBQ pit heats up far more quickly. Some afternoons you can even dispense with the wood or coals.

9. Roadwork and other construction projects aren’t delayed due to weather.

8. High pressure buffers the region from any hurricanes that might form.

7. Your dogs won’t track in mud from the back yard.

(Photo by Renzo D’souza on Unsplash)

6. If you have children, you can wow them with stories about the good old days when you played outside all summer long.

5. Sunshine and long days produce a solar energy bonanza.

4. Whenever it rains again, if it ever rains again, psychologically it’s going to feel so amazing.

3. Maybe, just maybe, the extreme heat radiating from roads and parking lots, even into the night, will discourage a few potential catalytic converter thieves. 

2. No rain means far fewer mosquitoes are out there biting in the evening.

And the no. 1 reason why this heat and drought ain’t such a bad thing is that you now have a medical reason to eat ice cream—hey doc, it’s helping to keep my core temperature down!

Wednesday

Tuesday’s high temperature “only” reached 98 degrees in Houston, and that’s probably about where things will top out today. Like on Tuesday, we should also see some isolated to perhaps scattered showers and thunderstorms later this morning and into the afternoon hours, with a 20 to 30 percent chance. Once again, if you get hit, you should feel fortunate, as the next street over might get nothing. Winds will be light, primarily out of the southeast.

Thursday

As high pressure intensifies it will tamp down on rain chances, but I think there’s still the potential for an isolated shower or two. Otherwise, expect mostly sunny skies and highs near 100 degrees.

Friday, Saturday, and Sunday

It will be hot, sunny, and humid. Look for highs of about 100 degrees each day, possibly in the low 100s for inland areas.

High temperature forecast for Saturday. (Weather Bell)

Next week

Conditions could start to change by Monday or Tuesday of next week, as a weak front moves into the area and brings more clouds, and perhaps nudges rain chances back up to 20 or 30 percent. This should be enough to bring high temperatures down into the upper 90s, or possibly even mid-90s with enough clouds. It is also possible that rain chances will be even higher, and temperatures lower, but it’s difficult to have too much confidence in such a hopeful forecast this far out. But some sort of change, at least, does appear to be on the way.

What you need to know about solar energy in Texas as summer heats up

Welcome to the summer solstice. Today is the longest day of the year, and the point at which the sun reaches the highest point in the sky. At 1:23 pm today, the sun will climb to an altitude of 84 degrees, reaching very nearly directly overhead. By contrast, in December, the sun’s peak altitude is as low as 37 percent. This high sun angle is why it is so important to wear sunscreen during the middle of the day in summer, but there is a brighter side of the story.

You may have noticed—it would be hard to miss, really—that it has been hot and sunny outside this summer. While summers are not always like this, when high pressure systems build over the Southern plains states, we see days and days of sunshine. This helps make Texas one of the sunniest states in the nation, with an average of 2,850 hours of sunshine a year.

The good news is that we can actually put that sunshine to use by gathering some of it for solar energy, and it is increasingly easy to do this. In this post, which is created through our partnership with Reliant, I want to talk about some of these ways you can put the Sun to work for you.

Put the sun to work. (Reliant Energy)

Solar stats

As the state grows, so does its energy needs. And especially during these warmer months, when the sun reaches the highest point in the sky, solar energy is helping to power our life-saving air conditioning units. Last summer, solar energy provided about 4 percent of the state’s electricity needs, and that may nearly double this summer. Let’s start with some basic information about solar power in Texas, sourced from the Solar Energy Industies Association:

  • $16.6 billion: Total solar energy investment in Texas
  • 1.7 million: Number of homes in Texas powered by solar energy
  • 506: Number of solar energy companies in Texas
  • 50 percent: Decrease in price of solar energy during the last decade
  • 4 GW: Amount of new solar energy capacity to be installed in the next five years
  • 1: State ranking in solar power generation in 2021

Understanding solar: utility scale vs. residential

There are two primary ways in which the sun’s energy is collected: residential solar power and utility-scale solar farms. Residential solar is spread around communities, made up of solar panels installed on roofs, or mounted on the ground, and is typically connected to the local utility distribution grid. Utility-scale means a centralized solar farm connected to the grid that sells generated energy back to the local wholesale utility, rather than end-use customers.

Both means of generating solar energy are helping to fuel growth in the state. And depending upon your home, lifestyle, and budget either might be an option for your electricity needs.

Solar hows

There are several ways for electricity consumers to get involved with solar energy, and save money in the process.

Without rooftop Panels

Support solar energy with your existing electricity plan. Reliant offers an easy option for customers to “Make It Solar” to support clean, renewable energy while keeping their existing plan the exact same. No panels required, you can live in a house or apartment that you own or rent. Customers simply add “Make It Solar” to their preferred electricity plan for $9.99/month. Or for those enrolling in any Truly Free electricity plan, it’s an automatic, free upgrade.

Enroll in a Plan Designed to Support Renewable Energy in Texas. Even without the commitment of installing panels, Reliant’s 100% Solar plan supports renewable energy and offers customers a stable fixed rate. When you sign up, 100 percent of your electricity usage will be matched with solar renewable energy certificates, which provide another source of revenue for solar power companies. These certificates signify that renewable energy equal to your usage has been generated and sent to the electric grid—so you’re not required to install solar panels on your roof.

With rooftop panels

Make the Most of Your Panels’ Power

If you already have solar panels, you can earn bill credits to make the most of your power. With the Reliant Solar Payback plan, you’ll receive bill credits up to your monthly usage when your solar panels generate excess energy and return it to the electric grid.

Backup Power Solutions

Harnessing Solar During an Outage

Solar can also provide peace-of-mind and serve as back-up power during an outage. Reliant’s sister company Goal Zero offers a range of solar-powered back-up power options, ranging from portable and mountable panels to portable chargers and generators. I used one of these to power some basic necessities, including a few laptops, during the Valentine’s Day freeze of 2021.

As we make it through this particularly sunny summer, know that the sun is helping create a cleaner, greener Texas. You can learn more about solar power and energy choices from Reliant at reliant.com/solar.