When Hurricane Harvey struck Texas, the company that supplies the vital network connections throughout Houston was ready. The category 4 hurricane struck Texas on August 25th with catastrophic flooding and 130 mph winds. Fiber-optic network infrastructure company, PS Lightwave, supplies Ethernet, internet, dark fiber, VoIP services, and managed services to a large customer base in the greater Houston area. As a result, their employees worked around-the-clock during the storm to ensure the network stayed up for the mission critical services their customers needed during the storm.
“PS Lightwave ensures our customers get the scalable bandwidth they need in Houston, Texas and beyond. No delays, no disruptions, just reliable connections all the time.” – Rhonda Cook, CEO and President, PS Lightwave
PS Lightwave’s facilities-based fiber network includes over 5,000 route miles of fiber cable with approximately 1,400 on-net locations. Through this network, coverage is provided across seven counties, including the Greater Houston Area where over 80% of the school districts, large and small municipalities, and enterprise customers are served. Extreme Networks is PS Lightwave’s primary network provider. With over 2,200 Extreme switches, PS Lightwave has created a redundant network using ring topology and pure layer 2 services to meet and exceed all of their customers’ needs.
Prior to the storm, PS Lightwave conducted numerous fire drills and had a disaster preparedness and recovery plan. Everyone on the team was ready for the storm. PS Lightwave sent communications and updates to customers via email, social media, and blogs. They also put their Extreme Networks account manager on notice so that in the event of the hurricane being catastrophic, they could place immediate and emergency orders for technology used in critical infrastructure across their network with no problems. The great relationship Extreme and PS Lightwave have cultivated over the last 20 years has made this process easier to prepare for and trust that the vendor will come through.
“Extreme is the only vendor that has volunteered to get product to us immediately. If I called and said I needed 50 switches air lifted here within 24 hours’ Extreme Networks would do it, no questions asked.” Robert Jacobs, Director of Network Services, PS Lightwave
Everything on PS Lightwave’s network runs Gig E, 10 gig, or 100 gig. The team was prepared for the storm because of the extra bandwidth they can manipulate, there is no need for them to go in and rip equipment out. They can take parts from some switches and move them to others to augment the network and ramp it up to get customers back online as quickly as possible when a long-term solution is not feasible in the short-term. Utilizing a single vendor for the network has resulted in PS Lightwave being able to switch parts quickly, have all staff members trained on the same technology, and cultivate a strong partnership.
At the peak of the storm, only 5% of PS Lightwave’s network was down and this was due to power outages, not hardware failures. After the storm, only 3 switches out of 2,200 are no longer operational. With this said, numerous switches were submerged in flooding for hours and continue to operate normally because power went out at the site before water hit the switch. Other switches experienced high temperatures as a result of air conditioners not functioning during the storm. Some switches recorded hitting 63 degrees Celsius when they should not go over 55 degrees. The switches worked double duty to keep the network working. According to PS Lightwave’s Director of Network Services, Robert Jacobs, “This goes to show how durable the switching technology from Extreme Networks is. Even though we will look into replacing the switches that were affected heavily during the storm, they made it through and we don’t need to replace them right away, we can do it on our time.”
“The durability and performance of the Extreme products are compelling reasons we continue to renew our commitment of using Extreme Networks. The elegant simplicity of the design has added to our success in consistently exceeding five 9s reliability in our core network. That is critical to us in managing the largest metropolitan area fiber network in our region.” – Rhonda Cook
Within two days of the hurricane, PS Lightwave was already getting requests from FEMA and others to setup emergency sites with connections. “We knew some sites were already flooded out or had extensive damages so we had already begun to dynamically move services around to get connections back up,” adds Robert.
PS Lightwave employees knew they would need to remain connected to the network to address customer needs, but they would not be able to come to the office because of flooding, fallen trees, and road closures. Many employees found creative ways to stay connected. Robert stayed connected by using the small inverter from his truck to keep a switch running because there was no power at his home. Some employees were getting to customer sites to address issues by boats and jet skis. “In some cases, where power was out, we installed generators at sites and every day our team would bring gas via a boat to ensure the generator and network remained operational,” notes Rhonda.
This hurricane was not an optimal environment for keeping customers connected to the network. It took a lot of planning and action to keep the network up throughout the storm. With every storm that affects PS Lightwave’s customers, the team has learned from previous experiences. “We are more intelligent about where we put aggregation nodes and pops and we have a lot more redundancy in the network,” says Rhonda, “Our preparedness, network nimbleness, and employee skill level is what kept critical technologies and infrastructure reliably connected throughout the storm.”
For more information on PS Lightwave’s response to Hurricane Harvey and their disaster preparedness plan, please view their press release.