Lebanon County DES maintains contact with callers during statewide 911 failure

The statewide intermittent 911 outage on Friday was the first of its kind in the Lebanon Valley and across Pennsylvania. 

About 2 p.m. Friday, the statewide system, which is serviced by call centers in eastern and western Pennsylvania, began to intermittently drop calls, including ones made in Lebanon County.  

Service was restored about 11 p.m. Friday, nine hours after the problem was first experienced, according to Bob Dowd, director of Lebanon County Emergency Management Services. The Pennsylvania Emergency Management Agency announced on Saturday an investigation into the cause is ongoing.

Pennsylvania Gov. Josh Shapiro said during a Saturday stop in Philadelphia that the problem was a technical issue and not sabotage. 

“The head of PEMA reported that it was not sabotage, and I think that should be comforting for folks,” Shapiro said.

It was the first time that there was a major issue within the statewide system, which is a few years old, according to Dowd.

“There has been a change to the way we do things,” he said. “Going back some years and prior, we got our 911 trunks from Verizon. When you called 911, no matter who your phone provider was, Verizon then gave the call to us. There were only so many phone service providers who were able to provide 911 trunks, and Verizon was one of the larger ones.”

Trunks are extra phone lines used specifically to receive incoming 911 calls.

Read More: [LebTown Exclusive] Lebanon County’s new 911 ops center set to open soon

Dowd noted that outages in the past, prior to the implementation of the statewide system, were specific to a given phone carrier or locale, meaning problems that occurred were never as widespread as the one that happened last week.

“If there was an outage, either it was specific to a carrier, so either Verizon went down or it was specific to an area,” Dowd said. “Or a certain exchange went down.”

Dowd said about three years ago the state changed the 911 emergency call services system. 

“What that means now is that we don’t have trunks with Verizon, we’ve got a fiber network between all of the (911) call centers that connects us to two data centers in the state for all of those calls,” he said. “They route it (a call) to us over that fiber network.”

Dowd emphasized this was the first time the state’s system has encountered a problem of this magnitude. On Friday, local residents were urged to call their local 10-digit phone number for 911 service.

“I think it’s three years ago now, and this is the first hiccup that has happened – and it’s a pretty big one,” Dowd told LebTown on Friday. “I’m not denying that and I am not downplaying it. The problem is the issue is very high level within that network and is why it is affecting everybody.”

Dowd said it’s a good idea for Lebanon County residents to save the 10-digit non-emergency number that dials into the county’s DES offices. Locally, that number is 717-708-2746.

“If there was a piece of advice here, there is no such thing as a 100 percent perfect system,” Dowd said.

He noted that the county’s emergency communications center personnel informed him that there was a problem shortly after 2 p.m. Friday. 

“The calls are still coming in,” he said Friday evening. “It’s intermittent, and what’s happening frequently is it doesn’t complete the connection to us. So we see it come in, but we can’t answer it. So what we’ve been doing is, we see the phone number and we call it back. Our outgoing calls are on a totally different system, so that’s how we’re responding and that has allowed us to continue to function.”

Dowd said that while calls were being dropped, Lebanon County emergency personnel were able to identify those callers and contact them using their independent redundant phone systems. Those systems include, for example, regular phone lines, which were never impacted by the intermittent disruptions to the statewide system.

Additionally, he noted the system has several protective layers, including alternate call routing.

“If something happens to our connection to the state network, the connection is monitored continuously, and if that network goes down, there’s a rule that kicks in automatically that starts sending 911 calls to a different phone number for us,” Dowd said. “So the end user, the person calling 911, doesn’t even know that something has changed. It’s automatic, it happens seamlessly.” 

That usual alternate routing, however, was problematic on Friday, added Dowd.

“The issue right now is that the alternate routing lives at the state’s data centers, and that’s also where this 911 problem is originating, so that alternate routing is not working correctly,” he said Friday evening. “So that switch that we’re talking about, which is that alternate routing, only works if the device that’s responsible for taking that call and sending it to the location is working correctly, and that’s what’s not working.”  

Another redundancy is what Dowd said is the Rapid SOS system, which is an additional layer so that emergency personnel know when a call comes into the communications center.

“It’s not a phone line, it’s a 911 information portal that gives us information about people calling 911,” he said. “It’s basically a web portal we keep open that’s independent of the 911 network, and that is the perfect example of why we have these redundancies.”

Dowd noted that while outages do occur, they are generally short-lived and previously localized. 

“This (outage) was different, it was the length of time,” he said. “I am sure things will change as a result of what happened.”