By Lee Feldman, Tschuna Patterson, and Dawn Thomas

On March 22, 2018, the city of Atlanta, Georgia, experienced one of the most high-profile ransomware cyberattacks to affect a U.S. city to date. The attack caused outages across internal and customer-facing applications, many of which affected critical city services, including law enforcement and courts.

In the days following the incident, city residents were unable to perform such simple tasks as paying parking tickets or utility bills. To restore city services, Atlanta has embarked on a months-long recovery process that could cost the city as much as $17 million.

The Atlanta cyberattack is indicative of the increasing prevalence of cyberattacks on city systems and infrastructure in recent years. As cyberattacks become more common, public sector attacks appear to be rising faster than those in the private sector, forcing city and county managers to consider how they would deal with the loss of critical systems and data for an undetermined period of time.

In response to the growing cyber threat, the city of Fort Lauderdale, Florida, took steps to enhance its ability to effectively mitigate respond to and recovery from a cyberattack. One of the authors of this article, Fort Lauderdale City Manager Feldman, worked with CNA—a nonprofit research and analysis organization—to design and conduct a cybersecurity exercise to identify gaps in the city’s plans, policies, and procedures.

The tabletop exercise brought together leadership and support personnel from across the city’s nine departments to discuss how the city would respond to and recover from a simulated cyberattack. Through the exercise, participants considered how their departments and the city as a whole would operate without access to critical systems and data for hours, days, and eventually several weeks.

While Fort Lauderdale’s Information Technology (IT) Department is responsible for preventing and protecting against attacks, participants who were part of the exercise quickly recognized that mitigation of, response to, and recovery from cyberattacks requires proactive planning by all city departments, including police, fire, public works, transportation, and finance to minimize adverse effects on the lives of the city’s residents.

Through the exercise, participants identified several opportunities for strengthening the city’s cyber preparedness. This article highlights five of those opportunities: ensuring continuity of operations during a cyber incident; developing a city-wide cyber incident response and recovery plan; planning for data recovery and restoration; ensuring delivery of prompt, reliable public information; and engaging with private sector partners.

Ensuring Continuity of Operations During a Cyber Incident

Continuity of operations (COOP) refers to the continued execution of essential functions during an emergency. Fort Lauderdale’s nine city departments each maintain their own COOP plans, which include information about essential functions, resources needed to perform those functions, alternate work sites for employees, and emergency communications procedures.

These plans primarily provide guidance during incidents that force employees to vacate their primary work sites, including fire, flooding, or power disruption. This results in COOP plans that focus on disruptions to the physical work environment rather than on disruptions to the virtual work environment.

Recognizing this gap, Fort Lauderdale departments have begun updating their COOP plans to provide guidance during cyber incidents. Topics that they are working to address include how to obtain clean laptops when employees’ primary desktops and laptops are infected; strategies for switching from automated to manual processes; processes for managing larger-than-usual amounts of paperwork; and understanding how legal and regulatory requirements will affect their work in a manual work environment.

The city’s IT department is also working to provide other departments with information about the city’s IT infrastructure to support COOP planning. The exercise, for example, revealed that most city employees are unsure whether the applications they use on a day-to-day basis are cloud-based, network-based, or a combination of the two.

Understanding where applications are hosted can be helpful, as data stored in cloud repositories may be insulated from malicious software affecting the city network.

Developing an Enterprise-wide Cyber Incident Response and Recovery Plan

While local governments have developed comprehensive emergency management plans, few of these plans address elements of response and recovery that are unique to cyber incidents. Due to Fort Lauderdale’s location on Florida’s Atlantic coast, the city is highly practiced in activating its emergency operations center (EOC) and conducting emergency management activities in response to hurricanes and severe weather events.

The exercise highlighted the need to consider how standard emergency management procedures will differ in the event of such a large-scale cyber incident as the one experienced by Atlanta.

Topics for communities to consider addressing in their cyber plans include triggers for determining whether an incident warrants emergency operations (EO) activation (e.g., severity, scope, anticipated duration, and concurrent events); procedures for determining which personnel should report to the emergency operations center to manage these types of response and recovery operations; procedures for deciding whether to pay in response to ransomware; and a prioritized list of critical applications to guide recovery efforts.

Planning for Data Recovery and Reconstruction

There are many forms of malware that can compromise computer functions or data. Ransomware, for example, restricts user access to files by encrypting them and displaying messages that are intended to force the user to pay the malware creator to remove the restrictions.

If a local government chooses not to pay a ransom—or if it pays and malware creators do not honor their promises to remove the restrictions—access to encrypted files can be lost forever.

Most local governments have processes in place for routinely backing up data, but the process of restoring data from backups can be time intensive, depending on the quantity of lost data and the length of time since the backups were stored.

If local governments lose large amounts of data, they will need to identify which critical assets should be prioritized for recovery and should document that information in the incident response and recovery plan discussed above.

Sophisticated malware can also infiltrate backups. If this occurs, data recovery will not be possible and local governments will instead need to reconstruct data.

During the exercise, participants considered various approaches they could take to reconstruct lost data. Potential approaches included transcribing hard copy data, piecing together information from emails, or asking vendors and customers to provide information from records like invoices or receipts.

Because these approaches are labor intensive and therefore costly, exercise participants identified the need to store their most critical datasets both on the city network and off the network (if possible).

Department directors also noted they needed to better account for a lengthy recovery period in their COOP plans, preparing their staff, the public, and city leaders for how long it will take to get back to normal, plus how much investment of time and money will be needed to do so.

Ensuring Delivery of Prompt, Reliable Public Information

Fort Lauderdale has ample experience developing and disseminating public information during severe weather emergencies. City departments are accustomed to working closely with the city’s Office of Strategic Communications to ensure that city officials speak with “one voice” when providing information to the public.

To disseminate public information, the city uses multiple channels, including traditional media as well as such social media platforms as Facebook and Twitter.

One challenge that Fort Lauderdale and other localities could face during a cyber incident is striking a balance between providing information as quickly as possible and ensuring that information is coordinated and reliable.

Some members of the public will become aware of a large-scale cyberattack—like the one experienced by Atlanta—almost immediately. In Fort Lauderdale, for example, many residents pay their utility bills in-person at city hall and would notice immediately if debit and credit card processing systems were unavailable.

In the initial stages of a cyber incident, it will be important to inform residents which services appear to be impacted and provide assurance that the local government is seeking a resolution. If emergency or nonemergency lines are down, community officials should also inform residents how to report emergency or non-emergency situations.

The Value of a Partnership

Private sector partners can play an important role in helping communities effectively respond to and recover from a cyberattack. Through its exercise with a private sector partner, Fort Lauderdale identified the need to pre-identify vendors to fill anticipated gaps. Exercise participants, for example, foresaw the need for supplemental staff during the recovery period to assist with data reconstruction.

Participants also noted that if phone systems were to become inoperable, they would consider working with a telecommunications company to establish temporary hotlines for city employees and residents.

Another private sector resource is cyber insurance, which can offer a range of data loss mitigation and incident response services. During the exercise, Fort Lauderdale identified the need to learn more about services available through its cyber insurance provider and how these services could support the city’s response to and recovery from a cyberattack.

Maintaining Preparedness for Evolving Cyber Threats

Maintaining cyber preparedness will require local governments to continually assess their response and recovery capabilities and take swift action to address any gaps or vulnerabilities that may appear.

Because system outages can have cascading effects across multiple city sectors—as the Atlanta cyberattack has shown—it is critical for local governments to engage all sectors, including private sector partners, in cyber preparedness activities.

Lessons learned during Fort Lauderdale’s cybersecurity exercise can serve as guidance to other local governments as they work to enhance cyber preparedness.

Lee Feldman is city manager, Fort Lauderdale, Florida (LFeldman@fortlauderdale.gov). Tschuna Patterson is senior research specialist, Safety and Security Division, CNA, Arlington, Virginia (pattersont@cna.org). Dawn Thomas is associate director, Safety and Security Division, CNA, Arlington, Virginia (thomasdh@cna.org).