It started out like a normal Thursday for Tony Mendoza, senior director of IT for Spectra Logic, a data warehousing company based in Boulder, Colorado. And then the ransomware attack began.
“We got some notifications of some system failures and they quickly turned into many unrelated system failures, which is really abnormal,” Mendoza says. He realized that the company was under attack and that his files were encrypted.
“When it arrived, we ran into our server room and data center and started pulling out the plugs so it couldn’t propagate, which took down our entire infrastructure,” he says.
WATCH: What is cyber insurance? Everything you need to know about what it covers and how it works
In total, three-quarters of the production environment was compromised with ransomware. The hackers left a ransom note demanding a payment of $ 3.6 million in bitcoins in exchange for the decryption key.
“Finding out what it was was quite simple, because they tell you who they are and they tell you where to send the money. It was NetWalker because it said so in the ransomware letter,” Mendoza explains.
Another problem: The attack occurred in May 2020, when many employees had just started working remotely due to the COVID-19 outbreak, so there was no way to easily communicate what was happening outside the building.
Despite that, the IT team had to assess the damage that had been done and what the options were for recovering the data, if possible. There was some hope: the company had backups, which were separate from the rest of the network and were safe from the incident.
“We are still under attack, we are still trying to stop the bleeding, we still don’t know what the extent of the damage was, but we knew we had data to work with,” Mendoza says.
Any organization that is the victim of a ransomware attack ultimately has to face an important question: Will they give in to the ransom demand to get their data back?
Cybersecurity companies and law enforcement agencies around the world argue against giving in to extortion around ransomware attacks, because it not only delivers hundreds of thousands or even millions of dollars in bitcoin to criminals, Rather, it demonstrates that the attacks work, encouraging ransomware attackers to continue their campaigns.
However, some victims feel they have no other choice and will pay the ransom, as they perceive it to be the fastest and easiest way to recover their data and recover the network, although that is not without its problems. . There are cases where the attackers took the money and fled, or took the ransom and then returned with a second attack.
Spectra Logic had cyber insurance, which could potentially have covered the cost of paying the ransom. That might have been the simplest short-term decision to restore the network, but it was quickly decided that with the backups still available, Spectra Logic would not give in to the ransom demand.
So instead of communicating with cybercriminals, Mendoza contacted the FBI.
“I went from being in a panic to being assured that they had seen it before, that we are not alone in this and that they are going to put the tools in place to start protecting us. That was the most important thing, getting protected,” he explained.
The FBI also assigned a team of specialists to help Spectra Logic deal with the immediate aftermath of the attack over the course of the following days.
Trying to restore the network turned out to be a 24/7 job for the small team over the course of the next week. For much of that time, people slept in the office to give themselves as much time as possible to focus on restoring the network.
“Starting Thursday morning, we spent 24 hours every day for the next five days working on this; we slept in shifts. Three of us worked through the night while two people slept for a few hours,” Mendoza said.
“There were no exits and returns, it was going to sleep on the sofa in case we need it. It was five days of hands-on work.”
In addition to this, he had to provide the board with an update on the current situation. They wanted answers on when the network was going to be restored and when business would return to normal.
“I’m dealing with the leadership of the company and I don’t want to lie to them and tell them that I know when it will happen; I had to tell them that I don’t know what’s going on or when the systems will be working,” he says.
It took round-the-clock workdays, but eventually the IT department, with the help of cybersecurity specialists, was able to restore some network functions a week after the ransomware attack, without paying the attackers.
WATCH: A winning strategy for cybersecurity (ZDNet Special Report) | Download the report in PDF (TechRepublic)
“Our cybersecurity team gave us the experience and the tools, the monitoring and the registry to remove the threat from our system. On Monday morning they gave us the green light; it is done, they detained it and eliminated it,” recalls Mendoza.
“The FBI told us we were going the hard way, but the right way, and it ended up being the easy way when we came back and said we were back eight days later; it was shocking for them.”
But that did not mean that everything returned to normal immediately; It took more weeks to recover non-business-critical systems, and throughout that time careful attention was required just to make sure the attackers had somehow failed to spread the ransomware again, which meant constantly monitoring everything. activity on the network for another month.
Many ransomware attacks never become public knowledge, and the examples of companies that go into detail about what happened are still few and far between.
But Mendoza says it is important to be transparent when dealing with a ransomware attack, because it is important to demonstrate that it is possible to recover from an attack without lining the pockets of cybercriminals.
“What we realized was that we protected our data and there is a way to thwart ransomware. We couldn’t find public information when we searched for it, so we wanted it to be common, that it’s okay to talk about being impacted by ransomware,” he said.
So what is the key lesson Mendoza would say other organizations should learn from the Spectra Logic experience? It’s all about backing up your systems, and doing it offline, so if the worst happens and your organization goes down, you still have offline backups.
“You have to limit the blast radius of your attack. Back up your data in multiple locations on multiple media and the key is to make an air gap. Be it a physical air gap or a virtual air gap, you have to put a wall between an attack and your data, “he said.
And how did the company end up falling victim to a ransomware attack in the first place? Analysis of the incident revealed that a phishing email sent to an employee working from home was the hackers’ way of gaining their initial access to the network.
In the aftermath of the ransomware attack, Spectra Logic has worked to improve its cybersecurity culture, both on-site and for remote workers, in an effort to learn from the incident. The company is now actively looking for potential cybersecurity threats that might have been overlooked before.
“Initially after the attack, when the wounds were fresh, we talked about security. Six months later, we are still concerned about security and are more aware of phishing attacks. Before we were a little complacent,” he says: Now the staff will notify you if the malware system does not detect a phishing email. “There is more awareness now.”