THE HUNT: A Cyber Attack in the Process Industry
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
#cybersecurity #thehunt
blog.f-secure.com/cyber-security… …
#cybersecurity #thehunt
Pulp Global.
Pulp Global Inc. is a leading producer of various types of pulp products
Number of employees: 12,500
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Martin, the CIO of Pulp Global, has a plan.
He wants to roll out reporting and manufacturing execution systems to all 25 of the company’s mills.
That’s where our story begins.
•Their industry is a highly competitive
•Producers need the lowest possible costs
•There are as many different ways of building and running a mill as there are mills
•Only 5 of its 25 mills use the same systems
#thehunt
Number One decides to do a little investigation of Pulp Global.
He calls this his “get-familiar phase.”
His team looks into vendors, suppliers, maintenance contractors, personnel, and information about the IT and OT systems.
One word keeps popping up: Segnen.
Built on industrial estate along the coast of the Baltic Sea, it has:
* 5 distributed control systems
* 8 SCADA systems
* 3 mill information systems
* And that’s just the beginning of the complexities
blog.f-secure.com/cyber-security…
#TheHunt
The files inside:
•Mill floor plans
•A list of equipment with brand, models & version #s
•A list of new software, OSes, network layouts, IP addresses, Modbus tags and I/O listings, and even and passwords
An engineer from Pulp Global’s third party integrator had received an email from containing only the zip file with the subject: “Important project documents.”
He decided to upload the zip file into a reputable malware scanning service.
Malware researchers occasionally have ties with the underground crime community.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Meet Peter.
Peter has just finalized his Master’s degree in Automation Technology and was almost immediately hired by Pulp Global.
His thesis just happens to be about – wait for it! – the Pulp Global's plant automation project.
The bad news for these hackers?
The Segnen plant hasn’t been modernized and has a limited attack surface.
Very limited.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Smart move for lowering costs, but there’s a problem.
The service also had a test environment where the service developers could test & deploy changes.
And there is an admin account
Login: admin
Password: admin
Score.
blog.f-secure.com/cyber-security…
#cybersecurity #thehunt
Number One decides it is time to launch a targeted spearphishing campaign.
Through the newsletter, he learns employees recently had been rewarded with a party.
He creates an email using Pulp Global’s template...
Subject: Employee Celebration Pics
Once executed, it would connect back to Number One’s command and control infrastructure and allow him to perform additional attacks.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Peter couldn’t go to the party but he’s interested in seeing what he missed.
Official template. Seems legit.
He clicks.
#thehunt
It only takes about an hour from sending the email to getting the initial foothold via Peter’s compromised computer.
Game ON.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Number One and the team spend the next two weeks mostly celebrating. No further attacks or trying to extend their foothold.
Instead, they monitor Peter’s laptop, logging keystrokes and capturing screenshots of the systems Peter accesses.
This allows the criminals to utilize Peter’s PC outside office hours.
They probe the network, launch attacks and extend their foothold.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
The amount of information and the level of detail he finds are unlike anything he’s come across before.
JACKPOT
#cybersecurity #thehunt
*network drawing
*system & shop floor layout
*project plan
*equipment list
*asset inventory report.
Their exfiltration of information detailing Pulp Global’s projects and sites totals tens of gigabytes.
Some comb through the millions of pages of documents. Others try to extend their foothold in Pulp Global’s network.
To remain undetected, they use all the same tools the company’s IT staff and system administrators use on a daily basis.
Pulp Global has no network-level visibility and very limited host-level visibility.
The attackers can run their tools, no matter how noisy, without a significant risk of getting caught.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
1.They're a mishmash built over 3 decades
2.They include everything from legacy applications and servers to new Windows systems
3.Many systems receive no updates, making them easy to infect with publicly available exploits
In a photo from IT support staff member, a computer monitor in the corner has a
Post-It note stuck to it.
The note contains the domain administrator credentials used by Pulp Global’s IT support staff.
Number One doesn’t believe they will work. But they do.
These credentials were reused so much within the corporate network that they also worked for Linux hosts, and even the routers and switches.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Meet Eric, a control systems engineer working for a third-party integrator.
He’s guy who uploaded the zip file to the reputable malware scanner
Remember that?
He’s responsible for the newly installed manufacturing execution system (MES) used to synch Pulp Global’s enterprise resource planning
(ERP) system to production.
MES also automates the flow of data between ERP and production.
#cybersecurity #thehunt
It has perfect access to both the corporate and production Incident Command System networks.
#cybersecurity #thehunt
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
*the different systems
*their software versions
*the PLCs (programmable logic computers)
*the DCS (distributed control system) equipment used in the mill
And for fun, they try to harvest more user credentials
#thehunt
BONUS: Local admins had written in the passwords for easy access for the employees.
However, one network is separate -- the engineering workstations that program the programmable logic computers (PLCs) and safety instrumentation systems (SIS) used to run the plant.
#thehunt
blog.f-secure.com/cyber-security…
Eventually, the engineering workstations is found via an unlikely source -- an operator station sitting in one of the control rooms.
#cybersecurity #thehunt
This makes it possible for automation engineers to access the engineering workstations while monitoring operations from the control room.
It's also the hackers' way in.
blog.f-secure.com/cyber-security…
After about a month of collecting information and extending the foothold in Pulp Global’s networks, Number One finally has enough information to prepare the targeted attack on the industrial control systems at the Segnen mill.
Fun!
In addition to generic resources, such as Google and various pulp industry trade publications, the attackers pour over Peter’s thesis on plant automation to plan the attack.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
This means they can test and validate their attack on the production systems with only a minor risk of getting caught.
It’s clear they need additional help – on the inside.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Playing the part of a new employee at one of the mill’s contractors, Number One asks questions that could only come from someone who knows the mill intimately.
The employees suspect nothing.
#thehunt
If successful, his attack would cause lasting reputational damage and financial loss.
If it fails, it will wipe out all PLCs, servers and hosts in the production environment.
Using the original source code, project files and HMI graphics, the group sets up a crude test bench for simulating and testing their attack payloads.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
This is easier said than done.
Segnen mill runs 24/7. Controllers can’t be taken offline or stopped for the code change without the attackers calling themselves out.
By changing small parts of the specific code blocks and extending the existing variable in data blocks, the controllers don’t need to be stopped.
The new code can be downloaded onto the controller and will start running during the next cycle.
They change six different PLCs, a few safety logics, and create new views for HMI panels used by the operators to monitor the process.
It’s go time.
blog.f-secure.com/cyber-security…
To maximize the impact, and make it extremely hard for the Segnen mill workers to pinpoint the problem, Number One attacks multiple parts of the pulping process using normal fluctuations to cloak the hack.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Temperatures used in the pulp cooking process begin to vary random intervals.
The fluctuations in temperature distort quality control system readings, which results in waste, quality problems, fines and -- ultimately --reputational damage.
#thehunt
#thehunt is on.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Due to recent attacks against the safety controllers, Eric, the control systems engineer, implemented a physical key attached to the controllers, putting them on run-only mode.
This stops anyone making software changes without actually turning the key.
The attack code is left running.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
The control systems engineers, including Eric, work overtime to find the cause of the random fluctuations.
#thehunt
A quality control system is running a hidden process.
That shouldn’t be happening.
The vendor rep sends the executable file to the Pulp Global IT team and a security company it trusts.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
The pros recommend a thorough sweep of the production hosts to see if any other implants are hidden in the network.
Eric contacts his boss, who notifies the CIO of Pulp Global:
The mill is under attack.
#thehunt
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
There it is: a huge number of alerts caused by one of the safety controllers on a valve.
But he can’t figure out why. Leaving the development environment open, he exits the control systems lab, puzzled.
#cybersecurity #thehunt
Normally reliable systems are now acting up, seemingly without any reason.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
He saw the development environment open with the very same part of the code they’d been trying to circumvent for the better part of a week.
“How did they find us?”
What could the group still do to inflict maximum damage with the foothold they still had?
“Time to say bye-bye to the Segnen mill operations,” Number One says, as he sends a command to all of the compromised hosts.
#cybersecurity #thehunt
Within two hours, the Segnen mill is completely halted.
The pulp cooking process has to be taken down using a manual override -- a process that can take weeks to reverse.
blog.f-secure.com/cyber-security…
Almost all of the Windows hosts in the mill are wiped clean.
Nearly 50 controllers, from two of their main vendors, suffer a similar fate.
Only a few controllers are unaffected.
#cybersecurity #thehunt
blog.f-secure.com/cyber-security…
Bored mill personnel begin posting pictures of blank control room displays on Facebook.
Reporters notice.
Eventually the news of the complete breakdown at the Segnen mill reaches the CIO–from a reporter looking for a comment.
#thehunt
After the call from the reporter, the CIO contacts Segnen to find out what's up.
The mill is in such chaos that it it’s impossible to get anyone to stay on the phone.
Pulp Global had no plans or procedures to guide the CIO in dealing with the incident.
He asks the vendors supplying the Segnen mill to send first responders on-site immediately.
#thehunt for the hackers is on.
#cybersecurity
blog.f-secure.com/cyber-security…
He just got an email.
“Segnen mill was just the beginning,” it reads. "Comply with our requests by the end of the day or this could get very expensive for you.”
#thehunt #cybersecurity
Meanwhile, the board of directors have called an emergency meeting about the ransom demand.
#thehunt #cybersecurity
blog.f-secure.com/cyber-security…
The analysts soon abandon the Segnen mill and turn their focus on the corporate network, as law enforcement investigates to see if it's an inside job.
#thehunt #cybersecurity
This means the hackers have nearly limitless possibilities to attack other targets within Pulp Global.
#thehunt
They aim to limit the access of the adversaries while being careful not to draw attention to their efforts.
#thehunt #cybersecurity
blog.f-secure.com/cyber-security…
An hour after the ransom deadline passes, nothing has happened.
#thehunt #cybersecurity
A quick check confirms it isn’t anyone within Pulp Global.
The adversaries have returned.
#thehunt
blog.f-secure.com/cyber-security…
The adversaries use a process running with system privileges to send commands to hosts within another mill’s production network.
blog.f-secure.com/cyber-security…
#thehunt #cybersecurity
Investigators find the same remote access trojan deployed to several other machines. Now the responders know what to look for.
Even better, the trojan keeps an easily decryptable log file of all of actions performed.
#thehunt #cybersecurity
blog.f-secure.com/cyber-security…
But the problems keep growing…
Pulp Global’s share price has plummeted more than 10% within a day.
#thehunt #cybersecurity
blog.f-secure.com/cyber-security…
Unhappy customers begin to demand compensation from Pulp Global due to the delays.
The local authorities investigate the environmental, safety and health hazards caused by the abrupt stop of mill operations.
#thehunt #cybersecurity
It takes nearly a year to regain the confidence of customers and investors and deal the resulting the safety and health hazards.
Although the direct cost of the forensics, equipment and labor are in the millions, the biggest cost can’t easily be measured -- the loss of customer & investor confidence.
With cyber attacks growing more advanced by the day, you cannot rely on stopping all threats before they infiltrate your infrastructure.
These days, a compromised network is not a question of “if,” but “when” and “what do we do about.
Speed matters.
A high-quality detection and response solution slashes attack detection from months to minutes.
Find out more about F-Secure’s Rapid Detection & Response:
f-secure.com/en/web/busines…