You see a weird openssl command running on one of your Linux systems. Here's how to investigate whether it's a bindshell backdoor operating on the box and hiding traffic inside an encrypted tunnel. Thread. #DFIR
The server and client to run the attack. The reverse bindshell causes openssl to connect back to us and is encrypted so network monitoring is blind to what is going on. Need to look at the host to figure it out.
We log into the host after seeing the weird outbound connection and need to investigate. Run ps -aux and lsof -p <PID> to see the process. Throw in netstat for good measure. We see openssl and /bin/sh -i running that look strange.
Once we see it's a PID of interest on Linux, we will go to /proc/<PID> to look around. Here we see the links to the executable and other data.
The process looks like standard openssl, so we'll now look at /proc/<PID>/cmdline and /proc/<PID>/environ to glean anything useful. These areas often leak important Linux attack information. #DFIR
We know this looks strange and has open network sockets. Let's look at the Linux maps to make sure it's not referencing any weird libraries inside /proc/<PID>/maps
So nothing is hiding that is obvious. Let's see what file descriptors it has open on Linux. This will show hidden data files, sockets, etc. the process has open that is of interest to us. *Let the process tell you what is interesting.* No need to break out the debugger. #DFIR
Right, we have a file of interest under /tmp/s. Let's look and see what it is. Appears to be a pipe with a single character name which is odd.
Doing the above steps to the /bin/sh -i process also. We see it has same links to open files so yes it's talking to openssl. They are clearly linked and referencing same Linux socket inode. They are talking to each other.
At this point we know this openssl client was not from a developer debugging a SSL certificate. It's malicious and time to initiate incident response procedures. Of course I think it's easier to use Sandfly to find this stuff automatically. #sandflysecurity#DFIR
Investigating a Linux host for backdoors manually is fun, but it's better to do it automatically. We have a free license to use our agentless Linux security platform. You can get it here instantly and use it for free on five or fewer hosts. #DFIR
Full article about how to use command line Linux forensics to investigate a suspicious openssl backdoor is here with more details than I can fit in a bunch of tweets:
@SandflySecurity was able to spot this malware very quickly and with multiple serious alerts. Let's have a walk-through about what it was up to and de-cloak it. #DFIR#sandflysecurity
Virus total shows very few results. But we saw many serious compromise tactics in use on this Linux system hit with the malware.
The top and ps commands show some odd things. Top is showing 100% of the CPU in use, but no process claiming responsibility. The ps listing shows nothing unusual. Something is hiding. Let's de-cloak it. #DFIR
Malware will name itself with [brackets] to impersonate a Linux kernel thread. Bracketed names mean no cmd line argument. Kernel threads almost always are in brackets.
Use ps with tree view to find our candidates for investigation:
Any #Linux process that looks like a [kernel thread] should have an empty maps file. If you look at the maps file for a process and it has data in it, it's not a kernel thread:
cat /proc/<PID>/maps
Our suspect below has entries under maps. Bad news.
Let's go step-by-step and do some basic live process forensics for #Linux. Today's contestant is a bindshell backdoor waiting for a connection on Ubuntu. We saw something odd when we ran:
netstat -nalp shows a process named "x7" PID with a listening port that we don't recognize. #DFIR
First thing we'll do is list out /proc/<PID> to see what is going on. Our PID is 5805:
ls -al /proc/5805
The current working directory is /tmp. The binary was in /tmp, but was deleted. A lot of exploits work out of /tmp and /dev/shm on Linux. This is a major red flag. #DFIR
Often, malware deletes itself after it starts so file scanners and integrity checks won't find it. It can make analysis harder if you can't get to the binary easily.
But if you remember /proc/<PID>/exe you can recover any deleted binary.