Thread by @notareverser

Let's talk about shellcode for a bit, shall we?

It used to be that back in the day, it was difficult to write YARA signatures for shellcode

This was before the xor keyword or anything like that

It was very common for malware authors to encode their shellcode payloads using trivial transforms

Single-byte XOR was extremely common, and relatively effective (still is, truth be told)

Writing signatures for XOR encoded payloads sucked quite a bit

Our friends @wxs and @plusvic helpfully added the 'xor' keyword to YARA a while back

This is great! It makes writing "normal" signatures for shellcode a bit easier

However, some of you may recall a little tool called Plug-X

The payload was typically a header-stripped PE file, wrapped up in shellcode

The shellcode decoded the payload's configuration, loaded the payload, and started 'er up

Annoying to write signatures for the shellcode but not impossible

But then, something changed

You may recall an extra layer of shellcode was added soon into the evolution

It had junk instructions, fake conditionals, and a new encoding on top of the original shellcode

It wasn't crazy, a series of add, xor, and sub

It was annoying because the keys changed every time

How to deal with this, especially to write YARA signatures?

Let's define an arbitrary stream of bytes B, and a function of successive single-byte operations D with fixed constants/keys

For all b in B, D(b) should always produce the same value d

The problem with this is that for any two input byte values b0 and b1

There doesn't have to be a relationship between the output values d0 and d1

The Plug-X add/xor/sub imposes this constraint on its encoded bytes

If you want a signature, what are you signaling on?

Enter our good friend "signal analysis"

No matter what linear single-byte operations are performed on the underlying byte stream

Or how many times such an encoding is imposed

The histogram of that underlying byte stream remains constant

The relative positions of the original bytes also remain constant

So we have two facts no matter how many single-byte fixed-key operations we impose

We already ruled out the histogram, because the relative outputs bear no resemblance to the inputs

How about positions?

Take an arbitrary byte value b from our original B, say 0x90

What will remain true is that the *location* of all underlying 0x90 bytes will remain constant

We also know that the encoded stream has to be reversible

We can use this to our advantage to write a signature

We can take an arbitrary byte value bx from the original byte stream

And we can encode information about the byte stream in terms of the relative positions of bx to all other instances of bx

And the sequences of byte values "not bx" between positions of bx

This can be encoded as a "yes/no" map, in terms of the counts of bx / not bx

We can then transform these counts into a regular expression

And we can emit one regex for each of all 256 possible yes/no maps for all possible single-byte encodings

Writing regular expressions is difficult enough

So I wrote a tool many years ago to create these for me

I've now rewritten it again so that I can share it

And here it is for #100DaysOfYARA

gist.github.com/notareverser/f…

Is this fragile to changes in shellcode? Absolutely

Will it fail a lot? You bet

Does it still work often enough to use? Natch

The reason is that people are lazy and the work is done by the software

Use this technique whenever you encounter shellcode encoded with single-byte operations

xor, add/xor/sub, etc

If it doesn't work for you, let me know, and be sure to 'yara -w'

If you're enjoying these tips and want to see more, retweet the first tweet and share your ideas for YARA!

Oh, by the way, as a postscript...

The real reason I was a YARA 1.7.2 die-hard forever was that I had many signatures that took advantage of this technique

Finally @wxs clued me in on the single-line modifier for YARA 2+ (s at the end of the regex)

So I grudgingly entered the modern world of YARA

Love ya @wxs