Thread by @EladErnst

1/ Our team at @dWalletLabs discovered a chain of vulnerabilities that could result in a loss of more than $1B in crypto assets. The full article here: Let's take a closer look0d.dwalletlabs.com/the-billion-do…

2/ So we simply scanned all SUI validators' ports. The list was extracted using the API in the SUI Explorer.

3/ On one of the validators, we discovered that port 55555 was open and exposed an interesting web service called "Tailon", an open source service that allows easy access to log files. There are some interesting features of this service, like tail, grep, and sed the log file.

4/ What makes it interesting? sed can get the parameter "r" and concatenate any file on the server. More than that, it supports the "e" flag, which lets us run any bash command and concatenate it to the output, take full control of the server and extract validator private keys

5/ What is this validator? This validator is managed by a company called InfStones, which supplies staking and private node services for many blockchain networks.

6/ With this understanding, we checked whether other validators around the world exposed the same vulnerability. We found many servers that looked similar but when we tried to access 55555 port, we were asked for a password.

7/ In order to gain a better understanding of how a dedicated node of InfStones operates, we created one ourselves. Our dedicated node also has port 55555 open, but it requires authentication.

8/ However, when we tried to access the "Logs" tab using the InfStones management interface, we found an embedded Tailon service. Digging into the rest API of InfStones, we found the following call:
cloud.infstones.com/backend/node_l…

9/ As far as I can tell, InfStones backend is performing a web proxy to make Tailon service accessible without authentication. We simply created a webserver and called the API with our IP address.

10/ As a result, InfStones BE attempted to connect to our server and sent us the credentials. Since the credentials were the same across all of the servers, we now have access to more than 100 servers, many of which are validators.

11/ While investigating our private node, we came across many interesting items such as the private keys of the validator and the AWS creds. e also found a Go-based web service named "infd" listening on port 12345 on our server, that exposed REST API to manage the server.

12/ After researching this rest API, we quickly discovered a Shell Injection. However, the rest API enforced JWT authentication. A closer look at the Auth handler revealed it is bypassed in two cases: calling from localhost, and when "CloudProvider" is set to "customer".

13/ So we performed a quick Censys search and found more than 1000 servers listening on 12345 with similar services, but after accessing the "data" route on all of them, we found one that did not enforce authentication (CloudProvider = customer).

14/ This might not have been as meaningful if it hadn't been for the @Aptos_Network validator of infStones ($145M staked).

15/ But it wasn't good enough for us, so we dug into the "ClientIP" function and found out a very cool thing - whenever the src IP address of socket is one of the "Known-Proxies", the web server will try to extract the "ClientIP" from the "X-Forwarded-For" header.

16/ However, the coolest part is that the default value of "Known-Proxies" is 0.0.0.0/0, so we were able to bypass authentication for all servers by using X-Forwarded-For: 127.0.0.1.

17/ As a result of the vulnerability, we were able to get full control on more than 1000 nodes in many blockchains, many of them validators, with a total stake worth more than $1B.