November 13, 2012 at 12:00Anatomy of Attack: How I Hacked Stackoverflow
- Transfer
Almost two years ago, I came across a fairly significant vulnerability in the StackExchange site network . I say “stumbled” because I did not try to hack the site. Circumstances opened the door for me. The vulnerability itself is quite interesting, and contains a lesson for everyone who creates and maintains sites or server infrastructure. So, here is the story of how I hacked StackOverflow ...
At that time I worked in a small company, where there was a firewall with completely draconian restrictions. He cut all request and response headers that did not meet the HTTP / 1.1 specification (and even cut valid HTTP / 1.1 headers). It played a trick on sites that rely on things like
I had several servers at the time, so I just installed Squid on one of them. Since I was thinking something, I only allowed proxy connections with
For those of you who want to tell me that I did wrong, I want to draw your attention to the fact that I had the opportunity to do this. And not just an opportunity, I was openly told to use proxies, since we had to work with some sites that did not work through our firewall. So I did not do anything “wrong.”
At that time, I often visited StackOverflow chat. Then he had just appeared, and there were a couple of bugs in it. One fine day, the site started showing me a call stack. I did not attach any importance to this, as I often met with this all over the Internet. Actually, almost every time I received an error message on a site written in ASP.NET, I saw a call stack.
And then I noticed a new menu item in the chat. This new menu item was called “Admin.” Out of curiosity, I clicked on the link, believing that I would be denied access. What happened next surprised me. I got full access to everything. I had a developer console where I could see who was doing what. I had an interface for working with a database, where I could directly query anything from any database. I got full admin rights.
The next thing I did was immediately report this to the moderator. A few minutes later I was in a private chat with a moderator, as well as two developers. The reason was found within approximately 10 minutes. After another 10 minutes, the problem was solved superficially. A complete correction took a couple of hours. They worked great. I still have the log of that chat, and I want to say that these developers deserve all praise. They answered quickly and professionally. And they solved the problem as soon as possible.
If you are smart, then you probably guessed what happened. Since I exited through a proxy, a header was added to my requests
But the most interesting thing was that ASP was showing in the logs. When they looked at the dumps of my requests, there was a record there
Never rely on
It is interesting to note that the developers did indeed use proper header checking. The bottom line is that you should never blindly trust your infrastructure. This hole appeared due to the configuration difference between the server and the application. Such little things happen every day. The application assumes one thing and the server another. The problem is that such trust can completely undermine security. In this case, the developers trusted the value
The StackOverflow team perfectly resolved this issue. Fast, responsive and reasonable. They asked me for help (which I gladly provided) and behaved professionally and respectfully. They did a fantastic job. We all need to learn a lesson. React to vulnerability reports seriously. Solve the issue professionally and quickly.
The most interesting thing here is that applications written in PHP can have the same vulnerability. We look Symfony2 Request class . It looks great. Until you notice that it uses a static variable to determine if a proxy should be trusted. This means that if any part of your application wants to receive headers from a proxy (for example, to write to the log), then your whole application will then receive headers from the proxy. To see if your code is vulnerable to this vulnerability, look for a call in it
Zend Framework 2 does the same. He has a classwhich behaves in a similar way (in terms of obtaining IP). Interestingly, in Zend Framework 1, the way to obtain an IP address was adequate. The calling code must explicitly indicate what it wants to receive, and by default there should be a safe option.
This problem was the result of a combination of two minor miscalculations. Separately, they are very easy to overlook. But if they converge in a certain way, you will receive a very serious security risk. And the biggest lesson is that you really cannot trust anything outside of your application. If you can get around this (for example, not trusting headers and variables, sort of
Initial data
At that time I worked in a small company, where there was a firewall with completely draconian restrictions. He cut all request and response headers that did not meet the HTTP / 1.1 specification (and even cut valid HTTP / 1.1 headers). It played a trick on sites that rely on things like
X-Requested-With. Therefore, for my "external adventures", I set up a proxy. I had several servers at the time, so I just installed Squid on one of them. Since I was thinking something, I only allowed proxy connections with
127.0.0.1. I raised the SSH tunnel to the server and pointed localhost as a proxy to the browser. The browser connected to the tunnel, which connected to Squid on the server. Everything was great. Besides the fact that all my traffic was encrypted, I got the opportunity to use the sites normally.For those of you who want to tell me that I did wrong, I want to draw your attention to the fact that I had the opportunity to do this. And not just an opportunity, I was openly told to use proxies, since we had to work with some sites that did not work through our firewall. So I did not do anything “wrong.”
Attack
At that time, I often visited StackOverflow chat. Then he had just appeared, and there were a couple of bugs in it. One fine day, the site started showing me a call stack. I did not attach any importance to this, as I often met with this all over the Internet. Actually, almost every time I received an error message on a site written in ASP.NET, I saw a call stack.
And then I noticed a new menu item in the chat. This new menu item was called “Admin.” Out of curiosity, I clicked on the link, believing that I would be denied access. What happened next surprised me. I got full access to everything. I had a developer console where I could see who was doing what. I had an interface for working with a database, where I could directly query anything from any database. I got full admin rights.
What happened next
The next thing I did was immediately report this to the moderator. A few minutes later I was in a private chat with a moderator, as well as two developers. The reason was found within approximately 10 minutes. After another 10 minutes, the problem was solved superficially. A complete correction took a couple of hours. They worked great. I still have the log of that chat, and I want to say that these developers deserve all praise. They answered quickly and professionally. And they solved the problem as soon as possible.
Vulnerability
If you are smart, then you probably guessed what happened. Since I exited through a proxy, a header was added to my requests
X-Forwarded-For. The value of this header was the IP from which the proxy was requested. But, since I connected to the proxy through the SSH tunnel, the IP was local. So Squid added X-Forwarded-For: 127.0.0.1... But the most interesting thing was that ASP was showing in the logs. When they looked at the dumps of my requests, there was a record there
REMOTE_ADDR: 127.0.0.1! In their application, all header checks were implemented correctly. But IIS was misconfigured and overwritten with a REMOTE_ADDRvalue X-Forwarded-Forif one was present. And thanks to such a configuration error, I was able to access the admin panel using my proxy server.conclusions
Never rely on
X-Forwarded-Forhow we see it is not safe. Always use REMOTE_ADDR. It is worth considering whether you need IP-based protection at all. Or, at least, you should not rely on it completely, but use it only as an additional measure. It is interesting to note that the developers did indeed use proper header checking. The bottom line is that you should never blindly trust your infrastructure. This hole appeared due to the configuration difference between the server and the application. Such little things happen every day. The application assumes one thing and the server another. The problem is that such trust can completely undermine security. In this case, the developers trusted the value
REMOTE_ADDR(justified), but the server was not configured correctly. Of course, there are situations when you must trust the server or other components, but remember that blind trust is not a good thing. Think about it, and try to insure yourself against such cases. The StackOverflow team perfectly resolved this issue. Fast, responsive and reasonable. They asked me for help (which I gladly provided) and behaved professionally and respectfully. They did a fantastic job. We all need to learn a lesson. React to vulnerability reports seriously. Solve the issue professionally and quickly.
Regarding PHP
The most interesting thing here is that applications written in PHP can have the same vulnerability. We look Symfony2 Request class . It looks great. Until you notice that it uses a static variable to determine if a proxy should be trusted. This means that if any part of your application wants to receive headers from a proxy (for example, to write to the log), then your whole application will then receive headers from the proxy. To see if your code is vulnerable to this vulnerability, look for a call in it
$request->trustProxy(). Also note that there is no reverse mechanism. You cannot “stop trusting” the proxy. I think this is a big architectural miscalculation ... Zend Framework 2 does the same. He has a classwhich behaves in a similar way (in terms of obtaining IP). Interestingly, in Zend Framework 1, the way to obtain an IP address was adequate. The calling code must explicitly indicate what it wants to receive, and by default there should be a safe option.
Conclusion
This problem was the result of a combination of two minor miscalculations. Separately, they are very easy to overlook. But if they converge in a certain way, you will receive a very serious security risk. And the biggest lesson is that you really cannot trust anything outside of your application. If you can get around this (for example, not trusting headers and variables, sort of
REMOTE_ADDR), then you can make your application more secure. But first of all, think about the code you write and the systems you build. And support them.