Too many cooks, or hacking the Internet with TR-069

Original author: Lior Oppenheim, Shahar Tal
  • Transfer
Shahar Tal: we will conduct a quick vote - which of you saw the series “Too many cooks?”. Good enough interest for such an audience. So, let's begin.

The screen saver for the film is shown on the screen, then portraits of speakers appear.

Further in the credits are indicated:

Denis DeVito - in the role of an affordable IPv4 toaster, Arnold Schwarzenegger - in the role of a TP-Link TD-W8961ND router and others ...

So, for those of you who have not seen the film “Too Many Cooks”, which was shown a few months ago, We advise you to watch it, we really liked it. If you watched it, you could appreciate our “funny pictures”, which we used for the presentation.

So, we are researchers of malware and vulnerabilities and work for a company that is a developer of CheckPoint IT security software located in Tel Aviv. The slogan of our company is “We protect the Internet”. Our task is to find the vulnerability of the equipment, tell the manufacturer about it and share information with the public.

Today we will talk about the following:

  • We will conduct a quick review of the CWMP TR-069 protocol specification, a kind of squeeze out of our speech at the DEFCON conference
  • the motivation for our research;
  • the latest edition of TR-069 from 2014, which will provide you with interesting information about our research and its technical details;
  • actually about our research;
  • mass users;
  • about a pessimistic view of things.

So, TR-069 is specification No. 69, which describes the CPE WAN control protocol (CPE - equipment installed in the client’s premises, your home router, WAN - an extensive Internet network with a large number of computers). Version 1.0 of this specification first appeared in 2004 at the Broadband Forum by a group of companies developing broadband standards. There were several editions of this specification, but remember that it was developed almost 10 years ago. In 2013, version 1.4 (amendment 5) appeared; perhaps in 2015 the 6th amendment of this specification will be released.

This is what ISPs use:

  • for initial installation (the so-called zero configuration);
  • To monitor your device for malfunctions and malicious activity
  • to configure your home router, including assigning it a MAC address, assigning host names for your network;
  • to create additional WI-Fi networks;
  • to update the firmware;
  • and many more additional operations.

The next slide shows what a work session looks like according to this specification.

On the right is the CPE user equipment — your home router with the TR-069 client, on the left is the TR-069 server, also called the ACS automatic configuration server. They “talk” to each other using SOAP RPS, using XML over HTTP.

The session is always initiated by the router, this is a single IP connection, through which the computer connects to the network. ASC uses the connection request and sends the router the commands “get parameter values” and “set parameter values”.

As you can see, this is quite simple. A dual authentication mechanism is involved. CPE must ensure that it contacts your trusted ASC, and ASC only allows authorized users to session.

At DEFCON 22, we talked about what we found in this process. Our study found flaws in the implementation and configuration produced by many ACS servers of network providers (ISPs):

  • ACSs are the only point of "ownership" of the Internet in modern ISP infrastructure;
  • many implementations of TR-069 are simply not serious enough - we found vulnerabilities in several products that could lead to hijacking of user routers.

Here is an excerpt from the TR-069 specification regarding the Connection Request: “ACS can at any time request that the CPE initiate a connection to it using the connection request notification mechanism. Support for this mechanism is mandatory for CPE. ” In fact, this means that any TR-069 client all over the world is also a connection request server on the port that we request, and for all TR-069 it is the same port 7547, it is used by default.

Last year, the guys from Dead Map conducted a very interesting study, it was Zachary Rumrich and his friends from the University of Michigan. They scanned 2 million random addresses on each of nearly 10 thousand ports, and determined that CWMP 7547 - the default port for TR-069 - is the second most popular open port in the world. The percentage of its use of the Hit Rate for the Internet is 1.12%. Let me remind you that this protocol was created 10 years ago - think about it! Approximately 45 million devices use this port to connect to the Internet, and they can be “tapped” through this open port.

We will conduct a short review of the most popular ports in the world, which various devices use to communicate with the Internet.

Port 80 is used by approximately 70 million devices, half - for communication with web servers on the global Internet. Most of them are Apache servers, followed by nginx, IIS 8, and others. The second half of the devices uses the 80th port for the Internet of things. These devices are routers, webcams, VoIP phones and toasters.

Now, finally, people are beginning to realize that open access to these things is simply dangerous. Therefore, they are looking for as many devices as possible that use only port 80 for local area networks.

Please note that the open port 7547 is used not only by ordinary users, its use is provided for by the software of the servers of providers that interact with user devices. And all 45 million devices using the TR-069 are the Internet of Things.

All of these devices go online using the Connection Request.
We decided to investigate security issues and find the number of victims of this vulnerability. We needed to stop guessing and move on to the facts. Last month (November 2014), we scanned 7547 ports several times over the entire IPv4 address space with the help of our friends Rapid 7 and guys from the University of Michigan. As a result, after a simple “slash” (/) was entered through port 7547, 1.18% of all devices of the public Internet answered us, the total number of which was 46 093 733 pieces all over the world. That is, this is not one country in which these ports are open, it is 189 countries around the world.

If you recall, the specification for TR-069 requires this port to be open for ASC requests. So, last year there were another 0.06% of new devices that satisfy this requirement, and this is as much as 2.2 million! That is, the growth trend is evident.

We dealt with the problem of implementing TR-069 on the client side, and here are the statistics we received.

We got 5 main types of servers using a connection request. More than half came from a thing called RomPager. This is a built-in HTTP server from Allegro Software, a company based in Massachusetts, which is included in the firmware of most routers. It is optimized for the minimum requirements - binary codes, minimum memory - and was first introduced in 1996. There have been many versions since then, the last one is 5.4. We decided to focus on it and found that the client equipment uses four versions of RomPager.

As a result, we found that 98.04% of devices use the outdated version 4.07, 1.44% use version 4.51, 0.51% - 4.03, and 0.01% 4.34. As stated in the series, "after that I became suspicious." What explains the incredible popularity of the only version of RomPager?

We bought a new TP Link router, unpacked it and connected it to the network. It was installed RomPager 4.07. We thought that it might be an old router with outdated firmware and downloaded the new firmware of 2014 from the TP Link website, updated it, rebooted the router, and as a result got version 4.07 again.

Thus, we got an explanation of the popularity of the old version - it was the only latest version of the firmware located on the websites of router manufacturers.

Which one here has an unpacked new router? It can’t be, this is good luck!

The guy from the hall passes the box with the router, and the speaker puts it on the table.

I assure you, this person does not work for me! We will deal with this box later.

We decided to explore what constitutes RomPager version 4.07. It was released in 2002 and has appeared in many modern devices:

  • 2,249,187 devices used it through port 80;
  • 11,328,029 devices used it through port 7547.

The study involved 200 different device models of more than 50 different brands. So, more than 11 million routers use an extremely outdated version of the built-in server, released in 2002, for the modern Internet. This is an excellent candidate for researching equipment vulnerabilities.

A very important point is that the most popular versions of specifications are available on the Internet.

And now let Lior continue - he will talk about how we analyzed this firmware.

Good day! My name is Lior and I analyzed how RomPager version 4.07 works and found interesting results. We had proprietary firmware downloaded from the site of TP Link router manufacturer. It looked like a compressed data array, so we used Binwalk, a tool for finding files and executable code in a given binary image. It is used to crack the archive and extract the packed files.

So, inside the firmware, we found the Bootloader bootloader, the manufacturer's logo in the .gif image format and the main binary Main binary.

For a more objective analysis, I decided to download all the RomPager firmware version 4.07 that I could find on the Internet, and found that they all contained the ZyNOS (mipsb32) header - an operating system that uses network devices manufactured by ZyXEL. I could not understand why devices from different manufacturers had the same firmware. I began to study what ZyNOS represents.

This is the main operating system in real time - Real Time OS, RTOS, without any file system and special permissions, it consists of one large binary file designed to meet any requirements.

This system has the well-known vulnerability “rom-0” (CVE-2014-4019). As of May 2014, 1,219,985 devices worldwide had this vulnerability. This vulnerability allows an attacker to take control of a router by simply downloading a configuration file from it, without any authorization, directly from a web browser panel through port 80. As a result, an attacker simply receives a password and username. And this vulnerability affects 1.2 million devices worldwide.

Consider what the attack interface looks like through port 80. We received an authorization request, and since we did not know the username and password, we tried to enter through port 7547.

Here we received a message that the object was not found at this address for any path except the correct address.

I decided to find the correct address, that is, send the request in the right way. Before starting to understand the code, I decided to perform manual testing through HTTP headers, and unexpectedly I managed to crack the router by sending the user digest name with the digest authentication header overflowing with the value 'a' # 600.

This led me to the first vulnerability. To understand why this is happening, let's look at the structure of the RomPager code. Each line here consists of an HTTP header and its producer function to process that header.

Consider the function that is responsible for the username.

You can see what leads to this vulnerability - it is an insecure strcpy, a standard library function for copying a null-terminated string (including a null-terminator) to the buffer. But what actually leads to the possibility of hacking the router is the lack of symbols and the possibility of dynamic analysis. This is very difficult to understand.

I opened the router and began to look for JTAG. For those who don’t know what it is, I’ll explain: JTAG is an interface for debugging and checking firmware. But when I removed the case, I didn’t find any JTAG connectors there, but I found there something that looked like a U-ART serial port on a separate chip - a connector designed for communication with other digital devices.

I started soldering and connected it to the router itself, and then through the USB adapter I connected this U-ART to my computer. When I downloaded the router, I found a very good opportunity to debug the router. I will show you what I got after I hacked the router - this is a very nice crash dump with all the registers and stacks, indicating the reason for the emergency recording - “TLB overflow”. At the top is an EPS line of instructions for the MIPS microprocessor. As you can see, it was rewritten by me, and this means that I control the router. Further analysis of the crash dump made it possible to fully determine the cause of the vulnerability.
So, insecure strcpy allows you to completely rewrite the EPC, which consists of 584 hexadecimal bytes, conveniently located after the username, so hacking the router is very easy.

This is exploit # 1:

  • need to send a long username;
  • rewrite the instruction pointer indicating your shellcode - binary executable code for the main processor;
  • Get the ability to remotely control the router!

The difficulty of this method of hacking is as follows:

  • Each device and each version of the firmware has a different location of the Nature's ASLR address space, that is, it is not known which value of the instruction pointer needs to be rewritten.
  • If you know where the memory is located in a specific firmware file, then you can easily run the code necessary to crack the “victim” router;
  • a hacker has only one chance for a successful attack, because after a "crash" the router will receive a new IP address, since there is a dynamic allocation of IP addresses.

Theoretically, these problems can be circumvented if you try to inject shellcode through another router information leakage vulnerability. Let's see how it works.

Since I did not have the ability to fully debug, I used the debugging option for the "poor", using the primitive debugging capabilities built into RomPager itself through the serial port. This made it possible to patch the firmware before downloading it, which was convenient enough, but very tiring.

After several reboots, I found a hidden function of the ZyNOS operating system, which allowed me to get to the router's memory via the Internet. This is ZynOs Remote Debugger, or ZORDON. This function allowed you to create breakpoints, view and edit memory, read and rewrite registers online.

Let's consider vulnerability No. 2.

If you remember, there is no dynamic memory allocation, so each incoming HTTP request fills the previously allocated “request structure”.

RomPager 4.07 processes up to 3 concurrent requests (3 pre-allocated structures).

When sending 3 consecutive requests, one of them can rewrite the HTTP structure that we saw earlier. It also became possible due to the insecure strcpy, and we again took control of the EPC.

So what is exploit number 2?

  • Blind reading of the memory by replacing the HTTP header line.
  • Problem: this technique only works with port 80, which also has a rom-0 vulnerability.

Let's move on to vulnerability 3.

RomPager supports cookies. As you remember, there is no dynamic memory allocation, so there is a preliminary memory allocation for each array of cookies with a total of 10 pieces, each 40 bytes long. The names of cookies are constant - from C0, C1, C2 ... C9. The next slide shows what C0 looks like.

Let's see how RomPager “steals cookies”, that is, processes the cookie file. At the top of the slide, you can see that he first checks the name “cookies” with a capital letter at the beginning. If so, it converts the rest of the cookie name to an integer and uses it as an index for the cookie array.

Then he loads it, multiplying by s3, this will be the index for 40, and uses it at the strncpy destination.

Thus, vulnerability No. 3 is as follows.

Performing an arbitrary write to the memory relative to a fixed point in the internal structure of the RomPager control, we gain control over everything that the RomPager does. Moreover, an additional bonus is the ability to overflow a 32-bit integer value in order to negatively affect the infrastructure.

If we send any cookies instead of C0, C1, and so on, we get the following answer:

This method works on any model of routers of any brand to which we have legal access. Thus, exploit # 3 is as follows: using several magic cookies added to your request, you bypass any authentication and view the configuration interface as an administrator through any open port.
Let's see a demonstration of what was said - we have a video of our actions. We launch an Internet browser and enter the IP address of our router and an authorization request in the address bar. If you write in the line: 7547, then we will get the message shown earlier.

Therefore, we use “our” Chrome browser plug-in, that is, an exploit built in by us, its icon is located to the right of the browser address bar (the plugin is called “Take control of all these things!”).

Click on the icon to activate this plugin.

Literally a second passes, and before us opens the TP-Link router settings window. We went around the authorization process!

Then you can click on the plugin again to restore the legal mode.

I will show you again how it works: enter the IP address of the router in the line, refresh the page with the plug-in connected ... it didn’t work ... we try to update again - it's done! We went to the router settings page without any authentication request, the window for entering the username and password did not appear at all.

I repeat once again: we have now connected a new router from the manufacturer, right out of the box, released in 2014, and it is subject to the same vulnerability that was discovered and described more than 10 years ago.

Let's get back to our presentation. We placed on this slide a link to the site, which contains information on the essence of the problem described by us.

This map shows which countries are affected by this vulnerability. These are 189 countries around the world and over 52% of all IP addresses.

I know what you’re thinking right now: you should immediately stop using port 7547 on your devices to access the Internet. But even if you try to deactivate the CWMP function in the router settings, port 7547 will still remain open.

You can counter this as follows:

Consider how the process of manufacturing routers goes.

Allegro Soft delivers RomPager to the chipset manufacturer, which sends them to manufacturers of ASUS, D-Link, Huawei, TP-Link, ZTE routers. They adapt the software to various models, insert their logo into the firmware and sell ready-made routers to the end user of the equipment.

Consider how slowly security updates occur in such a production chain.

Allegro Soft delivers a fixed version of RomPager to the chip manufacturer, which should include it in the SDK and transfer it to the router manufacturer. After that, you need to recompile the firmware for each product line and each model, and if you also perform regular updates, this will become a real nightmare for the manufacturer.

It is in this case that we say: “Too many cooks do spoil the broth”, that is, “Seven nannies have a child without an eye” (literal translation from English: “Too many cooks spoil the broth”).
I note that most people never update the firmware of their routers. This is one of the reasons why such vulnerabilities have not been fixed for months or even years.

Consider the issue of cooperation with manufacturers. We contacted Allegro Soft and the most influential manufacturers of routers, provided them with a full description of the vulnerability, and proposed non-destructive "patches" of the operating system (POS) to fix it. Despite our broken English, most heeded our recommendations and released firmware updates based on our recommendations, for example, Huawei.

But Allegro Soft told us that “we cannot force manufacturers to upgrade to the latest version”, despite the fact that they presented the latest updated version in 2005! Think about the fact that if the 2005 update has not yet gone through this whole chain, then something is clearly out of order.

Now we will answer the frequently asked questions:

Is it true that RomPager is so bad?

- No, it is very convenient, reliable and safe if the “patch” is used, we just investigated the old vulnerable version of the firmware, which is still extremely popular all over the world.

Is it possible that this “back door” was intentionally left open?

“No, that's not like that.”

Can you share your exploit?

- No we can not!

Can you tell me which IP addresses are vulnerable for this reason in my country?

- Scan ports 80 and 7547 and the ISP TR-069 user ports used to request a connection, as different ports may use different ports by default.

Conclusion: we discovered a very serious vulnerability of IPv4, the fourth version of the IP protocol, which today is the main one and serves most of the Internet.

Router manufacturers, you must fix it!

Thank you for your attention, now we are ready to answer your questions.


- I saw a D-Link GSL 320 router in your list, it uses a Linux-based OS, can you say that it meets modern security requirements?


- We do not know the features of each device presented in our list, because we did not have access to each device from the list, and we did not set ourselves the task of checking every router that provides Internet connection, but we can talk about this router later if you provide us with detailed information.


- When you published the list of vulnerable equipment , you were advised to use the ZoneAlarm firewall as protection. Can you advise me how to protect my computer without installing a personal firewall?


- This is a completely different way than the one we talked about here, and maybe we will talk about it later.


- Have you tested cable modems, because at least in Germany we use modems provided by our providers, these are special models, exploits for which are very well known, and you can also easily take control over them? I am interested because I can’t refuse to use this particular ISP.


- We did not try to classify cable or DSL modems, but if your model is on the list of potentially vulnerable equipment, it will not be difficult for me to test it.

We cannot know which devices in which countries are vulnerable, so we publish our studies in the hope of putting pressure on manufacturers to fix these problems as quickly as possible. Therefore, we ask - if you find a vulnerability, report it to your provider, because this is a very important aspect of your security.


- Maybe you should try to use gSOAP to solve the problem?


- We never used this SDK, because this is a completely different area of ​​research, but anyone interested can use it for such a case.


- Is it possible to use your exploit in order to crack the router and change the firmware?


- Sure you may!


- Is it possible to get rid of the vulnerability by blocking the use of port 7547?


- If you manage to block this port in some magical way, then you will deprive your router of the important functions that the Internet provider provides, including obtaining IP, and you may lose access to the WAN altogether. However, if you use this port to obtain IP in the local area network LAN, and use another port to access the Internet, then the vulnerability can be avoided. And some Internet service providers provide this opportunity to their customers.


- Can I get rid of the vulnerability if, right after I pull the router out of the box and connect it to the network, ACS will assign it a different default port?


- Perhaps that’s why we advise the provider to do just that, in any case, this will protect you from attacks by hackers who scan the entire Internet in search of the ports open by default.


- Are the latest firmware versions free from this vulnerability?


- Yes, many manufacturers took this problem into account and provided protection from it.

Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending it to your friends, a 30% discount for Habr users on a unique analogue of entry-level servers that was invented by us for you: The whole truth about VPS (KVM) E5-2650 v4 (6 Cores) 10GB DDR4 240GB SSD 1Gbps from $ 20 or how to divide the server? (options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

Dell R730xd 2 times cheaper? Only we have 2 x Intel Dodeca-Core Xeon E5-2650v4 128GB DDR4 6x480GB SSD 1Gbps 100 TV from $ 249 in the Netherlands and the USA! Read about How to Build Infrastructure Bldg. class c using Dell R730xd E5-2650 v4 servers costing 9,000 euros for a penny?

Also popular now: