A little more about Xiaomi phones and the fight against them. Updated
Honestly, I had no plans to write and publish this article, but, after two months in a close circle of colleagues saw 5 pieces of freshly purchased phones from Xiaomi, and a recent article on Geektimes, advertising Xiaomi smart home control, to me conscience came and,
First, a small introduction for those who are not in the subject. There is such a company Xiaomi, which makes good phones for filling and pours in them customized Android. The business model, as it was recently officially announced, “In fact, we are distributing our smartphones without making money from it. We are more concerned with long-term sources of income. We could sell 10 billion smartphones and not earn a cent on them. ” A sourceone and two .
Having glanced at the September article on the Security lab and even this complaint, I personally had the feeling that the Xiaomi phone is something like a leash on which the owner is driven by Big Brother (I exaggerate, of course).
This was the main motive for conducting a study of the behavior of the Xiaomi redmi 3S phone
with MIUI Global 8.1 firmware. Stable 8.1.1.0 (MALMIDI)
Research of the experimental rabbit and detection of the problem
I take a brand new phone out of the box. I turn it on and go through the initial setup wizard, after enabling the recording of traffic on a Wi-Fi router. Exactly two seconds after the phone connected to the access point, the downloading of a file of about 8 MB in size from one of the Xiaomi servers began. It was a regular zip archive, inside of which lay a bunch of everything, including the AnalyticsCore.apk file mentioned in the article on SecurityLab.
Further more. In total, for the entire time of observation, I counted a little less than eight dozen server names in different domains. Immediately make a reservation that in this number there are no Google and Facebook servers, the applications of which are also preinstalled. Just because I counted them separately. With them, too, everything is "fun."
Most of the connections to Xiaomi servers went through HTTPS, so it was not possible to figure out the details of what exactly was transmitted directly. Disabling all kinds of logins, synchronizations, etc. this traffic did not disappear.
Additionally, it was embarrassing that for the most part the requests were small (the amount of received transmitted TCP session traffic did not exceed 1-2Kb), but, because Since our mobile operators round up the traffic volume (for example, Tele2 up to 150Kb), then, if the match fails, you can “pump” significant traffic volumes in this way, and when roaming, you suddenly get money.
Those whom this fact does not bother may not read further, because Further there will be a description of the specifics of isolating traffic from applications sewn into the factory firmware.
Prerequisites
The first thing you need is to root the phone. I will not describe how this is done in the case of Xiaomi, I am sending out those who wish to go this way to the full version of this article (link at the end).
The second is to pour the firmware into the phone through the cable and erase ALL user data.
Thirdly, the phone SHOULD NOT have Internet access after a gulf of fresh firmware.
Update Until the installation of the restrictions described below, of course.
Disclamier You do all further manipulations on the phone at your own peril and risk.
The responsibility for any result lies with who exactly did the actions described below.
Small technical introduction
The servers that the phone accesses are mostly located in the Amazon cloud, so they are accessed by names that resolve through round-robin DNS to different IP addresses from different / 16 subnets. Blocking them all on subnets makes no sense - you can filter out half of the Internet, which is not good. Blocking by name is good, but not the fact that host names from L3 domains are not dynamically generated. It would be ideal to beat all the applications that access the Xiaomi servers, but, as practice has shown, the depth of their integration in Android is such that after removing some of them, the phone may simply refuse to boot.
Further. External servers are accessed by more than one process, but many, while the task is complicated by the presence of Android UID sharing, when different processes (applications) can generate network traffic under the same UID. Moreover, one of the useful processes (responsible for GPS) must be released to the outside world in order to download small updates, but at the same time he was sitting under the same UID as eight pieces of processes rushing to Xiaomi servers.
It is also necessary to mention the limitations of the tools available to solve the above problems, because most of the applications with the name firewall available on the Play Market work through the so-called VPN, i.e. they do not protect against information leakage before launching the application.
Most of what will be discussed later for professional Android developers is a commonplace truth, but for everyone else this will help to understand why filtering is constructed in this way.
Unlike regular Linux, where there are configuration files and startup scripts that lie in / etc, in Android everything is done a little differently. General network management is provided by Connection Manager, which is pulled by the netd system daemon, which, in turn, calls iptables with certain command line parameters. Accordingly, it does not make much sense to call IPtables from the boot script (init and others) - netd will still call iptables at startup, clear the rules and fill in its own.
The only way left by Google is to write the necessary iptables configuration commands in the script /system/bin/oem-iptables-init.sh. The path to this script and its name are hardcoded inside the source code of the netd daemon.
To filter static hostnames, you can edit the / etc / hosts file, but keep in mind their number and the possibility of their dynamic generation.
Then there will be a story of how all this was done.
Removing and freezing (if not sure) unnecessary programs
Using the free version of Titanium Backup, you can see the correspondence between the name of the program displayed on the system (Play Market), its code name (com.google.vending) and, if necessary, delete what is clearly not needed.
The disadvantage of the free version is that it does not know how to freeze programs, therefore we do freezing through the ADB shell using the package manager. Example:
root@land:/ # pm disable com.miui.analytics
pm disable com.miui.analytics
Package com.miui.analytics new state: disabled
root@land:/ # pm disable com.miui.systemAdSolution
pm disable com.miui.systemAdSolution
Package com.miui.systemAdSolution new state: disabled
root@land:/ # reboot
rebootNetwork Request Filtering
Disclamier 2. This article describes HOW you can filter the “left” network activity of the phone. What exactly to filter - everyone is free to decide for himself.
How can this be done.
1. The simplest is to fill in the / etc / hosts file with server name records with the IP address 127.0.0.1. My set of servers lies on Google Drive in the Files folder.
The disadvantage of this option is the impossibility of blocking unknown and dynamically generated host names and L3 / L4 domains.
Update Several times found strange behavior Netfilter / IPtables. After loading the phone, not all the rules listed in the script turned out to be in the table of current rules. If you restart the phone again - all the rules were in place.
2. we write the filtering packet sending commands on the / 16 and / 24 subnets using the standard Netfilter / IPtables rules in the oem-iptables-init.sh file. I don’t describe them here, those who wish to write them themselves, or find them in the full version of the article.
Update The disadvantage is that most servers are located in the Amazon cloud and have variable (round-robin DNS) IP addresses. To ensure their filtering, you will have to close more than a dozen subnets / 16, which is not good. You can inadvertently ban useful sites. But for static hosts (if there are not many), this solution is quite suitable.
3. We filter DNS queries to unnecessary domains. This is somewhat more complicated, so I will describe in more detail.
Updated.As part of IPtables, which is regularly running in Android, there are functional expansion modules, which we will use further. Remembering that DNS requests are sent by the system (UID 0), we write the rule:
$IPTABLES -A oem_out --protocol udp --dport 53 -m owner --uid-owner 0 -m string --algo bm --hex-string '|04|miui|03|com|00|' -m comment --comment "Deny UID 0 DNS queries for miui.com domain" -j DROP
#
$IPTABLES -A oem_out --protocol udp --dport 53 -m owner --uid-owner 0 -j ACCEPTUpdated. The first line will filter out all UDP packets sent by the system (UID 0) to the 53 UDP port of any IP address and containing bytes 046d69756903636f6d00 (queries to the DNS server containing .miui.com). IPtables independently convert the string | 04 | miui | 03 | com | 00 | in a purely hexadecimal form 046d69756903636f6d00.
The presence of a hexadecimal digit in the first position for the --hex-string parameter is required, otherwise IPtables will not accept the command. Separators in a domain name are converted into hexadecimal digits when generating a DNS query, indicating the number of bytes until the next separator. Therefore, the last byte is zero (00h).
The second line will skip all other DNS queries. I pointed out the comments for convenience, so that the iptables -L -v command displays the results of the locks more clearly.
4. Для работы Assited GPS необходимо дать возможность доступа к серверам QualComm процессу с UID 1000. Здесь всё сложнее, т.к. простая фильтрация пакетов по содержимому, как в случае DNS серверов, не сработает — начальные пакеты установления TCP соединения c флагами SYN, ACK ещё НЕ содержат в себе имя хоста, которое обязательно присутствует в HTTP запросе, а пакеты идущие после заголовка HTTP запроса уже могут не содержать в себе имя хоста. В результате фильтр пропустит из всей TCP сессии только часть пакетов, что равносильно её запрету или обрыву.
Поэтому рисуем вот такой костыль для фильтрации запросов седьмого уровня средствами 3-4 уровня:
# разрешаем инициировать установление TCP соединений на 80 порт всем процессам работающим под UID 1000.
$IPTABLES -A oem_out -m owner --uid-owner 1000 --protocol tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT
#проверяем наличие слова xtrapath в пакетах TCP соединений установленных на 80 порт процессами с UID 1000 и помечаем эти соединения шестнадцатиричным числом 5555.
$IPTABLES -A oem_out -m owner --uid-owner 1000 --protocol tcp --dport 80 -m conntrack --ctstate ESTABLISHED -m string --algo bm --string 'xtrapath' -j CONNMARK --set-xmark 0x5555
# убиваем пакеты всех установленных процессами с UID 1000 TCP соединений не имеющих нашей пометки число 5555
$IPTABLES -A oem_out -m owner --uid-owner 1000 --protocol tcp --dport 80 -m conntrack --ctstate ESTABLISHED -m connmark ! --mark 0x5555 -j DROP
5. Filter the Internet access by application (I had Google Chrome UID 10060). Allow access to the Internet Google Chrome and prohibit all other applications.
$IPTABLES -A oem_out -m owner --uid-owner 10060 -m comment --comment "Permit Google Chrome internet access" -j ACCEPT
#
# Block all other processes
#
$IPTABLES -A oem_out -m owner --uid-owner 0-9999 -m comment --comment "Block all other system processes internet access" -j DROP
$IPTABLES -A oem_out -m owner --uid-owner 10000-99999 -m comment --comment "Block all other user processes internet access" -j DROP
The weak point of this filtering method is its reliance on the presence of a UID mark on each specific packet when passing it through Netfilter / IPtables. This was discovered through incomprehensible TCP connections to Google servers whose packages did not contain UIDs. Research has shown that these packages are triggered by the Google Captive portal login process. I solved this problem in a workaround - just turning off these requests with commands in the ADB shell:
root@land:/ # settings put global captive_portal_detection_enabled 0
root@land:/ # reboot
Update Further observation showed how wrong I was. There are such "silent" processes, but some of them innocently communicate with each other through the address 127.0.0.1, which is not punishable.
Everything else must be banned.
Therefore, for their proper filtering, you need to add two more lines to the very beginning of the script:
$IPTABLES -A oem_out --protocol all --source 127.0.0.0/8 --destination 127.0.0.0/8 -m comment --comment "Accept internal traffic" --jump ACCEPT
$IPTABLES -A oem_out --protocol all -m owner ! --uid-owner 0-99999 -m comment --comment "Drop any traffic which does not have UID." --jump DROP
Update Already after the publication of the article, it finally became clear to me that the UID of applications set by the system when installing the application, when updating and / or reinstalling applications, can change in unpredictable ways. Consequently, the Internet access for the application will also fall off and it will be necessary to rewrite the Netfilter / IPtables rule again.
To solve this problem, I sketched a small piece of the script that reads the names of applications from the tail of the file, checks for their presence in the system database of the applications, and, if available, takes the application UID from there and dynamically (in the process of the script) forms the Netfilter / IPtables rule.
Strictly speaking - reading parameters from the script body is also a crutch. But it justifies me that at the time of the script, it is impossible to mount the file system accessible from the outside during normal phone operation. There are no corresponding device files in the / dev folder. I admit that this may be a feature of the firmware of a particular phone.
The text was specially made as detailed as possible for a better understanding.
# Permit intenet access for the packages listed at the end of this file. White list mode.
#
SU=`/system/bin/which su`
# changing reading file behavior (read whole file with \r\n into variable)
IFS=""
# reading first and second fields of every line of the system packages database into variable PACKAGESDB.
# Escalating privileges via su because of filesystem packages database file access limitations.
PACKAGESDB=`$SU -c "/system/bin/cut -d' ' -f 1,2 /data/system/packages.list"`
#
# Reading last lines of current script form the end till "exit 0" line
# Filtering empty lines, lines started with # and all symbols after # (comments) in every line.
#
# 's/#.*//' - remove all in every line after #
# '/^#/d' - remove lines staring with #
# '/./!d' - remove empty lines
# '/exit 0/,$ d' - remove all lines starting line with "exit 0"
# 's/ //g' - remove spaces from line
#
/system/bin/tac $0 | /system/bin/sed -e '/^#/d' -e 's/#.*//' -e '/exit 0/,$ d' -e '/./!d' -e 's/ //g'| while read line;
do
# Just in case 8-)
OUR_PACKAGE_NAME=$line
# Strict checking for existence of our package name in the system packages database. Checking first field.
PACKAGE_NAME_IN_DB=`echo $PACKAGESDB | /system/bin/cut -f 1 -d' ' | /system/bin/grep -Fx "$line"`
if
# Checking grep utility exit code. "0" means pattern found
test "$?" == "0"
then
#
# Looking for package UID in database. Checking second field. VERY important space after $line!!!
#
PACKAGE_UID=`echo $PACKAGESDB | /system/bin/grep "$line " | /system/bin/cut -f 2 -d' '`
else
# All other exit codes return us to the beginning of the cycle.
# echo "Package $OUR_PACKAGE_NAME not found"
$IPTABLES -A $CHAIN -m comment --comment "Package name $OUR_PACKAGE_NAME not found. Check package name." --jump LOG
continue
fi
#
# Set the package right for Internet access
#
$IPTABLES -A $CHAIN -m owner --uid-owner $PACKAGE_UID -m comment --comment "Permit $OUR_PACKAGE_NAME Internet access" -j ACCEPT
#
done
######
... skipped...
####
exit 0
####
####### Do NOT edit before this line #########
# Please add package names and comments after this line for granting them internet access.
#####
# Google Play Store and its companion processes
#
com.google.android.gms # Google Services Framework Internet access
com.android.vending # Google Play Market internet access
com.android.providers.downloads # Download manager service internet access
#
# Other Google apps
com.google.android.youtube # Youtube application internet access
com.google.android.apps.maps # Google Maps application internet access
com.google.android.googlequicksearchbox # Google Assistant internet access
#
#
com.android.chrome # Google Chrome browser internet access
After reinstalling / updating the application, you just need to restart the phone.
6. For the purpose of monitoring the operation of Netfilter / IPtables rules, you can add another line like this:
$IPTABLES -A oem_out --source 10.1.30.42 --protocol tcp --jump LOG --log-prefix "IPtables log:" --log-uidThe IP address of the sender parameter (--source 10.1.30.42) can be omitted, but in this case the log will be inundated with records of network activity of processes wrapped to the address 127.0.0.1 by the hosts file. The log can be read through the dmesg command (dmesg | grep IPtables) in the ADB Shell.
I posted the version of the article, which was written as a complete guide to solving this problem with Xioami Redmi 3S, on Google Drive . I did not dare to spread it here precisely because of the volume.
PS I'm not a developer of Android applications, just life made me deal with a saber phone for two months. Therefore, gentlemen, pros, if I'm wrong where - correct. I will be grateful.
PPSAs a means of interception, Zyxel Keenetic Extra was used. He has the ability to intercept Wi-Fi traffic and merge it onto a USB flash drive for later analysis.