Android accessibility - a wolf in sheep's clothing? Yandex lecture
A month ago, at a regular Droid Party, senior developer Danila Fetisov described in detail the principle of the service, which is responsible for the accessibility function of Android. You will learn how to use it to improve the availability of your projects, as well as a dangerous vulnerability called clickjacking.
- My name is Danila Fetisov, I am from the Moscow office of Yandex, more specifically, from Taxi, more specifically, from Taximeter. Today we will talk about what Android accessibility is and why I decided to call such a holy thing for people with disabilities a wolf in sheep's clothing.
So, plan a report. First we will see how this thing can be applied, created and customized. Maybe I will tell a couple of life hacking, grocery nishtyakov. And then about the most interesting - about why this thing is dangerous.
How can we apply this at all? In general, what is it for those who do not know? In the system there are any events, they fall into our service, and we process them.
Once we have something falling, we can take content from there and voice it using the same Yandex SpeechKit, Google Text-to-Speech or something else you like. But what is most interesting, we can not just get, but throw the event of the system. You can still say something. I threw an event, pressed a button, and everything is just fine, you enjoy life.
If we talk about these two points, they, in fact, are defaulted. Google launched this feature so that people with disabilities can use your application. But as everyone knows, we, the developers, do not want to be bored: we sit down, look at the code and think: how can we apply? And off we go.
Automation, testing using imitation of user actions, contextual advertising. You understand: since we get the content from the screen, we can understand how to use this user.
What else is interesting? Processing USSD requests. Unfortunately, the normal API appeared only from the 26th SDK, and developers somehow need to survive. And you imagine, they seriously parse the USSD requests, parse the UI using the AccessibilityService.
The sixth point is the dot. Why? Because after this report, you will develop an understanding of how to apply it and for what. And then you literally open your fantasy stream, take a cup of tea, coffee, and drive off to code your cool features.
Where to begin? The next slide will be painfully trivial.
We take the service, inherit from AccessibilityService and enjoy life.
Then we drop this service in Manifest, and what is so interesting about it? In fact, the usual service record. But pay attention to the label field. Now I will not tell what it is. Remember, it will be a little later.
Further on Manifest we have just a file of configs, nothing interesting either.
And the configs file itself looks a bit more terrible. From what you need, most likely, 85–90 percent is a filter setup field for inbound events.
Let's say it’s worth that we want to receive absolutely all events. You can only filter on clicks, on content changes, or on something else.
Next - filter on application packages. What is it for? You yourself are well aware that receiving events from absolutely all applications in the system would be too much for one application. So Google decided to limit it somehow. Added a filter.
Further description. There was a label field, and now use description. In fact, they create a pair, which then tells the user what to include your service for.
And the last point: you need or do not need content from the current open window. If you just want to understand that an event has occurred, put false and do not torment the user once again.
Enable service. Unfortunately, in order to turn on this piece, you will have to torture a half user and get him to go through seven circles of settings hell, relatively speaking.
First, we open special opportunities for him. The steep AccessibilityService is exactly the same label from manifest that I told you about in the previous slides.
Farther. The user needs to select your service, and then he starts reading the description, why do you want to enable this service and get some control over the device.
OK, includes. But it was not there. Some terrible dialogue comes out, which says: “Dude, if you turn it on now, we will take complete control over your application, conditionally speaking, everything will be scary”. Users do not always read it all, just click "OK", so that we are behind them. So we live.
Well, so, where is the fact handling of events in fact we get? The next slide is a bit smarter than the first slide about the service, but nonetheless.
Here is our DummyAccessibilityService, and we are throwing two methods there in which we will write everything. In fact, interrupt is a method in which you have to clean your references, stop subscriptions or anything else.
The most interesting thing is the AccessibilityEvent class, which falls to us. What can we get out of it?
For a start, we can understand what actually happened in your system. Take the EventType, and what's getting here?
And we get that we have some kind of Activity opened there, or Dialog, or Popup, or even a notification has collapsed. Right there everything falls on window_state_changed.
Further we understand that the content of some view has changed. They moved it, threw some line in the text view or something like that.
And then we will know when the user clicks, or selects any view. In my opinion, 85 and even 98, probably, you will cover the percent of the main product tasks with these event type.
But if you do not have enough of them - please, you can go to the documentation, I will show you a link at the end of the report, and, of course, you download the report, you can go. It's just a reading for the whole evening, there is a huge list. Seriously, you can't master it at once.
Now the question is: how to find the view? We found out what happened. And now let's understand with which view, with which component this happened.
We take our AccessibilityEvent.
We take from him the Source, and, voila, the meta information on the view is already with us.
But what to do if you are a hardcore developer, and you lack information about one view, do you want to understand what is happening on the screen now? There is a restriction on the SDK, but it is minor.
We take our AccessibilityService, and we get, in fact, a reference to the root element in the entire hierarchy on the current screen. And then we get our meta-information on the view.
This, of course, is cool, but now you should have a question in your head: well, we got a link to the root element. What should we do next?
If you want to go in a safe way, then please, the first option. Take the ID view, and just find one method.
If it seems to you that this is somehow boring, the hunt for something more fun, then we take the text from the view, we find the text.
If this is not interesting enough for you - well, okay, guys, take Child and Recursion, walk on it. Then you will understand everything that happens.
Fine. We understood how to find the view. How to interact with her now?
Again we take our AccessibilityNodeInfo, and now we throw Action into it. What action can we throw there?
For starters, we can throw a Click. Why in the end I put here not after just Click, but also Click and Select? Yes, because there is a lot of corner case, when the usual Click does not save you. Suppose a user chooses a text, and clicking on the view will cleanly remove the selection from this text. Or, on some devices, in principle, the view is not clickable until you call it to Select. In general, a lot of problems. And if you just want to steam, make a Select, click. Two Action, and everything, happiness is.
What's next with us? And then we have the text installation. Here, too, everything is simple. We take Action_set_text, we create Bundle with our line, and we live.
And the third point is go to the documentation. Again, this is the second evening you will have. A bunch of action games, in my opinion, even more than events.
Hooray! The most boring part of the report is over. Now is the time any nishtyachkov.
Suppose you created a service, set it up, threw some event handling. Now you need to understand how to force the user, how to ask him to still enable this service. To understand this, you need to know if service is currently enabled or not.
Everything is simple here. Take AccessibilityManager and ask him for all currently enabled services in the system, and there we find ours.
But everything would be great, of course, if AccessibilityManager was a normal guy, and there was no such thing that you ask him: “Dude, please give me all the available services,” and he says: “Sorry, today I’m not in a mood. Here's an empty list for you. ” And you sit and think like that: “Damn, you communicated normally”.
Well, well, there are good old friends - these are Settings.Secure.getInt and getString. First we ask if the services are included in the system at all. If included, everything is thrown into one line, and there we are looking for some of ours through some content.
And here we meet with the fact that we did everything correctly. We created the configuration, created the construction of any events, included the service. We see for sure that the service has been turned on, and events do not arrive to us. And so that something arrives, it does not arrive. At that moment I thought that in my life some kind of black stripe had arrived. Right there were no options.
I thought and thought, I edited and edited, and then, bingo, I realized that this is Xiaomi with their MIUI. Here are just guys, pain! This is serious.
Okay, understand what the problem is. How to solve it now? Unfortunately, on some firmware in MIUI everything is done in such a way that if your application is not in autostart, then the system will not even start your service in principle.
But here is one solution - ask the user to add your application to the autostart, and we live on, enjoy life.
And then came the cherry on the cake. Why is this service, so kind for people with disabilities, a wolf in sheep's clothing?
It seems to me that many of you have already guessed that if a user turns on this service, then we can easily steal a lot of confidential information from his device. Not only can we steal it, we can throw it somewhere, say, unlock the screen if it is locked with a pin-code. Well, ok, unlock the screen. What's next? We call on our advertisements, we will earn a lot of money, we will directly live. And, in the end, we can, in principle, simulate absolutely any actions of the user, and, say, put some application, give her admin rights, and tell the user: “Let's see you later. Say goodbye to your device. ”
OK, we can do all this, but now you will say: “You, of course, well done. He told us how we can steal what data. But how do you force the user to turn on your AccessibilityService, and if there is still some kind of left application? ”. Ok, the user, it seems to me, will simply open the settings, understand that some kind of nonsense is written there, close it, delete the application and forget it. And you know, you will be right that the first way to turn on through the system settings goes to "no", because, you know, this is some kind of nonsense. And here comes to our aid vulnerability, which, in fact, is called clickjacking. Guys, who of you is familiar with clickjacking? Great, it will be interesting.
So, on the basis of this vulnerability, some guys - I attached a small link, because all the following screens will be from the resource of these guys - they came up with an attack called Cloak and Dagger. In fact, with just two permission and minimal user interaction, you get access to the entire application directly from the word “absolutely”.
But ok, what do we need to do to still hack the user?
We cast our app on Google Play. We hope that the Android user has a version less than 8.
You know, a little bit - 95% at the moment. And that's all, the user downloads our application.
In short, this malware tells the user how to become a good guy, shows at the very end vidyushechku. Now see for yourself.
OK, let's start. A small text. We are told: "If you start the tutorial now, you will become a good guy." Okay, of course, we begin.
Some other text. Then people will be presented in the form of green men. Well, okay, let's click Next.
Another text. Okay. Now I will finish reading it and will definitely see Vidyushka, where I will be told how to become a good guy.
I click "OK", and there the vidyushka starts. It is not so important to us. It is important what actually happened on the device at that moment. The user starts the application, and since we put it from Google Play, we are automatically granted permission to overlay. Voila, we open a window of special features, and the top is just overlay. The user clicks on the Start Tutorial, and in fact selects our AccessibilityService. Well, you understand what went on.
Next we click Next, select the toggle switch.
At the very end, we turn on the AccessibilityService, and the user does not even realize that this has happened.
Do you know what the problem is? Overlay arranged in such a way that they either completely absorb all the events, all the touch and so far away, or they completely miss. And the trick of this particular attack is that we fill overlay, the entire screen except for one space. In this case, this is the OK button. This button is really from the dialogue. When we click on it, we get an event that calls for a touch outside, and everything is fine.
How does Google respond to all these problems? You understand that this is a rather cruel vulnerability.
It all started quite interesting - all with eight years of silence. Google released this little thing, said: “Guys, use. It's safe. Just saying. ” OK. Google threw. The same guys who came up with Cloak and Dagger, threw both support, and issue trackers, and bug trackers.
Then Google said, “Okay, okay, guys, take it easy. I'll close clickjacking in oreo. ” Well, closed. They are sitting still, thinking: “Well, damn it, guys, still, if the attackers force the user to turn on the accessibility, it will be bad. Let's make fun of the developers a little.
And they send a letter: “Do this, I don’t know what, otherwise your application will be deleted from Google Play”. In short, the letter contained the following: “Guys, tell us and the users why you need AccessibiiltyService”. How to tell? In what format? Where? There is no understanding at all. And in the letter, of course, it is not specified.
OK. Huge threads on reddit, a bunch of letters to support. And then some developer puts a letter on reddit, and it says: “Yes, there, guys, nothing complicated. Put in the description that I showed you in the manifest, drop it in the description on Google Play, and everything will be fine, I will leave you alone. ”
At that moment I thought: well, ok, I found this letter in some corridors of Reddit. And what if I'm a young developer, just met with the AccessibilityService and I want to release the application with this? From where should I find out that I need to perform such actions, I need to throw in the description, on Google Play, etc.? I thought, I thought, I searched, I searched and, you know, guys, I did not find anything, from the word at all.
Well, seriously, there is nothing. There is a simple description in the main documentation: “Guys, this is purely for people with disabilities and that’s it. More can not be used. And in fact, it turns out, you can.
Well, we have come to an end. What we could talk with you? We talked about how it can be used, understood how to create and customize it, understood how to handle UI events. And, of course, understood how you can, but do not need to interact with AccessibilityService. Of course, I do not recommend you to do so, it was just for information. Here are the promised sources:
- developer.android.com/guide/topics/ui/accessibility/services
- developer.android.com/reference/android/view/accessibility/AccessibilityEvent
-developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo
- cloak-and-dagger.org
Guys, thank you very much for your attention!
- My name is Danila Fetisov, I am from the Moscow office of Yandex, more specifically, from Taxi, more specifically, from Taximeter. Today we will talk about what Android accessibility is and why I decided to call such a holy thing for people with disabilities a wolf in sheep's clothing.
So, plan a report. First we will see how this thing can be applied, created and customized. Maybe I will tell a couple of life hacking, grocery nishtyakov. And then about the most interesting - about why this thing is dangerous.
How can we apply this at all? In general, what is it for those who do not know? In the system there are any events, they fall into our service, and we process them.
Once we have something falling, we can take content from there and voice it using the same Yandex SpeechKit, Google Text-to-Speech or something else you like. But what is most interesting, we can not just get, but throw the event of the system. You can still say something. I threw an event, pressed a button, and everything is just fine, you enjoy life.
If we talk about these two points, they, in fact, are defaulted. Google launched this feature so that people with disabilities can use your application. But as everyone knows, we, the developers, do not want to be bored: we sit down, look at the code and think: how can we apply? And off we go.
Automation, testing using imitation of user actions, contextual advertising. You understand: since we get the content from the screen, we can understand how to use this user.
What else is interesting? Processing USSD requests. Unfortunately, the normal API appeared only from the 26th SDK, and developers somehow need to survive. And you imagine, they seriously parse the USSD requests, parse the UI using the AccessibilityService.
The sixth point is the dot. Why? Because after this report, you will develop an understanding of how to apply it and for what. And then you literally open your fantasy stream, take a cup of tea, coffee, and drive off to code your cool features.
Where to begin? The next slide will be painfully trivial.
We take the service, inherit from AccessibilityService and enjoy life.
Then we drop this service in Manifest, and what is so interesting about it? In fact, the usual service record. But pay attention to the label field. Now I will not tell what it is. Remember, it will be a little later.
Further on Manifest we have just a file of configs, nothing interesting either.
And the configs file itself looks a bit more terrible. From what you need, most likely, 85–90 percent is a filter setup field for inbound events.
Let's say it’s worth that we want to receive absolutely all events. You can only filter on clicks, on content changes, or on something else.
Next - filter on application packages. What is it for? You yourself are well aware that receiving events from absolutely all applications in the system would be too much for one application. So Google decided to limit it somehow. Added a filter.
Further description. There was a label field, and now use description. In fact, they create a pair, which then tells the user what to include your service for.
And the last point: you need or do not need content from the current open window. If you just want to understand that an event has occurred, put false and do not torment the user once again.
Enable service. Unfortunately, in order to turn on this piece, you will have to torture a half user and get him to go through seven circles of settings hell, relatively speaking.
First, we open special opportunities for him. The steep AccessibilityService is exactly the same label from manifest that I told you about in the previous slides.
Farther. The user needs to select your service, and then he starts reading the description, why do you want to enable this service and get some control over the device.
OK, includes. But it was not there. Some terrible dialogue comes out, which says: “Dude, if you turn it on now, we will take complete control over your application, conditionally speaking, everything will be scary”. Users do not always read it all, just click "OK", so that we are behind them. So we live.
Well, so, where is the fact handling of events in fact we get? The next slide is a bit smarter than the first slide about the service, but nonetheless.
Here is our DummyAccessibilityService, and we are throwing two methods there in which we will write everything. In fact, interrupt is a method in which you have to clean your references, stop subscriptions or anything else.
The most interesting thing is the AccessibilityEvent class, which falls to us. What can we get out of it?
For a start, we can understand what actually happened in your system. Take the EventType, and what's getting here?
And we get that we have some kind of Activity opened there, or Dialog, or Popup, or even a notification has collapsed. Right there everything falls on window_state_changed.
Further we understand that the content of some view has changed. They moved it, threw some line in the text view or something like that.
And then we will know when the user clicks, or selects any view. In my opinion, 85 and even 98, probably, you will cover the percent of the main product tasks with these event type.
But if you do not have enough of them - please, you can go to the documentation, I will show you a link at the end of the report, and, of course, you download the report, you can go. It's just a reading for the whole evening, there is a huge list. Seriously, you can't master it at once.
Now the question is: how to find the view? We found out what happened. And now let's understand with which view, with which component this happened.
We take our AccessibilityEvent.
We take from him the Source, and, voila, the meta information on the view is already with us.
But what to do if you are a hardcore developer, and you lack information about one view, do you want to understand what is happening on the screen now? There is a restriction on the SDK, but it is minor.
We take our AccessibilityService, and we get, in fact, a reference to the root element in the entire hierarchy on the current screen. And then we get our meta-information on the view.
This, of course, is cool, but now you should have a question in your head: well, we got a link to the root element. What should we do next?
If you want to go in a safe way, then please, the first option. Take the ID view, and just find one method.
If it seems to you that this is somehow boring, the hunt for something more fun, then we take the text from the view, we find the text.
If this is not interesting enough for you - well, okay, guys, take Child and Recursion, walk on it. Then you will understand everything that happens.
Fine. We understood how to find the view. How to interact with her now?
Again we take our AccessibilityNodeInfo, and now we throw Action into it. What action can we throw there?
For starters, we can throw a Click. Why in the end I put here not after just Click, but also Click and Select? Yes, because there is a lot of corner case, when the usual Click does not save you. Suppose a user chooses a text, and clicking on the view will cleanly remove the selection from this text. Or, on some devices, in principle, the view is not clickable until you call it to Select. In general, a lot of problems. And if you just want to steam, make a Select, click. Two Action, and everything, happiness is.
What's next with us? And then we have the text installation. Here, too, everything is simple. We take Action_set_text, we create Bundle with our line, and we live.
And the third point is go to the documentation. Again, this is the second evening you will have. A bunch of action games, in my opinion, even more than events.
Hooray! The most boring part of the report is over. Now is the time any nishtyachkov.
Suppose you created a service, set it up, threw some event handling. Now you need to understand how to force the user, how to ask him to still enable this service. To understand this, you need to know if service is currently enabled or not.
Everything is simple here. Take AccessibilityManager and ask him for all currently enabled services in the system, and there we find ours.
But everything would be great, of course, if AccessibilityManager was a normal guy, and there was no such thing that you ask him: “Dude, please give me all the available services,” and he says: “Sorry, today I’m not in a mood. Here's an empty list for you. ” And you sit and think like that: “Damn, you communicated normally”.
Well, well, there are good old friends - these are Settings.Secure.getInt and getString. First we ask if the services are included in the system at all. If included, everything is thrown into one line, and there we are looking for some of ours through some content.
And here we meet with the fact that we did everything correctly. We created the configuration, created the construction of any events, included the service. We see for sure that the service has been turned on, and events do not arrive to us. And so that something arrives, it does not arrive. At that moment I thought that in my life some kind of black stripe had arrived. Right there were no options.
I thought and thought, I edited and edited, and then, bingo, I realized that this is Xiaomi with their MIUI. Here are just guys, pain! This is serious.
Okay, understand what the problem is. How to solve it now? Unfortunately, on some firmware in MIUI everything is done in such a way that if your application is not in autostart, then the system will not even start your service in principle.
But here is one solution - ask the user to add your application to the autostart, and we live on, enjoy life.
And then came the cherry on the cake. Why is this service, so kind for people with disabilities, a wolf in sheep's clothing?
It seems to me that many of you have already guessed that if a user turns on this service, then we can easily steal a lot of confidential information from his device. Not only can we steal it, we can throw it somewhere, say, unlock the screen if it is locked with a pin-code. Well, ok, unlock the screen. What's next? We call on our advertisements, we will earn a lot of money, we will directly live. And, in the end, we can, in principle, simulate absolutely any actions of the user, and, say, put some application, give her admin rights, and tell the user: “Let's see you later. Say goodbye to your device. ”
OK, we can do all this, but now you will say: “You, of course, well done. He told us how we can steal what data. But how do you force the user to turn on your AccessibilityService, and if there is still some kind of left application? ”. Ok, the user, it seems to me, will simply open the settings, understand that some kind of nonsense is written there, close it, delete the application and forget it. And you know, you will be right that the first way to turn on through the system settings goes to "no", because, you know, this is some kind of nonsense. And here comes to our aid vulnerability, which, in fact, is called clickjacking. Guys, who of you is familiar with clickjacking? Great, it will be interesting.
So, on the basis of this vulnerability, some guys - I attached a small link, because all the following screens will be from the resource of these guys - they came up with an attack called Cloak and Dagger. In fact, with just two permission and minimal user interaction, you get access to the entire application directly from the word “absolutely”.
But ok, what do we need to do to still hack the user?
We cast our app on Google Play. We hope that the Android user has a version less than 8.
You know, a little bit - 95% at the moment. And that's all, the user downloads our application.
In short, this malware tells the user how to become a good guy, shows at the very end vidyushechku. Now see for yourself.
OK, let's start. A small text. We are told: "If you start the tutorial now, you will become a good guy." Okay, of course, we begin.
Some other text. Then people will be presented in the form of green men. Well, okay, let's click Next.
Another text. Okay. Now I will finish reading it and will definitely see Vidyushka, where I will be told how to become a good guy.
I click "OK", and there the vidyushka starts. It is not so important to us. It is important what actually happened on the device at that moment. The user starts the application, and since we put it from Google Play, we are automatically granted permission to overlay. Voila, we open a window of special features, and the top is just overlay. The user clicks on the Start Tutorial, and in fact selects our AccessibilityService. Well, you understand what went on.
Next we click Next, select the toggle switch.
At the very end, we turn on the AccessibilityService, and the user does not even realize that this has happened.
Do you know what the problem is? Overlay arranged in such a way that they either completely absorb all the events, all the touch and so far away, or they completely miss. And the trick of this particular attack is that we fill overlay, the entire screen except for one space. In this case, this is the OK button. This button is really from the dialogue. When we click on it, we get an event that calls for a touch outside, and everything is fine.
How does Google respond to all these problems? You understand that this is a rather cruel vulnerability.
It all started quite interesting - all with eight years of silence. Google released this little thing, said: “Guys, use. It's safe. Just saying. ” OK. Google threw. The same guys who came up with Cloak and Dagger, threw both support, and issue trackers, and bug trackers.
Then Google said, “Okay, okay, guys, take it easy. I'll close clickjacking in oreo. ” Well, closed. They are sitting still, thinking: “Well, damn it, guys, still, if the attackers force the user to turn on the accessibility, it will be bad. Let's make fun of the developers a little.
And they send a letter: “Do this, I don’t know what, otherwise your application will be deleted from Google Play”. In short, the letter contained the following: “Guys, tell us and the users why you need AccessibiiltyService”. How to tell? In what format? Where? There is no understanding at all. And in the letter, of course, it is not specified.
OK. Huge threads on reddit, a bunch of letters to support. And then some developer puts a letter on reddit, and it says: “Yes, there, guys, nothing complicated. Put in the description that I showed you in the manifest, drop it in the description on Google Play, and everything will be fine, I will leave you alone. ”
At that moment I thought: well, ok, I found this letter in some corridors of Reddit. And what if I'm a young developer, just met with the AccessibilityService and I want to release the application with this? From where should I find out that I need to perform such actions, I need to throw in the description, on Google Play, etc.? I thought, I thought, I searched, I searched and, you know, guys, I did not find anything, from the word at all.
Well, seriously, there is nothing. There is a simple description in the main documentation: “Guys, this is purely for people with disabilities and that’s it. More can not be used. And in fact, it turns out, you can.
Well, we have come to an end. What we could talk with you? We talked about how it can be used, understood how to create and customize it, understood how to handle UI events. And, of course, understood how you can, but do not need to interact with AccessibilityService. Of course, I do not recommend you to do so, it was just for information. Here are the promised sources:
- developer.android.com/guide/topics/ui/accessibility/services
- developer.android.com/reference/android/view/accessibility/AccessibilityEvent
-developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo
- cloak-and-dagger.org
Guys, thank you very much for your attention!