Technical aspects of ensuring the non-visual availability of Android applications
- Tutorial
Perhaps, to a reader far from the issue under consideration, the name will seem absurd, because the design of the interface of both the Android system itself and the applications developed for it is focused primarily on visual clarity and attractiveness, which is aggravated by the use of the touch screen as the main organ of user interaction with device. However, there is a category of users, by the will of nature or a case deprived of the opportunity to fully enjoy all these charms. Due to the fact that Android provides alternative, or, better to say, additional, methods of interaction, the interface and the basic functionality of the system are by no means fundamentally inaccessible to this category of users.The TalkBack . As for the non-visual availability of third-party applications, it varies from case to case and sometimes requires the developer not only some special super-efforts, but at least minimal attention to the problem .
A list of Android applications tested for non-visual accessibility, with relevant comments, can be found, for example, here. Of course, this is not the only such list on the global network and probably not the most representative, but I refer to it primarily as a source of examples that clearly illustrate what is being discussed. Note that the non-visual accessibility of the interface of many of these applications is not due to the special efforts of their developers, but is a natural result of the work of the mechanisms built into the system. Application developers just do not interfere with this, which, however, I would also reward them a considerable merit.
We will not delve into the discussion of the appropriateness of caring for the non-visual accessibility of applications in principle. This has been said enough in other places. We only note that Android developers are paying some attention to this concern, which can be judged byhistory of the development of special access facilities . We will focus on purely technical aspects. Consider a number of typical problems and indicate ways to solve them. In other words, this essay is mainly aimed at developers of Android-applications , for one reason or another decided not to ignore the needs of users burdened with visual restrictions, and its purpose is to help them translate noble thoughts into life.
Since further exposition assumes the reader a more or less clear idea of the principles of non-visual access to the interfaceused in Android, from the point of view of both the user and the programmer, it is recommended that those who are new to this topic first of all familiarize themselves with some sources of fundamental information:
- Android accessibility basics
- Touchscreen OS Android 4.1 Jelly Bean
- Android OS availability in questions and answers
- Android OS: The basic part for blind users. Tutorial
- Recommendations for developing applications for the Android OS platform, taking into account availability
- Android Design: Accessibility
- Android accessibility API guide
The following considerations and recommendations will be illustrated and supported by specific examples, taken mainly from the TeamTalk project , my participation in which, not least, was connected with solving the problems of accessibility of the Android application.
Of course, as a rule, these will be not quite literal excerpts from the text. I will simplify them as much as possible and even modify them sometimes, so as not to bore the reader with irrelevant details and make the illustrated ideas the most convex. After all, the subject of our consideration is not this project itself, but the problems of non-visual accessibility, quite typical for Android applications in general, and possible solutions.
Those who want to read the source code, stingy excerpts from which will accompany the narrative, in its entirety, can easily satisfy its legitimate curiosity on Github .
The concept of universal design and the principle of healthy minimalism
I must say right away that I am far from preaching the non-visual accessibility of an interface to the detriment of its visual clarity or aesthetics, not to mention the functionality of the application. I only advocate that accessibility is also not forgotten, especially where it does not require any compromises from the developer or any noticeable special efforts.
I am a supporter of the concept of universal design , according to which, the application interface should ideally be equally accessible to all categories of users. And first of all, there is no need to interfere with the system itself to ensure such accessibility that entails the principle of healthy minimalism , which consists in the fact that you should not produce entities without real need .
That is, when there is a temptation to use any third-party library when developing an interface or to create your own completely original control, it would be nice to start to think: is it really necessary? The Android SDK provides the programmer with a very rich set of tools of this kind, and you should not go beyond it without serious reasons. By the way, this will positively affect not only the availability of the application, but also its compatibility.
About attribute contentDescription
The simplest and most obvious thing that an application developer can (and should, in my opinion) do for users with visual limitations without overworking and without sacrificing anything is to accurately sign all purely graphic interface elements through an attribute
contentDescription. However, unfortunately, very few people do this. And due respect for this attribute seems to be a happy exception rather than a common practice. The recommendations to use
contentDescriptionto increase the accessibility of the application interface are found both in governing documents of Google , and in other sources, so to be honest, it’s even awkward to remind you again. I would refrain if all of these recommendations were not ignored with a constancy worthy of clearly better application.Sometimes, in response to a direct request to sign graphic buttons from developers, I heard that, they say, there is not enough space on the screen for this. Of course, such an answer testifies first of all to the professional bankruptcy of the programmer, who, without even bothering to read the documentation in the slightest degree, figuratively speaking, does not write the program, but blunders. I would like to believe that there are not many illiterates among application developers, but just in case, I emphasize once again that the attribute is
contentDescriptioncompletely harmless, it does not affect the appearance of the application and does not require space on the screen . However, as with everything else, to fill
contentDescriptionmust be approached with understanding and without fanaticism. A mechanically thoughtless approach is likely to produce completely undesirable results. We illustrate what was said by example. Suppose we are going to display a list of users and for the list item we have the following scheme:
As you can see, a purely graphic element
ImageViewin this scheme does not have an attribute contentDescription. And this is completely conscious. The list element here is considered as a single whole, that is, its parts ( ImageViewand TextView) do not have an independent role: they do not have an attribute set clickable. The textual information required by the special access service is entirely contained in TextView, and ImageViewin this case plays for the most part a decorative role and from the point of view of non-visual access does not carry useful information. It is quite another matter if the element
ImageViewwere actually used as a button, clicking on which would cause some action. In this case, the attribute
contentDescriptionwould be extremely useful.Now suppose that the users in our list may be in a different state, say, "online" and "offline", and for their indication we will use different colors. To make this additional information also non-visually accessible to us
contentDescription, the attribute will help again , which this time we will set dynamically together with the color of the element in the list adapter. Here's how it can be implemented:
class UserListAdapter extends ArrayAdapter {
public UserListAdapter(Context context, int resource) {
super(context, resource);
}
@Override
public View getView(int position, View convertView, ViewGroup parent) {
Context context = getContext();
LayoutInflater inflater = LayoutInflater.from(context);
if (convertView == null)
convertView = inflater.inflate(R.layout.item_user, null);
User user = getItem(position);
TextView nickname = (TextView) convertView.findViewById(R.id.nickname);
nickname.setText(user.nickname);
if (user.stateOnline) {
convertView.setBackgroundColor(Color.rgb(133, 229, 141));
// Мы намеренно дублируем здесь основное текстовое содержание,
// так как при наличии атрибута contentDescription,
// служба специального доступа использует именно его,
// полностью игнорируя атрибут text.
nickname.setContentDescription(context.getString(R.string.user_state_online, user.nickname));
}
else {
convertView.setBackgroundColor(Color.rgb(0, 0, 0));
// Обнуляя contentDescription, мы заставляем
// службу специального доступа использовать атрибут text.
nickname.setContentDescription(null);
}
return convertView;
}
}
It is assumed that the string resource has a definition:
%1$s online Please note that we provide additional information to the special access service only when the user is in the "online" state. This helps to reduce the volume of voice messages without sacrificing informational content, since there are only two possible states, that is, no discrepancies arise.
Voice messages require time for perception , so their volume should be reduced wherever possible , without sacrificing useful information.
In addition, when composing a combined text for
contentDescription, we place the user name in front of the designation of his status, because for reasons of perception efficiency, the most popular information should be located at the beginning of the speech message .Lists with live elements
Continuing to consider the example from the previous paragraph, it will be logical to assume that the state of users changes for some reason external to the application, or, more precisely, its interface. And we need to regularly update the information on the screen so that it matches the real state of things.
For definiteness, suppose the following implementation:
public class MainActivity extends Activity {
private UserListAdapter userListAdapter;
private CountDownTimer listUpdateTimer;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
userListAdapter = new UserListAdapter(this, R.layout.item_user);
listUpdateTimer = new CountDownTimer(10000, 1000) {
@Override
public void onTick(long millisUntilFinished) {
userListAdapter.notifyDataSetChanged();
}
@Override
public void onFinish() {
start();
}
};
listUpdateTimer.start();
}
}
That is, the information on the screen will be updated approximately once per second. But with each such update, the list items will generate the corresponding events for the special access service, and if the availability focus is on one of the list items, this item will be constantly talked about, which will lead to the almost complete impossibility of normal user interaction with the application. There is a case when the excessive helpfulness of special access means is not for the future and frenzied fanaticism needs a reasonable restriction.
To this end, we introduce the auxiliary class:
public class AccessibilityAssistant extends AccessibilityDelegate {
private final Activity hostActivity;
private volatile boolean eventsLocked;
public AccessibilityAssistant(Activity activity) {
hostActivity = activity;
eventsLocked = false;
}
// Перед обновлением списка мы будем запрещать выдачу событий.
public void lockEvents() {
eventsLocked = true;
}
// После обновления списка мы будем вновь разрешать выдачу событий,
// однако реально это должно происходить лишь после того,
// как информация на экране действительно обновится.
public void unlockEvents() {
if (!hostActivity.getWindow().getDecorView().post(new Runnable() {
@Override
public void run() {
eventsLocked = false;
}
}))
eventsLocked = false;
}
@Override
public void sendAccessibilityEvent(View host, int eventType) {
if (!eventsLocked)
super.sendAccessibilityEvent(host, eventType);
}
@Override
public void sendAccessibilityEventUnchecked(View host, AccessibilityEvent event) {
if (!eventsLocked)
super.sendAccessibilityEventUnchecked(host, event);
}
}
Now we can easily implement continuous updating of information on the screen without sacrificing the non-visual accessibility of the interface:
public class MainActivity extends Activity {
private AccessibilityAssistant accessibilityAssistant;
private ArrayAdapter userListAdapter;
private CountDownTimer listUpdateTimer;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
accessibilityAssistant = new AccessibilityAssistant(this);
userListAdapter = new ArrayAdapter(this, R.layout.item_user) {
@Override
public View getView(int position, View convertView, ViewGroup parent) {
if (convertView == null)
convertView = LayoutInflater.from(getContext()).inflate(R.layout.item_user, null);
User user = getItem(position);
TextView nickname = (TextView) convertView.findViewById(R.id.nickname);
nickname.setText(user.nickname);
if (user.stateOnline) {
convertView.setBackgroundColor(Color.rgb(133, 229, 141));
nickname.setContentDescription(getString(R.string.user_state_online, user.nickname));
}
else {
convertView.setBackgroundColor(Color.rgb(0, 0, 0));
nickname.setContentDescription(null);
}
// Мы собираемся блокировать именно те события,
// источниками которых являются элементы списка.
convertView.setAccessibilityDelegate(accessibilityAssistant);
return convertView;
}
};
listUpdateTimer = new CountDownTimer(10000, 1000) {
@Override
public void onTick(long millisUntilFinished) {
// Запрещаем выдачу событий
accessibilityAssistant.lockEvents();
// Инициируем обновление списка на экране
userListAdapter.notifyDataSetChanged();
// Вновь разрешаем выдачу событий
// когда информация на экране обновится
accessibilityAssistant.unlockEvents();
}
@Override
public void onFinish() {
start();
}
};
listUpdateTimer.start();
}
}
In principle, the task could be solved by overriding the notifyDataSetChanged () method in the list adapter:
public void notifyDataSetChanged() {
accessibilityAssistant.lockEvents();
super.notifyDataSetChanged();
accessibilityAssistant.unlockEvents();
}
But this option is worse, because events that occur during any update of the list are blocked , even if it is triggered by any user actions. The special access system is designed to ensure that the user has an adequate response to their actions, so in general, such a lock is undesirable.
Complex list items and dynamic information
Now we will consider the situation when each element of the list has a button associated with it, that is, it is described, for example, by the following scheme:
When interacting with such a list in the research mode by touch, we can set the focus of accessibility both on the list elements themselves and on the buttons accompanying them.
Suppose further that in addition to the list, the screen still displays some constantly changing information. A simplified diagram looks something like this:
And, according to the tradition that has already been established, we will make every second updates:
public class MainActivity extends Activity {
private int counter;
private CountDownTimer updateTimer;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
counter = 0;
updateTimer = new CountDownTimer(10000, 1000) {
@Override
public void onTick(long millisUntilFinished) {
((TextView)findViewById(R.id.count_state)).setText(String.valueOf(++counter));
}
@Override
public void onFinish() {
start();
}
};
updateTimer.start();
}
}
In this situation, the problem arises when the focus of accessibility falls on one of the buttons inside the list. The fact is that after processing events that occur when updating information on the screen, the special access service restores its position on the list item, and not on the button . As a result, it turns out to be extremely difficult, and with more frequent updates it is completely impossible to press a button in non-visual access mode. And the event blocking considered in the previous paragraph, alas, does not help here.
To combat this nuisance, we will slightly expand the functionality of our helper class:
public class AccessibilityAssistant extends AccessibilityDelegate {
private final Activity hostActivity;
private volatile boolean eventsLocked;
private final AccessibilityManager accessibilityService;
// Этот флаг устанавливается тогда, когда обновление информации на экране
// может болезненно отразиться на положении фокуса доступности.
private volatile boolean discourageUiUpdates;
public AccessibilityAssistant(Activity activity) {
hostActivity = activity;
accessibilityService = (AccessibilityManager) activity.getSystemService(Context.ACCESSIBILITY_SERVICE);
discourageUiUpdates = false;
eventsLocked = false;
}
// Мы не рекомендуем обновлять экран, если это может сбить фокус доступности,
// но лишь тогда, когда режим специального доступа действительно используется.
public boolean isUiUpdateDiscouraged() {
return discourageUiUpdates && accessibilityService.isEnabled();
}
public void lockEvents() {
eventsLocked = true;
}
public void unlockEvents() {
if (!hostActivity.getWindow().getDecorView().post(new Runnable() {
@Override
public void run() {
eventsLocked = false;
}
}))
eventsLocked = false;
}
@Override
public void sendAccessibilityEvent(View host, int eventType) {
// Следим за тем, когда фокус доступности попадает на кнопки.
if (host instanceof Button)
checkEvent(eventType);
if (!eventsLocked)
super.sendAccessibilityEvent(host, eventType);
}
@Override
public void sendAccessibilityEventUnchecked(View host, AccessibilityEvent event) {
// Следим за тем, когда фокус доступности попадает на кнопки.
if (host instanceof Button)
checkEvent(event.getEventType());
if (!eventsLocked)
super.sendAccessibilityEventUnchecked(host, event);
}
// Реакция на события, связанные с перемещением фокуса доступности
private void checkEvent(int eventType) {
switch (eventType) {
case AccessibilityEvent.TYPE_VIEW_ACCESSIBILITY_FOCUSED:
discourageUiUpdates = true;
break;
case AccessibilityEvent.TYPE_VIEW_ACCESSIBILITY_FOCUS_CLEARED:
discourageUiUpdates = false;
break;
default:
break;
}
}
}
And we will avoid updating information on the screen when the focus of accessibility falls on the buttons built into the list, but, of course, only when the non-visual access mode is used:
public class MainActivity extends Activity {
private AccessibilityAssistant accessibilityAssistant;
private ArrayAdapter userListAdapter;
private CountDownTimer updateTimer;
private int counter;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
accessibilityAssistant = new AccessibilityAssistant(this);
counter = 0;
userListAdapter = new ArrayAdapter(this, R.layout.item_user) {
@Override
public View getView(int position, View convertView, ViewGroup parent) {
if (convertView == null)
convertView = LayoutInflater.from(getContext()).inflate(R.layout.item_user, null);
User user = getItem(position);
TextView nickname = (TextView) convertView.findViewById(R.id.nickname);
Button button = (Button) convertView.findViewById(R.id.msg_btn);
nickname.setText(user.nickname);
if (user.stateOnline) {
convertView.setBackgroundColor(Color.rgb(133, 229, 141));
nickname.setContentDescription(getString(R.string.user_state_online, user.nickname));
}
else {
convertView.setBackgroundColor(Color.rgb(0, 0, 0));
nickname.setContentDescription(null);
}
// Мы собираемся следить за тем,
// когда фокус доступности попадает на кнопку.
button.setAccessibilityDelegate(accessibilityAssistant);
convertView.setAccessibilityDelegate(accessibilityAssistant);
return convertView;
}
};
updateTimer = new CountDownTimer(10000, 1000) {
@Override
public void onTick(long millisUntilFinished) {
// Выполняем обновление лишь тогда,
// когда это не повредит доступности
// или когда режим специального доступа не используется.
if (!accessibilityAssistant.isUiUpdateDiscouraged())
((TextView)findViewById(R.id.count_state)).setText(String.valueOf(++counter));
}
@Override
public void onFinish() {
start();
}
};
updateTimer.start();
}
}
I note that this is the only recipe given here that has visible consequences. That is, when the focus of accessibility falls on some elements of the interface, the screen updates will freeze. But this will happen only in special access mode. In normal mode, there will be no side effects.
Events on invisible pages
Consider another interesting case, namely, the use of switchable tabs (or pages):
The fact is that for the sake of smooth switching of pages, the system keeps in working order not only the one that is displayed on the screen, but also those adjacent to it. And in case of updating information on these neighboring pages that are outside the visible area, the corresponding events for the special access service are generated. This sometimes happens when moving from one page to another, when the system prepares the next and events are already triggered from it. As a result, the voice response does not match what is displayed on the screen.
To get rid of this undesirable effect, we recall that the decision to trigger an event for the special access service is made at the top level of the hierarchy, and we will develop our auxiliary class as follows:
public class AccessibilityAssistant extends AccessibilityDelegate {
private final Activity hostActivity;
private final AccessibilityManager accessibilityService;
// Все контролируемые страницы
private SparseArray monitoredPages;
// Страница, видимая в данный момент
private View visiblePage;
// Номер видимой страницы
private int visiblePageId;
private volatile boolean discourageUiUpdates;
private volatile boolean eventsLocked;
public AccessibilityAssistant(Activity activity) {
hostActivity = activity;
accessibilityService = (AccessibilityManager) activity.getSystemService(Context.ACCESSIBILITY_SERVICE);
monitoredPages = new SparseArray();
visiblePage = null;
visiblePageId = 0;
discourageUiUpdates = false;
eventsLocked = false;
}
public boolean isUiUpdateDiscouraged() {
return discourageUiUpdates && accessibilityService.isEnabled();
}
public void lockEvents() {
eventsLocked = true;
}
public void unlockEvents() {
if (!hostActivity.getWindow().getDecorView().post(new Runnable() {
@Override
public void run() {
eventsLocked = false;
}
}))
eventsLocked = false;
}
// Каждая страница должна регистрироваться при помощи этого метода
// в момент своего возникновения. Это логично делать, например,
// в методе onCreateView() или onViewCreated() соответствующего фрагмента.
public void registerPage(View page, int id) {
monitoredPages.put(id, page);
if (id == visiblePageId)
visiblePage = page;
page.setAccessibilityDelegate(this);
}
public void setVisiblePage(int id) {
visiblePageId = id;
visiblePage = monitoredPages.get(id);
}
@Override
public boolean onRequestSendAccessibilityEvent(ViewGroup host, View child, AccessibilityEvent event) {
return ((monitoredPages.indexOfValue(host) < 0) || (host == visiblePage)) && super.onRequestSendAccessibilityEvent(host, child, event);
}
@Override
public void sendAccessibilityEvent(View host, int eventType) {
if (host instanceof Button)
checkEvent(eventType);
if (!eventsLocked)
super.sendAccessibilityEvent(host, eventType);
}
@Override
public void sendAccessibilityEventUnchecked(View host, AccessibilityEvent event) {
if (host instanceof Button)
checkEvent(event.getEventType());
if (!eventsLocked)
super.sendAccessibilityEventUnchecked(host, event);
}
private void checkEvent(int eventType) {
switch (eventType) {
case AccessibilityEvent.TYPE_VIEW_ACCESSIBILITY_FOCUSED:
discourageUiUpdates = true;
break;
case AccessibilityEvent.TYPE_VIEW_ACCESSIBILITY_FOCUS_CLEARED:
discourageUiUpdates = false;
break;
default:
break;
}
}
}
Now it remains only in time to report a page change:
public class MainActivity extends Activity implements ViewPager.OnPageChangeListener {
private AccessibilityAssistant accessibilityAssistant;
private ViewPager viewPager;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
accessibilityAssistant = new AccessibilityAssistant(this);
viewPager = (ViewPager) findViewById(R.id.pager);
viewPager.setOnPageChangeListener(this);
}
@Override
public void onPageScrolled(int position, float positionOffset, int positionOffsetPixels) {
}
@Override
public void onPageSelected(int position) {
accessibilityAssistant.setVisiblePage(position);
}
@Override
public void onPageScrollStateChanged(int state) {
}
}
And each fragment responsible for the page should register it :
public class PageFragment extends Fragment {
@Override
public void onViewCreated(View view, Bundle savedInstanceState) {
((MainActivity)getActivity()).accessibilityAssistant.registerPage(view, PAGE_NUMBER);
super.onViewCreated(view, savedInstanceState);
}
}
The parameter
PAGE_NUMBERhere actually means the page position number. The same as the parameter of the FragmentPagerAdapter.getItem () method .Conclusion
It may seem that the methods presented here for the most part are not intended to help, but just to prevent the special access system from bringing this or that information to the user's consciousness. In essence, the way it is. But an excess of information sometimes harms no less than its lack. Especially when it is clearly superfluous and irrelevant. I never tire of repeating that a good speech interface should speak as little as possible, but always in a timely manner and always in essence.
The built-in screenreader TalkBack in Android , alas, is far from perfect and, unfortunately, is developing much less dynamically than the accessibility API in the system itself. The participation of the community in its development is hampered by the fact that published sourcesusually irrelevant, and the development team simply ignores the appeals of enthusiasts and constructive proposals.
However, this topic, which deserves separate consideration, lies beyond the scope of this essay. I just wanted to draw the attention of developers to the problem of accessibility of applications and, on the one hand, to dispel some common fears, on the other hand, to show how, by using simple gestures, one can sometimes significantly improve the situation. I hope I succeeded at least to some extent.