Pagination lists in Android with RxJava. Part II

xoxol_89 November 30, 2015 at 09:11

Pagination lists in Android with RxJava. Part II

Good day to all!
About a month ago, I wrote an article about organizing list pagination (RecyclerView) using RxJava. What is pagination simple? This is an automatic loading of data to the list when scrolling through it.
The solution that I presented in that article was quite working, resistant to errors in answers to data loading requests and resistant to screen reorientation (correct state saving).
But thanks to the comments of the Habrovsk citizens, their comments and suggestions, I realized that the solution has a number of shortcomings that can be completely eliminated.
Many thanks to Matvey Malkov for detailed comments and great ideas. Without it, refactoring a past decision would not have taken place.
I ask everyone interested under cat.

And so, what were the disadvantages of the first option:

The appearance of custom AutoLoadingRecyclerViewand AutoLoadingRecyclerViewAdapter. That is, just like that, you can’t insert this solution into the already written code. Have to work a little. And this, of course, somewhat binds hands in the future.
When initializing the AutoLoadingRecyclerViewneed to explicitly call the methods setLimit, setLoadingObservable, startLoading. And this is in addition to standard RecyclerViewmethods, type setAdapter, setLayoutManagerand others. Also, keep in mind that the method startLoadingmust be called last. Yes, all these methods are marked with comments on how and in what order they should be called, but this is not very intuitive, and you can easily get confused.
The pagination mechanism was implemented in AutoLoadingRecyclerView. A brief summary of it is as follows:
- There is PublishSubject, attached to RecyclerView.OnScrollListener, and which, accordingly, “emits” certain elements upon the occurrence of an event (when the user has twisted to a certain position).
- There is Subscriberone who listens to the above PublishSubject, and when an element c arrives at it PublishSubject, he unsubscribes from it and calls the special Observableone responsible for loading new elements.
- And there is Observable, loading new items, updating the list, and then reconnecting Subscriberto PublishSubjectto listen to the scrolling list.
The biggest drawback of this algorithm is its use PublishSubject, which is generally recommended for use in exceptional situations and which somewhat breaks the whole concept of RxJava. As a result, we get a bit of “crutch reactivity”.

Refactoring
And now, using the above disadvantages, we will try to develop a more convenient and beautiful solution.

First of all, we’ll get rid of itPublishSubject , and create a place for it Observablethat will “emit” upon the occurrence of a given condition, that is, when the user scrolls to a certain position.
The method for obtaining this Observable(for simplicity we will call it - scrollObservable) will be as follows:

private static Observable getScrollObservable(RecyclerView recyclerView, int limit, int emptyListCount) {
        return Observable.create(subscriber -> {
            final RecyclerView.OnScrollListener sl = new RecyclerView.OnScrollListener() {
                @Override
                public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
                    if (!subscriber.isUnsubscribed()) {
                        int position = getLastVisibleItemPosition(recyclerView);
                        int updatePosition = recyclerView.getAdapter().getItemCount() - 1 - (limit / 2);
                        if (position >= updatePosition) {
                            subscriber.onNext(recyclerView.getAdapter().getItemCount());
                        }
                    }
                }
            };
            recyclerView.addOnScrollListener(sl);
            subscriber.add(Subscriptions.create(() -> recyclerView.removeOnScrollListener(sl)));
            if (recyclerView.getAdapter().getItemCount() == emptyListCount) {
                subscriber.onNext(recyclerView.getAdapter().getItemCount());
            }
        });
    }

Let's go through the parameters:

RecyclerView recyclerView - our search list :)
int limit- the number of loaded items at a time. I added this parameter here for the convenience of determining the “position X”, after which it Observablestarts to “emit”. The position is determined by this expression:
```
int updatePosition = recyclerView.getAdapter().getItemCount() - 1 - (limit / 2);
```
As I said in a previous article, it was revealed empirically, and you can already change it yourself depending on the problem you are solving.
int emptyListCountIs already a more interesting parameter. Remember, I said that in the previous version, after initialization with the latest one, you need to call the method startLoadingfor the primary boot. So now, if the list is empty and it is not scrolled, then it scrollObservableautomatically “emits” the first element, which serves as the starting point for starting pagination:
```
if (recyclerView.getAdapter().getItemCount() == 0) {
    subscriber.onNext(recyclerView.getAdapter().getItemCount());
}
```
But, what if the list already contains some elements "by default" (for example, one element). But pagination must somehow begin. The parameter also helps with this emptyListCount.
```
int emptyListCount = 1;
if (recyclerView.getAdapter().getItemCount() == emptyListCount) {
    subscriber.onNext(recyclerView.getAdapter().getItemCount());
}
```

The resulting scrollObservable"emit" number equal to the number of elements in the list. The same number is a shift (or “offset”).

subscriber.onNext(recyclerView.getAdapter().getItemCount());

When scrolling after reaching a certain position, it scrollObservablebegins to massively “emit” elements. We need only one “emit” with a changed “offset”. Therefore, we add an operator distinctUntilChanged()that cuts off all repeating elements.
The code:

getScrollObservable(recyclerView, limit, emptyListCount)
    .distinctUntilChanged();

It is also necessary to remember that we work with the UI element and track changes in its state. Therefore, all the work of “listening” for scrolling the list should occur in the UI thread:

getScrollObservable(recyclerView, limit, emptyListCount)
    .subscribeOn(AndroidSchedulers.mainThread())
    .distinctUntilChanged();

Now it is necessary to correctly load this data.
To do this, create an interface PagingListener, implementing which, the developer sets Observable, responsible for loading data:

public interface PagingListener {
    Observable> onNextPage(int offset);
}

Switching to "loading" Observableis possible using the operator switchMap. Also remember that it is desirable to perform data loading not in the UI stream.
Attention to the code:

getScrollObservable(recyclerView, limit, emptyListCount)
                .subscribeOn(AndroidSchedulers.mainThread())
                .distinctUntilChanged()
                .observeOn(Schedulers.from(BackgroundExecutor.getSafeBackgroundExecutor()))
                .switchMap(pagingListener::onNextPage);

We subscribe to this Observablealready in a fragment or activity, where the developer decides how to deal with the newly downloaded data. Or immediately into the list, or filter, and only then the list. The great thing is that we can easily redesign Observableas we want. In this, of course, RxJava is wonderful, but Subject, which was in the last article, is not an assistant.

Error Handling
But what if there was some short-term error while loading the data, such as a “lost network”, etc.? We should be able to retry requesting data. Of course, the operator begs the question retry(long count)( retry()I avoid the operator because of the possibility of freezing if the error is not short-term). Then:

getScrollObservable(recyclerView, limit, emptyListCount)
                .subscribeOn(AndroidSchedulers.mainThread())
                .distinctUntilChanged()
                .observeOn(Schedulers.from(BackgroundExecutor.getSafeBackgroundExecutor()))
                .switchMap(pagingListener::onNextPage)
                .retry(3);

But here is the problem. If an error occurs and the user scrolls to the end of the list, nothing will happen, a second request will not be sent. The thing is that the operator retry(long count)re-signs Subscriberto in case of an error Observable, and we again “listen” to the scrolling list. And the list has reached the end, so there is no repeated request. It is treated only by “twitching” the list so that scrolling works. But this, of course, is not correct.

Therefore, I had to dodge so that in case of an error, the request would still be resent regardless of the scrolling of the list and no more than what the developer asked.
The solution is:

int startNumberOfRetryAttempt = 0;
getScrollObservable(recyclerView, limit, emptyListCount)
    .subscribeOn(AndroidSchedulers.mainThread())
    .distinctUntilChanged()
    .observeOn(Schedulers.from(BackgroundExecutor.getSafeBackgroundExecutor()))
    .switchMap(offset -> getPagingObservable(pagingListener, pagingListener.onNextPage(offset), startNumberOfRetryAttempt, offset, retryCount))
private static  Observable> getPagingObservable(PagingListener listener, Observable> observable, int numberOfAttemptToRetry, int offset, int retryCount) {
    return observable.onErrorResumeNext(throwable -> {
        // retry to load new data portion if error occurred
        if (numberOfAttemptToRetry < retryCount) {
            int attemptToRetryInc = numberOfAttemptToRetry + 1;
            return getPagingObservable(listener, listener.onNextPage(offset), attemptToRetryInc, offset, retryCount);
        } else {
            return Observable.empty();
        }
    });
}

The parameter is retryCountset by the developer. This is the maximum number of retries in case of an error. That is, this is not the maximum number of attempts for all requests, but the maximum - only for a specific request.
How does this code work, or rather a method getPagingObservable?
TO

Observable observable применяем оператор onErrorResumeNext, который в случае ошибки подставляет другой Observable. Внутри данного оператора мы сначала проверяем количество уже совершенных попыток. Если их еще меньше retryCount:
if (numberOfAttemptToRetry < retryCount) {


, то мы инкрементируем счетчик совершенных попыток:
int attemptToRetryInc = numberOfAttemptToRetry + 1;


, и рекурсивно вызываем этот же метод с обновленным счетчиком попыток, который снова осуществляет тот же запрос через listener.onNextPage(offset):
return getPagingObservable(listener, listener.onNextPage(offset), attemptToRetryInc, offset, retryCount);


Если количество попыток превысило максимально допустимое, то просто возвращает пустой Observable:
return Observable.empty();


Пример

А теперь вашему вниманию полный пример использования PaginationTool.
PaginationTool/**
 * @author e.matsyuk
 */
public class PaginationTool {
    // for first start of items loading then on RecyclerView there are not items and no scrolling
    private static final int EMPTY_LIST_ITEMS_COUNT = 0;
    // default limit for requests
    private static final int DEFAULT_LIMIT = 50;
    // default max attempts to retry loading request
    private static final int MAX_ATTEMPTS_TO_RETRY_LOADING = 3;
    public static  Observable> paging(RecyclerView recyclerView, PagingListener pagingListener) {
        return paging(recyclerView, pagingListener, DEFAULT_LIMIT, EMPTY_LIST_ITEMS_COUNT, MAX_ATTEMPTS_TO_RETRY_LOADING);
    }
    public static  Observable> paging(RecyclerView recyclerView, PagingListener pagingListener, int limit) {
        return paging(recyclerView, pagingListener, limit, EMPTY_LIST_ITEMS_COUNT, MAX_ATTEMPTS_TO_RETRY_LOADING);
    }
    public static  Observable> paging(RecyclerView recyclerView, PagingListener pagingListener, int limit, int emptyListCount) {
        return paging(recyclerView, pagingListener, limit, emptyListCount, MAX_ATTEMPTS_TO_RETRY_LOADING);
    }
    public static  Observable> paging(RecyclerView recyclerView, PagingListener pagingListener, int limit, int emptyListCount, int retryCount) {
        if (recyclerView == null) {
            throw new PagingException("null recyclerView");
        }
        if (recyclerView.getAdapter() == null) {
            throw new PagingException("null recyclerView adapter");
        }
        if (limit <= 0) {
            throw new PagingException("limit must be greater then 0");
        }
        if (emptyListCount < 0) {
            throw new PagingException("emptyListCount must be not less then 0");
        }
        if (retryCount < 0) {
            throw new PagingException("retryCount must be not less then 0");
        }
        int startNumberOfRetryAttempt = 0;
        return getScrollObservable(recyclerView, limit, emptyListCount)
                .subscribeOn(AndroidSchedulers.mainThread())
                .distinctUntilChanged()
                .observeOn(Schedulers.from(BackgroundExecutor.getSafeBackgroundExecutor()))
                .switchMap(offset -> getPagingObservable(pagingListener, pagingListener.onNextPage(offset), startNumberOfRetryAttempt, offset, retryCount));
    }
    private static Observable getScrollObservable(RecyclerView recyclerView, int limit, int emptyListCount) {
        return Observable.create(subscriber -> {
            final RecyclerView.OnScrollListener sl = new RecyclerView.OnScrollListener() {
                @Override
                public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
                    if (!subscriber.isUnsubscribed()) {
                        int position = getLastVisibleItemPosition(recyclerView);
                        int updatePosition = recyclerView.getAdapter().getItemCount() - 1 - (limit / 2);
                        if (position >= updatePosition) {
                            subscriber.onNext(recyclerView.getAdapter().getItemCount());
                        }
                    }
                }
            };
            recyclerView.addOnScrollListener(sl);
            subscriber.add(Subscriptions.create(() -> recyclerView.removeOnScrollListener(sl)));
            if (recyclerView.getAdapter().getItemCount() == emptyListCount) {
                subscriber.onNext(recyclerView.getAdapter().getItemCount());
            }
        });
    }
    private static int getLastVisibleItemPosition(RecyclerView recyclerView) {
        Class recyclerViewLMClass = recyclerView.getLayoutManager().getClass();
        if (recyclerViewLMClass == LinearLayoutManager.class || LinearLayoutManager.class.isAssignableFrom(recyclerViewLMClass)) {
            LinearLayoutManager linearLayoutManager = (LinearLayoutManager)recyclerView.getLayoutManager();
            return linearLayoutManager.findLastVisibleItemPosition();
        } else if (recyclerViewLMClass == StaggeredGridLayoutManager.class || StaggeredGridLayoutManager.class.isAssignableFrom(recyclerViewLMClass)) {
            StaggeredGridLayoutManager staggeredGridLayoutManager = (StaggeredGridLayoutManager)recyclerView.getLayoutManager();
            int[] into = staggeredGridLayoutManager.findLastVisibleItemPositions(null);
            List intoList = new ArrayList<>();
            for (int i : into) {
                intoList.add(i);
            }
            return Collections.max(intoList);
        }
        throw new PagingException("Unknown LayoutManager class: " + recyclerViewLMClass.toString());
    }
    private static  Observable> getPagingObservable(PagingListener listener, Observable> observable, int numberOfAttemptToRetry, int offset, int retryCount) {
        return observable.onErrorResumeNext(throwable -> {
            // retry to load new data portion if error occurred
            if (numberOfAttemptToRetry < retryCount) {
                int attemptToRetryInc = numberOfAttemptToRetry + 1;
                return getPagingObservable(listener, listener.onNextPage(offset), attemptToRetryInc, offset, retryCount);
            } else {
                return Observable.empty();
            }
        });
    }
}
PagingException/**
 * @author e.matsyuk
 */
public class PagingException extends RuntimeException {
    public PagingException(String detailMessage) {
        super(detailMessage);
    }
}
PagingListener/**
 * @author e.matsyuk
 */
public interface PagingListener {
    Observable> onNextPage(int offset);
}
PaginationFragment/**
 * A placeholder fragment containing a simple view.
 */
public class PaginationFragment extends Fragment {
    private final static int LIMIT = 50;
    private PagingRecyclerViewAdapter recyclerViewAdapter;
    private Subscription pagingSubscription;
    @Override
    public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
        View rootView = inflater.inflate(R.layout.fmt_pagination, container, false);
        setRetainInstance(true);
        init(rootView, savedInstanceState);
        return rootView;
    }
    @Override
    public void onResume() {
        super.onResume();
    }
    private void init(View view, Bundle savedInstanceState) {
        RecyclerView recyclerView = (RecyclerView) view.findViewById(R.id.RecyclerView);
        GridLayoutManager recyclerViewLayoutManager = new GridLayoutManager(getActivity(), 1);
        recyclerViewLayoutManager.supportsPredictiveItemAnimations();
        // init adapter for the first time
        if (savedInstanceState == null) {
            recyclerViewAdapter = new PagingRecyclerViewAdapter();
            recyclerViewAdapter.setHasStableIds(true);
        }
        recyclerView.setLayoutManager(recyclerViewLayoutManager);
        recyclerView.setAdapter(recyclerViewAdapter);
        // if all items was loaded we don't need Pagination
        if (recyclerViewAdapter.isAllItemsLoaded()) {
            return;
        }
        // RecyclerView pagination
        pagingSubscription = PaginationTool
                .paging(recyclerView, offset -> EmulateResponseManager.getInstance().getEmulateResponse(offset, LIMIT), LIMIT)
                .observeOn(AndroidSchedulers.mainThread())
                .subscribe(new Subscriber>() {
                    @Override
                    public void onCompleted() {
                    }
                    @Override
                    public void onError(Throwable e) {
                    }
                    @Override
                    public void onNext(List items) {
                        recyclerViewAdapter.addNewItems(items);
                        recyclerViewAdapter.notifyItemInserted(recyclerViewAdapter.getItemCount() - items.size());
                    }
                });
    }
    @Override
    public void onDestroyView() {
        if (pagingSubscription != null && !pagingSubscription.isUnsubscribed()) {
            pagingSubscription.unsubscribe();
        }
        super.onDestroyView();
    }
}
PagingRecyclerViewAdapter/**
 * @author e.matsyuk
 */
public class PagingRecyclerViewAdapter extends RecyclerView.Adapter {
    private static final int MAIN_VIEW = 0;
    private List listElements = new ArrayList<>();
    // after reorientation test this member
    // or one extra request will be sent after each reorientation
    private boolean allItemsLoaded;
    static class MainViewHolder extends RecyclerView.ViewHolder {
        TextView textView;
        public MainViewHolder(View itemView) {
            super(itemView);
            textView = (TextView) itemView.findViewById(R.id.text);
        }
    }
    public void addNewItems(List items) {
        if (items.size() == 0) {
            allItemsLoaded = true;
            return;
        }
        listElements.addAll(items);
    }
    public boolean isAllItemsLoaded() {
        return allItemsLoaded;
    }
    @Override
    public long getItemId(int position) {
        return getItem(position).getId();
    }
    public Item getItem(int position) {
        return listElements.get(position);
    }
    @Override
    public int getItemCount() {
        return listElements.size();
    }
    @Override
    public RecyclerView.ViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
        if (viewType == MAIN_VIEW) {
            View v = LayoutInflater.from(parent.getContext()).inflate(R.layout.recycler_item, parent, false);
            return new MainViewHolder(v);
        }
        return null;
    }
    @Override
    public int getItemViewType(int position) {
        return MAIN_VIEW;
    }
    @Override
    public void onBindViewHolder(RecyclerView.ViewHolder holder, int position) {
        switch (getItemViewType(position)) {
            case MAIN_VIEW:
                onBindTextHolder(holder, position);
                break;
        }
    }
    private void onBindTextHolder(RecyclerView.ViewHolder holder, int position) {
        MainViewHolder mainHolder = (MainViewHolder) holder;
        mainHolder.textView.setText(getItem(position).getItemStr());
    }
}


Также данный пример и пример из предыдущей статьи доступны на GitHub.


Спасибо за внимание! Буду рад замечаниям, предложениям и, конечно же, благодарностям.