Android Native - How to use RemoteMediator

Introduction

The Android Paging 3 library can operate in two modes, with an offline Room database as a source of truth or without one. In this tutorial, we will look at how to use Paging 3 with a local Room database.

This tutorial expands on the previously published Paging 3 tutorial without an offline database here. Paging 3 is somewhat complex, so if you are new to the library, I would recommend checking out the tutorial without RemoteMediator first.

Goals

At the end of the tutorial, you would have learned:

1. How to use RemoteMediator with Paging 3.

Tools Required

1. Android Studio. The version used in this tutorial is Bumblebee 2021.1.1 Patch 3.

Prerequisite Knowledge

1. Intermediate Android.
2. MVVM.
3. Hilt.
4. Kotlin Flow.
5. Retrofit.
6. Moshi.
7. ViewBinding.
8. DataBinding.
9. Room.
10. Basic SQL.

Project Setup

To follow along with the tutorial, perform the steps below:

1. Clone the sample app called CatFacts from https://github.com/dmitrilc/CatFacts.git
2. Switches to the With_RemoteMediator branch.

Project Overview

The project has been completed, but we will walk through the important classes to understand how RemoteMediator is used.

In the project structure below, the classes and packages in red can be ignored, while the ones in green are important to the topic.

The App is very simple. It only loads Cat Facts from the Cat Facts API and displays them in a RecyclerView.

There are also some TextViews at the top of the screen to show statistics about the pages. This is only for demonstration purposes.

App Architecture

The diagram PagingSource With RemoteMediator below attempts to explain the relationships among the classes in the current App. I have also included the diagram without RemoteMediator to make it easier for you to see the differences between the two.

RemoteMediator is required if we want to use Paging 3 with a local database. It has a couple of responsibilities:

1. Determine what to do based on the LoadType of APPEND, PREPEND, or REFRESH.
2. Queries more data from the WebService.
3. Commit CRUDs on this new data to the local Room database. How Room talks to the Pager object is mostly implementation details.

The Pager only reaches out to the RemoteMediator if there is no more data to load from the database. You might have also noticed that we do not have to implement our own PagingSource in this case. This is because Room provides its own implementation for us, with both load() and getRefreshKey() implemented.

Implementing RemoteMediator

First, let us walk through the CatFactRemoteMediator class to see how it was implemented. When you provide an implementation of RemoteMediator, you are required to override two functions, load() and initialize().

In our App, the implementation of load() is as below.

override suspend fun load(
   loadType: LoadType,
   state: PagingState<Int, CatFact>
): MediatorResult {
   return try {
       val loadKey = when (loadType) {
           LoadType.REFRESH -> {
               catFactDao.deleteAll()
               remoteKeyDao.deleteAll()
               CAT_FACTS_STARTING_PAGE_INDEX
           }
           LoadType.PREPEND -> return MediatorResult.Success(true)
           LoadType.APPEND -> {
               val remoteKey = remoteKeyDao.get()!!

               if (remoteKey.currentPage == remoteKey.lastPage){
                   return MediatorResult.Success(true)
               }

               remoteKey.currentPage.plus(1)
           }
       }

       val response = webService.getCatFactPage(loadKey)

       database.withTransaction {
           remoteKeyDao.insertOrReplace(
               RemoteKey(
                   currentPage = response.current_page,
                   lastPage = response.last_page
               ))

           catFactDao.insertAll(response.data)
       }

       MediatorResult.Success(false)
   } catch (e: IOException) {
       MediatorResult.Error(e)
   } catch (e: HttpException) {
       MediatorResult.Error(e)
   }
}

In the function above, the most important things that we need to understand are:

1. Because the load() function has so many responsibilities, it is often easy to get lost if you are looking at someone else’s implementation because their app logic, business logic, and remote endpoints are different from yours. It is in your best interest to only understand how each function argument can be used to return the appropriate return value for your app.

2. The LoadType parameter: this is just an enum with 3 values, APPEND, PREPEND, and REFRESH. APPEND and PREPEND represent the direction in which the user is scrolling, so you can handle this however you want based on your App behavior, such as loading the previous page or the next page. REFRESH is a little bit different, but you are in control of this as well; you can start loading from the beginning or resuming at a specific page. In this sample app, we clear everything and start from the beginning for REFRESH. For PREPEND, we return MediatorResult.Success with endOfPaginationReached set to true; this is akin to saying you have reached the end of pagination in this direction. For APPENDING, we are just incrementing the key by one.

3. If the function has not been short-circuited by one of the return statements in the when expression’s body, then we will request new data from the API and insert them into the Room database.

4. We did not make use of the PagingState parameter at all. This is the same object that is passed to PagingSource#getRefreshKey(). It includes useful information such as the anchorPosition and the loaded pages. Again, you do not have to use it if it does not provide any useful information for your use case.

5. We must return a MediatorResult object in the load() function as well. Do not confuse it with the LoadResult object from the PagingSource’s load() function. MediatorResult is a sealed class, so you cannot instantiate it directly. You must instantiate its subclasses Error and Success instead. If you are returning an Error, then you need to provide it with a Throwable object. If you are returning a Success, then you need to provide it with a boolean, indicating whether the end of paging has been reached for the current LoadType.

Next, we will need to implement initialize().

override suspend fun initialize(): InitializeAction {
   val remoteKey = remoteKeyDao.get()
   return if (remoteKey == null){
       InitializeAction.LAUNCH_INITIAL_REFRESH
   } else {
       InitializeAction.SKIP_INITIAL_REFRESH
   }
}

In this function, you can decide whether to perform an initial load or not. Fortunately, InitializeAction is just a simple enum with two constants. You do not have any dependency passed to initialize(), so you will have to rely on the WebService or the database objects in your own RemoteMediator class to decide whether you would like to refresh.

Creating the Pager

Creating a Pager when using a RemoteMediator is almost the same as without one, the only extra step that you need to do is to provide a RemoteMediator object to the Pager’s remoteMediator parameter.

fun getCatFactPagerWithRemoteMediator(): Flow<PagingData<CatFact>> =
   Pager(
       PagingConfig(1),
       remoteMediator = catFactRemoteMediator
   ) {
       catFactDao.pagingSource()
   }
       .flow

Returning PagingSource<K, V> from Room

Another very important step that you need to do is to return a PagingSource from your DAO query. Our query is located in the CatFactDao.kt file.

@Query("SELECT * FROM cat_fact")
fun pagingSource(): PagingSource<Int, CatFact>

This part is a bit weird because we are not in control of how PagingSource is created by the generated DAO. It is a bit clearer when we look at the generated source code though.

@Override
public PagingSource<Integer, CatFact> pagingSource() {
 final String _sql = "SELECT * FROM cat_fact";
 final RoomSQLiteQuery _statement = RoomSQLiteQuery.acquire(_sql, 0);
 return new LimitOffsetPagingSource<CatFact>(_statement, __db, "cat_fact") {
   @Override
   protected List<CatFact> convertRows(Cursor cursor) {
     final int _cursorIndexOfId = CursorUtil.getColumnIndexOrThrow(cursor, "id");
     final int _cursorIndexOfFact = CursorUtil.getColumnIndexOrThrow(cursor, "fact");
     final int _cursorIndexOfLength = CursorUtil.getColumnIndexOrThrow(cursor, "length");
     final List<CatFact> _result = new ArrayList<CatFact>(cursor.getCount());
     while(cursor.moveToNext()) {
       final CatFact _item;
       final int _tmpId;
       _tmpId = cursor.getInt(_cursorIndexOfId);
       final String _tmpFact;
       if (cursor.isNull(_cursorIndexOfFact)) {
         _tmpFact = null;
       } else {
         _tmpFact = cursor.getString(_cursorIndexOfFact);
       }
       final int _tmpLength;
       _tmpLength = cursor.getInt(_cursorIndexOfLength);
       _item = new CatFact(_tmpId,_tmpFact,_tmpLength);
       _result.add(_item);
     }
     return _result;
   }
 };
}

The code above is generated (Java), so it is a bit hard to read, but not that hard. After investigating, I have broken it down to several steps:

1. It provides a concrete implementation of LimitOffsetPagingSource, overriding the method convertRows(Cursor cursor).
2. LimitOffsetPagingSource is a subclass of PagingSource.
3. The Cursor is looped through, creating CatFact objects and then place everything inside an ArrayList.

Managing the remote key

When using RemoteMediator with an offline database, Google recommends developers to create another entity for managing remote keys. How you build your key is highly dependent on what the remote data APIs return.

For the Cat Facts API specifically, the prev and next keys are provided with a paginated query, but they also provided a last_page key, and I have decided use that for simplicity instead of the prev and next keys, which are in nullable string form and changes for every query. You can check out the files RemoteKey and RemoteKeyDao to see how the keys are stored.

Unlike RemoteMediator, the Paging library is not at all aware of the key table, so you are in control of storing and using them however you want. Because the data is stored locally, it makes a lot of sense to create another table for managing the keys as well. The benefit with this approach is that you can apply data integration features that Room provides (foreign key, indexes, constraints, relationships, etc).

Consuming in Activity

To consume the Flow exposed from the Repository, we would consume it the same way when not using RemoteMediator as well.

lifecycleScope.launch {
   viewModel.pagerWithRemoteMediator.collectLatest {
       adapter.submitData(it)
   }
}

Summary

Congratulations! We have learned how to use Paging 3 with a RemoteMediator in this tutorial. The full project code can be found at https://github.com/dmitrilc/CatFacts/tree/With_RemoteMediator.