Android Paging Library: How to intelligently switch Between online and offline data?

I am following this tutorial by Raywenderlich on paging-library-for-android-with-kotlin on how to use android paging library. This is one of the easiest tutorials on the net and I have followed it thoroughly. However, I would like to make some changes so that I can intelligently switch between online data and offline data.

That is, I have old some posts in my database. Initially I have internet connection. So I load latest data from internet, then insert it into my database. Finally, I show this latest data in my recyclerView / PagedListAdapter. If for some reason, there is no internet connection after sometime, I should show the old posts from database.

How can I do this?

My attempts:

This is my code on github repository.

Here, I tried to create a factory pattern. It checks if initially I have internet, the factory returns pagedList from online dataSource. ELse, the factory returns pagedList from offline dataSource. But this doesnot intelligently switch between the 2 states.

I tried some random codes such as creating a boundary callback. But I am not sure how to make the necessary changes. I am not adding codes here (at least for now) to keep it short and precise.

Can anyone help me?

Edit:

To be specific, I am loading paged data primarily from network. If there is a network error, I don’t want to show the user an error. Instead I load paged data from cache / database and continuously show it to my user as long as possible. If the network is back,switch back to network paged data. (that’s what instagram / facebook does I think). What is the appropriate way to implement this? See my code / attemp in the answer.

Answer

Okay, so after trying out some codes for 2 days, this is what I came up with. However, I really don’t know if this is a good pratice or not. So I am open to any acceptable answers.

Explanation:

Since I have multiple data sources(network and database), I created ProfilePostDataSource: PageKeyedDataSource<Pair<Long, Long>, ProfilePost> here the key is a pair, the 1st one for network pagination, the 2nd one is for database pagination.

I used kotlin’s Coroutine to write some asynchronous codes in a simple if-else like manner. So we can write it in a psudo-code like this:

Database db;
Retrofit retrofit;

inside loadInitial/loadBefore / loadAfter:
  currNetworkKey = params.key.first;
  currDBKey = params.key.second;
  
  ArrayList<Model> pagedList;

  coroutine{
    ArrayList<Model> onlineList = retrofit.getNetworkData(currNetworkKey);  // <-- we primarily load data from network
    if(onlineList != null) {
      pagedList = onlineList;
      db.insertAll(onlineList);  // <-- update our cache
    }else{
      ArrayList<Model> offlineList = db.getOfflineData(currDBKey); // <-- incase the network fails, we load cache from database  
      if(offlineList !=null){
           pagedList = offlineList;
      }
    }
    if(pagedList != null or empty) {
      nextNetworkKey = // update it accordingly
      nextDBKey = // update it accordingly
      Pair<int, int> nextKey = new Pair(nextNetworkKey, nextDBKey);
      
      pagingLibraryCallBack.onResult(pagedList, nextKey); // <-- submit the data to paging library via callback. this updates your adapter, recyclerview etc...
    }
  }

So in apps like facebook, instagram etc, we see them primarily loading data from network. But if the network is down, they show you a cashed data. We can intelligently make this switch like this code.

Here is a relevant code snippet, the PageKeyedDataSource written in kotlin:

ProfilePostDataSource.kt

/** @brief: <Key, Value> = <Integer, ProfilePost>. The key = pageKey used in api. Value = single item data type in the recyclerView
 *
 * We have a situation. We need a 2nd id to fetch profilePosts from database.
 * Change of plan:  <Key, Value> = < Pair<Int, Int>, ProfilePost>. here the
 *
 *                    key.first = pageKey used in api.      <-- Warning: Dont switch these 2!
 *                     Key.second = db last items id
 *                                   used as out db page key
 *
 * Value = single item data type in the recyclerView
 *
 * */
class ProfilePostDataSource: PageKeyedDataSource<Pair<Long, Long>, ProfilePost> {

  companion object{
    val TAG: String = ProfilePostDataSource::class.java.simpleName;
    val INVALID_KEY: Long = -1;
  }

  private val context: Context;
  private val userId: Int;
  private val liveLoaderState: MutableLiveData<NetworkState>;
  private val profilePostLocalData: ProfilePostLocalDataProvider;

  public constructor(context: Context, userId: Int, profilePostLocalData: ProfilePostLocalDataProvider, liveLoaderState: MutableLiveData<NetworkState>) {
    this.context = context;
    this.userId = userId;
    this.profilePostLocalData = profilePostLocalData;
    this.liveLoaderState = liveLoaderState;
  }

  override fun loadInitial(params: LoadInitialParams<Pair<Long, Long>>, pagingLibraryCallBack: LoadInitialCallback<Pair<Long, Long>, ProfilePost>) {
    val initialNetworkKey: Long = 1L;  // suffix = networkKey cz later we'll add dbKey
    var nextNetworkKey = initialNetworkKey + 1;
    val prevNetworkKey = null; // cz we wont be using it in this case

    val initialDbKey: Long = Long.MAX_VALUE; // dont think I need it
    var nextDBKey: Long = 0L;

    GlobalScope.launch(Dispatchers.IO) {
      val pagedProfilePosts: ArrayList<ProfilePost> = ArrayList(); // cz kotlin emptyList() sometimes gives a weird error. So use arraylist and be happy
      val authorization : String = AuthManager.getInstance(context).authenticationToken;

      try{
        setLoading();
        val res: Response<ProfileServerResponse> = getAPIService().getFeedProfile(
          sessionToken = authorization, id = userId, withProfile = false, withPosts = true, page = initialNetworkKey.toInt()
        );

        if(res.isSuccessful && res.body()!=null) {
          pagedProfilePosts.addAll(res.body()!!.posts);
        }

      }catch (x: Exception) {
        Log.e(TAG, "Exception -> "+x.message);
      }

      if(pagedProfilePosts.isNotEmpty()) {
        // this means network call is successfull
        Log.e(TAG, "key -> "+initialNetworkKey+" size -> "+pagedProfilePosts.size+" "+pagedProfilePosts.toString());

        nextDBKey = pagedProfilePosts.last().id;
        val nextKey: Pair<Long, Long> = Pair(nextNetworkKey, nextDBKey);

        pagingLibraryCallBack.onResult(pagedProfilePosts, prevNetworkKey, nextKey);
        // <-- this is paging library's callback to a pipeline that updates data which inturn updates the recyclerView. There is a line: adapter.submitPost(list) in FeedProfileFragment. this callback is related to that line...
        profilePostLocalData.insertProfilePosts(pagedProfilePosts, userId); // insert the latest data in db
      }else{
        // fetch data from cache
        val cachedList: List<ProfilePost> = profilePostLocalData.getProfilePosts(userId);
        pagedProfilePosts.addAll(cachedList);

        if(pagedProfilePosts.size>0) {
          nextDBKey = cachedList.last().id;
        }else{
          nextDBKey = INVALID_KEY;
        }
        nextNetworkKey = INVALID_KEY; // <-- probably there is a network error / sth like that. So no need to execute further network call. thus pass invalid key
        val nextKey: Pair<Long, Long> = Pair(nextNetworkKey, nextDBKey);
        pagingLibraryCallBack.onResult(pagedProfilePosts, prevNetworkKey, nextKey);

      }
      setLoaded();

    }
  }

  override fun loadBefore(params: LoadParams<Pair<Long, Long>>, pagingLibraryCallBack: LoadCallback<Pair<Long, Long>, ProfilePost>) {}  // we dont need it in feedProflie

  override fun loadAfter(params: LoadParams<Pair<Long, Long>>, pagingLibraryCallBack: LoadCallback<Pair<Long, Long>, ProfilePost>) {
    val currentNetworkKey: Long = params.key.first;
    var nextNetworkKey = currentNetworkKey; // assuming invalid key
    if(nextNetworkKey!= INVALID_KEY) {
      nextNetworkKey = currentNetworkKey + 1;
    }

    val currentDBKey: Long = params.key.second;
    var nextDBKey: Long = 0;

    if(currentDBKey!= INVALID_KEY || currentNetworkKey!= INVALID_KEY) {
      GlobalScope.launch(Dispatchers.IO) {
        val pagedProfilePosts: ArrayList<ProfilePost> = ArrayList(); // cz kotlin emptyList() sometimes gives a weird error. So use arraylist and be happy
        val authorization : String = AuthManager.getInstance(context).authenticationToken;

        try{
          setLoading();
          if(currentNetworkKey!= INVALID_KEY) {
            val res: Response<ProfileServerResponse> = getAPIService().getFeedProfile(
                    sessionToken = authorization, id = userId, withProfile = false, withPosts = true, page = currentNetworkKey.toInt()
            );

            if(res.isSuccessful && res.body()!=null) {
              pagedProfilePosts.addAll(res.body()!!.posts);
            }
          }

        }catch (x: Exception) {
          Log.e(TAG, "Exception -> "+x.message);
        }

        if(pagedProfilePosts.isNotEmpty()) {
          // this means network call is successfull
          Log.e(TAG, "key -> "+currentNetworkKey+" size -> "+pagedProfilePosts.size+" "+pagedProfilePosts.toString());

          nextDBKey = pagedProfilePosts.last().id;
          val nextKey: Pair<Long, Long> = Pair(nextNetworkKey, nextDBKey);

          pagingLibraryCallBack.onResult(pagedProfilePosts,  nextKey);
          setLoaded();
          // <-- this is paging library's callback to a pipeline that updates data which inturn updates the recyclerView. There is a line: adapter.submitPost(list) in FeedProfileFragment. this callback is related to that line...
          profilePostLocalData.insertProfilePosts(pagedProfilePosts, userId); // insert the latest data in db
        }else{
          // fetch data from cache
//          val cachedList: List<ProfilePost> = profilePostLocalData.getProfilePosts(userId);
          val cachedList: List<ProfilePost> = profilePostLocalData.getPagedProfilePosts(userId, nextDBKey, 20);
          pagedProfilePosts.addAll(cachedList);

          if(pagedProfilePosts.size>0) {
            nextDBKey = cachedList.last().id;
          }else{
            nextDBKey = INVALID_KEY;
          }

          nextNetworkKey = INVALID_KEY; // <-- probably there is a network error / sth like that. So no need to execute further network call. thus pass invalid key
          val nextKey: Pair<Long, Long> = Pair(nextNetworkKey, nextDBKey);
          pagingLibraryCallBack.onResult(pagedProfilePosts, nextKey);
          setLoaded();
        }
      }
    }
  }

  private suspend fun setLoading() {
    withContext(Dispatchers.Main) {
      liveLoaderState.value = NetworkState.LOADING;
    }
  }

  private suspend fun setLoaded() {
    withContext(Dispatchers.Main) {
      liveLoaderState.value = NetworkState.LOADED;
    }
  }

}

Thank you for reading this far. If you have a better solution, feel free to let me know. I’m open to any working solutions.