Client cache & Retrofit

Or how to create your own cache, when server can not help you

Small preface: Ho-ho-ho my dear friends. I am happy to wish you a happy comming New Year. I hope in this year we all will make even more cool things and produce even more clean and stable code. I wish you to be or to remain a good programmers and to share your knowledges with other, because I’m sure they are precious.

On my recent working place a stunned with project, where (at the first glance) database was actively used. It was similar to Clean Architecture approach, so the structure of project included , that was responsible to provide data and internally it decided whether it need to be use database or network. Apparently overwhelming majority of cases used db as just a cache for data.

On one of projects, I’d participated, we had our own network layer, so we had full control over features, it can support, including of course data caching. But now I faced with Retrofit. Doing caching by your own hands seems too laborious as for me, so I decided to investigate and understand how we can level this shortcoming

Known approach (cache-control)

Surfing though groups of search services, gives us known result: cache-control. You can read more about instructions here and how you can support it on the client with Retrofit here.

Basically it is bunch of standardized headers. They have info about staling time and expiration time. To start mechanism working we need to support of that instruction both in client and server sides.

Unfortunately, that this was not my case. Our server time has no time or wish (I don’t know exactly) to do that. Let’s not get into reasons and details. We just have no support of cache-control on server side.

Wrapping Api interface

Next Idea, that comes in mind is: hey, we have an interface to declare request. We can just wrap an Retrofit’s implementation with our implementation and make caching for needed requests.

It is a good idea, since we are using Decorator pattern, so we not change the logic, we just extend it through object composition. The only flaw of such approach is that we also need to write many things by our hands for each query that supports caching. We can generalize that approach and exclude common logic in a separate file/class/function. We can use only one table to store caching data. We provide a friendly function that will wrap a Request with some caching mechanic, but still we will need to implement all function and write at least one line for them:

interface RocketApi {
@GET
fun search(@Query("filter") filter: String): Single<SearchResult>

@GET
fun getById(@Query("rocket_id") id: String): Single<Rocket>
}

class CacheRocketApi(
//provide DB, repository, file or any other local storage dependency
private val origin: RocketApi
) : RocketApi {
override fun search(filter: String) = origin.search(filter).wrapWithCaching(HOUR, 15 * SECOND)

override fun getById(id: String) = origin.getById(id).wrapWithCaching(HOUR, 5 * MINUTE)

private fun <T> Single<T>.wrapWithCaching(expirationTimeMs: Long, staleTimeMs: Long): Single<T> {
//provide your cache strategy here
}

companion object {
private const val SECOND = 1000L
private const val MINUTE = 60 * SECOND
private const val HOUR
= 60 * MINUTE
}
}

class SearchResult
class Rocket

-is time of freshness of response. If you have an internet and response was received less than ago, then request will not be invoked and instead of that cached response will be taken from local storage. If you have no Internet, then you will get cached response, unless it was received from the server more than ago.

As you can see, we need to implement every function from Api interface and write additional mostly duplicated code. As for me more declarative way is to have annotations that are built in interface:

interface WitcherApi {
@WithCache(stalesAfterMs = 15 * SECOND, expiresAfterMs = HOUR)
@GET
fun searchBeast(@Query("smell") smell: String): Single<Beast>

@WithCache(stalesAfterMs = 5 * MINUTE, expiresAfterMs = HOUR)
@GET
fun getPosionById(@Query("posion_id") id: String): Single<Posion>
}

You may not write names of parameters of annotation. I’ve written that just for more readability in Medium :)

Client-side approach

So Retrofit does not allow us to write own annotations and provide logic for them, unfortunately. But we can still implement it by workarounds.

Conception

The key of solution lies beneath Retrofit library. It is method of interface implementation by java Proxy. The idea is to take an implementation of generated by Retrofit and create our own implementation of by java Proxy. Our own implementation will use Retrofit’s implementation internally and wrap it somehow with caching and restoring cached data. So let’s start with providing of base interfaces and implementations of cache and network.

Providing a Cache

For cache we will create next interface:

interface CacheProvider {
fun obtainFresh(key: String): String?
fun obtain(key: String): String?
fun store(key: String, value: String, stalesAfterMs: Long, expiresAfterMs: Long)
}

It’s really simple and straightforward. Just one thing. Let’s talk about difference between and . As we already mentioned in examples above there are two parameters for cache: staling and expiration.

The first one is about data, when you have an internet connection. So when you have internet connection and cached data is not staled, then you will not make any internet request and instead of that you will take data just from the cache.

The second one is about data, when you have no internet connection. So when you have no any connection and cached data is not expired, then you will take it just from the cache. But if data is expired, then you will receive an internet connection error, like you did if you’d called an ordinary request without internet connection.

So will return cached data if it is not expired and will return cached data if it is not staled. If you interested in implementation of this interface, you can check out this link. I’ve decided to not put whole portion of code in an article.

Last thing according to is what is a . Well, basically it is just unique representation of Request. In my implementation it is combination of Api interface class canonical name + method name + arguments (type + value):

private fun generateKey(method: Method, args: Array<out Any>) =
with(method) { (declaringClass.canonicalName ?: "") + "." + name } +
args.joinToString(prefix = "_", separator = "_") { it.javaClass.canonicalName + "&" + it.toString() }

You can create you own generating method. Moreover my approach is very strict. So if you will create two different methods, but for the same Api call:

interface WitcherApi {
@WithCache(stalesAfterMs = 15 * SECOND, expiresAfterMs = HOUR)
@GET("beasts/")
fun searchBeast(@Query("smell") smell: String): Single<Beast>

@GET("beasts/")
fun searchBeastWithoutCache(@Query("smell") smell: String): Single<Beast>
}

Then in my case those methods will generate different for same Requests. So in my case it is not possible, for example, to write two requests: one with caching and one strict without caching to provide force getting data from network.

Providing a Network

For network things are just self obvious. We just have an interface :

interface NetProvider {
val isOnline: Boolean
}

With simpliest implementation:

class ContextNetProvider(
private val context: Context
) : NetProvider {
override val isOnline: Boolean get() = context.isOnline()
}

Where is:

@SuppressLint("MissingPermission")
fun Context.isOnline() = connectivityManager.activeNetworkInfo?.isConnectedOrConnecting == true

val
Context.connectivityManager get() = getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager

To be honnest is deprecated method. So you should be aware of that and try to provide a not deprecated approach with registering a network callback. If you can provide an example related to the article, please, fill free to share it in a comments below. ;)

Implementation for Single

For now we have and , and implementations (last one we get by ). Now it is time to wrap all those instances and provide a simple implementation for rx Single:

In the method we create a Proxy implementation. Then, when 's method invoked, a method is called. Here we try to find out, is called method was annotated by annotation or not. If not, then we just invoke this method on Retrofit’s implementation ( )by calling .

If method was annotated by then we try to get from our . Depending on situation we invoke or . If we have no or no caching was provided, then we call Retrofit’s with wrapping by invoking method.

In we adding storing result data in cache, if data was recieved from request call and also check an if it was found. If error relates to network, then we try to data from cache and if there is no cached data we just pass on this error. If there is data, then just return it.

Now: when you want to use it, you just need to write:

val interfaceCls = WitcherApi::class.java
wrapper.wrap(interfaceCls, retrofit.create(interfaceCls))

Now I will make things clear: we didn’t include Retrofit dependency into because I don’t want to have external dependencies. But wait, we have dependency in it. Yes, you absolutely right. In ideal way we need to replace it with interface:

interface Serializer {
fun serialize(obj: Any): String
fun deserialize(plain: String, objType: Type): Any
}

Limitations

For now this approach has next limitations:

  • We forced to use Gson, for caching and restoring objects. As I mentioned earlier, we can eliminate that by introducing high level interface and make implementation depending on serialization library: Gson, Jackson and e.t.c
  • We have to make annotated request return . I will stop your right here and say, that if you want to see generalization of method for not only single, but also for and , then please go to the BONUS section.
  • We have to configure Retrofit in the way of using Rx. If we will use courutines or just s, then we will need to provide a method for them. I think we can do that, but for me and for now it is just fine, that we working only with Rx.

Afterwords

As I said above, we wrote cache wrapping only for methods with as a return type. But we can generalize this. Jump on to BONUS section to see that.

So now we have a special wrapper. We can define a way for caching data. We can define a way to determine net working. And we have general caching approach, which helps us to elimintate manual written boilerplate code (when you manualy save data from requests).

Need to mention that I your server has supporting of cache-control then, please, implement it also. Good new, it is not hard to do. But if for some reason your server can not provide that, you can consider to use approach presented in this article. :)

Bonus: RxTemplate

So last and additional thing, I want to mention, is working not only with , but also with and . To do that we need to create a general template with rx chaining, that will use similar general operators like: , and also general factory methods: , , .

So let’s jump straight to proposed solution. First of all let’s look on base two interfaces:

internal interface RxTemplate<Actual, T> {
val actual: Actual
fun doOnNext(onNext: (T) -> Unit): RxTemplate<Actual, T>
fun onErrorResumeNext(onResumeNext: (Throwable, RxTemplateFactory<Actual, T>) -> RxTemplate<Actual, T>): RxTemplate<Actual, T>
}
internal interface RxTemplateFactory<Actual, T> {
fun just(item: T): RxTemplate<Actual, T>
fun error(thr: Throwable): RxTemplate<Actual, T>
fun defer(supplier: (RxTemplateFactory<Actual, T>) -> RxTemplate<Actual, T>): RxTemplate<Actual, T>
fun create(actual: Actual): RxTemplate<Actual, T>
}

First interface represents generalization of , or and second represent their creation methods. I must admit, that those interfaces and my solution are far from ideal. At the minimum, there is a question why we need to to say factory the type of item ? Don’t we may just to have one factory for every doesn't matter about type ?

I can say, that yes, I need to work on that interfaces and more proper generalization. But for now we have that approach, and it works and suits our needs in article. That’s why I made them .

Now let’s look an implementations for and for :

And now we need to rewrite our method:

Basically nothing changed except adding rx templates instead of hardcoded

Now thats all. Happy coding! :)

Love being creative to solve some problems with an simple and elegant ways