Gson Unsafe Problem

And how to invent a workaround for that

I love Gson library. It’s very small and stable in terms of parsing Json objects. We can talk about advantages of using Gson pretty long and compare it with other libraries like Jackson. But it is a topic of another article. I want to discuss a one problem that Gson have. More precise, this problem with not Gson (as lack of something or library bug), but not ability of library to work with custom creators of objects (like invoking specific constructors and e.t.c). I’ve seen much of questions about that in stackoverflow (link to example), so I think it is pretty insteresting and needed topic. So let’s try to undestand what’s happening and how we can resolve or try to resolve this problem.

The Problem

Example

So the problem appears when you have a class with fields, which have a default values, and you use that class to create object and parse Json data in it. So for example, we have a Parameter.class:

public class Parameter {
    private final int id;
    private final String name;
    private boolean isRequired = true;

    public Parameter(int id, String name) {
        this.id = id;
        this.name = name;
    }

    public int getId() {
        return id;
    }

    public String getName() {
        return name;
    }

    public boolean isRequired() {
        return isRequired;
    }
}

Then we retrieve from server a json string: json = “{\”id\”: 1, \”name\”: \”something\”}” and parse it through: Parameter param = new Gson().fromJson(json, Parameter.class). As we might naively suggest param.isReuired() will return true, since server didn’t pointed exact value for isRequired and we need to initialize it with default value, which is true.

Unfortunately, we will retrieve false. Why? How? For what?

Description

This happens because of using java.unsafe package to construct of object. You see, Gson, when it tries to create an instance of target class, firstly tries to find an empty constructor and if there is no one, it will use UnsafeAllocator to create an object. Unsafe is a dark side of java programming. It is a place for the dark programmers who don’t be afraid to be pursued by good programmers and punished by them. It’s very dangerous and mysterious place.

When object is created by Unsafe mechanism it will not created through any constructors at all, and that’s why fields not initialized with default values. So every boolean will be false, and every complex field will be null, no matter initialized you it or not.

And it can become really pain in terms Kotlin’s data classes. For example:

data class Car(
        val id: Int,
        val name: String,
        val isPopular: Boolean = true,
        val brand: Brand = UnknownBrand()
)

We can not be sure that third and fourth field will be properly initialized when we will retrieve data from server. And if isPopular will just return wrong value (which is of course bad), then invoking car.brand will produce NPE which is unacceptable and is lack of type null safety.

Other example is model with system fields, like:

data class Cars(
        private val data: List<Car>
) {
    val sorted: List<Car> by lazy { data.sortedBy { it.name } }
}

In this example, sorted is a special system field with predefined behavior. It show sort data list in first invocation and then return it. But when we parse a server string with data and then call `cars.sorted` we will receive NPE.

Popular Possible Way

One of the simplest and as I saw most popular solution is to add an empty constructor for your model. Thus, Gson will use it to produce an instance of target class and then fill it with server data. So all additional fields will be initialized.

This is a pretty good solution in terms of not changing Gson to Jackson :). It is fast, small and matches for most of cases. But it have serious problem, when you have data validation in client side (and besides, when you writing your models as Kotlin’s data classes, this solution spoils their design).

Data validation is a special mechanic for checking consistency of data returning from server. For example, you have aforementioned Car.class and you receive from the server json = “{\”id\”: 3}”. If you will not have validation mechanism, you will successfully construct a Car’s instance and you will receive NPE when invoke: `cat.name`. Because of no direct link between NPE and wrong server answer you will spend additional time to find out why this happening.

As opposite, with validation mechanism you will find out that server retrieved not full data and you will be able to handle it, for example send an error message from you api layout to view and show retry button to the user, for example.

Now lets see once again on Car.class but now with default constructor:

data class Car(
        val id: Int,
        val name: String,
        val isPopular: Boolean = true,
        val brand: Brand = UnknownBrand()
) {
    private constructor() : this(0, "") //or other default values
}

We can not pass to primary constructor null values, so we can not now check that data from server didn’t get received, and thus our validator will be not working. So, how we can resolve that now?

Solution

What’s happening in Jackson

In Jackson there is a pretty good solution for such cases. It has two special annotations: @JsonCreator and @JsonProperty

Here is an example how you can use them (link to article):

public class NonDefaultBean {
    private final String name;
    private final int age;

    private String type;

    @JsonCreator
    public NonDefaultBean(@JsonProperty("name") String name, @JsonProperty("age") int age)
    {
        this.name = name;
        this.age = age;
    }

    public void setType(String type) {
        this.type = type;
    }
}

In this case Unsafe will not be used. Instead of that bad approach (here example link, why is that) a normal constructor NonDefaultBean(String name, int age) will be invoked.

We don’t know how it is working inside a Jackson and this is not so important. Main thing: we want same behavior by using Gson, which is unfortunately can call only non arguments constructors by default or use Unsafe.

Main Solution Frame

So Gson has one strong thing that allows you customize (de)serialization process. It is a type adapters. We can use that approach for our purposes. Base solution in that case is:

class ConstructorInvocationDeserializer<T>(
        private val fields: Array<String>
) : JsonDeserializer<T> {
    override fun deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): T {
        val clazz = TypeToken.get(typeOfT).rawType
        var args: Array<Any?> = arrayOf()
        val appropriateConstructor = clazz.constructors
                .filter { it.parameterTypes.size == fields.size }
                .find {
                    try {
                        args = deserializeEach(json!!.asJsonObject, it.parameterTypes, context)
                        true
                    } catch (e: JsonSyntaxException) {
                        false
                    }
                }
        return appropriateConstructor!!.newInstance(*args) as T
    }

    @Throws(JsonSyntaxException::class)
    private fun deserializeEach(json: JsonObject, fieldTypes: Array<Class<*>>, context: JsonDeserializationContext): Array<Any?> {
        val result = Array<Any?>(fields.size, {null})
        fields.forEachIndexed { index, name ->
            result[index] = context.deserialize(json.get(name), fieldTypes[index])
        }
        return result
    }
}

Now if we remember above Car.class, we just need to register type adapter for it like:

val gson = GsonBuilder()
        .registerTypeAdapter(Car::class.java, ConstructorInvocationDeserializer<Car>(arrayOf("id", "name")))
        .create()

Also, we will have to mark primary constructor of Car.class with @JvmOverloads annotation in order to produce 3 different constructors for java, and, thus, Gson will be able to recognize needed constructor Car(int id, String name)

That’s all.

Problems with that solution

This solutions works in the way to find appropriate constructor and just create an object with it, nothing more. So if Json has more field for target type, they will not be affected to this object. Moreover, for this solution we will need to register such adapter for any type that should use such functionality (creating from constructor). Those we have at least 3 main problems:

1. Necessity to register Adapter for each new type (you may forget to do that and thus the bug will be produced)
2. Lack of constructor variations. If we have a model with 2 or 3 constructors. How we will tell to our adapter to find appropriate constructor according to received Json
3. Fields injected only through constructor. So if Json has any additional fields they will be not applicable for created object, that has appropriate fields but not in the constructor.

I think all that 3 problems we can solve, but main point was to show main skeleton of solution. If you will like that article I will try to write a continuation with optimizations and enhancements for this solution core to transform it from core to fully workable solution.

Thank you.