Gson with Scala causes StackOverflow for Enumerations - scala

Question

I have an enum defined in Scala class as follows
// define compression types as enumerator
object CompressionType extends Enumeration
{
type CompressionType = Value
val None, Gzip, Snappy, Lz4, Zstd = Value
}
and I have class that I want to Serialize in JSON
case class ProducerConfig(batchNumMessages : Int, lingerMs : Int, messageSize : Int,
topic: String, compressionType: CompressionType.Value )
That class includes the Enum object. It seems that using GSON to serialize causes StackOverflow due to some circular dependency.
val gson = new Gson
val jsonBody = gson.toJson(producerConfig)
println(jsonBody)
Here is the stack trace I get below. I saw this question here and answer except the solution seems to be Java solution and didn't work for scala. Can someone clarify?
17:10:04.475 [ERROR] i.g.a.Gatling$ - Run crashed
java.lang.StackOverflowError: null
at com.google.gson.stream.JsonWriter.beforeName(JsonWriter.java:617)
at com.google.gson.stream.JsonWriter.writeDeferredName(JsonWriter.java:400)
at com.google.gson.stream.JsonWriter.value(JsonWriter.java:526)
at com.google.gson.internal.bind.TypeAdapters$7.write(TypeAdapters.java:233)
at com.google.gson.internal.bind.TypeAdapters$7.write(TypeAdapters.java:218)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:127)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:245)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:127)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:245)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:127)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:245)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:127)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:245)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:127)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:245)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:127)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:245)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:127)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:245)

Answer 1

I'm not a Scala guy but I think Gson is a wrong tool to use here.
Firstly, Gson is not aware of scala.Enumeration therefore handling it as a regular data bag that's traversable using reflection.
Secondly, there is no an easy (if any?) way of deserializing to the original value state (can be ignored if you're going only to produce, not consume, JSON documents).
Here is why:
object Single
extends Enumeration {
val Only = Value
}
final class Internals {
private Internals() {
}
static void inspect(final Object o, final Excluder excluder, final boolean serialize)
throws IllegalAccessException {
inspect(o, clazz -> !excluder.excludeClass(clazz, serialize), field -> !excluder.excludeField(field, serialize));
}
static void inspect(final Object o, final Predicate<? super Class<?>> inspectClass, final Predicate<? super Field> inspectField)
throws IllegalAccessException {
for ( Class<?> c = o.getClass(); c != null; c = c.getSuperclass() ) {
if ( !inspectClass.test(c) ) {
continue;
}
System.out.println(c);
for ( final Field f : c.getDeclaredFields() ) {
if ( !inspectField.test(f) ) {
continue;
}
f.setAccessible(true);
System.out.printf("\t%s: %s\n", f, f.get(o));
}
}
}
}
final Object value = Single.Only();
Internals.inspect(value, gson.excluder(), true);
produces:
class scala.Enumeration$Val
private final int scala.Enumeration$Val.i: 0
private final java.lang.String scala.Enumeration$Val.name: null
class scala.Enumeration$Value
private final scala.Enumeration scala.Enumeration$Value.scala$Enumeration$$outerEnum: Single
class java.lang.Object
As you can see, there are two crucial fields:
private final java.lang.String scala.Enumeration$Val.name gives null unless named (the enumeration element can be obtained using toString though).
private final scala.Enumeration scala.Enumeration$Value.scala$Enumeration$$outerEnum is actually a reference to the concrete enumeration outer class (that's actually the cause of the infinite recursion and hence stack overflow error).
These two prevent from proper deserialization.
The outer enum type can be obtained in at least three ways:
either implement custom type adapters for all types that can contain such enumerations (pretty easy for data bags (case classes in Scala?) as fields already contain the type information despite Gson provides poor support of this; won't work for single primitive literals like the above or collections);
or bake the outer enumeration name to JSON holding two entries for the name and outer type.
The latter could be done like this (in Java, hope it's easy to simplify it in Scala):
final class ScalaStuff {
private static final Field outerEnumField;
private static final Map<String, Method> withNameMethodCache = new ConcurrentHashMap<>();
static {
try {
outerEnumField = Enumeration.Value.class.getDeclaredField("scala$Enumeration$$outerEnum");
outerEnumField.setAccessible(true);
} catch ( final NoSuchFieldException ex ) {
throw new RuntimeException(ex);
}
}
private ScalaStuff() {
}
#Nonnull
static String toEnumerationName(#Nonnull final Enumeration.Value value) {
try {
final Class<? extends Enumeration> aClass = ((Enumeration) outerEnumField.get(value)).getClass();
final String typeName = aClass.getTypeName();
final int length = typeName.length();
assert !typeName.isEmpty() && typeName.charAt(length - 1) == '$';
return typeName.substring(0, length - 1);
} catch ( final IllegalAccessException ex ) {
throw new RuntimeException(ex);
}
}
#Nonnull
static Enumeration.Value fromEnumerationValue(#Nonnull final String type, #Nonnull final String enumerationName)
throws ClassNotFoundException, NoSuchMethodException {
// using get for exception propagation cleanliness; computeIfAbsent would complicate exception handling
#Nullable
final Method withNameMethodCandidate = withNameMethodCache.get(type);
final Method withNameMethod;
if ( withNameMethodCandidate != null ) {
withNameMethod = withNameMethodCandidate;
} else {
final Class<?> enumerationClass = Class.forName(type);
withNameMethod = enumerationClass.getMethod("withName", String.class);
withNameMethodCache.put(type, withNameMethod);
}
try {
return (Enumeration.Value) withNameMethod.invoke(null, enumerationName);
} catch ( final IllegalAccessException | InvocationTargetException ex ) {
throw new RuntimeException(ex);
}
}
}
final class ScalaEnumerationTypeAdapterFactory
implements TypeAdapterFactory {
private static final TypeAdapterFactory instance = new ScalaEnumerationTypeAdapterFactory();
private ScalaEnumerationTypeAdapterFactory() {
}
static TypeAdapterFactory getInstance() {
return instance;
}
#Override
#Nullable
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
if ( !Enumeration.Value.class.isAssignableFrom(typeToken.getRawType()) ) {
return null;
}
#SuppressWarnings("unchecked")
final TypeAdapter<T> typeAdapter = (TypeAdapter<T>) Adapter.instance;
return typeAdapter;
}
private static final class Adapter
extends TypeAdapter<Enumeration.Value> {
private static final TypeAdapter<Enumeration.Value> instance = new Adapter()
.nullSafe();
private Adapter() {
}
#Override
public void write(final JsonWriter out, final Enumeration.Value value)
throws IOException {
out.beginObject();
out.name("type");
out.value(ScalaStuff.toEnumerationName(value));
out.name("name");
out.value(value.toString());
out.endObject();
}
#Override
public Enumeration.Value read(final JsonReader in)
throws IOException {
in.beginObject();
#Nullable
String type = null;
#Nullable
String name = null;
while ( in.hasNext() ) {
switch ( in.nextName() ) {
case "type":
type = in.nextString();
break;
case "name":
name = in.nextString();
break;
default:
in.skipValue();
break;
}
}
in.endObject();
if ( type == null || name == null ) {
throw new JsonParseException("Insufficient enum data: " + type + ", " + name);
}
try {
return ScalaStuff.fromEnumerationValue(type, name);
} catch ( final ClassNotFoundException | NoSuchMethodException ex ) {
throw new JsonParseException(ex);
}
}
}
}
The following JUnit 5 test will passed:
private static final Gson gson = new GsonBuilder()
.disableHtmlEscaping()
.registerTypeAdapterFactory(ScalaEnumerationTypeAdapterFactory.getInstance())
.create();
#Test
public void test() {
final Enumeration.Value before = Single.Only();
final String json = gson.toJson(before);
System.out.println(json);
final Enumeration.Value after = gson.fromJson(json, Enumeration.Value.class);
Assertions.assertSame(before, after);
}
where the json variable would hold the following JSON payload:
{"type":"Single","name":"Only"}
The ScalaStuff class above is most likely not complete. See more at how to deserialize a json string that contains ## with scala' for Scala and Gson implications.
Update 1
Since you don't need to consume the produced JSON documents assuming the JSON consumers can deal with the enumeration deserialization themselves, you can produce an enumeration value name that's more descriptive than producing nameless ints. Just replace the Adapter above:
private static final class Adapter
extends TypeAdapter<Enumeration.Value> {
private static final TypeAdapter<Enumeration.Value> instance = new Adapter()
.nullSafe();
private Adapter() {
}
#Override
public void write(final JsonWriter out, final Enumeration.Value value)
throws IOException {
out.value(value.toString());
}
#Override
public Enumeration.Value read(final JsonReader in) {
throw new UnsupportedOperationException();
}
}
Then following test will be green:
Assertions.assertEquals("\"Only\"", gson.toJson(Single.Only()));