Kryo serialisation in Chronicle Map - slow byte reading - scala

Question

I'm using Chronicle Map extensively in Scala and recently decided to experiment with Kryo serialisation. I added custom marshallers (code below) and while it reduced the size of my stores by 14G (roughly 62%), and everything still works, the speed was unacceptable.
I created a sample usecase and did a few runs on the same data
[Using kryo] took 6883, and then 7187, 6954, 7225, 13051
[Not using kryo] took 2326, and then 1352, 1493, 1500, 1187
So it's several times slower. Here's the marshaller for reading:
class KryoMarshallerReader[T] extends BytesReader[T] {
val kryo = // Reference to KryoPool from Twitter's Chill library
override def read(in: Bytes[_], using: T): T = {
val bytes = benchmark("array allocation") {
new Array[Byte](in.readRemaining().toInt)
}
benchmark("reading bytes") {
in.read(bytes)
}
benchmark("deserialising") {
kryo.fromBytes(bytes).asInstanceOf[T]
}
}
override def readMarshallable(wire: WireIn): Unit = {}
override def writeMarshallable(wire: WireOut): Unit = {}
}
I then averaged execution times (all in ms) over those three stages and realised that reading the bytes is the slowest:
stage Average time (ms)
(fctr) (dbl)
1 [array allocation] 0.9432907
2 [deserialising] 0.9944112
3 [reading bytes] 13.2367265
Now the question is - what am I doing wrong?
I looked through the interface of Bytes[_] and it looks like it reads bytes one by one - is there a way to use a buffer or something magically capable of bulk loading?
Update: Eventually I changed array allocation + reading bytes to in.toByteArray but it's still slow because under the hood it copies bytes one by one. Just running reads on the map shows that byte reading is the bottleneck:

Answer 1

in.readRemaining() of the Bytes, passed into BytesReader.read(), is not your object in serialized form, it's more than that. The serialized form of your object is guaranteed to start from in.readPosition(), but it typically end much earlier than in.readLimit() (as readRemaining() = readLimit() - readPosition()). Generally BytesReader/BytesWriter pair of implementations should care of determining the end of the object bytes itself (if it needs), e. g. see the implementation of CharSequenceArrayBytesMarshaller in BytesReader and BytesWriter section of the Chronicle Map tutorial:
public final class CharSequenceArrayBytesMarshaller
implements BytesWriter<CharSequence[]>, BytesReader<CharSequence[]> {
...
#Override
public void write(Bytes out, #NotNull CharSequence[] toWrite) {
out.writeInt(toWrite.length); // care about writing the size ourselves!
...
}
#NotNull
#Override
public CharSequence[] read(Bytes in, #Nullable CharSequence[] using) {
int len = in.readInt(); // care about reading the size ourselves!
...
}
}
But since your are implementing Kryo serialization which should be conceptually similar to Java standard serialization, you should probably take the source of SerializableReader and SerializableDataAccess and modify it to use Kryo instead of standard Java serialization (but note that those sources are LGPLv3 licensed). In particular those implementations use Bytes.inputStream() and Bytes.outputStream() to bridge the standard Java serialization, which doesn't know about Bytes, but knows about InputStream/OutputStream, without needlessly copying bytes. I'm pretty sure Kryo supports InputStream/OutputStream as well.
Be very careful with kryo as an instance field of any serializer interface (BytesReader in your case) without implementing StatefulCopyable. You may easily introduce concurrency bottlenecks, or concurrency bugs (data races). Check Understanding StatefulCopyable section in the Chronicle Map tutorial and Chronicle Map custom serialization checklist.