Akka Http Error: InvalidContentLengthException - scala

Question

We have an Akka HTTP Server serving some scala JS content on AWS. We've noticed that after some time, the server starts throwing the error below in the logs and although, the certain URL's work, but some files fail to be downloaded correctly with the Failed to load resource: net::ERR_CONTENT_LENGTH_MISMATCH error in the browser. The server logs looks like below (it's exactly the same error in all instances):
[ERROR] [09/29/2016 21:29:22.150] [designer-actor-system-akka.actor.default-dispatcher-56831] [akka.actor.ActorSystemImpl(designer-actor-system)] Outgoing response stream error
akka.http.scaladsl.model.InvalidContentLengthException: HTTP message had declared Content-Length 997 but entity data stream amounts to 164 bytes less
at akka.http.scaladsl.model.InvalidContentLengthException$.apply(ErrorInfo.scala:50)
at akka.http.impl.engine.rendering.RenderSupport$CheckContentLengthTransformer$$anon$2.onUpstreamFinish(RenderSupport.scala:130)
at akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:732)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:616)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:471)
at akka.stream.impl.fusing.GraphInterpreterShell.receive(ActorGraphInterpreter.scala:433)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:603)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:618)
at akka.actor.Actor$class.aroundReceive(Actor.scala:484)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:529)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
We are running Akka version 2.4.7 and OpenJDK version 1.8.0_101-b13.The error goes away if we restart the Akka HTTP server, but comes back again after a few hours.
Not sure what is causing the issue. Any help would be greatly appreciated.

Answer 1

I think I figured out what was causing it. We had an automated deployment process that deploys HTML/CSS/Javascript files to the Akka server. It seems like the Akka server caches content size and if the static files are updated without restarting, it would give InvalidContentLengthException. We included the restart as part of deployment process and the problem seems to have been resolved.