How to speed up sbt scala build in docker? - scala

Question

I have docker file with:
FROM amazonlinux:2017.03
ENV SCALA_VERSION 2.11.8
ENV SBT_VERSION 0.13.13
# Install Java8
RUN yum install -y java-1.8.0-openjdk-devel
# Install Scala and SBT
RUN yum install -y https://downloads.lightbend.com/scala/2.11.8/scala-2.11.8.rpm
RUN yum install -y https://dl.bintray.com/sbt/rpm/sbt-0.13.13.rpm
RUN sbt sbtVersion
COPY . /root
WORKDIR /root
# Exposing port 80
EXPOSE 80
RUN sbt compile
CMD sbt run
And sbt configuration file with:
name := "hello"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies += "com.fasterxml.jackson.core" % "jackson-databind" % "2.9.5"
libraryDependencies += "com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.9.5"
Each time when I build docker container sbt download jackson library anew. How can I speed up this process, may execute part of sbt file before compilation.
Before I have add RUN sbt sbtVersion to Dockerfile sbt have download itself completely and after I add this command have cached and not run each time when I build docker container.
May be there is same tricks with caching in docker downloading libraries in sbt?

Answer 1

First, you don't need to install the scala RPM as SBT itself downloads Scala for you (whichever version is configured in your build).
Second, each RUN command creates a new layer which you want to avoid usually. Combine them:
RUN cmd1 \
&& cmd2 \
&& cmd3
Why do you want to build an image for each of your builds? That seems wasteful. Usually you build your stuff outside of a Docker image and only package up the results.
My advice would be to use the sbt-native-packager SBT plugin with its Docker integration to simply build an image from your artefacts after you have build them. That way you would only need a JRE in your image, not a JDK, not SBT. Also, you would not need to wait for SBT to initialize when starting your image.
You could use multi-stage builds if you have a new Docker version installed.

Answer 2

You can reference to travis's solution:
.travis.yaml:
# These directories are cached to S3 at the end of the build
cache:
directories:
- $HOME/.ivy2/cache
- $HOME/.sbt/boot/
Travis will backup the above two folders to s3, then every time before user rebuild, it will fetch the two folders from cache.
So, we can know the possible way for you to not download jackson library again could be separate the .ivy2 & .sbt folder and put them in something like a cache.
For your situation, I think the best solution is store these in a base image, the base image's Dockerfile could simply same as the one you are using now(of course, you can simple it, just add jackson library define in build.sbt, the aim just want the .ivy2 have the jackson library), then E.g. tag it as mybaseimage:v1
Then your new project's Dockerfile could use base mybaseimage:v1, as mybaseimage already have the library in .ivy2& .sbt, so no need to download jackson again every time you build your project's Dockerfile.
The solution maybe ugly, but I think could act as a workaround, just FYI.