I am trying to run Scala application using docker. I've created a sample project with the following structure.
build.sbt
name := "test"
version := "1.0"
scalaVersion := "2.12.2"
project/build.properties
sbt.version = 0.13.15
src/main/scala/HelloWorld.scala
object HelloWorld {
def main(args: Array[String]): Unit = {
println("Hello, world!")
}
}
Now if I do sbt run, everything works fine and I have the lovely hello-world greeting.
$ sbt run
...
[info] Set current project to test (in build file:/Users/yuchen/Documents/test/)
[info] Running HelloWorld
Hello, world!
[success] Total time: 1 s, completed 22-May-2017 4:30:28 PM
I aded a Dockerfile:
FROM openjdk:8
ENV SCALA_VERSION 2.12.2
ENV SBT_VERSION 0.13.15
RUN touch /usr/lib/jvm/java-8-openjdk-amd64/release
RUN \
curl -fsL http://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz | tar xfz - -C /root/ && \
echo >> /root/.bashrc && \
echo 'export PATH=~/scala-$SCALA_VERSION/bin:$PATH' >> /root/.bashrc
RUN \
curl -L -o sbt-$SBT_VERSION.deb http://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb && \
apt-get update && \
apt-get install sbt && \
sbt sbtVersion
WORKDIR /root
CMD sbt run
They are all copied from https://hub.docker.com/r/hseeberger/scala-sbt/~/dockerfile/ and with the additional last line CMD sbt run.
And I tried running docker commands:
docker build -t test .
docker tag test test/test:1.0
docker run test/test:1.0
However, I keep getting this error:
[info] Set current project to root (in build file:/root/)
[info] Updating {file:/root/}root...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
java.lang.RuntimeException: No main class detected.
at scala.sys.package$.error(package.scala:27)
[trace] Stack trace suppressed: run last compile:run for the full output.
[error] (compile:run) No main class detected.
[error] Total time: 0 s, completed May 23, 2017 4:00:19 AM
What does it mean and how do I resolve this?
FYI: I am aware of http://www.scala-sbt.org/sbt-native-packager/formats/docker.html, but just want to try writing Dockerfile as I am learning this.
I don't really know sbt. But this appears that you are not copying the project into the image (there is no COPY or ADD in the Dockerfile).
Try doing something like:
FROM openjdk:8
ENV SCALA_VERSION 2.12.2
ENV SBT_VERSION 0.13.15
RUN touch /usr/lib/jvm/java-8-openjdk-amd64/release
RUN \
curl -fsL http://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz | tar xfz - -C /root/ && \
echo >> /root/.bashrc && \
echo 'export PATH=~/scala-$SCALA_VERSION/bin:$PATH' >> /root/.bashrc
RUN \
curl -L -o sbt-$SBT_VERSION.deb http://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb && \
apt-get update && \
apt-get install sbt && \
sbt sbtVersion
WORKDIR /myapp
CMD sbt run
COPY . /myapp
The addition here is just a COPY to copy all the code to /myapp and to changing the working directory to this folder as well. This would copy the build context to /myapp and then the WORKDIR should run sbt run from the same folder.
Related
I am deploying Jenkins Using GitHub Action Runner using Skaffold.
While the Skaffold is installed over the default image of GitHub Runner
The pod is restarting due to crash loop back off error and causing it to restart.
I am not sure why it is happening.
When I am deploying runner over Google Kubernetes Engine my runner is failing because of following error:
'''A runner exists with the same name
√ Successfully replaced the runner
√ Runner connection is good
# Runner settings
√ Settings Saved.
√ Connected to GitHub
Current runner version: '2.294.0'
2022-12-01 06:03:57Z: Listening for Jobs
Runner update in progress, do not shutdown runner.
Downloading 2.299.1 runner
Waiting for current job finish running.
Generate and execute update script.
Runner will exit shortly for update, should be back online within 10 seconds.
Runner update process finished.
Runner listener exit because of updating, re-launch runner in 5 seconds
Restarting runner...
/home/docker/actions-runner/run-helper.sh: line 20: /home/docker/actions-runner/bin/Runner.Listener: No such file or directory
Exiting with unknown error code: 127
Exiting runner...
'''
Following is the Dockerfile used for runner :
'''FROM ubuntu:22.04
#instalIng skaffold
RUN apt-get update -y && apt-get upgrade -y sudo
RUN apt-get install -y curl
RUN curl -LO https://storage.googleapis.com/skaffold/releases/v2.0.2/skaffold-linux-amd64 \
&& sudo chmod +x skaffold-linux-amd64 \
&& sudo mv skaffold-linux-amd64 /usr/local/bin/skaffold
#install
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update -y && apt-get upgrade -y && useradd -m docker
RUN apt-get install -y curl jq build-essential libssl-dev libffi-dev python3 python3-venv python3-dev ca-certificates gnupg2 iputils-ping software-properties-common apt-transport-https lsb-release git zip unzip postgresql-client python3-pip npm
RUN ln -sf /usr/bin/python3 /usr/bin/python
# set the github runner version
ARG RUNNER_VERSION="2.294.0"
# cd into the user directory, download and unzip the github actions runner
RUN cd /home/docker && mkdir actions-runner && cd actions-runner \
&& curl -O -L https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz \
&& tar xzf ./actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
# install some additional dependencies
RUN chown -R docker ~docker && /home/docker/actions-runner/bin/installdependencies.sh
#Docker
RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - && \
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable" && \
apt-get update && \
apt-get -y install docker-ce
# Downloading gcloud package
RUN curl https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz > /tmp/google-cloud-sdk.tar.gz
# Installing the package
RUN mkdir -p /usr/local/gcloud \
&& tar -C /usr/local/gcloud -xvf /tmp/google-cloud-sdk.tar.gz \
&& /usr/local/gcloud/google-cloud-sdk/install.sh
# Adding the package path to local
ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin
#Install Kubectl
RUN curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl" \
&& chmod +x ./kubectl \
&& mv ./kubectl /usr/local/bin/kubectl
# copy over the start.sh script
COPY start.sh start.sh
# make the script executable
RUN chmod +x start.sh && mv start.sh /home/docker
# since the config and run script for actions are not allowed to be run by root,
# set the user to "docker" so all subsequent commands are run as the docker user
USER docker
# set the entrypoint to the start.sh script
ENTRYPOINT ["/home/docker/start.sh"] '''
Below is the startup script :
#!/bin/bash
SNAPTIME=`date '+%Y%m%d%H%M%S'`
echo "Started $SNAPTIME"
ORGANIZATION=$ORGANIZATION
ACCESS_TOKEN=`cat /etc/pat/pat`
GH_PROJECT=$GH_PROJECT
RUNNER_NAME="${RUNNER_NAME:-RUN$SNAPTIME}"
RUNNER_LABELS="${RUNNER_LABELS:-simple}"
REG_TOKEN=$(curl -sX POST -H "Authorization: token ${ACCESS_TOKEN}" https://api.github.com/repos/${ORGANIZATION}/$GH_PROJECT/actions/runners/registration-token | jq .token --raw-output)
# gcloud auth activate-service-account --key-file=${GOOGLE_APPLICATION_CREDENTIALS}
cd /home/docker/actions-runner
./config.sh --name $RUNNER_NAME --labels ${RUNNER_LABELS} --url https://github.com/${ORGANIZATION}/${GH_PROJECT} --unattended --replace --token ${REG_TOKEN}
cleanup() {
echo "Removing runner..."
./config.sh remove --unattended --token ${REG_TOKEN}
}
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM
./run.sh & wait $!
The pod is restarting whenever the load is coming into it:
runner-automation-dev-docker-595f48c7dc-k2wbz 1/2 CrashLoopBackOff 7 (67s ago) 18m
I am not sure what exactly is causing this issue.
I am currently constructing a Makefile and one of the things it will be doing is downloading and building postgres from source. Before starting to write the makefile file I always used the following set of commands to do this:
curl -LJ https://github.com/postgres/postgres/archive/refs/tags/REL_13_3.zip -o postgres.zip
unzip postgres.zip
rm postgres.zip
cd postgres-REL_13_3
./configure --prefix "`pwd`" --without-readline --without-zlib
make
make install
Executing the above listed commands in the terminal results in the successful installation of postgres. Then I translated these into a makefile which looks as follows:
build:
curl -LJ https://github.com/postgres/postgres/archive/refs/tags/REL_13_3.zip -o postgres.zip
unzip postgres.zip
rm postgres.zip
cd postgres-REL_13_3 \
&& ./configure --prefix "`pwd`" --without-readline --without-zlib \
&& $(MAKE) \
&& $(MAKE) install
Running this Makefile results in the error:
../../src/include/utils/elog.h:71:10: fatal error: 'utils/errcodes.h' file not found
It seems that something about calling the make from another Makefile causes a referencing issue with the files during the build process, but I just can figure out for the life of me what I have to change to fix this.
It appears to be the influence of Makefile enviromental variables. I didn't discover the exact mechanism, but unsetting them helps.
build:
curl -LJ https://github.com/postgres/postgres/archive/refs/tags/REL_13_3.zip -o postgres.zip
unzip postgres.zip
rm postgres.zip
unset MAKELEVEL && unset MAKEFLAGS && unset MFLAGS && cd postgres-REL_13_3 \
&& ./configure --prefix "`pwd`" --without-readline --without-zlib \
&& $(MAKE) \
&& $(MAKE) install
Trying to execute integration tests using docker.
Below is the Dockerfile:
FROM openjdk:8u232
ARG SBT_VERSION=1.2.8
# Install sbt
RUN \
curl -L -o sbt-$SBT_VERSION.deb https://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb && \
apt-get update && \
apt-get install sbt && \
sbt sbtVersion
ENV WORK_DIR="/test"
USER root
COPY . ${WORK_DIR}/
RUN cd ${WORK_DIR}
RUN sbt clean it:test
Below is the docker-compose.yml:
version: '3.7'
services:
test:
build:
context: ../..
dockerfile: Dockerfile.test
restart: always
when I run docker-compose -f test.yaml up :
it gives me error:
Step 10/10 : RUN sbt clean it:test
---> Running in d9513c01439b
[warn] No sbt.version set in project/build.properties, base directory: /
[info] Set current project to root (in build file:/)
[success] Total time: 0 s, completed Jun 6, 2020 7:20:37 AM
[error] Expected ';'
[error] No such setting/task
[error] it:test
to check it i removed this line from docker file and built the image and exec into container and my tests ran fine.
The structure of my project is:
src
main
it
test
I tried executing the unit tests as well
but it gave me:
Step 10/10 : RUN sbt clean test
---> Running in c300fcf2bf53
[warn] No sbt.version set in project/build.properties, base directory: /
[info] Set current project to root (in build file:/)
[success] Total time: 0 s, completed Jun 6, 2020 7:19:13 AM
[success] Total time: 10 s, completed Jun 6, 2020 7:19:24 AM
Removing intermediate container c300fcf2bf53
---> c0215c62663d
Successfully built c0215c62663d
Successfully tagged it_it:latest
Creating it_it_1 ... done
Attaching to it_it_1
when i changed it to:
Step 10/10 : CMD sbt clean it:test
---> Running in 719b239fb7e4
Removing intermediate container 719b239fb7e4
---> f7adb19f2cb6
Successfully built f7adb19f2cb6
Successfully tagged it_it:latest
Creating it_it_1 ... done
Attaching to it_it_1
it_1 | [warn] No sbt.version set in project/build.properties, base directory: /
it_1 | [info] Set current project to root (in build file:/)
[success] Total time: 0 s, completed Jun 6, 2020 7:32:24 AM
[error] Expected ';'
it_1 | [error] No such setting/task
it_1 | [error] it:test
it_1 | [error]
How can i execute sbt it:test from dockerfile using compose?
If it was working inside the container then this must work:
ENTRYPOINT cd ${WORK_DIR} && \
sbt clean it:test
Change to next:
RUN cd ${WORK_DIR} && sbt clean it:test
Different RUN in dockerfile won't affect eachother.
I have an sbt project producing my artifact xyz.
I would like to put it along with all its dependencies in the docker container so it can be used using
coursier launch --mode offline xyz
The important part is that preparation should take use of local cursier cache from host.
I tried
executing sbt publishLocal,
then resolving my artifact dependencies (cursier resolve xyz),
then preparing to directories - local & cache - by copying resolved artifact into them
then copying those directories into docker container (as coursier cache and ivy local respectively).
This didn't work because coursier doesn't list .pom and .xml files in its output. I tried copying whole directories (abc/1.0.0 instead of abc/1.0.0/some.jar) but AFAIK there is no reliable way to know how many folders up one has to go because maven and ivy have different dir structures.
while my usecase is not quite identical to yours -- I figure I'd write up my findings and maybe my solution works for you as well!
here's my sample dockerfile, I used this to install scalafmt in an offline-compatible way
FROM ubuntu:jammy
RUN : \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates \
curl \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* # */ stackoverflow highlighting bug
ARG CS=v2.1.0-RC4
ARG CS_SHA256=176e92e08ab292531aa0c4993dbc9f2c99dec79578752f3b9285f54f306db572
ARG JDK_SHA256=aef49cc7aa606de2044302e757fa94c8e144818e93487081c4fd319ca858134b
ENV PATH=/opt/coursier/bin:$PATH
RUN : \
&& curl --location --silent --output /tmp/cs.gz "https://github.com/coursier/coursier/releases/download/${CS}/cs-x86_64-pc-linux.gz" \
&& echo "${CS_SHA256} /tmp/cs.gz" | sha256sum --check \
&& curl --location --silent --output /tmp/jdk.tgz "https://download.java.net/openjdk/jdk17/ri/openjdk-17+35_linux-x64_bin.tar.gz" \
&& echo "${JDK_SHA256} /tmp/jdk.tgz" | sha256sum --check \
&& mkdir -p /opt/coursier \
&& tar --strip-components=1 -C /opt/coursier -xf /tmp/jdk.tgz \
&& gunzip /tmp/cs.gz \
&& mv /tmp/cs /opt/coursier/bin \
&& chmod +x /opt/coursier/bin/cs \
&& rm /tmp/jdk.tgz
ENV COURSIER_CACHE=/opt/.cs-cache
RUN : \
&& cs fetch scalafmt:3.6.1 \
&& cs install scalafmt:3.6.1 --dir /opt/wd/bin
the key to offline execution for me was to use cs fetch and set COURSIER_CACHE
here's the offline execution succeeding:
$ docker run --net=none --rm -ti cs /opt/wd/bin/scalafmt --version
scalafmt 3.6.1
I am building docker image of my application. And I would like to run jar file when I run my docker image. However, I get this error:
Could not find or load main class
Main class is set in the manifest file of the jar file. If I run my jar file from terminal or bash script it works fine. So this error is only observed while running docker:
docker run -v my-volume:/workdir container-name
Are there some configurations missing in my Dockerfile or jar file should be copied/added?
Here is my Dockerfile:
FROM java:8
ENV SCALA_VERSION 2.11.8
ENV SBT_VERSION 1.1.1
ENV SPARK_VERSION 2.2.0
ENV SPARK_DIST spark-$SPARK_VERSION-bin-hadoop2.6
ENV SPARK_ARCH $SPARK_DIST.tgz
WORKDIR /opt
# Install Scala
RUN \
cd /root && \
curl -o scala-$SCALA_VERSION.tgz http://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz && \
tar -xf scala-$SCALA_VERSION.tgz && \
rm scala-$SCALA_VERSION.tgz && \
echo >> /root/.bashrc && \
echo 'export PATH=~/scala-$SCALA_VERSION/bin:$PATH' >> /root/.bashrc
# Install SBT
RUN \
curl -L -o sbt-$SBT_VERSION.deb https://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb
# Install Spark
RUN \
cd /opt && \
curl -o $SPARK_ARCH http://d3kbcqa49mib13.cloudfront.net/$SPARK_ARCH && \
tar xvfz $SPARK_ARCH && \
rm $SPARK_ARCH && \
echo 'export PATH=$SPARK_DIST/bin:$PATH' >> /root/.bashrc
EXPOSE 9851 9852 4040 9092 9200 9300 5601 7474 7687 7473
VOLUME /workdir
CMD java -cp "target/scala-2.11/demo_consumer.jar" consumer.SparkConsumer
I believe this is because the command you execute from the Docker container are not in the right folder. You could try to execute the commands from the workdir:
docker run -v my-volume:/workdir -w /workdir container_name
If that does not work, you could inspect what's inside the container. Either with a ls:
docker run -v my-volume:/workdir -w /workdir container_name bash -c 'ls -lah'
Or by accessing its bash session:
docker run -v my-volume:/workdir -w /workdir container_name bash
p.s: if bash does not work, try with sh.