I am using https://sbt-native-packager.readthedocs.io/en/stable/introduction.html to build my scala application and would like the publish the application as a local docker container.
I entered the following command:
sbt docker:publishLocal
I think, everything went fine and I've got an image generated:
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
sapmock 0.1 db261ab225b5 About a minute ago 664MB
<none> <none> 0940c75d8e5a About a minute ago 703MB
openjdk 8 bec43387959a 10 days ago 625MB
Why did sbt create an image <none>?
And when I run the image as a container, it stops immediately:
~/scala/sapmock$ docker run -d -p 8080:80 sapmock:0.1
91abf0e00e3a46baa78a09b9eccd3c562ae82d5b058d5e122e8927f8e7307afd
~/scala/sapmock$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
91abf0e00e3a sapmock:0.1 "/opt/docker/bin/sap…" 10 seconds ago Exited (0) 8 seconds ago upbeat_brattain
~/scala/sapmock$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Why?
Resources can be find here: https://gitlab.com/sweetsoft/sapmock
In addition, the content of the generated docker file:
FROM openjdk:8 as stage0
WORKDIR /opt/docker
COPY opt /opt
USER root
RUN ["chmod", "-R", "u=rX,g=rX", "/opt/docker"]
RUN ["chmod", "u+x,g+x", "/opt/docker/bin/sapmock"]
FROM openjdk:8
USER root
RUN id -u demiourgos728 2> /dev/null || (( getent group 0 || groupadd -g 0 root ) && useradd --system --create-home --uid 1001 --gid 0 demiourgos728 )
WORKDIR /opt/docker
COPY --from=stage0 --chown=demiourgos728:root /opt/docker /opt/docker
EXPOSE 8080
USER 1001:0
ENTRYPOINT ["/opt/docker/bin/sapmock"]
CMD []
Related
I want to to build a PostgreSQL image that only contains some extra .sql files to be executed at starting time
Dockerfile:
FROM postgres:11.9-alpine
USER postgres
WORKDIR /
COPY ddl/*.sql /docker-entrypoint-initdb.d/
Then I build the image:
docker build -t my-postgres:1.0.0 -f Dockerfile .
And run the container
docker run -d --name my-database \
-e POSTGRES_PASSWORD=abc123 \
-p 5432:5432 \
my-postgres:1.0.0
The output of it is the container id
33ed596792a80fc08f37c7c0ab16f8827191726b8e07d68ce03b2b5736a6fa4e
Checking the running containers returns nothing:
Docker container ls
But if I explicitly start it, it works
docker start my-postgres
In the original PostgreSQL image the Docker run command already starts the database. Why after building my own image it doesn't?
It turned out that one of the copied .sql files was failing to execute and, based on this documentation, it makes the entrypoint script to exit. Fixing the SQL solved the issue and the container started normally with Docker run
using vscode and wsl2, I have tried to launch a container using the default method and no customization. This generated the same error as below.
so following vscode docs I set a "workspaceMount" in devcontainer.json
"workspaceMount": "source=${localWorkspaceFolder},target=/workspaces/myRepo,type=bind,consistency=delegated",
"workspaceFolder": "/workspaces",
I select Reopen in container, the launch sequence happens but an error is generated
a mount config is invalid, make sure it has the right format and a source folder that exists on the machine where the Docker daemon is running
the log error is
Command failed: docker run -a STDOUT -a STDERR --mount source=d:\git\myRepo,target=/workspaces/myRepo,type=bind,consistency=delegated --mount type=volume,src=vscode,dst=/vscode -l vsch.quality=stable -l vsch.remote.devPort=0 -l vsch.local.folder=d:\git\myRepo --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --entrypoint /bin/sh vsc-myRepo-a878aa9edbcf04f717c76e764dabcde6 -c echo Container started ; trap "exit 0" 15; while sleep 1 & wait $!; do :; done
by launching the container from docker desktop I can confirm
cd /workspaces
ls -l
drwxr-xr-x 2 root root 4096 Dec 3 11:48 myRepo
Is this issue due to owner root:root ?
Should this be changed by chown in the Dokerfile? if so could you provide a sample code to do this, is it by RUN chown ...?
I guess you followed the documentation in https://code.visualstudio.com/docs/remote/containers-advanced
The source should contains the subfolder "myRepo" and the target only "workspaces"
"workspaceMount": "source=${localWorkspaceFolder}/myRepo,target=/workspaces,type=bind,consistency=delegated",
"workspaceFolder": "/workspaces",
I am trying out OWASP/ZAP to see if it is something we can use for our project, but I cannot make it work I don't know what I am doing wrong and the documentation really does not help. What I am trying is to run a scan on my api running in a docker container locally on my windows machine so I run the command:
docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-baseline.py -t http://172.21.0.2:8080/swagger.json -g gen.conf -r testreport.html the ip 172.21.0.2 is the IPAddress of my api container even tried with localhost and 127.0.0.1
but it just hangs in the following log message:
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
Feb 14, 2019 1:43:31 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Nothing happens and my zap docker container is in a unhealthy state, after some time it just crashes and ends up with a bunch of NullPointerExceptions. Is zap docker only working for linux, something specifically I need to do when running it on a windows machine? I don't get why this is not working even when I am following specifically the guideline in https://github.com/zaproxy/zaproxy/wiki/Docker
Edit 1
My latest try where I am trying to target my host ip address directly and the port that I am exposing my api to gives me the following error:
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
Feb 14, 2019 2:12:07 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Total of 3 URLs
ERROR Permission denied
2019-02-14 14:12:57,116 I/O error(13): Permission denied
Traceback (most recent call last):
File "/zap/zap-baseline.py", line 347, in main
with open(base_dir + generate, 'w') as f:
IOError: [Errno 13] Permission denied: '/zap/wrk/gen.conf'
Found Java version 1.8.0_151
Available memory: 3928 MB
Setting jvm heap size: -Xmx982m
213 [main] INFO org.zaproxy.zap.DaemonBootstrap
When you run docker with: docker run -v $(pwd):/zap/wrk/:rw ...
you are mapping the /zap/wrk/ directory in the docker image to the current working directory (cwd) of the machine in which you are running docker.
I think the problem is that your current user doesn't have write access to the cwd.
Try below command, hope it resolves issue.
$docker run --user $(id -u):$(id -g) -v $(pwd):/zap/wrk/:rw --rm -t owasp/zap2docker-stable zap-baseline.py -t https://your_url -g gen.conf -r testreport.html
The key error here is:
IOError: [Errno 13] Permission denied: '/zap/wrk/gen.conf'
This means that the script cannot write to the gen.conf file that you have mounted on /zap/wrk
Do you have write access to the cwd when its not mounted?
The reason for that is, if you use -r parameter, zap will attempt to generate the file report.html at location /zap/wrk/. In order to make this work, we have to mount a directory to this location /zap/wrk.
But when you do so, it is important that the zap container is able to perform the write operations on the mounted directory.
So, below is the working solution using gitlab ci yml. I started with this approach of using image: owasp/zap2docker-stable however then had to go to the vanilla docker commands to execute it.
test_site:
stage: test
image: docker:latest
script:
# The folder zap-reports created locally will be mounted to owasp/zap2docker docker container,
# On execution it will generate the reports in this folder. Current user is passed so reports can be generated"
- mkdir zap-reports
- cd zap-reports
- docker pull owasp/zap2docker-stable:latest || echo
- docker run --name zap-container --rm -v $(pwd):/zap/wrk -u $(id -u ${USER}):$(id -g ${USER}) owasp/zap2docker-stable zap-baseline.py -t "https://example.com" -r report.html
artifacts:
when: always
paths:
- zap-reports
allow_failure: true
So the trick in the above code is
Mount local directory zap-reports to /zap/wrk as in $(pwd):/zap/wrk
Pass the current user and group on the host machine to the docker container so the process is using the same user / group. This allows writing of reports on the directory mounted from local host. This is done by -u $(id -u ${USER}):$(id -g ${USER})
Below is the working code with image: owasp/zap2docker-stable
test_site:
variables:
GIT_STRATEGY: none
stage: test
image:
name: owasp/zap2docker-stable:latest
before_script:
- mkdir -p /zap/wrk
script:
- zap-baseline.py -t "https://example.com" -g gen.conf -I -r testreport.html
- cp /zap/wrk/testreport.html testreport.html
artifacts:
when: always
paths:
- zap.out
- testreport.html
I'm new to Docker.
I'm trying to run my node app tests in a Docker container.
I want to run the tests with a real postgres db.
I'm creating this container with the following Dockerfile:
# Set image
FROM postgres:alpine
# Install node latest
RUN apk add --update nodejs nodejs-npm
# Set working dir
WORKDIR .
# Copy the current directory contents into the container at .
ADD src src
ADD .env.testing .env
ADD package.json .
ADD package-lock.json .
# Run tests
CMD npm install && npm run coverage
From the image docs, when I run the container with:
$ docker run build-name -d postgres
I see that the container takes time to start postgresql service.
When I run the container without the "-d postgres" param:
$ docker run build-name
The service does not start and the tests fail due to "could not connect to server".
Questions:
A. How can I run the tests AFTER the postgresql service starts?
B. I saw some examples using docker-composer but can I do this without composer?
Thanks
Thanks to #Bogdan I found the complete solution:
Dockerfile should be:
# Set image
FROM postgres:alpine
# Install node latest
RUN apk add --update nodejs nodejs-npm
# Set working dir
WORKDIR .
# Copy the current directory contents into the container at .
ADD src src
ADD .env.testing .env
ADD package.json .
ADD package-lock.json .
# Install
RUN npm install
# Init container
CMD psql -U postgres -c "SELECT 1;" postgres
Build container:
$ docker build -t test .
Run container:
$ docker run --name startedtest -d test -d postgres
Run tests after conatiner is running:
$ docker exec startedtest some_create_schema_script && npm run coverage
If the goal is just to run the tests in the Postgres container, one solution could be to install NodeJs in your postgres:alpine derived image and run the container normally. Once the database is up, you can run npm using docker exec like this:
docker exec <container_id> npm run coverage
Here is my docker file:
FROM ubuntu:14.04
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" > /etc/apt/sources.list.d/pgdg.list
RUN apt-get update && apt-get -y -q install python-software-properties software-properties-common \
&& apt-get -y -q install postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3
USER postgres
RUN /etc/init.d/postgresql start \
&& psql --command "CREATE USER pguser WITH SUPERUSER PASSWORD 'pguser';" \
&& createdb -O pguser pgdb
USER root
RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/9.3/main/pg_hba.conf
RUN echo "listen_addresses='*'" >> /etc/postgresql/9.3/main/postgresql.conf
EXPOSE 5432
RUN mkdir -p /var/run/postgresql && chown -R postgres /var/run/postgresql
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
USER postgres
CMD ["/usr/lib/postgresql/9.3/bin/postgres", "-D", "/var/lib/postgresql/9.3/main", "-c", "config_file=/etc/postgresql/9.3/main/postgresql.conf"]
Here is what I did...
I build the docker image:
docker build --rm=true -t my_image/postgresql:9.3
Then, I create a new directory called data in my current directory and ran the following command:
docker run -i -t -v="data:/data" -p 5432:5432 my_image/postgresql:9.3
I open another terminal and enter the postgres shell by running:
psql -h my_docker_ip -p 5432 -U pguser -W pgdb
and I create a table:
pgdb=# create table test (test_id bigserial primary key);
I verify the table exist using \dt and exit the postgres shell
I terminate the docker process and rerun the following:
docker run -i -t -v="data:/data" -p 5432:5432 my_image/postgresql:9.3
I enter the posgrest shell again and run \dt
I notice
there are no tables.
in the data directory there are no files.
I must be doing something wrong since I am assuming that the table I created will persist. Can someone point out my mistake?
There is something that confused me and for me was not very clear in the official documentation.
To my knowledge, persistent volumes can be created in three ways.
At container invocation time including full path ( -v ~/database:/data ): makes an external folder from the host available inside the docker container. Both can modify it.
At container invocation time using a volume name ( -v datamysql:/data ): makes a volume that is persistent available inside the container. It is created it if it did not exist. You can list them by name with docker volume ls. Internally, it will be stored in a place such as /var/lib/docker/volumes/ae4445f7c9317a22fe84726fb894c47754f38a7fd150c00fd877024889968750/_data.
At container build time ( VOLUME ["/database/data"] in Dockerfile). Every invocation of docker run will create a new volume that will persist even if you delete the container. This can be confusing becausee subsequent invocations will result in different volumes being created that will not be reused.
You can list both named (second case) and unnamed (third case) volumes with
$ docker volume ls
DRIVER VOLUME NAME
local 064593b3e65977097d4d0c8402a6c633f1af69be2937bf118678ab8f97ee9a7e
local 4753ad0437d13e54c76d9c34a30a1843396a1866a0cf9237d500fdcca0d78c5f
local 8d7a35354f666b2e8a26866a35bbae36bb9601701d4c6b505ab8ce6629f69415
local db48eefe8f189b36107ca9c4eebb792690590ab0ba055e7e4e2c9adfd1765b7e
local datamysql
You can see the exact location of a container's volume by using docker inspect mycontainer
{
"Type": "volume",
"Name": "8d7a35354f666b2e8a26866a35bbae36bb9601701d4c6b505ab8ce6629f69415",
"Source": "/media/USBdrive/docker/volumes/8d7a35354f666b2e8a26866a35bbae36bb9601701d4c6b505ab8ce6629f69415/_data",
"Destination": "/var/lib/mysql",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
},
It might be handy to remove unused volumes (for the third case, specially).
$ docker volume prune
WARNING! This will remove all volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Volumes:
4753ad0437d13e54c76d9c34a30a1843396a1866a0cf9237d500fdcca0d78c5f
Total reclaimed space: 205MB
Because you used the VOLUME directive in your Dockerfile, you are in the third case. Inspect your container to look for the file, and specify the volume from the command line if you want repeated sessions to persist data.
Based on your comment:
the data persisted, but I still can't find the persist data in my host ./data directory
and running this command:
docker run -i -t -v="data:/data" -p 5432:5432 my_image/postgresql:9.3
You appear to be confusing a named volume and a host volume. The named volume is used when you give the volume a name without a path, like data. The named volume stores the data using the docker driver (typically local) under a given name that you can reuse. It has the advantage of being listed in docker volume ls, and being initialized to the content of the image at the mounted location.
If you include a full path, like /home/username/data that would mount the directory from the docker host instead of using the named volume. The biggest disadvantage is that you don't get the directory initialized with the contents from the image, and you will likely encounter permission issues where the uid of the container process won't match the uid you use on your host.
For more details, see https://docs.docker.com/engine/tutorials/dockervolumes/