I'm trying to run a bare bones version of Hyperledger Sawtooth using Docker CE on a Mac. The docker-compose.yaml has containers running the base images from Sawtooth.
I'm unable to access the Sawtooth REST API from the host machine even though there are ports published for it when I run docker ps. The docker-compose file has worked on other Macs running Docker CE so I'm suspecting it may be a configuration or setup issue.
The contents of the docker-compose.yaml are below:
version: '2.1'
services:
settings-tp:
image: 'hyperledger/sawtooth-settings-tp:1.1.3'
container_name: sawtooth-settings-tp
depends_on:
- validator
entrypoint: settings-tp --connect tcp://validator:4004
identity-tp:
image: 'hyperledger/sawtooth-identity-tp:1.1.3'
container_name: sawtooth-identity-tp
depends_on:
- validator
entrypoint: identity-tp -vv --connect tcp://validator:4004
rest-api:
image: 'hyperledger/sawtooth-rest-api:1.1.3'
container_name: sawtooth-rest-api
ports:
- '8008:8008'
depends_on:
- validator
entrypoint: sawtooth-rest-api --connect tcp://validator:4004 --bind rest-api:8008
validator:
image: 'hyperledger/sawtooth-validator:1.1.3'
container_name: sawtooth-validator
ports:
- '4004:4004'
command: |
bash -c "
if [ ! -f /etc/sawtooth/keys/validator.priv ]; then
sawadm keygen
sawtooth keygen my_key
sawset genesis -k /root/.sawtooth/keys/my_key.priv
sawset proposal create \
-k /root/.sawtooth/keys/my_key.priv \
sawtooth.consensus.algorithm.name=Devmode \
sawtooth.consensus.algorithm.version=0.1 \
-o config.batch && \
sawadm genesis config-genesis.batch config.batch
fi;
sawtooth-validator -vvv \
--endpoint tcp://validator:8800 \
--bind component:tcp://eth0:4004 \
--bind network:tcp://eth0:8800 \
--bind consensus:tcp://eth0:5050 \
"
devmode-engine:
image: 'hyperledger/sawtooth-devmode-engine-rust:1.1.3'
container_name: sawtooth-devmode-engine-rust-default
depends_on:
- validator
entrypoint: devmode-engine-rust -C tcp://validator:5050
If you cannot access the port from the host, the container must not be running correctly. Look for error messages for that container when starting docker-compose
What does docker ps -a show?
Can you connect to the port? Try something like telnet localhost 8008
Related
Running $ docker compose up -d on the following docker-compose.yml
version: '3'
services:
web:
platform: linux/arm/v6
build: ./web
restart: always
environment:
DATABASE_URL: ${DATABASE_URL}
SECRET_KEY: ${SECRET_KEY}
TZ: Europe/Amsterdam
ports:
- ${PORT}:5000
depends_on:
- db
volumes:
- web-data:/data
db:
platform: linux/arm/v6
image: arm32v6/postgres:15.1-alpine
restart: always
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
ports:
- ${POSTGRES_PORT}:5432
volumes:
- postgres-data:/var/lib/postgresql/data
volumes:
postgres-data:
web-data:
should create two docker containers. However, it produces
[+] Building 6.5s (3/3) FINISHED
=> [internal] load build definition from Dockerfile 0.5s
=> => transferring dockerfile: 32B 0.2s
=> [internal] load .dockerignore 0.2s
=> => transferring context: 2B 0.1s
=> ERROR resolve image config for docker.io/docker/dockerfile:1 2.4s
------
> resolve image config for docker.io/docker/dockerfile:1:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend
dockerfile.v0: failed to solve with frontend gateway.v0: no match for platform in manifest
sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc: not found
I verified that both services (web and db) in my docker-compose file can be build using docker build, which excludes any platform issues from the services.
What does no match for platform in manifest mean? More importantly, how can I fix the error?
A temporary solution is to replace docker-compose.yml with explicit docker commands.
# Create the network and volumes
docker network create randpion-app
docker volume create postgres-data
docker volume create web-data
# Build the web service
docker build -t randpion/web web
# Load the environment variables from the .env file
export $(cat .env | xargs)
# Start the database service
docker run -d \
--network randpion-app --network-alias db --name randpion-db \
--platform "linux/arm/v6" \
-v postgres-data:/var/lib/postgresql/data \
-e POSTGRES_USER=${POSTGRES_USER} \
-e POSTGRES_PASSWORD=${POSTGRES_PASSWORD} \
-e POSTGRES_DB=${POSTGRES_DB} \
-p ${POSTGRES_PORT}:5432 \
arm32v6/postgres:15.1-alpine
# Start the web service
docker run -d \
--network randpion-app --network-alias web --name randpion-web \
--platform "linux/arm/v6" \
-v web-data:/data \
-e DATABASE_URL=${DATABASE_URL} \
-e SECRET_KEY=${SECRET_KEY} \
-e TZ=Europe/Amsterdam \
-p ${PORT}:5000 \
randpion/web
This runs fine for now, but the solution is not ideal. For example, the containers must restart after every reboot.
I am able to run Apache Pulsar using this docker command:
docker run -it \
-p 6650:6650 \
-p 8080:8080 \
--mount source=pulsardata,target=/pulsar/data \
--mount source=pulsarconf,target=/pulsar/conf \
apachepulsar/pulsar:2.6.0 \
bin/pulsar standalone
I am trying to convert this to docker-compose and I use the docker-compose.yml file below. When I run the command:
docker-compose up
I get the error:
Attaching to pulsar
pulsar | Error: Could not find or load main class "
pulsar exited with code 1
What am I doing wrong here?
Thanks in advance.
version: '3.1'
services:
standalone:
image: apachepulsar/pulsar:2.6.0
container_name: pulsar
ports:
- 8080:8080
- 6650:6650
environment:
- PULSAR_MEM=" -Xms512m -Xmx512m -XX:MaxDirectMemorySize=1g"
volumes:
- pulsardata:/pulsar/data
- pulsarconf:/pulsar/conf
command: /bin/bash -c "bin/pulsar standalone"
volumes:
pulsardata:
pulsarconf:
The issue is with the env variable. It should work if you specify it in the following way:
version: '3.1'
services:
standalone:
image: apachepulsar/pulsar:2.6.0
ports:
- 8080:8080
- 6650:6650
environment:
PULSAR_MEM: " -Xms512m -Xmx512m -XX:MaxDirectMemorySize=1g"
command: bin/pulsar standalone
# ... other parameters
When trying to translate the following 2 docker commands into a docker-compose.yml using Compose version 3
docker run \
--name timescaledb \
--network timescaledb-net \
-e POSTGRES_PASSWORD=insecure \
-e POSTGRES_INITDB_WALDIR=/var/lib/postgresql/data/pg_wal \
-e PGDATA=/var/lib/postgresql/data/pg_data \
timescale/timescaledb:latest-pg11 postgres \
-cwal_level=archive \
-carchive_mode=on \
-carchive_command="/usr/bin/wget wale/wal-push/%f -O -" \
-carchive_timeout=600 \
-ccheckpoint_timeout=700 \
-cmax_wal_senders=1
and
docker run \
--name wale \
--network timescaledb-net \
--volumes-from timescaledb \
-v ./backups:/backups \
-e WALE_LOG_DESTINATION=stderr \
-e PGWAL=/var/lib/postgresql/data/pg_wal \
-e PGDATA=/var/lib/postgresql/data/pg_data \
-e PGHOST=timescaledb \
-e PGPASSWORD=insecure \
-e PGUSER=postgres \
-e WALE_FILE_PREFIX=file://localhost/backups \
timescale/timescaledb-wale:latest
we get the following error when running docker-compose up:
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.wale: 'volumes_from'
How can we translate the 2 Docker commands correctly to use Compose version 3? We will need to be able to specify the location of the volumes on the host (i.e. ./timescaledb).
Using Mac OS X 10.15.3, Docker 19.03.8, Docker Compose 1.25.4
docker-compose.yml
version: '3.3'
services:
timescaledb:
image: timescale/timescaledb:latest-pg11
container_name: timescaledb
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=insecure
- POSTGRES_INITDB_WALDIR=/var/lib/postgresql/data/pg_wal
- PGDATA=/var/lib/postgresql/data/pg_data
command: -cwal_level=archive -carchive_mode=on -carchive_command="/usr/bin/wget wale/wal-push/%f -O -" -carchive_timeout=600 -ccheckpoint_timeout=700 -cmax_wal_senders=1
volumes:
- ./timescaledb:/var/lib/postgresql/data
networks:
- timescaledb-net
wale:
image: timescale/timescaledb-wale:latest
container_name: wale
environment:
- WALE_LOG_DESTINATION=stderr
- PGWAL=/var/lib/postgresql/data/pg_wal
- PGDATA=/var/lib/postgresql/data/pg_data
- PGHOST=timescaledb
- PGPASSWORD=insecure
- PGUSER=postgres
- WALE_FILE_PREFIX=file://localhost/backups
volumes_from:
- tsdb
volumes:
- ./backups:/backups
networks:
- timescaledb-net
depends_on:
- timescaledb
networks:
timescaledb-net:
In the container timescaledb you are actually mounting the /var/lib/postgresql/data to ./timescaledb and hence, if you want to use the same volume for the wale container, you can edit the wale container like:
...
volumes:
- ./backups:/backups
- ./timescaledb:/var/lib/postgresql/data
...
In this case, both of the containers will be able to read and write from the same mounted volume to your local machine.
Also, remember to remove this part as it is not a valid command in docker-compose:
volumes_from:
- tsdb
I have an image (gepick:latest) with node app created from Dockerfile:
FROM centos:7
# Create app directory
WORKDIR /usr/src/app
RUN curl --silent --location https://rpm.nodesource.com/setup_8.x | bash -
RUN yum install -y nodejs
RUN curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | tee /etc/yum.repos.d/yarn.repo
RUN rpm --import https://dl.yarnpkg.com/rpm/pubkey.gpg
RUN yum install -y yarn
RUN yarn
COPY . .
EXPOSE 8080
CMD [ "yarn", "test-matches-collecting-job"]
My goal is run tests in docker. But it requires mongodb
docker run gepick:latest :
...
Mongoose default connection error: MongoError: failed to connect to server [localhost:27017] on first connect [MongoError: connect ECONNREFUSED 127.0.0.1:27017]
...
I tried link mongo:4 images container docker run --link 0d24c3a35d5a gepick:latest but get same error.
When you launch your container using a docker-compose yaml file Docker bridges the containers together and allows you to have it launch the mongo container before other containers which rely on mongo to be active ... try something like this
cat my-docker-compose.yml
version: '3'
services:
my-gepick:
image: gepick:latest
container_name: blah_gepick
restart: always
depends_on:
- loudmongo
volumes:
- /cryptdata5/var/log/blobs:/blobs
- /webapp/enduser/bundle:/tmp
environment:
- MONGO_SERVICE_HOST=loudmongo
- MONGO_SERVICE_PORT=$GKE_MONGO_PORT
- MONGO_URL=mongodb://loudmongo:$GKE_MONGO_PORT/test
- METEOR_SETTINGS=${METEOR_SETTINGS}
- MAIL_URL=smtp://support#${GKE_DOMAIN_NAME}:blah#loudmail:587/
links:
- loudmongo
ports:
- 127.0.0.1:3000:3000
working_dir: /tmp
command: /usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf
loudmongo:
image: mongo
container_name: loud_mongo
restart: always
ports:
- 127.0.0.1:$GKE_MONGO_PORT:$GKE_MONGO_PORT
volumes:
- /cryptdata7/var/data/db:/data/db
so your launch sequence may look like
docker-compose -f /somedir/my-docker-compose.yml pull
docker-compose -f /somedir/my-docker-compose.yml up -d
I want to start using Flyway to version our database changes. I am trying to create a Postgres Docker container with seeded data, TAG and publish that docker image to be used in automated testing.
I tried using docker-compose, however I haven't figured a way to TAG and publish after Flyway runs.
Repository with test project
https://github.com/bigboy1122/flyway_postgres
Here is a docker-compose I created
version: '3.7'
services:
flyway:
image: boxfuse/flyway
restart: always
command: -url=jdbc:postgresql://db:5432/foo -user='postgres' -password='test123' -schemas='bar' migrate
volumes:
- .:/flyway/sql
depends_on:
- db
db:
image: tgalati1122/flyway_seeded_postgres
build:
context: .
dockerfile: ./Dockerfile
restart: always
environment:
POSTGRES_PASSWORD: 'test123'
POSTGRES_USER: 'postgres'
POSTGRES_DB: 'foo'
ports:
- 5432:5432
adminer:
image: adminer
restart: always
ports:
- 8080:8080
depends_on:
- db
Here is trying to use Docker multi build feature.
In the example below, the database I think spins up but I can't get flyway to access it.
FROM postgres:10.5-alpine as donar
ENV PGDATA=/pgdata
ENV POSTGRES_PASSWORD='test123'
ENV POSTGRES_USER='postgres'
ENV POSTGRES_DB='foo'
EXPOSE 5432:5432
RUN /docker-entrypoint.sh --help
FROM debian:stretch-slim as build_tools
ENV FLWAY_VERSION='5.2.0
RUN set -ex; \
if ! command -v gpg > /dev/null; then \
apt-get update; \
apt-get install -y --no-install-recommends \
gnupg \
dirmngr \
wget \
; \
rm -rf /var/lib/apt/lists/*; \
fi
VOLUME flyway/sql
RUN wget --no-check-certificate https://repo1.maven.org/maven2/org/flywaydb/flyway-commandline/5.2.0/flyway-commandline-5.2.0-linux-x64.tar.gz -O - | tar -xz
RUN pwd; \
ls -l; \
cd flyway-5.2.0; \
pwd; \
ls -l; \
sh ./flyway -url=jdbc:postgresql://localhost:5432/optins -user='postgres' -password='test123' -schemas='bar' migrate; \
FROM postgres:10.5-alpine
ENV PGDATA=/pgdata
ENV POSTGRES_PASSWORD='test123'
ENV POSTGRES_USER='postgres'
ENV POSTGRES_DB='foo'
EXPOSE 5432:5432
COPY --chown=postgres:postgres --from=donor /pgdata /pgdata
The idea I am going for as database changes occur, I want to automatically build a new lightweight test database as well as update the persisted databases throughout the enterprise.