Drone.io | Create docker-compose settings - github

I can't connect to Drone.io with my GitHub.
And have several problems with the app:
1) drone-agent can't connect to server
dodge#comp:$drone agent
28070:M 15 Nov 22:04:01.906 * connecting to server http://<my_ip>
28070:M 15 Nov 22:04:01.906 # connection failed, retry in 15s. websocket.Dial http://<my_ip>: bad scheme
2) I can't add Postgresql to docker-compose.
When I add this text from your site
DRONE_DATABASE_DRIVER: postgres
DRONE_DATABASE_DATASOURCE: postgres://root:password#1.2.3.4:5432/postgres?sslmode=disable
I have this error
INFO: 2017/11/15 19:42:33 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: Error while dialing dial tcp 172.18.0.2:9000: getsockopt: connection refused"; Reconnecting to {drone-server:9000 <nil>}
3) When I use only a server and an agent in docker-compose I have this error
dodge#comp:$drone server
ERRO[0000] sql: unknown driver "sqlite3" (forgotten import?)
FATA[0000] database connection failed
docker-compose.yml
version: '2'
services:
drone-server:
image: drone/drone:0.8
ports:
- 80:8000
- 9000
volumes:
- /var/lib/drone:/var/lib/drone/
- ./drone:/var/lib/drone/
restart: always
environment:
- DRONE_DEBUG=true
- DRONE_OPEN=true
- DRONE_HOST=http://172.18.0.2
- DRONE_GITHUB=true
- DRONE_GITHUB_CLIENT=secretid
- DRONE_GITHUB_SECRET=secretpass
- DRONE_SECRET=password
drone-agent:
image: drone/agent:0.8
command: agent
restart: always
depends_on: [ drone-server ]
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- DRONE_SERVER=drone-server:9000
- DRONE_SECRET=password
4) I cannot start tests in my project. Maybe I missed something during the setup.

$ drone server
$ drone agent
I see the above commands in your examples. These commands are only available in drone 0.7 and below. Drone 0.8 uses drone-server and drone-agent binaries. There seems to be some version disconnect here.
connection failed, retry in 15s. websocket.Dial
drone 0.7 and below used websockets. I see in the docker-compose example you are using drone 0.8 which uses http2 and grpc. There seems to be a disconnect in your configuration vs the version of drone you are using.
sql: unknown driver "sqlite3"
this happens when you compile drone with CGO disabled, or use a version of drone that has been compiled with CGO disabled. If CGO is disabled the sqlite3 driver is not compiled into the binary. Are you trying to build drone from source?
grpc: addrConn.resetTransport failed to create client transport
This error comes from the agent, and is therefore unrelated to a postgres configuration. You should not be providing your agent with a postgres configuration, only the server.

version: '2'
services:
drone-server:
image: drone/drone:latest
ports:
- 80:8000
- 9000:9000
volumes:
- /var/lib/drone:/var/lib/drone/
- ./drone:/var/lib/drone/
restart: always
environment:
- DRONE_DEBUG=true
- DRONE_HOST=http://<container_ip_server>
- DRONE_OPEN=true
- DRONE_GITHUB=true
- DRONE_GITHUB_CLIENT=<client_git>
- DRONE_GITHUB_SECRET=<secret_git>
- DRONE_SECRET=<secret_drone>
- DRONE_GITHUB_MERGE_REF=true
drone-agent:
image: drone/agent:latest
command: agent
restart: always
depends_on: [ drone-server ]
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- DRONE_SERVER=drone-server:9000
- DRONE_SECRET=<drone_secret>
This workes fine.

Related

Docker compose read connection reset by peer error on pipeline

when running a docker compose in a pipeline I'm getting this error when the tests on the pipleine are making use of mycontainer's API.
panic: Get "http://localhost:8080/api/admin/folder": read tcp 127.0.0.1:60066->127.0.0.1:8080: read: connection reset by peer [recovered]
panic: Get "http://localhost:8080/api/admin/folder": read tcp 127.0.0.1:60066->127.0.0.1:8080: read: connection reset by peer
This is my docker copose file:
version: "3"
volumes:
postgres_vol:
driver: local
driver_opts:
o: size=1000m
device: tmpfs
type: tmpfs
networks:
mynetwork:
driver: bridge
services:
postgres:
image: postgres:14
container_name: postgres
restart: always
environment:
- POSTGRES_USER=xxx
- POSTGRES_PASSWORD=xxx
- POSTGRES_DB=newdatabase
volumes:
#- ./postgres-init-db.sql:/docker-entrypoint-initdb.d/postgres-init-db.sql
- "postgres_vol:/var/lib/postgresql/data"
ports:
- 5432:5432
networks:
- mynetwork
mycontainer:
image: myprivaterepo/mycontainer-image:1.0.0
container_name: mycontainer
restart: always
environment:
- DATABASE_HOST=postgres
- DATABASE_PORT=5432
- DATABASE_NAME=newdatabase
- DATABASE_USERNAME=xxx
- DATABASE_PASSWORD=xxx
- DATABASE_SSL=false
depends_on:
- postgres
ports:
- 8080:8080
networks:
- mynetwork
mycontainer is listening on port 8080 and locally everything works fine.
However, when I run the pipeline which is initiating this docker compose is where I'm getting the error.
Basically, I'm running some tests in the pipeline that make use of mycontainer API (http://localhost:8080/api/admin/folder).
If I run the docker compose locally and I reproduce the steps followed on my pipeline to make use of the API everything is working fine. I can comunicate locally with both containers through localhost.
Also, I tried using healthchecks on the containers and 127.0.0.1:8080:8080 on mycontainer & 127.0.0.1:5432:5432 in postgres (including 0.0.0.0:8080:8080 & 0.0.0.0:5342:5432 just in case).
Any idea about that?
I was able to reproduce your error in a pipeline.
Make sure that you are not catching anything (e.g the code that's interacting with your container's API).
You did not mention anything related to your pipeline but just in case, delete the catching on your pipeline.

What can be the reason why I can't connect to CateDB with grafana and locally with Dbeaver?

Cheers
I am currently trying to reproduce the tutorial to develop an application / solution with Fiware seen in: Desarrolla tu primer aplicaciĆ³n en Fiware and I am having difficulty connecting Grafana with the Crate database (recommended by Fiware).
My docker-compose file configuration is:
version: "3.5"
services:
orion:
image: fiware/orion
hostname: orion
container_name: fiware-orion
networks:
- fiware_network
depends_on:
- mongo-db
ports:
- "1026:1026"
command: -dbhost mongo-db -logLevel DEBUG -noCache
mongo-db:
image: mongo:3.6
hostname: mongo-db
container_name: db-mongo
networks:
- fiware_network
ports:
- "27017:27017"
command: --bind_ip_all --smallfiles
volumes:
- mongo-db:/data
grafana:
image: grafana/grafana:8.2.6
container_name: fiware-grafana
networks:
- fiware_network
ports:
- "3000:3000"
depends_on:
- crate
quantumleap:
image: fiware/quantum-leap
container_name: fiware-quantumleap
networks:
- fiware_network
ports:
- "8668:8668"
depends_on:
- mongo-db
- orion
- crate
environment:
- CRATE_HOST=crate
crate:
image: crate:1.0.5
networks:
- fiware_network
ports:
# Admin UI
- "4200:4200"
# Transport protocol
- "4300:4300"
command: -Ccluster.name=democluster -Chttp.cors.enabled=true -Chttp.cors.allow-origin="*"
volumes:
- cratedata:/data
volumes:
mongo-db:
cratedata:
networks:
fiware_network:
driver: bridge
After starting the containers, I have a positive response from OCB (Orion), from Quantumleap and even after creating the subscription between Orion and quantumleap, in the Crate database, the data is stored and updated correctly.
Unfortunately I am not able to get the visualization of the data in grafana.
I thought that the reason was the fact that crateDB was removed as a grafana plugin versions ago, but after researching how to connect crateDB with grafana, through a postgres data source (I read on: Visualizing time series data with Grafana and CrateDB), I'm still having difficulty getting the connection, getting in grafana "Query error
dbquery error: EOF"
grafana settings img
The difference with respect to the guide is the listening port, since with input port 5432 I get a response indicating that it is not possible to connect to the server. I'm using port 4300.
After configuring, and trying to query from grafana, I get the mentioned EOF error
EOF error in grafana img
I tried to connect from a database IDE (Dbeaver) and I get exactly the same problem.
EOF error in Dbeaver img
Is there something I am missing?
What should I change in my docker configuration, or ports, or anything else to fix this?
I think it is worth mentioning that I am studying this because I am being asked in a project to visualize context switches in real time with Fiware and grafana.
Thanks in advance
The PostgreSQL port 5432 must be exposed as well by the CrateDB docker image.
Additionally, I highly recommend to use a recent (or just the latest) CrateDB version, current stable is 5.0.0, your used version 1.0.5 is very old and not maintained anymore.
Full example entry:
crate:
image: crate:latest
networks:
- fiware_network
ports:
# Admin UI
- "4200:4200"
# Transport protocol
- "4300:4300"
# PostgreSQL protocol
- "5432:5432"
command: -Ccluster.name=democluster -Chttp.cors.enabled=true -Chttp.cors.allow-origin="*"
volumes:
- cratedata:/data

docker-compose - PHP instance seems not to communicate with database service

I'm developing a project based on the Github template dunglas/symfony-docker to which I want to add a postgres database..
It seems that my docker compose.yml file is incorrectly configured because the communication between PHP and postgres is malfunctioning.
Indeed when I try to perform a symfony migration, doctrine returns me the following error :
password authentication failed for user "postgres"
When I inspect the PHP logs I notice that PHP is waiting after the database
php_1 | Still waiting for db to be ready... Or maybe the db is not reachable.
My docker-compose.yml :
version: "3.4"
services:
php:
links:
- database
build:
context: .
target: symfony_php
args:
SYMFONY_VERSION: ${SYMFONY_VERSION:-}
SKELETON: ${SKELETON:-symfony/skeleton}
STABILITY: ${STABILITY:-stable}
restart: unless-stopped
volumes:
- php_socket:/var/run/php
healthcheck:
interval: 10s
timeout: 3s
retries: 3
start_period: 30s
environment:
# Run "composer require symfony/orm-pack" to install and configure Doctrine ORM
DATABASE_URL: ${DATABASE_URL}
# Run "composer require symfony/mercure-bundle" to install and configure the Mercure integration
MERCURE_URL: ${CADDY_MERCURE_URL:-http://caddy/.well-known/mercure}
MERCURE_PUBLIC_URL: https://${SERVER_NAME:-localhost}/.well-known/mercure
MERCURE_JWT_SECRET: ${CADDY_MERCURE_JWT_SECRET:-!ChangeMe!}
caddy:
build:
context: .
target: symfony_caddy
depends_on:
- php
environment:
SERVER_NAME: ${SERVER_NAME:-localhost, caddy:80}
MERCURE_PUBLISHER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET:-!ChangeMe!}
MERCURE_SUBSCRIBER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET:-!ChangeMe!}
restart: unless-stopped
volumes:
- php_socket:/var/run/php
- caddy_data:/data
- caddy_config:/config
ports:
# HTTP
- target: 80
published: 80
protocol: tcp
# HTTPS
- target: 443
published: 443
protocol: tcp
# HTTP/3
- target: 443
published: 443
protocol: udp
###> doctrine/doctrine-bundle ###
database:
image: postgres:${POSTGRES_VERSION:-13}-alpine
environment:
POSTGRES_DB: ${POSTGRES_DB:-app}
# You should definitely change the password in production
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-ChangeMe}
POSTGRES_USER: ${POSTGRES_USER:-symfony}
volumes:
- db-data:/var/lib/postgresql/data:rw
# You may use a bind-mounted host directory instead, so that it is harder to accidentally remove the volume and lose all your data!
# - ./docker/db/data:/var/lib/postgresql/data:rw
###< doctrine/doctrine-bundle ###
volumes:
php_socket:
caddy_data:
caddy_config:
###> doctrine/doctrine-bundle ###
db-data:
###< doctrine/doctrine-bundle ###
extract of my .env file :
POSTGRES_DB=proximityNL
POSTGRES_PASSWORD=postgres
POSTGRES_USER=postgres
DATABASE_URL="postgresql://postgres:postgres#database:5432/proximityNL?serverVersion=13&charset=utf8"
Can you help me ?
Best regards ..
UPDATE :
Indeed I understood on Saturday that it was just necessary to remove orphan ..
docker-compose down --remove-orphans --volumes
When running in a container, 127.0.0.1 refers to the container itself. Docker compose creates a virtual network where each container has its own IP address. You can address the containers by their service names.
So your connection string should point to database:5432 instead of 127.0.0.1:5432 like this
DATABASE_URL="postgresql://postgres:postgres#database:5432/proximityNL?serverVersion=13&charset=utf8"
You use database because that's the service name of your postgresql container in your docker compose file.
In docker you can call images via the name of it.
So try to use the name of the docker image for your config
DATABASE_URL="postgresql://postgres:postgres#database:5432/proximityNL?serverVersion=13&charset=utf8"
and maybe add an link between your php and database image
services:
php:
links:
- database
This is the way how i am connect a java app with an mysql db.
Docker should map DNS resolution from the Docker Host into your containers.
Networking in Compose link
Because of that, you DB URL should look like:
"postgresql://postgres:postgres#database:5432/..."
I understood on Saturday that it was just necessary to remove orphan
docker-compose down --remove-orphans --volumes

TeamCity not connecting to database while running in swarm mode

I'm trying to run a TeamCity in a Docker Swarm but it can not authenticate with the external AWS RDS Postgres instance. The strange part to me is that this issue does not occur when running as docker-compose. I've run this locally and on an AWS EC2 to double check that it is not related to something going on with the machine. Both times I get the same results.
The error message is:
Could not connect to PostgreSQL server.
the connection attempt failed.: org.postgresql.util.PSQLException: The connection attempt failed. Caused by: java.net.UnknownHostException: rds_url.com
docker-compose.yaml:
version: "3"
services:
teamcity-server:
privileged: true
image: jetbrains/teamcity-server:2020.2.2
hostname: teamcity-server
ports:
- "8111:8111"
- "5432:5432"
volumes:
- ./data_dir:/data/teamcity_server/datadir
- ./log_dir:/opt/teamcity/logs
teamcity-agent:
privileged: true
image: jetbrains/teamcity-agent:2020.2.2
environment:
- SERVER_URL=http://teamcity-server:8111
- AGENT_NAME=regular_agent
- DOCKER_IN_DOCKER=start
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
teamcity-minimal-agent:
privileged: true
image: jetbrains/teamcity-minimal-agent:2020.2.2
environment:
- SERVER_URL=http://teamcity-server:8111
- AGENT_NAME=minimal_agent
- DOCKER_IN_DOCKER=start
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
Talking with TeamCity support I was able to figure it out. In short I needed to set the DNS server to the VPC DNS server. I also set the network mode to host.
dns: 169.254.169.253
network_mode: host
Locally I never solved it. To hit my rds server I need to be on a VPN and docker swarm was unable to resolve the IP address with the vpn running.

why can't the first container "see" the later containers

I have a series of containers that are started by docker-compose. Specifically they are multiple zookeeper containers:
zk1:
image: seven10/zookeeper:3.4.6
container_name: zk1
hostname: zk1
restart: always
ports:
- "2181:2181"
- "2888:2888"
- "3888:3888"
environment:
- ZOOKEEPER_ID=1
net: ${MY_NETWORK_NAME}
volumes:
- /seven10/zk/zk1/data:/opt/zookeeper-3.4.6/data
zk2:
image: seven10/zookeeper:3.4.6
container_name: zk2
restart: always
hostname: zk2
ports:
- "2182:2181"
- "2889:2888"
- "3889:3888"
environment:
- ZOOKEEPER_ID=2
net: ${MY_NETWORK_NAME}
volumes:
- /seven10/zk/zk2/data:/opt/zookeeper-3.4.6/data
zk3:
image: seven10/zookeeper:3.4.6
container_name: zk3
hostname: zk3
restart: always
ports:
- "2183:2181"
- "2890:2888"
- "3890:3888"
environment:
- ZOOKEEPER_ID=3
net: ${MY_NETWORK_NAME}
volumes:
- /seven10/zk/zk3/data:/opt/zookeeper-3.4.6/data
So when I go to start the containers, zk1 gives me this warning at the start:
WARN [WorkerSender[myid=1]:QuorumCnxManager#382] - Cannot open channel to 3 at
election address zk3:3888
java.net.UnknownHostException: zk3
but then doesn't say anything else about zk3 after a couple of seconds.
However, zk1 gives the following error for zk2 continuously:
2016-02-18 15:28:57,384 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Learner#233] - Unexpected exception, tries=0, connecting to zk2:2888
java.net.UnknownHostException: zk2
zk2 doesn't say ANYTHING ever about zk1, but briefly complains with the "cannot open channel" error for zk3.
zk3 doesn't every mention zk1 or zk2.
So the big problem is that zk1 can't find zk2 ever. It just spams the logs and refuses connections from kafka. Why is this so and how should I go about solving this problem?
My dev box is using docker version 1.9.1 and docker-compose version 1.5.1 on ubuntu 14.04 (Mint Rafello I think?), although the target environment will be ubuntu 15.10.
Does your host system know how to link zk1/2/3 to an IP address? If you're launching all three servers on the same node, you should use localhost as the host name (the server name should still be unique)