Filter or map in jinja2 - docker-compose

I'm trying to filter or map this yml file. Up to now I was able to access it with this
{{services[0].version}}
But I need to access it with it name, without the position.
Yml file
"services":
- "service": "front"
"image": "acalls-caselog-web-app"
"version": "latest"
- "service": "back"
"image": "acalls-caselog-web-service"
"version": "latest"
docker-compose.yml.j2
version: "3.3"
services:
front:
image: url/{{services[0].image}}:{{services[0].version}}
ports:
- "81:81"
extra_hosts:
- "backend:172.32.3.46"
environment:
profile: preproduction
back:
image: url/ {{services[1].image}} : {{services[1].version}}
ports:
- "82:82"
extra_hosts:
- "backend:172.32.3.46"
environment:
profile: preproduction
I really don't now If I need to use map, filter or there is another form like service.service(front).image

You can do this using the selectattr filter:
{{services | selectattr("service", "equalto", "front")}}
In your YAML, you can apply it like this:
version: "3.3"
services:
front:
image: {% with front=services|selectattr("service", "equalto", "front") -%}
url/{{front.image}}:{{front.version}}
{%- endwith %}
ports:
- "81:81"
extra_hosts:
- "backend:172.32.3.46"
environment:
profile: preproduction
back:
image: {% with back = services|selectattr("service", "equalto", "back") -%}
url/ {{back.image}} : {{back.version}}
{%- endwith %}
ports:
- "82:82"
extra_hosts:
- "backend:172.32.3.46"
environment:
profile: preproduction

Related

How to fix "Error: getaddrinfo ENOTFOUND redis" on docker?

I am using Redis using NestJS and I see following error. I am going through different articles like here and looks like I am following the same but still getting this error.
Steps:
I used docker compose up command
Made sure that host in redis.module.ts is same as service name in docker-compose.yml which is redis.
What am I missing here?
Error:
Error: getaddrinfo ENOTFOUND redis
at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:71:26)
Code:
redis.module.ts
import { CacheModule, Module } from '#nestjs/common';
import { ConfigModule, ConfigService } from '#nestjs/config';
import { RedisService } from './redis.service';
import * as redisStore from 'cache-manager-redis-store';
import { envVariables } from '../env.variables';
#Module({
imports: [
CacheModule.registerAsync({
imports: [ConfigModule],
inject: [ConfigService],
useFactory: async (configService: ConfigService) => ({
store: redisStore,
host: process.env.REDIS_HOST,
port: configService.get('REDIS_PORT'),
ttl: configService.get('CACHE_TTL'),
max: configService.get('MAX_ITEM_IN_CACHE'),
}),
}),
],
providers: [RedisService],
exports: [RedisService],
})
export class RedisModule {}
.env
#REDIS
REDIS_HOST=redis
docker-compose.yml
version: "3.8"
services:
partnersusers:
image: partnersusers
build:
context: .
dockerfile: ./Dockerfile
environment:
- RUN_ENV=dev
- NODE_ENV=development
ports:
- "4000:4000"
networks:
- default
redis:
image: 'redis:alpine'
ports:
- "6379:4000"
networks:
default:
driver: bridge
Error in Docker:
I'm not an expert, but I notice a couple of things on your docker-compose.yml file.
First your redis service is missing the network assignation:
networks:
- default
Without this, redis-commander won't be able to find it as it's not on the same network.
Second redis by default runs on port: 6379 if you want it to run on port 4000 I believe you will need to specify an env var for it.
Or here maybe you just confused the order of the port matching which should've been: 4000:6379 (host_port:container_port).
this is my working docker-compose.yml for reference:
---
version: '3.8'
services:
...
redis:
image: redis
container_name: redis
hostname: redis
environment:
- ALLOW_EMPTY_PASSWORD=yes
ports:
- '6379:6379'
networks:
- my-net
redis-commander:
depends_on:
- redis
container_name: redis-commander
hostname: redis-commander
image: rediscommander/redis-commander:latest
restart: always
environment:
- REDIS_HOSTS=local:redis:6379 # note: this has to be the port the redis container exposes.
ports:
- "8081:8081"
networks:
- my-net
networks:
my-net:
Hope this helps :)

How to get Dapr Service to Service Invocation to work when running under docker-compose?

I am receiving the following error when trying to call a service using Dapr SDK.
System.Net.Http.HttpRequestException: Connection refused (127.0.0.1:3500)
---> System.Net.Sockets.SocketException (111): Connection refused
Here is my docker-compose settings of the service I am trying to call:
quest-service:
image: ${DOCKER_REGISTRY-gamification}/quest-service:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
build:
context: .
dockerfile: Services/LW.Gamification.QuestService/Dockerfile
ports:
- "5110:80"
- "50010:50001"
quest-service-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Quest-Service",
"-app-port", "80",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- quest-service
network_mode: "service:quest-service"
And the settings for the caller:
player-service:
image: ${DOCKER_REGISTRY-gamification}/player-service:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
build:
context: .
dockerfile: Services/LW.Gamificaiton.PlayerService/Dockerfile
ports:
- "5109:80"
- "50009:50001"
player-service-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Player-Service",
"-app-port", "80",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- player-service
network_mode: "service:player-service"
And here is the code that is failing to work:
// demo service to service call
var httpClient = DaprClient.CreateInvokeHttpClient("Quest-Service");
var requestUri = $"api/v1/Quest";
var result = await httpClient.GetFromJsonAsync<IEnumerable<string>>(requestUri);
Note: Messaging is working fine. :-)
I am new to Dapr so I must be doing something silly wrong, maybe something to do with ports.. I just don't know!
From following this question :Dapr Client Docker Compose Issue
I managed to get this partly working using the following docker-compose config:
services:
placement:
image: "daprio/dapr"
command: ["./placement", "-port", "50000", "-log-level", "debug"]
ports:
- "50000:50000"
quest-service:
image: ${DOCKER_REGISTRY-gamification}/quest-service:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
- DAPR_GRPC_PORT=50010
build:
context: .
dockerfile: Services/LW.Gamification.QuestService/Dockerfile
ports:
- "5110:80"
- "50010:50010"
depends_on:
- placement
- rabbitmq
- redis
- seq
- zipkin
quest-service-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Quest-Service",
"-app-port", "80",
"-placement-host-address", "placement:50000",
"-dapr-grpc-port", "50010",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- quest-service
network_mode: "service:quest-service"
generatetraffic:
image: ${DOCKER_REGISTRY-gamification}/generatetraffic:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
- DAPR_GRPC_PORT=50017
build:
context: .
dockerfile: Services/LW.Gamification.GenerateTraffic/Dockerfile
ports:
- "5117:80"
- "50017:50017"
depends_on:
- placement
- rabbitmq
- redis
- seq
- zipkin
generatetraffic-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Generate-Traffic",
"-app-port", "80",
"-placement-host-address", "placement:50000",
"-dapr-grpc-port", "50017",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- generatetraffic
network_mode: "service:generatetraffic"
However I still have issues with some of the documented APIs not working!.
var httpClient = DaprClient.CreateInvokeHttpClient("Quest-Service");
var requestUri = $"api/v1/Quest";
var result = await httpClient.GetAsync(requestUri);
Still fails?

Create a docker-compose with loop in ansible

I need to build a docker compose based on a yml file. In the next yml it will be the name, the image and the version of each service.
"services":
- "service": "front"
"image": "acalls-caselog-web-app"
"version": "latest"
- "service": "back"
"image": "acalls-caselog-web-service"
"version": "latest"
- "service": "vb"
"image": "acalls-caselog-vb-service"
"version": "latest"
- "service": "salesforce"
"image": "acalls-caselog-salesforce-app-service"
"version": "latest"
- "service": "tts"
"image": "ydilo-tts-service"
"version": "latest"
- "service": "ai classifier"
"image": "acalls-caselog-ai-classifier-service"
"version": "latest"
Up to now I had an array to set each image in the docker compose, like this
version: "3.3"
services:
front:
image: url/{{services[0].image}}:{{services[0].version}}
ports:
- "81:81"
extra_hosts:
- "backend:172.32.3.46"
environment:
profile: preproduction
back:
image: url/{{services[1].image}}:{{services[1].version}}
ports:
- "20101:20101"
environment:
profile: preproduction
saleforce:
image: url/{{services[2].image}}:{{services[2].version}}
ports:
- "20103:20103"
environment:
profile: preproduction
But I need to find a way to make this dynamically with a loop in the ansible task, for example, without the array position in the docker-compose file.
Main.yml
---
- name: stop container
ignore_errors: yes
become: True
shell:
cmd: "docker-compose down"
chdir: dir
- name: set docker-compose
template:
src: docker-compose-acalls.yml.j2
dest: dir/docker-compose.yml
mode: 0700
- name: Run container
become: True
shell:
cmd: "nohup docker-compose -f docker-compose.yml up -d"
chdir: dir
you create a template file: you have to play with whitespace , %- and -% to adjust the position, i have just given the general idea
version: "3.3"
services:
{% for item in services %}
{{ item.service }}:
{% if item.image is defined %}
image: url/{{item.image}}:{{item.version}}
{%- endif %}
{% if item.ports is defined %}
ports:
- "{{ item.ports[0] }}"
{%- endif %}
{% if item.extra_hosts is defined %}
extra_hosts:
- "{{ item.extra_hosts[0] }}"
{%- endif %}
{% if item.environment is defined %}
environment:
- profile: {{ item.environment.profile }}
{% endif %}
{% endfor %}
your playbook:
- name: test
hosts: localhost
vars_files:
- reference.yml
tasks:
template:
src: fileconf.j2
dest: composedocker.yml
and your reference.yml file:
services:
- service: "front"
image: "acalls-caselog-web-app"
ports:
- "81:81"
extra_hostss:
- "backend:172.32.3.46"
environment:
profile: preproduction
version: "latest"
- service: "back"
image: "acalls-caselog-web-service"
version: "latest"
ports:
- "20101:20101"
environment:
profile: preproduction
- service: "vb"
image: "acalls-caselog-vb-service"
version: "latest"
- service: "salesforce"
image: "acalls-caselog-salesforce-app-service"
version: "latest"
ports:
- "20103:20103"
environment:
profile: preproduction
- service: "tts"
image: "ydilo-tts-service"
version: "latest"
- service: "ai classifier"
image: "acalls-caselog-ai-classifier-service"
version: "latest"
result:
version: "3.3"
services:
front:
image: url/acalls-caselog-web-app:latest
ports:
- "81:81"
extra_hosts:
- "backend:172.32.3.46"
environment:
- profile: preproduction
back:
image: url/acalls-caselog-web-service:latest
ports:
- "20101:20101"
environment:
- profile: preproduction
vb:
image: url/acalls-caselog-vb-service:latest
salesforce:
image: url/acalls-caselog-salesforce-app-service:latest
ports:
- "20103:20103"
environment:
- profile: preproduction
tts:
image: url/ydilo-tts-service:latest
ai classifier:
image: url/acalls-caselog-ai-classifier-service:latest

Single Sign on Keyrock-Grafana doesn't work

I'm trying to use Keyrock to offer Single Sign-on on different platforms. Specifically, I want to offer that service in Grafana. I've seen the configuration to be changed in Grafana and my docker-compose is like this:
version: "3.1"
services:
grafana:
image: grafana/grafana:5.1.0
ports:
- 3000:3000
networks:
default:
ipv4_address: 172.18.1.4
environment:
- GF_AUTH_GENERIC_OAUTH_CLIENT_ID=90be8de5-69dc-4b9a-9cc3-962cca534410
- GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET=9e98964b-5043-4086-9657-51f1d8c11fe0
- GF_AUTH_GENERIC_OAUTH_ENABLED=true
- GF_AUTH_GENERIC_OAUTH_AUTH_URL=http://172.18.1.5:3005/oauth2/authorize
- GF_AUTH_GENERIC_OAUTH_TOKEN_URL=http://172.18.1.5:3005/oauth2/token
- GF_AUTH_GENERIC_OAUTH_API_URL=http://172.18.1.5:3005/v1/users
- GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP = true
- GF_Server_DOMAIN=172.18.1.4
- GF_Server_ROOT_URL=http://172.18.1.4:3000
keyrock:
image: fiware/idm:7.5.1
container_name: fiware-keyrock
hostname: keyrock
networks:
default:
ipv4_address: 172.18.1.5
depends_on:
- mysql-db
ports:
- "3005:3005"
- "3443:3443"
environment:
- DEBUG=idm:*
- DATABASE_HOST=mysql-db
- IDM_DB_PASS_FILE=/run/secrets/my_secret_data
- IDM_DB_USER=root
- IDM_HOST=http://localhost:3005
- IDM_PORT=3005
- IDM_HTTPS_ENABLED=false
- IDM_HTTPS_PORT=3443
- IDM_ADMIN_USER=admin
- IDM_ADMIN_EMAIL=admin#test.com
- IDM_ADMIN_PASS=test
secrets:
- my_secret_data
healthcheck:
test: curl --fail -s http://localhost:3005/version || exit 1
mysql-db:
restart: always
image: mysql:5.7
hostname: mysql-db
container_name: db-mysql
expose:
- "3306"
ports:
- "3306:3306"
networks:
default:
ipv4_address: 172.18.1.6
environment:
- "MYSQL_ROOT_PASSWORD_FILE=/run/secrets/my_secret_data"
- "MYSQL_ROOT_HOST=172.18.1.5"
volumes:
- mysql-db-sso:/var/lib/mysql
- ./mysql-data:/docker-entrypoint-initdb.d/:ro
secrets:
- my_secret_data
networks:
default:
ipam:
config:
- subnet: 172.18.1.0/24
volumes:
mysql-db-sso:
secrets:
my_secret_data:
file: ./secrets.txt
I have the Grafana application registered in Keyrock and has as callback http://172.18.1.4:3000/login. When I try to Sign-in in Grafana through Oauth it redirects me to the keyrock page to Sign-in, but when entering the credentials it returns me an invalid client_id, but it is the same one that returns Keyrock to me when obtaining the application information.
Is it possible that I lack something to configure or should it be done in another way?
Here is the working configuration for Keyrock 7.5.1 and Grafana 6.0.0
Grafana:
[auth.generic_oauth]
enabled = true
allow_sign_up = true
client_id = ${CLIENT_ID}
client_secret = ${CLIENT_SECRET}
scopes = permanent
auth_url = ${KEYROCK_URL}/oauth2/authorize
token_url = ${KEYROCK_URL}/oauth2/token
api_url = ${KEYROCK_URL}/user
App in Keyrock:
url - ${GRAFANA_ROOT_URL}
callback_url - ${GRAFANA_ROOT_URL}/login/generic_oauth
Token types - Permanent
So you need to fix env variable
GF_AUTH_GENERIC_OAUTH_API_URL
to
http://172.18.1.5:3005/user
and callback url
http://172.18.1.4:3000/login
to
http://172.18.1.4:3000/login/generic_oauth
and add oauth2 scopes

Multiple Orderer redundancy in Kafka based network

We're stuck configuring a fabric network based on 3 orgs with 1 peer each and 2 kafka-based orderers. For kafka ordering we use 4 kafka nodes with 3 zookeepers. It's deployed on some AWS ec2 instances as follows:
1: Org1
2: Org2
3: Org3
4: orderer0, orderer1, kafka0, kafka1, kafka2, kafka3, zookeeper0, zookeeper1, zookeeper2
The whole of the ordering nodes plus kafka cluster is located in the same machine for connectivity reasons (read somewhere they must be in the same machine to avoid these problems)
During our test, we try taking down the first orderer (orderer0) for redundancy testing with docker stop. We expected the network to continue working through orderer1, but instead it dies and stops working.
Looking at the peer's console, I can see some errors.
Could not connect to any of the endpoints: [orderer0.example.com:7050, orderer1.example.com:8050]
Find attached the content of the files related to the configuration of the system.
Orderer + kafka + zk
#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
services:
zookeeper0.example.com:
container_name: zookeeper0.example.com
extends:
file: docker-compose-base.yaml
service: zookeeper0.example.com
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
zookeeper1.example.com:
container_name: zookeeper1.example.com
extends:
file: docker-compose-base.yaml
service: zookeeper1.example.com
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
zookeeper2.example.com:
container_name: zookeeper2.example.com
extends:
file: docker-compose-base.yaml
service: zookeeper2.example.com
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
kafka0.example.com:
container_name: kafka0.example.com
extends:
file: docker-compose-base.yaml
service: kafka0.example.com
depends_on:
- zookeeper0.example.com
- zookeeper1.example.com
- zookeeper2.example.com
- orderer0.example.com
- orderer1.example.com
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
kafka1.example.com:
container_name: kafka1.example.com
extends:
file: docker-compose-base.yaml
service: kafka1.example.com
depends_on:
- zookeeper0.example.com
- zookeeper1.example.com
- zookeeper2.example.com
- orderer0.example.com
- orderer1.example.com
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
kafka2.example.com:
container_name: kafka2.example.com
extends:
file: docker-compose-base.yaml
service: kafka2.example.com
depends_on:
- zookeeper0.example.com
- zookeeper1.example.com
- zookeeper2.example.com
- orderer0.example.com
- orderer1.example.com
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
kafka3.example.com:
container_name: kafka3.example.com
extends:
file: docker-compose-base.yaml
service: kafka3.example.com
depends_on:
- zookeeper0.example.com
- zookeeper1.example.com
- zookeeper2.example.com
- orderer0.example.com
- orderer1.example.com
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
orderer0.example.com:
container_name: orderer0.example.com
image: hyperledger/fabric-orderer:x86_64-1.1.0
environment:
- ORDERER_GENERAL_LOGLEVEL=debug
- ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
- ORDERER_LISTEN_PORT=7050
- ORDERER_GENERAL_GENESISMETHOD=file
- ORDERER_GENERAL_GENESISFILE=/etc/hyperledger/configtx/genesis.block
- ORDERER_GENERAL_LOCALMSPID=OrdererMSP
- ORDERER_GENERAL_LOCALMSPDIR=/etc/hyperledger/crypto/orderer/msp
- ORDERER_GENERAL_TLS_ENABLED=true
- ORDERER_GENERAL_TLS_PRIVATEKEY=/etc/hyperledger/crypto/orderer/tls/server.key
- ORDERER_GENERAL_TLS_CERTIFICATE=/etc/hyperledger/crypto/orderer/tls/server.crt
- ORDERER_GENERAL_TLS_ROOTCAS=[/etc/hyperledger/crypto/orderer/tls/ca.crt, /etc/hyperledger/crypto/peerOrg1/tls/ca.crt, /etc/hyperledger/crypto/peerOrg2/tls/ca.crt, /etc/hyperledger/crypto/peerOrg3/tls/ca.crt]
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/orderers
command: orderer
ports:
- 7050:7050
volumes:
- ./channel:/etc/hyperledger/configtx
- ./channel/crypto-config/ordererOrganizations/example.com/orderers/orderer0.example.com/:/etc/hyperledger/crypto/orderer
- ./channel/crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/:/etc/hyperledger/crypto/peerOrg1
- ./channel/crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/:/etc/hyperledger/crypto/peerOrg2
- ./channel/crypto-config/peerOrganizations/org3.example.com/peers/peer0.org3.example.com/:/etc/hyperledger/crypto/peerOrg3
depends_on:
- kafka0.example.com
- kafka1.example.com
- kafka2.example.com
- kafka3.example.com
orderer1.example.com:
container_name: orderer1.example.com
image: hyperledger/fabric-orderer:x86_64-1.1.0
environment:
- ORDERER_GEN ERAL_LOGLEVEL=debug
- ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
- ORDERER_LISTEN_PORT=8050
- ORDERER_GENERAL_GENESISMETHOD=file
- ORDERER_GENERAL_GENESISFILE=/etc/hyperledger/configtx/genesis.block
- ORDERER_GENERAL_LOCALMSPID=OrdererMSP
- ORDERER_GENERAL_LOCALMSPDIR=/etc/hyperledger/crypto/orderer/msp
- ORDERER_GENERAL_TLS_ENABLED=true
- ORDERER_GENERAL_TLS_PRIVATEKEY=/etc/hyperledger/crypto/orderer/tls/server.key
- ORDERER_GENERAL_TLS_CERTIFICATE=/etc/hyperledger/crypto/orderer/tls/server.crt
- ORDERER_GENERAL_TLS_ROOTCAS=[/etc/hyperledger/crypto/orderer/tls/ca.crt, /etc/hyperledger/crypto/peerOrg1/tls/ca.crt, /etc/hyperledger/crypto/peerOrg2/tls/ca.crt, /etc/hyperledger/crypto/peerOrg3/tls/ca.crt]
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/orderers
command: orderer
ports:
- 8050:7050
volumes:
- ./channel:/etc/hyperledger/configtx
- ./channel/crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/:/etc/hyperledger/crypto/orderer
- ./channel/crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/:/etc/hyperledger/crypto/peerOrg1
- ./channel/crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/:/etc/hyperledger/crypto/peerOrg2
- ./channel/crypto-config/peerOrganizations/org3.example.com/peers/peer0.org3.example.com/:/etc/hyperledger/crypto/peerOrg3
depends_on:
- kafka0.example.com
- kafka1.example.com
- kafka2.example.com
- kafka3.example.com
Peer and Ca from Org2
#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
services:
ca.org2.example.com:
image: hyperledger/fabric-ca:x86_64-1.1.0
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
- FABRIC_CA_SERVER_CA_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem
- FABRIC_CA_SERVER_CA_KEYFILE=/etc/hyperledger/fabric-ca-server-config/efa7d0819b7083f6c06eb34da414acbcde79f607b9ce26fb04dee60cf79a389a_sk
- FABRIC_CA_SERVER_TLS_ENABLED=true
- FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem
- FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/efa7d0819b7083f6c06eb34da414acbcde79f607b9ce26fb04dee60cf79a389a_sk
ports:
- "8054:7054"
command: sh -c 'fabric-ca-server start -b admin:adminpw -d'
volumes:
- ./channel/crypto-config/peerOrganizations/org2.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
container_name: ca_peerOrg2
peer0.org2.example.com:
container_name: peer0.org2.example.com
extends:
file: base.yaml
service: peer-base
environment:
- CORE_PEER_ID=peer0.org2.example.com
- CORE_PEER_LOCALMSPID=Org2MSP
- CORE_PEER_ADDRESS=peer0.org2.example.com:7051
ports:
- 8051:7051
- 8053:7053
volumes:
- ./channel/crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/:/etc/hyperledger/crypto/peer
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
extra_hosts:
- "orderer0.example.com:xxx.xxx.xxx.xxx"
- "orderer1.example.com:xxx.xxx.xxx.xxx"
- "kafka0.example.com:xxx.xxx.xxx.xxx"
- "kafka1.example.com:xxx.xxx.xxx.xxx"
- "kafka2.example.com:xxx.xxx.xxx.xxx"
- "kafka3.example.com:xxx.xxx.xxx.xxx"
Orderer base
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
services:
orderer-base:
image: hyperledger/fabric-orderer:$IMAGE_TAG
environment:
- ORDERER_GENERAL_LOGLEVEL=error
- ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
- ORDERER_GENERAL_GENESISMETHOD=file
- ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
- ORDERER_GENERAL_LOCALMSPID=OrdererMSP
- ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
# enabled TLS
- ORDERER_GENERAL_TLS_ENABLED=true
- ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
- ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
- ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
# kafka
- CONFIGTX_ORDERER_ORDERERTYPE=kafka
- CONFIGTX_ORDERER_KAFKA_BROKERS=[kafka0.example.com,kafka1.example.com,kafka2.example.com,kafka3.example.com]
- ORDERER_KAFKA_RETRY_SHORTINTERVAL=1s
- ORDERER_KAFKA_RETRY_SHORTTOTAL=30s
- ORDERER_KAFKA_VERBOSE=true
working_dir: /opt/gopath/src/github.com/hyperledger/fabric
command: orderer
Kafka base
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
services:
zookeeper:
image: hyperledger/fabric-zookeeper
environment:
- ZOO_SERVERS=server.1=zookeeper0.example.com:2888:3888 server.2=zookeeper1.example.com:2888:3888 server.3=zookeeper2.example.com:2888:3888
restart: always
kafka:
image: hyperledger/fabric-kafka
restart: always
environment:
- KAFKA_MESSAGE_MAX_BYTES=103809024 # 99 * 1024 * 1024 B
- KAFKA_REPLICA_FETCH_MAX_BYTES=103809024 # 99 * 1024 * 1024 B
- KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE=false
- KAFKA_MIN_INSYNC_REPLICAS=2
- KAFKA_DEFAULT_REPLICATION_FACTOR=3
- KAFKA_ZOOKEEPER_CONNECT=zookeeper0.example.com:2181,zookeeper1.example.com:2181,zookeeper2.example.com:2181
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "3"
configtx.yaml
Organizations:
- &OrdererOrg
Name: OrdererMSP
ID: OrdererMSP
MSPDir: crypto-config/ordererOrganizations/example.com/msp
- &Org1
Name: Org1MSP
ID: Org1MSP
MSPDir: crypto-config/peerOrganizations/org1.example.com/msp
AnchorPeers:
- Host: peer0.org1.example.com
Port: 7051
- &Org2
Name: Org2MSP
ID: Org2MSP
MSPDir: crypto-config/peerOrganizations/org2.example.com/msp
AnchorPeers:
- Host: peer0.org2.example.com
Port: 7051
- &Org3
Name: Org3MSP
ID: Org3MSP
MSPDir: crypto-config/peerOrganizations/org3.example.com/msp
AnchorPeers:
- Host: peer0.org3.example.com
Port: 7051
################################################################################
Orderer: &OrdererDefaults
OrdererType: kafka
Addresses:
- orderer0.example.com:7050
- orderer1.example.com:7050
BatchTimeout: 2s
BatchSize:
MaxMessageCount: 10
AbsoluteMaxBytes: 98 MB
PreferredMaxBytes: 512 KB
Kafka:
Brokers:
- kafka0.example.com:9092
- kafka1.example.com:9092
- kafka2.example.com:9092
- kafka3.example.com:9092
Organizations:
################################################################################
Application: &ApplicationDefaults
Organizations:
################################################################################
Profiles:
ThreeOrgsOrdererGenesis:
Orderer:
<<: *OrdererDefaults
Organizations:
- *OrdererOrg
Consortiums:
SampleConsortium:
Organizations:
- *Org1
- *Org2
- *Org3
ThreeOrgsChannel:
Consortium: SampleConsortium
Application:
<<: *ApplicationDefaults
Organizations:
- *Org1
- *Org2
- *Org3
May it be a configuration error? Connection problems are almost discarded because running the same network on a local machine gives the same result.
Thanks in advance
Regards
Finally got it running smooth. Turns out the problem wasn't in docker-compose files, but in the version of fabric sdk for the web service. I was using fabric-client and fabric-ca-client both on version 1.1, and this was missing until 1.2. (More info https://jira.hyperledger.org/browse/FABN-90)
Just to clarify, I was able to see transactions happening on both orderers because of the interconnection between them, but I was only attacking the first one. When that orderer went down, network would go dark.
I understood the way fabric deals with orderers, it points to the first orderer of the list, if it is down or unreachable, moves it to the bottom of the list and targets the next one. This is what's happening since 1.2, in older versions you have to code your own error controller so that it changes to the next orderer.
I'm not sure but it could be because of different network layer. Since it's a different compose file , docker do create different network layer for each composer.
Also, I don't see network mentioned in the yaml files.
Please check list of network layer using "docker network list".