Hyperledger peers with TLS in kubernetes cluster constantly keep throwing TLS handshake errors - kubernetes

Question

Below are the peer logs:
2019-12-06 07:00:31.121 UTC [core.comm] ServerHandshake -> ERRO fa975 TLS handshake failed with error EOF server=ChaincodeServer remoteaddress=192.168.131.215:25731
2019-12-06 07:00:31.215 UTC [core.comm] ServerHandshake -> ERRO fa976 TLS handshake failed with error EOF server=ChaincodeServer remoteaddress=192.168.131.215:20784
2019-12-06 07:00:31.301 UTC [core.comm] ServerHandshake -> ERRO fa977 TLS handshake failed with error EOF server=PeerServer remoteaddress=192.168.131.215:8059
2019-12-06 07:00:31.512 UTC [core.comm] ServerHandshake -> ERRO fa978 TLS handshake failed with error EOF server=ChaincodeServer remoteaddress=192.168.163.185:46359
2019-12-06 07:00:31.768 UTC [core.comm] ServerHandshake -> ERRO fa979 TLS handshake failed with error EOF server=PeerServer remoteaddress=192.168.131.215:34603
Everything is working fine. We are able to do transactions on the chaincode.
Can anyone please help us on this issue?
EDITED: 9th Dec. 2019
Below is the peer deployment yaml file
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: korg60
name: peer1-korg60
spec:
replicas: 1
strategy: {}
selector:
matchLabels:
app: hyperledger
role: peer
org: korg60
name: peer1-korg60
template:
metadata:
labels:
app: hyperledger
role: peer
org: korg60
name: peer1-korg60
spec:
containers:
- name: couchdb
image: hyperledger/fabric-couchdb:latest
ports:
- containerPort: 5984
- name: peer1-korg60
image: hyperledger/fabric-peer:1.4
env:
- name: FABRIC_CA_CLIENT_HOME
value: /opt/gopath/src/github.com/hyperledger/fabric/peer
- name: FABRIC_CA_CLIENT_TLS_CERTFILES
value: /data/korg60-ca-chain.pem
- name: ENROLLMENT_URL
value: http://peer1:peer1pw#ica-korg60.korg60:7054
- name: PEER_NAME
value: peer1-korg60
- name: PEER_HOME
value: /opt/gopath/src/github.com/hyperledger/fabric/peer
- name: PEER_HOST
value: some.domain.com:7051
- name: PEER_NAME_PASS
value: peer1:peer1pw
- name: CORE_PEER_ADDRESSAUTODETECT
value: "true"
- name: CORE_PEER_ID
value: peer1-korg60
- name: CORE_PEER_ADDRESS
value: some.domain.com:7051
- name: CORE_PEER_LOCALMSPID
value: korg60MSP
- name: CORE_PEER_MSPCONFIGPATH
value: /opt/gopath/src/github.com/hyperledger/fabric/peer/msp
- name: CORE_VM_ENDPOINT
value: unix:///host/var/run/docker.sock
- name: CORE_VM_DOCKER_ATTACHSTDOUT
value: "true"
- name: FABRIC_LOGGING_SPEC
value: "peer=INFO"
- name: CORE_PEER_TLS_ENABLED
value: "true"
- name: CORE_PEER_TLS_CERT_FILE
value: /opt/gopath/src/github.com/hyperledger/fabric/peer/tls/server.crt
- name: CORE_PEER_TLS_KEY_FILE
value: /opt/gopath/src/github.com/hyperledger/fabric/peer/tls/server.key
- name: CORE_PEER_TLS_ROOTCERT_FILE
value: /data/korg60-ca-chain.pem
- name: CORE_PEER_TLS_CLIENTAUTHREQUIRED
value: "false"
- name: CORE_PEER_TLS_CLIENTROOTCAS_FILES
value: /data/korg60-ca-chain.pem
- name: CORE_PEER_TLS_CLIENTCERT_FILE
value: /data/tls/peer1-korg60-client.crt
- name: CORE_PEER_TLS_CLIENTKEY_FILE
value: /data/tls/peer1-korg60-client.key
- name: CORE_PEER_GOSSIP_USELEADERELECTION
value: "true"
- name: CORE_PEER_GOSSIP_ORGLEADER
value: "false"
- name: CORE_PEER_GOSSIP_EXTERNALENDPOINT
value: some.domain.com:7051
- name: CORE_PEER_GOSSIP_SKIPHANDSHAKE
value: "true"
- name: CORE_PEER_CHAINCODELISTENADDRESS
value: 0.0.0.0:7052
- name: CORE_LEDGER_STATE_STATEDATABASE
value: CouchDB
- name: CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS
value: localhost:5984
- name: ORG
value: korg60
- name: ORG_ADMIN_CERT
value: /data/orgs/korg60/msp/admincerts/cert.pem
- name: GODEBUG
value: "netdns=go"
ports:
- containerPort: 7051
- containerPort: 7052
- containerPort: 7053
command: ["sh"]
args: ["-c", "/scripts/start-peer.sh 2>&1"]
volumeMounts:
- mountPath: /scripts
name: rca-scripts
- mountPath: /data
name: rca-data
- mountPath: /host/var/run/
name: run
volumes:
- name: rca-scripts
persistentVolumeClaim:
claimName: rca-scripts-korg60-pvc
- name: rca-data
persistentVolumeClaim:
claimName: rca-data-korg60-pvc
- name: run
hostPath:
path: /run
---
apiVersion: v1
kind: Service
metadata:
namespace: korg60
name: peer1-korg60
spec:
selector:
app: hyperledger
role: peer
org: korg60
name: peer1-korg60
type: NodePort
ports:
- name: endpoint
protocol: TCP
port: 7051
targetPort: 7051
nodePort: 30401
- name: endpoint-chaincode
protocol: TCP
port: 7052
targetPort: 7052
nodePort: 30402
Below is the ordere yaml file.
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: koinearth
name: orderer1-koinearth
spec:
replicas: 1
strategy: {}
selector:
matchLabels:
app: hyperledger
role: orderer
org: koinearth
name: orderer1-koinearth
template:
metadata:
labels:
app: hyperledger
role: orderer
org: koinearth
name: orderer1-koinearth
spec:
containers:
- name: orderer1-koinearth
image: hyperledger/fabric-orderer:1.4
env:
- name: FABRIC_CA_CLIENT_HOME
value: /etc/hyperledger/orderer
- name: FABRIC_CA_CLIENT_TLS_CERTFILES
value: /data/koinearth-ca-chain.pem
- name: FABRIC_LOGGING_SPEC
value: "peer=INFO"
- name: ENROLLMENT_URL
value: http://orderer1:orderer1pw#ica-koinearth.koinearth:7054
- name: ORDERER_HOME
value: /etc/hyperledger/orderer
- name: ORDERER_HOST
value: orderer1-koinearth.koinearth
- name: ORDERER_GENERAL_LISTENADDRESS
value: 0.0.0.0
- name: ORDERER_GENERAL_GENESISMETHOD
value: file
- name: ORDERER_GENERAL_GENESISFILE
value: /data/genesis.block
- name: ORDERER_GENERAL_LOCALMSPID
value: koinearthMSP
- name: ORDERER_GENERAL_LOCALMSPDIR
value: /etc/hyperledger/orderer/msp
- name: ORDERER_GENERAL_TLS_ENABLED
value: "true"
- name: ORDERER_GENERAL_TLS_CLIENTAUTHREQUIRED
value: "false"
- name: ORDERER_GENERAL_TLS_PRIVATEKEY
value: /etc/hyperledger/orderer/tls/server.key
- name: ORDERER_GENERAL_TLS_CERTIFICATE
value: /etc/hyperledger/orderer/tls/server.crt
- name: ORDERER_GENERAL_LOGLEVEL
value: debug
- name: ORDERER_DEBUG_BROADCASTTRACEDIR
value: data/logs
- name: ORG
value: koinearth
- name: ORG_ADMIN_CERT
value: /data/orgs/koinearth/msp/admincerts/cert.pem
- name: ORDERER_GENERAL_TLS_ROOTCAS
value: '[/data/koinearth-ca-chain.pem]'
- name: ORDERER_GENERAL_TLS_CLIENTROOTCAS
value: '[/data/koinearth-ca-chain.pem]'
- name: ORDERER_KAFKA_VERBOSE
value: "true"
- name: ORDERER_KAFKA_VERSION
value: 1.0.0
- name: GODEBUG
value: "netdns=go"
ports:
- containerPort: 7050
command: ["sh"]
args: ["-c", "/scripts/start-orderer.sh 2>&1"]
volumeMounts:
- mountPath: /etc/hyperledger/fabric-ca
name: orderer
- mountPath: /scripts
name: rca-scripts
- mountPath: /data
name: rca-data
volumes:
- name: orderer
persistentVolumeClaim:
claimName: orderer-koinearth-pvc
- name: rca-scripts
persistentVolumeClaim:
claimName: rca-scripts-koinearth-pvc
- name: rca-data
persistentVolumeClaim:
claimName: rca-data-koinearth-pvc
---
apiVersion: v1
kind: Service
metadata:
namespace: koinearth
name: orderer1-koinearth
spec:
selector:
app: hyperledger
role: orderer
org: koinearth
name: orderer1-koinearth
type: NodePort
ports:
- name: endpoint
protocol: TCP
port: 7050
targetPort: 7050
nodePort: 30300
Peer and orderer identity is created in the startup scripts and stored locally in the container.

Answer 1

This happens when you are using wrong certificates.
What are the two parties?
2 peers or 1 peer 1 orderer?
Or maybe the client?
The two parties must have valid TLS certificates, here you are using some wrong ones.