I have setup for 3 organisations on Kubernetes cluster and it is giving me following error when I restart peer node pod and tried command -> peer channel list on bash shell,
[comm.tls] ClientHandshake -> ERRO 026 Client TLS handshake failed after 2.997205009s with error: context canceled remoteaddress=10.0.94.178:7051
[grpc] WarningDepth -> DEBU 027 [core]grpc: addrConn.createTransport failed to connect to {peer0-org1:7051 peer0-org1:7051 <nil> 0 <nil>}. Err: connection error: desc = "transport: authentication handshake failed: context canceled". Reconnecting...
Error: error getting endorser client for channel: endorser client failed to connect to peer0-org1:7051: failed to create new connection: context deadline exceeded
After some time (nearly 10-15 minutes) if I tried same command,
[comm.tls] ClientHandshake -> DEBU 024 Client TLS handshake completed in 1.48399ms remoteaddress=10.0.94.178:7051
[grpc] InfoDepth -> DEBU 025 [core]Subchannel Connectivity change to READY
It seems working and give me channel list. I am not able to find out what is reason behind this, Please help for this.
Related
I have Hyperledger Fabric (1.3) network which had expired certificates.
I was not able to execute peer chaincode commands.
I have generated certificates using same ca server and replaced. Now I am able to run query commands but still getting following error on peer for invoke,
2022-11-23 15:07:01.440 UTC [grpc] createTransport -> DEBU 0be grpc:
addrConn.createTransport failed to connect to {orderer1:7050 0 <nil>}. Err :connection error:
desc = "transport: authentication handshake failed: remote error: tls: bad certificate". Reconnecting...
Kindly help. Any suggestion will be appreciated.
I have a hyperledger fabric network version 2.4.4 running on Kubernetes. The peers and other components are running behind istio ingress. The chaincode is running on dind (docker-in-docker) container and connects to peer through its URL. The problem is the chaincode connection is being dropped after few minutes. Below are the logs:
2022-07-14T04:31:13.057Z info [c-api:lib/handler.js] [assetschannel-ddc183b4] Calling chaincode Invoke() succeeded. Sending COMPLETED message back to peer
2022-07-14T04:33:04.197Z error [c-api:lib/handler.js] Chat stream with peer - on error: %j "Error: 14 UNAVAILABLE: Connection dropped\n at Object.callErrorFromStatus (/usr/local/src/node_modules/#grpc/grpc-js/build/src/call.js:31:26)\n at Object.onReceiveStatus (/usr/local/src/node_modules/#grpc/grpc-js/build/src/client.js:391:49)\n at Object.onReceiveStatus (/usr/local/src/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:328:181)\n at /usr/local/src/node_modules/#grpc/grpc-js/build/src/call-stream.js:187:78\n at processTicksAndRejections (node:internal/process/task_queues:78:11)"
I did set the following environment variables in the peer pod to keep the connection alive:
CORE_CHAINCODE_KEEPALIVE: 60000
CORE_PEER_KEEPALIVE_CLIENT_INTERVAL: 600s
CORE_PEER_KEEPALIVE_CLIENT_TIMEOUT: 2s
CORE_PEER_KEEPALIVE_DELIVERYCLIENT_INTERVAL: 20s
CORE_PEER_KEEPALIVE_MININTERVAL: 15s
but this did not resolve the issue.
Any suggestions would be appreciated.
It appears to be an issue with aws elb. The idle timeout was set to 60s which was breaking the connection between chaincode and peer when there was no communication between them. Increasing this time fixed the issue.
I have been trying to deploy a hyperledger fabric model with 3 CAs 1 orderer and 2 peer nodes. I am able to create the channel with OSADMIN command of fabric but when I try to join the channel with peer node, I get Error: error getting endorser client for channel: endorser client failed to connect to peer-govt:7051: failed to create new connection: context...... .
Here are the logs from terminal (local host machine):
2021-06-01 06:38:54.509 UTC [common.tools.configtxgen] main -> INFO 001 Loading configuration
2021-06-01 06:38:54.522 UTC [common.tools.configtxgen.localconfig] completeInitialization -> INFO 002 orderer type: etcdraft
2021-06-01 06:38:54.522 UTC [common.tools.configtxgen.localconfig] completeInitialization -> INFO 003 Orderer.EtcdRaft.Options unset, setting to tick_interval:"500ms" election_tick:10 heartbeat_tick:1 max_inflight_blocks:5 snapshot_interval_size:16777216
2021-06-01 06:38:54.522 UTC [common.tools.configtxgen.localconfig] Load -> INFO 004 Loaded configuration: /etc/hyperledger/clipod/configtx/configtx.yaml
2021-06-01 06:38:54.712 UTC [common.tools.configtxgen] doOutputBlock -> INFO 005 Generating genesis block
2021-06-01 06:38:54.712 UTC [common.tools.configtxgen] doOutputBlock -> INFO 006 Creating application channel genesis block
2021-06-01 06:38:54.712 UTC [common.tools.configtxgen] doOutputBlock -> INFO 007 Writing genesis block
cli-dd4cc5fbf-pdcgb
Status: 201
{
"name": "commonchannel",
"url": "/participation/v1/channels/commonchannel",
"consensusRelation": "consenter",
"status": "active",
"height": 1
}
cli-dd4cc5fbf-pdcgb
Error: error getting endorser client for channel: endorser client failed to connect to peer-govt:7051: failed to create new connection: context deadline exceeded
command terminated with exit code 1
Error: error getting endorser client for channel: endorser client failed to connect to peer-general:9051: failed to create new connection: context deadline exceeded
command terminated with exit code 1
One thing to note down here is I am using Kubernetes and service CLUSTER_IP for all the PODS.
here are logs from one of the PEER POD (same for other)
2021-06-01 06:38:42.180 UTC [nodeCmd] registerDiscoveryService -> INFO 01b Discovery service activated
2021-06-01 06:38:42.180 UTC [nodeCmd] serve -> INFO 01c Starting peer with ID=[peer-govt], network ID=[dev], address=[peer-govt:7051]
2021-06-01 06:38:42.180 UTC [nodeCmd] func6 -> INFO 01d Starting profiling server with listenAddress = 0.0.0.0:6060
2021-06-01 06:38:42.180 UTC [nodeCmd] serve -> INFO 01e Started peer with ID=[peer-govt], network ID=[dev], address=[peer-govt:7051]
2021-06-01 06:38:42.181 UTC [kvledger] LoadPreResetHeight -> INFO 01f Loading prereset height from path [/var/hyperledger/production/ledgersData/chains]
2021-06-01 06:38:42.181 UTC [blkstorage] preResetHtFiles -> INFO 020 No active channels passed
2021-06-01 06:38:56.006 UTC [core.comm] ServerHandshake -> ERRO 021 Server TLS handshake failed in 24.669µs with error tls: first record does not look like a TLS handshake server=PeerServer remoteaddress=172.17.0.1:13258
2021-06-01 06:38:57.007 UTC [core.comm] ServerHandshake -> ERRO 022 Server TLS handshake failed in 17.772µs with error tls: first record does not look like a TLS handshake server=PeerServer remoteaddress=172.17.0.1:29568
2021-06-01 06:38:58.903 UTC [core.comm] ServerHandshake -> ERRO 023 Server TLS handshake failed in 13.581µs with error tls: first record does not look like a TLS handshake server=PeerServer remoteaddress=172.17.0.1:32615
To overcome this issue, I tried disabling the TLS by setting CORE_PEER_TLS_ENABLED to FALSE
then the proposal gets submitted but the orderer POD throws the same error of TLS handshake failed.........
Here are the commands I am using to join the channel from cli pod:
kubectl -n hyperledger -it exec $CLI_POD -- sh -c "export FABRIC_CFG_PATH=/etc/hyperledger/clipod/config && export CORE_PEER_LOCALMSPID=GeneralMSP && export CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/clipod/organizations/peerOrganizations/general.example.com/peers/peer0.general.example.com/tls/ca.crt && export CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/clipod/organizations/peerOrganizations/general.example.com/users/Admin#general.example.com/msp && export CORE_PEER_ADDRESS=peer-general:9051 && peer channel join -b /etc/hyperledger/clipod/channel-artifacts/$CHANNEL_NAME.block -o orderer:7050 --tls --cafile /etc/hyperledger/clipod/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem"
I am stuck on this problem, any help will be appreciated.
Thank you
I have fixed it. The issue I was facing was because of not setting the CORE_PEER_TLS_ENABLED = true for CLI pod.
One thing I have got learn from this whole model, whenever you see TLS issue, first to check for would be checking CORE_PEER_TLS_ENABLED variable. Make sure you have set it for all the pods or containers you are trying to interact with. The case can be false(for no TLS) or true(for using TLS) depending on your deployment.
Other things to keep in mind is using the correct variables of fabric including FABRIC_CFG_PATH, CORE_PEER_LOCALMSPID, CORE_PEER_TLS_ROOTCERT_FILE, CORE_PEER_MSPCONFIGPATH and some others depending on your command.
I am trying to install a HLF (v2) network on 2 physical machines.
Machine 1:
1 orderer 2 peers 2 couchdb 2 zookeeper 2 kafka
Machine 2:
1 orderer 2 peers 2 couchdb 1 zookeeper 2 kafka
The containers is running on both machines.
However, when i try to create channel on machine 1
peer channel create -o orderer.example.com:7050 -c $CHANNEL_NAME -f ./channel-artifacts/channel.tx --tls --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
I receive this error:
Error: failed to create deliver client for orderer: orderer client failed to connect to orderer.example.com:7050: failed to create new connection: context deadline exceeded
docker logs orderer.example.com -f
output:
FO 00c [channel: byfn-sys-channel] Setting up the topic for this channel...
2020-06-09 20:10:31.316 UTC [orderer.consensus.kafka.sarama] Open -> DEBU 00d ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2020-06-09 20:10:31.320 UTC [orderer.consensus.kafka.sarama] withRecover -> DEBU 00e Failed to connect to broker kafka0.example.com:9092: dial tcp 172.20.0.5:9092: connect: connection refused
2020-06-09 20:10:31.321 UTC [orderer.consensus.kafka.sarama] Open -> DEBU 00f ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2020-06-09 20:10:31.323 UTC [orderer.consensus.kafka.sarama] withRecover -> DEBU 010 Failed to connect to broker kafka1.example.com:9092: dial tcp 172.20.0.9:9092: connect: connection refused
2020-06-09 20:10:31.323 UTC [orderer.consensus.kafka.sarama] Open -> DEBU 011 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2020-06-09 20:10:31.342 UTC [orderer.consensus.kafka.sarama] withRecover -> DEBU 012 Failed to connect to broker kafka2.example.com:9092: dial tcp: lookup kafka2.example.com on 127.0.0.11:53: no such host
2020-06-09 20:10:31.342 UTC [orderer.consensus.kafka.sarama] Open -> DEBU 013 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2020-06-09 20:10:31.355 UTC [orderer.consensus.kafka.sarama] withRecover -> DEBU 014 Failed to connect to broker kafka3.example.com:9092: dial tcp: lookup kafka3.example.com on 127.0.0.11:53: no such host
2020-06-09 20:10:36.355 UTC [orderer.consensus.kafka.sarama] Open -> DEBU 015 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
You need to configure a unique "clientid" per consumer in Sarama.
Kafka has the concept of "consumer groups" and "consumers" (aka clientid). Each consumer must have a unique id within the consumer group. Kafka will assign partition(s) to consumers. Kafka will keep track of the offset position of all consumers using the consumer group.
Hello Everyone i am working on setup of fabric default first-network in kubernetes. But when i am instantiate the chaincode it gives me error. Please check below are my peer logs.
2019-07-22 07:25:02.134 UTC [endorser] SimulateProposal -> ERRO 066 [mychannel][c4b4e2ae] failed to invoke chaincode name:"lscc" , error: container exited with 0
github.com/hyperledger/fabric/core/chaincode.(*RuntimeLauncher).Launch.func1
/opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/runtime_launcher.go:63
runtime.goexit
/opt/go/src/runtime/asm_amd64.s:1333
chaincode registration failed
Getting this error on Cli :-
2019-07-22 07:24:58.263 UTC [chaincodeCmd] checkChaincodeCmdParams -> INFO 001 Using default escc
2019-07-22 07:24:58.264 UTC [chaincodeCmd] checkChaincodeCmdParams -> INFO 002 Using default vscc
Error: could not assemble transaction, err proposal response was not successful, error code 500, msg chaincode registration failed: container exited with 0
Once check if all your docker containers are up and running and if you are simply running the sample network without making any changes to the smart contract and the docker files then you can simply stop your network and freshly start the network(it worked in my case).
I have check with my configuration files it was due to wrong CORE_PEER_CHAINCODELISTENADDRESS env variable value for the peer.