Using the last version of openshift origin v3.10.0 i run the following command on centos VM:
oc cluster up --public-hostname=192.168.56.15 --http-proxy=http://proxy.ip:port --https-proxy=https://proxy.ip:port --no-proxy=[192.168.56.0/24,172.0.0. 0/8,192.168.56.15,192.168.56.15,localhost]
In result i get:
Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.10 is available ...
Checking type of volume mount ...
Determining server IP ...
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.10 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.10 ...
I1003 10:58:00.643521 3446 flags.go:30] Running "create-kubelet-flags"
I1003 10:58:01.314805 3446 run_kubelet.go:48] Running "start-kubelet"
I1003 10:58:01.549316 3446 run_self_hosted.go:172] Waiting for the kube-apiserver to be ready ...
E1003 11:03:01.559324 3446 run_self_hosted.go:542] API server error: Get https://127.0.0.1:8443/healthz?timeout=32s: dial tcp 127.0.0.1:8443: getsockopt: connection refused ()
Error: timed out waiting for the condition
And while following the log of docker i notice the following error:
E1003 github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/kubelet/kubelet.go:452: Failed to list *v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp [::1]:8443: getsockopt: connection refused
Which is a normal behavior since netstat shows only one port opened:
tcp6 0 0 :::10250 :::* LISTEN 3894/hyperkube
PS:
As you can see i use proxy.
I tried to use a local resolving, using dns instead of ip# and since i don't have a DNS server i used /etc/hosts, same problem.
Related
I am deploying kubernetes in Cloud and I'm trying to call another container inside the same pod through an API.
I am using localhost but also I treid with 127.0.0.1. Also, I tried with the container's name.
2022/11/04 15:50:47 dial tcp [::1]:4245: connect: connection refused
2022/11/04 15:50:47 Successfully processed file.json file
2022/11/04 15:50:47 Get "http://localhost:4245/api/admin/projects/default": dial tcp [::1]:4245: connect: connection refused
panic: Get "http://localhost:4245/api/admin/projects/default": dial tcp [::1]:4245: connect: connection refused
goroutine 1 [running]:
log.Panic({0xc000119dc8?, 0xc000166000?, 0x6aaaea?})
/opt/app-root/src/sdk/go1.19.2/src/log/log.go:388 +0x65
main.StatusServer({0xc000020570?, 0x30?}, {0x0, 0x0})
/build/script.go:197 +0x1ee
main.ProcessData({0xc000020041, 0x15}, {0x0, 0x0}, {0xc00002000f?, 0x43ce05?})
/build/script.go:291 +0xa6
main.main()
/build/script.go:443 +0xc5
Any idea if I can call the container like that?
You get a connection refused means you reached localhost and it decided to refuse the connection.
This is most likly because nothing is listening on the port.
If it was a firewall issue the request would timeout.
You can check listening ports with command like:
netstat -an
If not installed maybe you can try it from the workernode where the pod is running.
Another method of testing is to use
curl http://127.0.0.1:4245
This will probably result in same connection refused.
Are you really sure the container is running in same pod?
Please check your deployment and service.
If you cant find the failure please come back with more information so it can be analysed.
I am trying to connect to a nfs server. where i seeing the below issue.
mount.nfs: timeout set for Mon May 11 19:27:01 2020
mount.nfs: trying text-based options 'nfsvers=3,addr=IP'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: portmap query retrying: RPC: Timed out
mount.nfs: prog 100003, trying vers=3, prot=17
mount.nfs: portmap query failed: RPC: Unable to receive - Connection refused
nfs-utils is installed and running in the client.
rpcbind is running in the client machine.
Ports 111, 2049 and 892 are opened to the NFS server.
However i am not sure if i am missing anything to resolve this issue.
When ever i try to get the cluster up using "oc cluster up"
Below is the error I get. Kindly help on how to fix this
[mano#mano ~]$ oc cluster up
Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.11 is available ...
Checking type of volume mount ...
Determining server IP ...
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.11 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.11 ...
I0923 13:40:32.364326 15396 config.go:40] Running "create-master-config"
I0923 13:40:59.938492 15396 config.go:46] Running "create-node-config"
I0923 13:41:10.721711 15396 flags.go:30] Running "create-kubelet-flags"
I0923 13:41:18.241285 15396 run_kubelet.go:49] Running "start-kubelet"
I0923 13:41:23.016238 15396 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...
E0923 13:46:23.023479 15396 run_self_hosted.go:571] API server error: Get https://127.0.0.1:8443/healthz?timeout=32s: dial tcp 127.0.0.1:8443: connect: connection refused ()
Error: timed out waiting for the condition
OC version
[mano#mano` ~]$ oc version
oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO
followed the article :https://github.com/openshift/origin/blob/release-3.11/docs/cluster_up_down.md
yet no luck
Filebeat is running on Machine B which read logs and push to ELK logstash on Machine A.
But in the Machine B filebeat log, it shows the error i/o timeout
2019-08-24T12:13:10.065+0800 ERROR pipeline/output.go:100 Failed to connect to backoff(async(tcp://example.com:5044)): dial tcp xx.xx.xx.xx:5044: i/o timeout
2019-08-24T12:13:10.065+0800 INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://example.com:5044)) with 1 reconnect attempt(s)
I've check the logstash on Machine A which running well, can listening on 0 0.0.0.0:5044
Here is the logstash log
[INFO ] 2019-08-24 12:09:35.217 [[main]-pipeline-manager] beats - Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
And here is netstat output,
$ sudo netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:5044 0.0.0.0:* LISTEN 20668/java
I also check the firewall on Machine A is closed.
$ firewall-cmd --list-all
FirewallD is not running
$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy DROP)
target prot opt source destination
I also use telnet to connect Machine A, But I get this,
$ telnet example.com 5044
Trying xx.xx.xx.xx...
telnet: connect to address xx.xx.xx.xx: Connection timed out
I run the filebeat with same config on Machine A(local) to check it the config for filebeat on Machine B(remote) is wrong, it works well.
2019-08-24T14:17:35.195+0800 INFO pipeline/output.go:95 Connecting to backoff(async(tcp://localhost:5044))
2019-08-24T14:17:35.198+0800 INFO pipeline/output.go:105 Connection to backoff(async(tcp://localhost:5044)) established
At last I find it's caused by the VPS Provider aliyun, it only open some common port such 22, 80,443.
I need to login to aliyun VPS management page, and open 5044 to make VPS Provider bypass the 5044 port.
*Note: * Attachment: some other issues I encountered when config filebeat with ELK.
**Issue 1: ** Failed to connect to backoff(async(tcp://ip:5044)): dial tcp ip:5044: connect: connection refused
2019-08-26T10:25:41.955+0800 ERROR pipeline/output.go:100 Failed to connect to backoff(async(tcp://example.com:5044)): dial tcp xx.xx.xx.xx:5044: connect: connection refused
2019-08-26T10:25:41.955+0800 INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://example:5044)) with 2 reconnect attempt(s)
Issue 2: Failed to publish events caused by: write tcp ip:46890->ip:5044: write: connection reset by peer
2019-08-26T10:28:32.274+0800 ERROR logstash/async.go:256 Failed to publish events caused by: write tcp xx.xx.xx.xx:46890->xx.xx.xx.xx:5044: write: connection reset by peer
2019-08-26T10:28:33.311+0800 ERROR pipeline/output.go:121 Failed to publish events: write tcp xx.xx.xx.xx:46890->xx.xx.xx.xx:5044: write: connection reset by peer
Issue 3: Filebeat error: lumberjack protocol error and Logstash error: OPENSSL_internal:WRONG_VERSION_NUMBER
Filebeat log error,
2019-08-26T08:49:09.505+0800 INFO pipeline/output.go:95 Connecting to backoff(async(tcp://example.com:5044))
2019-08-26T08:49:09.588+0800 INFO pipeline/output.go:105 Connection to backoff(async(tcp://example.com:5044)) established
2019-08-26T08:49:09.605+0800 ERROR logstash/async.go:256 Failed to publish events caused by: lumberjack protocol error
2019-08-26T08:49:09.606+0800 ERROR logstash/async.go:256 Failed to publish events caused by: client is not connected
Logstash log,
[INFO ] 2019-08-26 08:49:29.444 [defaultEventExecutorGroup-4-2] BeatsHandler - [local: 0.0.0.0:5044, remote: undefined] Handling exception: javax.net.ssl.SSLHandshakeException: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER
[WARN ] 2019-08-26 08:49:29.445 [nioEventLoopGroup-2-7] DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
...
All the three issues are caused by miss configuration, here is the workable config,
logstash version,
/usr/share/logstash/bin/logstash -V
logstash 7.3.1
filebeat version,
/usr/share/filebeat/bin/filebeat version
filebeat version 7.3.1 (amd64), libbeat 7.3.1 [a4be71b90ce3e3b8213b616adfcd9e455513da45 built 2019-08-19 19:30:50 +0000 UTC]
logstash conf file /etc/logstash/conf.d/beat.conf
input {
beats {
port => 5044
ssl => true
ssl_certificate_authorities => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
ssl_verify_mode => "peer"
}
}
output {
elasticsearch {
hosts => "http://127.0.0.1:9200"
manage_template => false
index => "%{[#metadata][beat]}-%{[#metadata][version]}-%{+YYYY.MM.dd}"
document_type => "%{[#metadata][type]}"
}
}
filebeat conf file /etc/filebeat/filebeat.yml
#=========================== Filebeat inputs =============================
filebeat.inputs:
# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.
- type: log
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /data/error_logs/Log_error_201908
#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["example.com:5044"]
# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
ssl.certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
# Certificate for SSL client authentication
ssl.certificate: "/etc/pki/tls/certs/logstash-forwarder.crt"
# Client Certificate Key
ssl.key: "/etc/pki/tls/private/logstash-forwarder.key"
Hi I am trying to install kubernetes on a vm with ubuntu 14.04, using manual from http://kubernetes.io/docs/getting-started-guides/ubuntu/
I tried
KUBERNETES_PROVIDER=ubuntu ./kube-up.sh
I am getting following error:
etcd start/pre-start, process 1557
Error: client: etcd cluster is unavailable or misconfigured
error #0: dial tcp 127.0.0.1:4001: getsockopt: connection refused
error #1: dial tcp 127.0.0.1:2379: getsockopt: connection refused
Following is the content of /var/log/upstart/etcd.log:
2016-10-24 13:28:54.269743 I | etcdmain: listening for peers on http://localhost:2380
2016-10-24 13:28:54.269852 I | etcdmain: listening for peers on http://localhost:7001
2016-10-24 13:28:54.269921 I | etcdmain: listening for client requests on http://127.0.0.1:4001
2016-10-24 13:28:54.269994 I | etcdmain: stopping listening for client requests on http://127.0.0.1:4001
2016-10-24 13:28:54.270017 I | etcdmain: stopping listening for peers on http://localhost:7001
2016-10-24 13:28:54.270052 I | etcdmain: stopping listening for peers on http://localhost:2380
I am using it behind corporate proxy, http_proxy, https_proxy and no_proxy has been set
I tried versions KUBE_VERSION=1.2.0, FLANNEL_VERSION=0.5.0, ETCD_VERSION=2.2.0
I even tried different version for KUBE 1.1.8, 1.3.0, 1.4.0, 1.4.4 But ended up is same error.
Kindly Help
I found out solution, Kindly refer this page https://github.com/kubernetes/kubernetes/issues/19235#issuecomment-255987755