I have install rabbitmq cluster using rabbitmq cluster operator. I have also added rabbitmq topology operator. I am trying to create queues using rabbitmq topology operator using following yml file
kind: Queue
metadata:
name: software-results
namespace: rabbitmq-system
spec:
name: software-results # name of the queue
type: quorum # without providing a queue type, rabbitmq creates a classic queue
autoDelete: false
durable: true # seting 'durable' to false means this queue won't survive a server restart
rabbitmqClusterReference:
name: client-queues
I am getting error as
Error from server (InternalError): error when creating "singleQueue.yml": Internal error occurred: failed calling webhook "vqueue.kb.io": failed to call webhook: Post "https://webhook-service.rabbitmq-system.svc:443/validate-rabbitmq-com-v1beta1-queue?timeout=10s": dial tcp 10.97.65.156:443: connect: connection refused
I tried to search for the same but didn't find much. Can anyone help me what exactly is going wrong here ?
Related
We are configuring KEDA to auto-scale our deployment based on the RabbitMQ queue. Our setup makes TLS mandatory, and we have configured the RabbitMQ management plugin on HTTPS. We are trying to connect with the RabbitMQ host on https like:
"https://user:password#host:15671/", but the Rabbit MQ scaler is not able to connect to the Rabbit MQ host, and we are getting the following error: "context deadline exceeded (Client.Timeout exceeded while awaiting headers)".
We are able to connect with RabbitMQ host on http protocol like "http://user:password#host:15672/". Below is our trigger definition in scaled object:
triggers:
- type: rabbitmq
metadata:
host: https://user:password#host:15671/
mode: QueueLength
queueName: MyQueueName
value: "5"
excludeUnacknowledged: "true"
NOTE: We can't use the AMQP protocol in the host because we're using the "excludeUnacknowledged" property, which isn't applicable to the AMQP protocol.
We are facing this issue with KEDA version 2.8 and 2.9.
Keda does not support HTTPS yet: https://github.com/kedacore/keda/pull/1073
I am trying to set up a multi-cluster architecture. I have a Spring Boot API that I want to run on a second cluster (for isolation purposes). I have set that up using the gateway.networking.k8s.io API. I am using a Gateway that has an SSL certificate and matches an IP address that's registered to my domain in the DNS registry. I am then setting up an HTTPRoute for each service that I am running on the second cluster. That works fine and I can communicate between our clusters and everything works as intended but there is a problem:
There is a timeout of 30s by default and I cannot change it. I want to increase it as the application in the second cluster is a WebSocket and I obviously would like our WebSocket connections to stay open for more than 30s at a time. I can see that in the backend service that's created from our HTTPRoute there is a timeout specified as 30s. I found a command to increase it gcloud compute backend-services update gkemcg1-namespace-store-west-1-8080-o1v5o5p1285j --timeout=86400
When I run that command it would increase the timeout and the webSocket connection will be kept alive. But after a few minutes this change gets overridden (I suspect that it's because it's managed by the yaml file). This is the yaml file for my backend service
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: public-store-route
namespace: namespace
labels:
gateway: external-http
spec:
hostnames:
- "my-website.example.org"
parentRefs:
- name: external-http
rules:
- matches:
- path:
type: PathPrefix
value: /west
backendRefs:
- group: net.gke.io
kind: ServiceImport
name: store-west-1
port: 8080
I have tried to add either a timeout, timeoutSec, or timeoutSeconds under every level with no success. I always get the following error:
error: error validating "public-store-route.yaml": error validating data: ValidationError(HTTPRoute.spec.rules[0].backendRefs[0]): unknown field "timeout" in io.k8s.networking.gateway.v1beta1.HTTPRoute.spec.rules.backendRefs; if you choose to ignore these errors, turn validation off with --validate=false
Surely there must be a way to configure this. But I wasn't able to find anything in the documentation referring to a timeout. Am I missing something here?
How do I configure the timeout?
Edit:
I have found this resource: https://cloud.google.com/kubernetes-engine/docs/how-to/configure-gateway-resources
I have been trying to set up a LBPolicy and attatch it it the Gateway, HTTPRoute, Service, or ServiceImport but nothing has made a difference. Am I doing something wrong or is this not working how it is supposed to? This is my yaml:
kind: LBPolicy
apiVersion: networking.gke.io/v1
metadata:
name: store-timeout-policy
namespace: sandstone-test
spec:
default:
timeoutSec: 50
targetRef:
name: public-store-route
group: gateway.networking.k8s.io
kind: HTTPRoute
I am running into the below error when creating compute IP.
Config connector is already enabled, and it is a private cluster hosted on a shared network.
Version 1.17.15-gke.800
$ kubectl apply -f webapp-compute-ip. yaml
Error from server (InternalError): error when creating "webapp-compute-ip.yaml": Internal error occurred: failed calling webhook "annotation-defaulter.cnrm.cloud.google.com": Post https://cnrm-validating-webhook.cnrm-system.svc:443/annotation-defaulter?timeout=30s: dial tcp 192.168.66.130:9443: i/o timeout
$cat webapp-compute-ip.yaml
apiVersion: compute.cnrm.cloud.google.com/v1beta1
kind: ComputeAddress
metadata:
name: webapp-ip-test
namespace: sandbox
labels:
app: webapp
environment: test
annotations:
cnrm.cloud.google.com/project-id: "cluster-name"
spec:
location: global`
This problem was due to a config connector version issue.
There was a change in the webhook default port, from 443 to 9443, see
Config connector version depends on GKE version, I did not have any control over it, moreover there no is public documentation available on what config connector version is available with the GKE version. There is an existing request here.
The solution was for me to add port 9443 in the firewall rule.
I've got an InfluxDB as database service in the default namespace.
It's service is called influxdb and works fine with chronograf to visualize the data.
Now i'd like to connect with an other deployment from the namspace test to this service. It's a python application. The normal python Influxdb Lib uses Requests to connect to the db.
Architecture Overview
Istio is also installed.
Namspace: Default
Influxdb Deployment
Influxdb Service
Chronograf Deployment (visualise influxdb)
Chronograf Service to Ingress(for external web access)
Namespace: test
Python App which should connect to influxdb for processing etc.
Influxdb Service (which points to influxdb.default.svc.cluster.local)
Therefore i created a service in the Namespace test which points to the service of influxdb in the default namespace.
apiVersion: v1
kind: Service
metadata:
name: influxdb
labels:
app: pythonapp
namespace: test
spec:
type: ExternalName
externalName: influxdb.default.svc.cluster.local
ports:
- port: 8086
name: http
- port: 8088
name: http-flux
And now deployed the python app which points to the influxdb Service. Which keeps getting a http connection error.
2020-07-03 13:02:05 - db.meterdb [meterdb.__init__:57] - ERROR - Oops, something wen't wrong during init of db. message: HTTPConnectionPool(host='influxdb', port=8086): Max retries exceeded with url: /query?q=SHOW+DATABASES (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6863ed6310>: Failed to establish a new connection: [Errno 111] Connection refused'))
2020-07-03 13:02:05 - db.meterdb [meterdb.check_connection:113] - ERROR - can't reach db server... message: HTTPConnectionPool(host='influxdb', port=8086): Max retries exceeded with url: /ping (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6863e57650>: Failed to establish a new connection: [Errno 111] Connection refused'))
When I visualise the traffic with kiali I see, that the Python app tries to connect to the influxdb service, but which is unknown with http traffic.
I don't know how to get it work to use the created influxdb Service.
Connection settings for python influxdb Client library. Link to python influxdb lib
host=influxdb
port=8086
Traffic from Kiali
How can i route the traffik to the right service ?
It seems to me, that it routes the traffik to an unknown service because it's http and not tcp.
You don't need the
kind: Service
metadata:
name: influxdb
labels:
app: pythonapp
namespace: test
Just access the service directly in your python request:
requests.get('influxdb.default.svc.cluster.local:8086')
And this can be more configurable.
# Kubernetes deployment
containers:
- name: pythonapp
env:
- name: DB_URL
value: influxdb.default.svc.cluster.local:8086
# python
DB = os.environ['DB_URL']
requests.get(DB)
I'm trying to apply the following StorageBucket resource from Google's sample manifest:
apiVersion: storage.cnrm.cloud.google.com/v1alpha2
kind: StorageBucket
metadata:
labels:
label-one: "value-one"
name: dmacthedestroyer-hdjkwhekhjewkjeh-storagebucket-sample
spec:
lifecycleRule:
- action:
type: Delete
condition:
age: 7
versioning:
enabled: true
cors:
- origin: ["http://example.appspot.com"]
responseHeader: ["Content-Type"]
method: ["GET", "HEAD", "DELETE"]
maxAgeSeconds: 3600
The response times out with the following errors:
$ kubectl apply -f sample.yaml
Error from server (Timeout): error when creating "sample.yaml": Timeout: request did not complete within requested timeout 30s
UPDATE:
For some unknown reason, the error message has changed to this:
Error from server (InternalError): error when creating "sample.yaml": Internal error occurred: failed calling webhook "cnrm-deny-unknown-fields-webhook.google.com": Post https://cnrm-validating-webhook-service.cnrm-system.svc:443/deny-unknown-fields?timeout=30s: net/http: TLS handshake timeout
I've tested this on two different networks, with the same error result.
I installed the Config Connector components as described in their documentation, using a dedicated service account with the roles/owner permissions, exactly as stated in the above instructions.
I have successfully deployed IAMServiceAccount and IAMServiceAccountKey resources with this setup.
How should I proceed to troubleshoot this?
My issue was due to an incorrect service account configuration.
In particular, I was assigning the owner role to a different project.
After properly configuring my service account, the timeout errors are resolved.