I'm trying to deploy a cloned website in a second namespace. But I forgot to change the URL for Ambassador's mapping resource. So both clones are the same URL https://mywebsite.dev which supposed to be https://mywebsite.dev and https://testing.mywebsite.dev. The main website was taken down soon after I kubectl apply the minor website. Both sites are offline now. Basically that means I ran the mapping.yaml twice in different namespaces.
Is there any chance that duplicated mapping causes the error? How to fix that?
This is the yaml file:
apiVersion: getambassador.io/v1
kind: Mapping
metadata:
name: mywebsite
spec:
cors:
credentials: true
headers: x-csrf-token,Content-Type,Authorization
methods: POST, PATCH, GET, OPTIONS, PUT, DELETE
origins:
- https://mywebsite.dev
host: mywebsite.dev
load_balancer:
cookie:
name: stickyname
policy: ring_hash
prefix: /
resolver: endpoint
service: http://my-website-service.default
timeout_ms: 60000
It seems an error from Ambassador 0.86 when it has a duplicated/bad record of mapping, then got stuck there, doesn't accept any new mapping record. I fixed it by delete ambassador pods then let the deployment recreate them. It works fine later.
Related
argoproj/argocd:v1.8.7
Have a helm chart (1 with ingress, 1 with deployment/service/cm).
It has sync policies of automated (prune and self-heal). When trying to delete them from the argocd dashboard, they are getting deleted (no more on the k8s cluster), however the status on the dashboard has been stuck at Deleting.
If I try to click sync, it shows -> Unable to deploy revision: application is deleting.
Any ideas why it's stuck on Deleting status even though all resources have been deleted already ? Is there a way to refresh the status in the dashboard to reflect that actual state?
Thanks!
================
Update:
After doing cascade delete, this is the screenshot (i've removed the app names that why it's white for some part)
Doing kubectl get all -A shows all resources isn't present anymore (e.g even the cm, svc, deploy, etc)
I was actually able to make this work by updating the Application yaml:
Add spec.syncPolicy.allowEmpty: true
Remove metadata.finalizers
The working version without getting stuck at Deleting status:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: service-name
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
namespace: argocd
project: proj-name
source:
path: service-name
repoURL: ssh://...git
targetRevision: dev
helm:
valueFiles:
- ../values.yaml
- ../values_version_dev.yaml
syncPolicy:
automated:
prune: true
allowEmpty: true
selfHeal: true
This has happened to me several times. In every case it was because I had two declarations of applications of the same name.
I am interested in using Istio in a use case where I spin up a pod based on some event (user starting a game for example), and allow that user to connect to the specific pod through a unqiue URL. Then when this game is over, I can spin the pod down.
I am trying to configure Istio in such a way that either the subdomain or URL can indicate a specific pod to send that request.
Maybe some way to dynamically control label matching? Like <pod_label>.api.com routes to pod matching label label_name: <pod_label> without having to update VirtualServices and DestinationRoutes every time a pod is created?
Edit:
A psuedo config would look similar to:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: dynamic-route
spec:
hosts:
- r.prod.svc.cluster.local
http:
- match:
uri:
prefix: "/pod/{pod_name}"
ignoreUriCase: true
route:
- destination:
host: {pod_name}.prod.svc.cluster.local
(not sure if this would belong in the virtual service or the destination route)
I'm adding an Ingress as follows:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: cheddar
spec:
rules:
- host: cheddar.213.215.191.78.nip.io
http:
paths:
- backend:
service:
name: cheddar
port:
number: 80
path: /
pathType: ImplementationSpecific
but the logs complain:
W0205 15:14:07.482439 1 warnings.go:67] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
time="2021-02-05T15:14:07Z" level=info msg="Updated ingress status" namespace=default ingress=cheddar
W0205 15:18:19.104225 1 warnings.go:67] networking.k8s.io/v1beta1 IngressClass is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 IngressClassList
Why? What's the correct yaml to use?
I'm currently on microk8s 1.20
I have analyzed you issue and came to the following conclusions:
The Ingress will work and these Warnings you see are just to inform you about the available api versioning. You don't have to worry about this. I've seen the same Warnings:
#microk8s:~$ kubectl describe ing
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
As for the "why" this is happening even when you use apiVersion: networking.k8s.io/v1, I have found the following explanation:
This is working as expected. When you create an ingress object, it can
be read via any version (the server handles converting into the
requested version). kubectl get ingress is an ambiguous request,
since it does not indicate what version is desired to be read.
When an ambiguous request is made, kubectl searches the discovery docs
returned by the server to find the first group/version that contains
the specified resource.
For compatibility reasons, extensions/v1beta1 has historically been
preferred over all other api versions. Now that ingress is the only
resource remaining in that group, and is deprecated and has a GA
replacement, 1.20 will drop it in priority so that kubectl get ingress would read from networking.k8s.io/v1, but a 1.19 server
will still follow the historical priority.
If you want to read a specific version, you can qualify the get
request (like kubectl get ingresses.v1.networking.k8s.io ...) or can
pass in a manifest file to request the same version specified in the
file (kubectl get -f ing.yaml -o yaml)
Long story short: despite the fact of using the proper apiVersion, the deprecated one is still being seen as the the default one and thus generating the Warning you experience.
I also see that changes are still being made recently so I assume that it is still being worked on.
I had the same issue and was unable to update the k8s cluster which was subscribed to release channel.
One of the reasons for this log warning generation is the ClusterRole definition of external-dns. The external-dns keep querying the ingresses in k8s cluster as per the rules defined in the Cluster role
- apiGroups: ["extensions", "networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get","watch","list"]
Found in the helm chart in here
It queries the old extensions of ingress as well which keeps on generating those logs. Please update the cert-manager.
I have a RESTful service within a spring boot application. This spring boot app is deployed inside a kubernetes cluser and we have Istio as a service mesh attached to the sidecar of each container pod in the cluster. Every request to my service first hits the service mesh i.e Istio and then gets routed accordingly.
I need to put a validation for a request header and if that header is not present then randomly generate a unique value and set it as a header to the request. I know that there is Headers.HeaderOperations which i can use in the destination rule but how can i generate a unique value every time the header is missing? I dont want to write the logic inside my application as this is a general rule to apply for all the applications inside the cluster
There is important information that needs to be said in this subject. And it looks to me like You are trying to make a workaround tracing for an applications that does not forward/propagate headers in Your cluster. So I am going to mention few problems that can be encountered with this solution (just in case).
As mentioned in answer from Yuri G. You can configure unique x-request-id headers but they will not be very useful in terms of tracing if the requests are passing trough applications that do not propagate those x-request-id headers.
This is because tracing entire request paths needs to have unique x-request-id though out its entire trace. If the x-request-id value is different in various parts of the path the request takes, how are We going to put together the entire trace path?
In a scenario where two requests are received in application pod at the same time even if they had unique x-request-id headers, only application is able to tell which inbound request matches with which outbound connection. One of the requests could take longer to process and without forwarded trace header we can't tell which one is which.
Anyway for applications that do support forwarding/propagating x-request-id headers I suggest following guide from istio documentation.
Hope it helps.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: enable-envoy-xrequestid-in-response
namespace: istio-system
spec:
configPatches:
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.http_connection_manager"
patch:
operation: MERGE
value:
typed_config:
"#type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
always_set_request_id_in_response: true
From reading the documentation of istio and envoy it seems like this is not supported by istio/envoy out of the box. As a workaround you have 2 options
Option 1: To set the x-envoy-force-trace header in virtual service
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews-route
spec:
hosts:
- reviews.prod.svc.cluster.local
http:
- headers:
request:
set:
x-envoy-force-trace: true
It will generate a header x-request-id if it is missing. But it seems like abuse of tracing mechanism.
Option 2: To use consistentHash balancing based on header, e.g:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: bookinfo-ratings
spec:
host: ratings.prod.svc.cluster.local
trafficPolicy:
loadBalancer:
consistentHash:
httpHeaderName:
name: x-custom-request-id
It will generate the header x-custom-request-id for any request that doesn't have this header. In this case the requests with same x-custom-request-id value will go always to the same pod that can cause uneven balancing.
The answer above works well! I have updated it for the latest istio (filter name is in full):
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: enable-envoy-xrequestid-in-response
namespace: istio-system
spec:
configPatches:
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
patch:
operation: MERGE
value:
typed_config:
"#type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
always_set_request_id_in_response: true
Using the v1.7.9 in kubernetes I'm facing this issue:
if I set a rate limit (traefik.ingress.kubernetes.io/rate-limit) and custom response headers (traefik.ingress.kubernetes.io/custom-response-headers) then when a request gets rate limited, the custom headers won't be set. I guess it's because of some ordering/priority among these plugins. And I totally agree that reaching the rate-limit should return the response as soon as is possible, but it would be nice, if we could modify the priorities if we need.
The question therefore is: will we be able to set priorities for the middlewares?
I couldn't find any clue of it in the docs nor among the github issues.
Concrete use-case:
I want CORS-policy headers to always be set, even if the rate-limiting kicked in. I want this because my SPA won't get the response object otherwise, because the browser won't allow it:
Access to XMLHttpRequest at 'https://api.example.com/api/v1/resource' from origin 'https://cors.exmaple.com' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
In this case it would be a fine solution if i could just set the priority of the headers middleware higher than the rate limit middleware.
For future reference, a working example that demonstrates such an ordering is here:
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: ratelimit
spec:
rateLimit:
average: 100
burst: 50
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: response-header
spec:
headers:
customResponseHeaders:
X-Custom-Response-Header: "value"
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroute
spec:
# more fields...
routes:
# more fields...
middlewares: # the middlewares will be called in this order
- name: response-header
- name: ratelimit
I asked the same question on the Containous' community forum: https://community.containo.us/t/can-we-set-priority-for-the-middlewares-in-v2/1326
Regular web pages can use the XMLHttpRequest object to send and receive data from remote servers, but they're limited by the same origin policy. Extensions aren't so limited. An extension can talk to remote servers outside of its origin, as long as it first requests cross-origin permissions.
1. Try while testing on you local machine, replaced localhost with your local IP. You had to achieve CORS by the following line of code request.withCredentials = true; where request is the instance of XMLHttpRequest. CORS headers has to be added to the backend server to allow cross origin access.
2. You could just write your own script which will be responsible for executing rate limit middleware after headers middleware.
In the v2, the middlewares can ordered in the order you want, you can put the same type of middleware several times with different configurations on the same route.
https://docs.traefik.io/v2.0/middlewares/overview/