I am using istio.1.6 and i was trying to store metrics from istio prometheus to external prometheus based on istio best practise doc.But in the first step, I have to edit my configuration and add recording rules.I tried to edit the configmap of istio prometheus and added the recording rules.Edit is successful but when i try to see the rules in prometheus dashboard ,they donot appear(which i believe means the config didnot apply).I also tried to just delete the pod and see if the new pod has new configurations but still the problem.
What am i doing wrong? Any suggestions and answers is appreciated.
The problem was that the way I added the recording rules.I added rules in rules.yaml but forgot to mention it in rule_files field of the prometheus config file.I didn't know how to do prometheus configuration and that was the problem.
I also refered this github example
Also check out this post on prometheus federation
Related
I'm desperate with activating alerts in grafana (running in k8s).
Are there standard alerts (like a default file or so), which I can import into grafana?
For instance for getting notified when there is something with the k8s cluster?
I've downloaded a file from awesome-prometheus-alerts, which provides a starting set of rules. According to the grafana doc, it can also be used in grafana.
Unfortunately, I'm unable to get it running. Any ideas, how this can be accomplished?
Our setting:
We are using helm for deploying
Use of loki-stack
Configurations are being made in a values.yml file, overwriting values in grafana.ini
Grafana Loki supports prometheus queries and you can update the queries in values.yaml file and apply it for getting the changes applied. Follow this document for more information. If you want the list of promQl queries follow this document.
Azure defender is showing vulnerabilities in the Nginx ingress image
ingress images are in ACR.
I did update the helm repo but it's still showing same issue
I am happy to provide more information if needed
As a first step I would try this:
https://kubernetes.io/blog/2022/04/28/ingress-nginx-1-2-0/#skip-the-talk-what-do-i-need-to-use-this-new-approach
Second, you can read this to figure out what is the best solution for you.
https://support.f5.com/csp/article/K01051452
You can also take a look here for security issues:
https://github.com/kubernetes/ingress-nginx/issues/8372
Summarize the problem:
Any way we can add an ENV to a pod or a new pod in kubernetes?
For example, I want to add HTTP_PROXY to many pods and the new pods it will generate in kubeflow 1.4. So these pods can be access to internet.
Describe what you’ve tried:
I searched and found istio maybe do that, but it's too complex for me.
The second, there are too many yamls in kubeflow, as to I cannot modify them one by one to use configmap or add ENV just in them.
So anyone has a good simle way to do this? Like doing this in kubernetes configuation.
Use "PodPreset" object to inject common environment variables and other params to all the matching pods.
Please follow below article
https://v1-19.docs.kubernetes.io/docs/tasks/inject-data-application/podpreset/
If PodPreset is indeed removed from v1.20, then you seem to need a webhook.
You will have to run an additional service in your cluster that will change the configuration of the pods.
Here is an example, on the basis of which I created my webhook, which changed the configuration of the pods in the cluster, in this example the developer used the logic adding a sidecar to the pod, but you can set your own to forward the required ENV:
https://github.com/morvencao/kube-mutating-webhook-tutorial/blob/master/medium-article.md
I am facing difficulty in working with jaeger and Istio.
Can anyone please describe the steps that are to be followed in configuring jaeger and istio for any demo application. I have tried a few blogs and sites but unfortunately, nothing worked for me. if anyone could help me in this that would be great.
I hope you have followed the official documentation of the jager with istio.
If you are using the helm chart make the following changes required.
In main values.yaml file
tracing:
enabled: true
In tracing/values.yaml
provider: jaeger
Export the dashboard via Kube port-forward or ingress.
Official Documentation.
https://istio.io/docs/tasks/telemetry/distributed-tracing/jaeger/
NOTE: The important thing by default jaeger will trace something like 0.1% request i.e. 1 request out of 100 so put a lot of requests only then you can see a trace in UI.
I have a setup with scdf-server on kubernetes working fine, it deploys each task in an on-demand pod on the very same default namespace, the one that hosts the scdf-server pod.
Now, I need to deploy a pod in another namespace and I can't find the argument/property to use in the scdf server dashboard for the pod to be created in the given namespace. Does anybody know how to find that? I tried spring.cloud.deployer.kubernetes.namespace, deployer.kubernetes.namespace, spring.cloud.deployer.kubernetes.environmentVariables, deployer.<app>.kubernetes.namespace, spring.cloud.dataflow.task.platform.kubernetes.namespace, scheduler.kubernetes.environmentVariables SPRING_CLOUD_SCHEDULER_KUBERNETES_NAMESPACE... as both 'properties' and 'arguments' text boxes...
This seems like a duplicate thread that was posted in SCDF gitter channel. The properties were described and pointed out in the commentary - more details here.