I have just set up a kubernetes cluster on bare metal using kubeadm, Flannel and MetalLB. Next step for me is to install ArgoCD.
I installed the ArgoCD yaml from the "Getting Started" page and logged in.
When adding my Git repositories ArgoCD gives me very weird error messages:
The error message seems to suggest that ArgoCD for some reason is resolving github.com to my public IP address (I am not exposing SSH, therefore connection refused).
I can not find any reason why it would do this. When using https:// instead of SSH I get the same result, but on port 443.
I have put a dummy pod in the same namespace as ArgoCD and made some DNS queries. These queries resolved correctly.
What makes ArgoCD think that github.com resolves to my public IP address?
EDIT:
I have also checked for network policies in the argocd namespace and found no policy that was restricting egress.
I have had this working on clusters in the same network previously and have not changed my router firewall since then.
I solved my problem!
My /etc/resolv.conf had two lines that caused trouble:
domain <my domain>
search <my domain>
These lines were put there as a step in the installation of my host machine's OS that I did not realize would affect me in this way. After removing these lines, everything is now working perfectly.
Multiple people told me to check resolv.conf, but I didn't realize what these two lines did until now.
That looks like argoproj/argo-cd issue 1510, where the initial diagnostic was that the cluster is blocking outbound connections to GitHub. And it suggested to check the egress configuration.
Yet, the issue was resolved with an ingress rule configuration:
need to define in values.yaml.
argo-cd default provide subdomain but in our case it was /argocd
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: HTTP
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
path: /argocd
hosts:
- www.example.com
and this I have defined under templates >> argocd-server-deployment.yaml
containers:
- name: argocd-server
image: {{ .Values.server.image.repository }}:{{ .Values.server.image.tag }}
imagePullPolicy: {{ .Values.server.image.pullPolicy }}
command:
- argocd-server
- --staticassets - /shared/app - --repo-server - argocd-repo-server:8081 - --insecure - --basehref - /argocd
The same case includes an instance very similar to yours:
In any case, do check your git configuration (git config -l) as seen in the ArgoCD cluster, to look for any insteadOf which would change automatically github.com into a local URL (as seen here)
Related
I have 2 AKS clusters, Cluster 1 and Cluster 2 both running Istio 1.14 minimal out-of-the-box (default configs).
Everything on Cluster 1 works as expected (after deploying Istio).
On Cluster 2, all HTTPS outbound connections initiated from my services (injected with istio-proxy) fail.
curl http://www.google.com #works
curl https://www.google.com #fails
If I create a service entry for google, then the https curl works:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: google
spec:
hosts:
- www.google.com
ports:
- number: 443
name: https
protocol: HTTPS
resolution: DNS
location: MESH_EXTERNAL
Both Istio installations are out-of-the-box so meshConfig.outboundTrafficPolicy.mode is set to ALLOW_ANY (double-checked).
I read online that there were some Istio bugs that would cause this behavior, but I don't think it's the case here. I also compared the Istio configs between the 2 clusters and they really seem to be the same.
I'm starting to think the problem may lie in some cluster configs because I know there are some differences between the 2 clusters here.
How would you go about troubleshooting this? Do you think the issue is related to Istio or Cluster services/configs? What should I look into first?
You are correct. By default ALLOW_ANY is value set for meshConfig.outboundTrafficPolicy.mode. This can be verified in the cluster by running below command.
kubectl get configmap istio -n istio-system -o yaml | grep -o "mode: ALLOW_ANY"
Please also refer the istio documentation for the options available in accessing external services
I'm new to Kubernetes and Helm and want to create a SSO with OIDC using vouch-proxy
I found a tutorial which explains how to do it and was able to write some helmfiles that were accepted by kubernetes.
I added the ingress configuration to the values.yaml that I load in my helmfile.yaml.
helmfile.yaml
bases:
- environments.yaml
---
releases:
- name: "vouch"
chart: "halkeye/vouch"
version: {{ .Environment.Values.version }}
namespace: {{ .Environment.Values.namespace }}
values:
- values.yaml
values.yaml
# vouch config
# bare minimum to get vouch running with OpenID Connect (such as okta)
config:
vouch:
some:
other:
values:
# important part
ingress:
enabled: true
hosts:
- "vouch.minikube"
paths:
- /
With this configuration helmfile creates an Ingress for the correct host, but when I open the URL in my Browser it returns a 404 Not Found which makes sense, since I didn't specify the correct port (9090).
I tried some notations to add the port but it lead to either helmfile not updating the pod or 500 Internal Server Errors.
How can I add a port in the configuration? And is it the "correct" way to do it? Or should ingresses be handled by kubectl still?
I am new at kubernetes and I am trying to setup a clsuter on a remote server.
For this I use microk8s and a server from hetzner-cloud (https://www.hetzner.com/de/cloud).
I entered the server with ssh and followd the microk8s installation instructions for linux(https://microk8s.io/). That all seemed to work fine. My problem is now that I have not found a way to access the kubernetes-dashboard.
I've tryed the workaround with NodePort and microk8s kubectl proxy --disable-filter = true but it does not work and is not recommended for security reasons. With the disable Filter Method it is possible to access the login page but it does not respond.
I've also tryed to access the dashbourd from outside using a ssh tunnel and this tutorial: How can I remotely access kubernetes dashboard with token
The tunnel seems to works fine, but I still cannot access the port.
Now I have two main questions:
1: How do you usualy use kubernetes, if kubernetes does not wan you to access the dashboard from outside. Because don't you run your services usualy on a rented server that is not in you living room? What's the point I simply do not get?
2: How can I access the Dashborad?
I would be really happy, if anybody could help me. I am already struggling with this problem since a month now. :)
best greetings,
mamo
In order to access the K8s services from outside using HTTP, you should configure and use ingress controller.
after ingress is running, you will be able to specify a "path" or route and a port and name that points to your service.
Once this is done, you should be able to access the dashboard
Sample configuration (reference)
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: dashboard-google
namespace: kube-system
annotations:
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
tls:
- hosts:
- kube.mydomain.com
secretName: tls-secret
rules:
- host: kube.mydomain.com
http:
paths:
- path: /
backend:
serviceName: kubernetes-dashboard
servicePort: 443
I have a website running inside a kubernetes cluster.
I can access it localy, but want to make it available over the internet. (I have a registered domain), but the external IP keeps pending
I worked with this instruction: https://dev.to/peterj/expose-a-kubernetes-service-on-your-own-custom-domain-52dd
This is the code for the service and ingress
kind: Service
apiVersion: v1
metadata:
name: app-service
spec:
selector:
app: website
ports:
- name: http
protocol: TCP
port: 3000
targetPort: 3000
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: app-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: www.carina.bernrieder.de
http:
paths:
- path: /
backend:
serviceName: app-service
servicePort: 3000
So I'm using helm to install the nginx-controller, but after that Kubectl get all the external IP of the nginx controller keeps pending.
EXTERNAL-IP is expected to be pending in a non cloud environment such as minikube. You should be able to access the application using curl www.carina.bernrieder.de
Here is guide on using nginx ingress to expose an application on minikube
As #Arghya Sadhu mentioned, in local environment it is the expected behaviour. Maybe it will be easier to understand when you look a bit more deeply on how it works in cloud environments. Without going into details, if you apply an Ingress resource on GKE, EKS or AKS, a few more things happen "under the hood". A loadbalancer with an external IP is automatically created so your ingress can use it to forward external traffic to Pods deployed on your kubernetes cluster.
Minikube doesn't have such capabilities as it cannot make any call to any API for additional infrastructure resources to be created, as it happens on cloud environments.
But let's start from the beginning. You didn't mention in your question anything about your external IP or domain configuration. If you don't have an external static IP to which your domain has been redirected, it have no chances to work anyway.
As to this point, I won't fully agree:
You should be able to access the application using curl
www.carina.bernrieder.de
Yes, you will be able to access it via your domain (actually via any domain that you don't even need to own) provided you add the following entry in your /etc/hosts file so DNS won't be used and it will be resolved based on this locally defined mapping:
172.17.0.15 www.carina.bernrieder.de
As you can read here:
Note: If you are running Minikube locally, use minikube ip to get the
external IP. The IP address displayed within the ingress list will be
the internal IP.
But keep in mind that both those IPs will be private IPs. The one, that is displayed within the ingress list will be internal cluster ip and the external one will be extarnal only from your Minikube cluster perspective. It will be still the IP in your local network assigned to your Minikube vm.
And as you said in your question you want to make it available over the Internet. As you can see it has no chances to work without additional configuration.
Another important thing. You didn't mention where your Minikube is actually installed, so I guess you set it up on your local computer and most probably you're behind NAT router. If this is your case, it won't be so easy to expose it on a public internet. You will need to configure proper port forwarding rules on your router and of course you need a static IP or you need to configure dynamic DNS to be able to access your computer on the Internet via your dynami public IP.
Minikube was designed mainly for playing locally with kubernetes and not for production environments. Of course you can use it to run your small app, but then you may think about installing it on a VM in a cloud environment or some sort of VPS server.
I'm getting no healthy upstream error. when accessing ambassador. Pods/Services and Loadbalancer seems to be all fine and healthy. Ambassador is on top of aks.
At the moment I have got multiple services running in the Kubernetes cluster and each service has it's on Mapping with its own prefix. Is it possible to point out multiple k8s services to the same mapping so that I don't have too many prefixes? And all my k8s services will be under the same ambassador prefix?
By Default ambassador is taking me through https which is creating certificate issues, although I will be bringing https in near future for now I'm just looking to prove the concept so how can I disable HTTPS and do HTTP only ambassador?
No healthy upstream typically means that, for whatever reason, Ambassador cannot find the service listed in the mapping. The first thing I usually do when I see this is to run kubectl exec -it -n ambassador {my_ambassador_pod_name} -- sh and try to curl -v my-service where "my-service" is the Kube DNS name of the service you are trying to hit. Depending on the response, it can give you some hints on why Ambassador is failing to see the service.
Mappings work on a 1-1 basis with services. If your goal, however, is to avoid prefix usage, there are other ways Ambassador can match to create routes. One common way I've seen is to use host-based routing (https://www.getambassador.io/docs/latest/topics/using/headers/host/) and create subdomains for either individual or logical sets of services.
AES defaults to redirecting to HTTPS, but this behavior can be overwritten by applying a host with insecure routing behavior. A very simple one that I commonly use is this:
---
apiVersion: getambassador.io/v2
kind: Host
metadata:
name: wildcard
namespace: ambassador
spec:
hostname: "*"
acmeProvider:
authority: none
requestPolicy:
insecure:
action: Route
selector:
matchLabels:
hostname: wildcard