I have set up my master nodes using kubeadm.
Now I want to run the join command on my nodes so that the later join the cluster.
All I have to do is run
kubeadm join --token <token> --discovery-token-ca-cert-hash <sha256>
where <token> and are values previously returned by the command below:
kubeadm init
I am also trying to script the above process and I see that parsing the actual tokens from the last command is kinda difficult;
So I was wandering whether there is a way to explicitly specify the <token> and the <sha256> during cluster initialization, to avoid having to perform hacky parsing of the init command.
I was trying to make a script for it as well.
In order to get the values needed I am using these commands:
TOKEN=$(sshpass -p $PASSWORD ssh -o StrictHostKeyChecking=no root#$MASTER_IP sudo kubeadm token list | tail -1 | cut -f 1 -d " ")
HASH=$(sshpass -p $PASSWORD ssh -o StrictHostKeyChecking=no root#$MASTER_IP openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //' )
Basically I use this commands to ssh on master and get this values.
I have not found a easier way to achieve this.
Actually there seems to be a way around this:
(I am putting this in ansible tasks cause this is where I am planning to use it)
- name: kubernetes.yml --> Initiate kubernetes cluster
shell: 'kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address={{ ansible_facts[if_name]["ipv4"]["address"] }}'
become: yes
when: inventory_hostname in groups['masters']
- name: kubernetes.yml --> Get the join command
shell: kubeadm token create --print-join-command
register: rv_join_command
when: inventory_hostname in (groups['masters'] | last)
become: yes
- name: kubernetes.yml --> Print the join command
debug:
var: rv_join_command.stdout
Output:
TASK [kubernetes.yml --> Print the join command] *******************************
ok: [kubernetes-master-1] =>
rv_join_command.stdout: 'kubeadm join 192.168.30.1:6443 --token ah0dbr.grxg9fke3c28dif3i --discovery-token-ca-cert-hash sha256:716712ca7f07bfb4aa7df9a8b30ik3t0k3t2259b8c6fc7b68f50334356078 '
Related
Suppose i've got a zip file available under some URL. I need to get its hash, which should be identical to the one output by nix-prefetch-url --unpack <URL>, but without a working Nix installation. How can one do it?
Seems there is no easy way, as nix-prefetch-url adds the file to the store. More details here: https://discourse.nixos.org/t/generate-a-file-hash-similar-to-the-one-output-by-nix-prefetch-url/19907 (many thanks to prompt and thorough community member's response)
Use Docker.
Demo:
$ nix-prefetch-url --unpack https://github.com/hraban/git-hly/archive/06ff628d5f2b02d1a883c94b01d58187d117f4f3.tar.gz
path is '/nix/store/gxx1pfp19s3a39j6gl0xw197b4409cmp-06ff628d5f2b02d1a883c94b01d58187d117f4f3.tar.gz'
164gyvpdm6l6rdvn2rwjz95j1jz0w2igcbk9shy862sdx2rdw9hn
$ # Or .zip: it's the same, because of --unpack:
$ nix-prefetch-url --unpack https://github.com/hraban/git-hly/archive/06ff628d5f2b02d1a883c94b01d58187d117f4f3.zip
path is '/nix/store/1bpjlzknnmq1x3hq213r44jwag1xkaqs-06ff628d5f2b02d1a883c94b01d58187d117f4f3.zip'
164gyvpdm6l6rdvn2rwjz95j1jz0w2igcbk9shy862sdx2rdw9hn
Download to a local directory
$ cd "$(mktemp -d)"
$ curl -sSL --fail https://github.com/hraban/git-hly/archive/06ff628d5f2b02d1a883c94b01d58187d117f4f3.tar.gz | tar xz
$ cd *
And test it:
$ # Using the modern nix command:
$ nix hash path --base32 .
164gyvpdm6l6rdvn2rwjz95j1jz0w2igcbk9shy862sdx2rdw9hn
$ # Or the same, using nix-hash:
$ nix-hash --type sha256 --base32 .
164gyvpdm6l6rdvn2rwjz95j1jz0w2igcbk9shy862sdx2rdw9hn
Same in Docker:
$ docker run --rm -v "$PWD":/data nixos/nix nix --extra-experimental-features nix-command hash path --base32 /data
164gyvpdm6l6rdvn2rwjz95j1jz0w2igcbk9shy862sdx2rdw9hn
$ docker run --rm -v "$PWD":/data nixos/nix nix-hash --type sha256 --base32 /data
164gyvpdm6l6rdvn2rwjz95j1jz0w2igcbk9shy862sdx2rdw9hn
P.S.: I'm not a huge fan of nix-prefetch-url's default output (base32). The default output of nix hash path is better, if you can use it:
$ nix hash path .
sha256-FibesuhNC4M81Gku9qLg4MsgS/qSZ2F3y4aa2u72j5g=
$ # Sanity check:
$ nix-hash --type sha256 --to-base32 $(<<<"FibesuhNC4M81Gku9qLg4MsgS/qSZ2F3y4aa2u72j5g=" base64 -d | hexdump -v -e '/1 "%02x"' )
164gyvpdm6l6rdvn2rwjz95j1jz0w2igcbk9shy862sdx2rdw9hn
I am using kubernetes and its resources like secrets. During deployment one secret has been created (say test-secret) with some values inside it.
Now I need to renamed this secretes (dev-secret) within the same namespace.
How can I rename the secret or how can I copy test-secret value to dev-secret.
Please let me know the correct approach for this.
There is no specific way to do this. The Kubernetes API does not have "rename" as an operation. In this particular case you would kubectl get server test-secret -o yaml, clean up the metadata: sections that don't apply anymore, edit the name, and kubectl apply it again.
Extending #coderanger answer:
If you still have secret config yaml file you can do
kubectl delete -f </path/to/secret-config-yaml>
change metadata.name object and issue
kubectl apply -f </path/to/secret-config-yaml>
I needed to do something similar: rename K8s secrets.
I searched everywhere, but could not find a good way to do it.
So I wrote a bash script for copying secrets into new secrets with a new name.
In my case, I also wanted to do this in batch, as I had many secrets with the same prefix that I needed to change.
I don't work with bash all the time, so there might be better ways... but it did the trick for me.
I hope it helps!
#!/bin/bash
# Copies K8s secrets with names containing the NAME_PART into new
# secrets where the NAME_PART was replaced with NEW_NAME_PART.
# i.e. if NAME_PART is "test-abc" and NEW_NAME_PART is "test-xyz", a secret names test-abc-123
# will be copied into a new secret named test-xyz-123
#
# Pre-requisites:
# - have kubectl installed and pointing to the cluster you want to alter
#
# NOTE: tested with kubectl v1.18.0 and K8s v1.21.5-eks-bc4871b
# configure the NAME_PARTs here
NAME_PART=test-abc
NEW_NAME_PART=test-xyz
WORK_DIR=work_secret_copy
mkdir -p $WORK_DIR
echo "Getting secrets from K8s..."
allSecrets=`kubectl get secrets | tail -n +2 | cut -d " " -f1`
matchingSecrets=`echo $allSecrets | tr ' ' '\n' | grep $NAME_PART`
#printf "All secrets:\n $allSecrets \n"
#printf "Secrets:\n $secrets \n"
for secret in $matchingSecrets; do
newSecret=${secret/$NAME_PART/$NEW_NAME_PART}
echo "Copying secret $secret to $newSecret"
# skip this secret if one with the new name already exists
if [[ $(echo $allSecrets | tr ' ' '\n' | grep -e "^$newSecret\$") ]]; then
echo "Secret $newSecret already exists, skipping..."
continue
fi
kubectl get secret $secret -o yaml \
| grep -v uid: \
| grep -v time: \
| grep -v creationTimestamp: \
| sed "s/$secret/$newSecret/g" \
> $WORK_DIR/$newSecret.yml
kubectl apply -f $WORK_DIR/$newSecret.yml
done
I'm having a trouble setting up access to kubernetes cluster from the outside. This is what I'm trying to achieve:
- Have ability to access to kube cluster from the outside (from nodes that are not "master" and even from any remote) to be able to do kube actions only on specific namespace.
My logic was to do following:
Create new namespace (let's call it testns)
Create service account (testns-account)
Create role which gives access for creating any type of kube resource inside testns namespace
Create role binding which binds service account with role
Generate token from service account
Now, my logic was that I need to have token + api server URL to access kube cluster with limited "permissions" but that doesnt seem like it is enough.
What would be the easiest way to achieve this? For start, I could have access with kubectl just to verify that limited permissions on namespace work but eventually, I would have some client side code which doing the access and creates kube resources with these limited permissions.
You need to generate a kubeconfig from the token. There are scripts to handle this. Here it is for posterity:
!/usr/bin/env bash
# Copyright 2017, Z Lab Corporation. All rights reserved.
# Copyright 2017, Kubernetes scripts contributors
#
# For the full copyright and license information, please view the LICENSE
# file that was distributed with this source code.
set -e
if [[ $# == 0 ]]; then
echo "Usage: $0 SERVICEACCOUNT [kubectl options]" >&2
echo "" >&2
echo "This script creates a kubeconfig to access the apiserver with the specified serviceaccount and outputs it to stdout." >&2
exit 1
fi
function _kubectl() {
kubectl $# $kubectl_options
}
serviceaccount="$1"
kubectl_options="${#:2}"
if ! secret="$(_kubectl get serviceaccount "$serviceaccount" -o 'jsonpath={.secrets[0].name}' 2>/dev/null)"; then
echo "serviceaccounts \"$serviceaccount\" not found." >&2
exit 2
fi
if [[ -z "$secret" ]]; then
echo "serviceaccounts \"$serviceaccount\" doesn't have a serviceaccount token." >&2
exit 2
fi
# context
context="$(_kubectl config current-context)"
# cluster
cluster="$(_kubectl config view -o "jsonpath={.contexts[?(#.name==\"$context\")].context.cluster}")"
server="$(_kubectl config view -o "jsonpath={.clusters[?(#.name==\"$cluster\")].cluster.server}")"
# token
ca_crt_data="$(_kubectl get secret "$secret" -o "jsonpath={.data.ca\.crt}" | openssl enc -d -base64 -A)"
namespace="$(_kubectl get secret "$secret" -o "jsonpath={.data.namespace}" | openssl enc -d -base64 -A)"
token="$(_kubectl get secret "$secret" -o "jsonpath={.data.token}" | openssl enc -d -base64 -A)"
export KUBECONFIG="$(mktemp)"
kubectl config set-credentials "$serviceaccount" --token="$token" >/dev/null
ca_crt="$(mktemp)"; echo "$ca_crt_data" > $ca_crt
kubectl config set-cluster "$cluster" --server="$server" --certificate-authority="$ca_crt" --embed-certs >/dev/null
kubectl config set-context "$context" --cluster="$cluster" --namespace="$namespace" --user="$serviceaccount" >/dev/null
kubectl config use-context "$context" >/dev/null
cat "$KUBECONFIG"
I've lost the original 'kubeadm join' command when I previously ran kubeadm init.
How can I retrieve this value again?
kubeadm token create --print-join-command
To print a join command for a new worker node use:
kubeadm token create --print-join-command
But if you need to join a new control plane node, you need to recreate a new key for the control plane join command. This can be done with three simple steps:
Re upload certificates in the already working master node with kubeadm init phase upload-certs --upload-certs. That will generate a new certificate key.
Print join command in the already working master node with kubeadm token create --print-join-command.
Join a new control plane node with $JOIN_COMMAND_FROM_STEP2 --control-plane --certificate-key $KEY_FROM_STEP1.
This might not work for the old Kubernetes versions but I tried with the new version and it worked for me.
To create kubeadm join command, please run the following commands:
Step 1 - Retrieve Token CA Hash:
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt \
| openssl rsa -pubin -outform der 2>/dev/null \
| openssl dgst -sha256 -hex \
| sed 's/^.* //'
This command will provide you public key.
Step 2 - Retrieve bootstrap Tokens:
kubeadm token list
This will print all tokens, so copy the token value under TOKEN with the description "The default bootstrap token generated by kubeadm init."
Step 3 - Creates kubeadm init command:
Now use following syntax to create join command without creating a new token:
kubeadm join <ip-address>:6443\
--token=<token-from-step-2> \
--discovery-token-ca-cert-hash sha256:<ca-hash-from-step-1>
kubeadm token create command creates a new token, in this case without any description, so for you not to create any additional tokens, just pick the token which has a DESCRIPTION as mentioned in Step 2.
Run the below command on your master node machine.
kubeadm token create --print-join-command
This command will generate the new token as well as the join command which you can use at your worker node to join the cluster.
Building off #Abhishek Jain's answer, here's a script to print the kubeadm join command with a little help from jq:
# get the join command from the kube master
CERT_HASH=$(openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt \
| openssl rsa -pubin -outform der 2>/dev/null \
| openssl dgst -sha256 -hex \
| sed 's/^.* //')
TOKEN=$(kubeadm token list -o json | jq -r '.token' | head -1)
IP=$(kubectl get nodes -lnode-role.kubernetes.io/master -o json \
| jq -r '.items[0].status.addresses[] | select(.type=="InternalIP") | .address')
PORT=6443
echo "sudo kubeadm join $IP:$PORT \
--token=$TOKEN --discovery-token-ca-cert-hash sha256:$CERT_HASH"
If you are joining control plane nodes, you will need a certificate key in the command too:
kubeadm token create \
--print-join-command \
--certificate-key \
$(kubeadm alpha certs certificate-key)
The kubeadm alpha certs certificate-key command will generate a new certificate key on demand as per the documentation here
To Join a worker node, the command kubeadm token create --print-join-command given in the accepted answer is sufficient
Here is a bash script that automate this task
read -p 'master ip address : ' ipaddr
sha_token = "$(openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //')"
token = "$(kubeadm token list | awk '{print $1}' | sed -n '2 p')"
echo "kubeadm join $ipaddr:6443 --token=$token --discovery-token-ca-cert-hash sha256:$sha_token"
In writing my deployment script, I want to set a git checkout url that I want to be secret. I want to create a Travis job to test out my playbook. The easiest approach that I can think of now is by setting my global_vars to look for an env variable say DEPLOYMENT_GIT_URL. I then encrypt this env variable in travis and pass it to docker exec when I am building the docker image to test against my playbook.
Question:
Can I pass my encrypted Travis variable to the instance via docker exec ? Something like sudo docker exec ... export DEPLOYMENT_GIT_URL=$TRAVIS_ENV ansible-playbook -i ....
While this seems the simplest way to do it, appreciate comments on this method.
Thanks
You can pass variables directly to Ansible. If you want nested or complex variables, use a JSON string.
docker exec <container> ansible-playbook -e GITURL="$GITURL" whatever.yml
As gogstad mentions, Ansible has the facility to manage secrets via ansible-vault. Unless this URL is information that Travis needs to be the source of, it might be easier storing it directly in Ansible. Otherwise openssl can manage the secret:
secret=$(echo -n "yourdata" | openssl enc -e -aes-256-cbc -a -k 'passpasspass')
echo $secret | openssl enc -d -aes-256-cbc -a -k 'passpasspass'
If you really want to pass the data with an environment variable you would need to do that at container creation with docker run and -e
docker run -e GITURL="what" busybox sh -c 'echo $GITURL'