I want to know whether metadata is also created using rados put {object-name} {file-path} --pool=data command in ceph?
If not then how metadata can be created and viewed?
The omapval and xattr commands of rados allow you to attach key/value pairs to the object. They are not included in the content of the object itself, they are metadata. By default there are no omapval or xattr when you create a new object. Here are a few examples:
$ rados put FOO /etc/group --pool rbd
$ rados listomapvals FOO --pool rbd
$ rados setomapval FOO frob nitz --pool rbd
$ rados listomapvals FOO --pool rbd
frob
value: (4 bytes) :
0000 : 6e 69 74 7a : nitz
$ rados listxattr FOO --pool rbd
$ rados setxattr FOO fi bou --pool rbd
$ rados listxattr FOO --pool rbd
fi
$ rados getxattr FOO fi --pool rbd
bou
Related
i am new to kubernetes/helm.
I'm trying to configure encryption TLS/SSL for Cassandra in K8ssandra?
I've seen this example but can't get any further.
Click here!
I perform the following steps as described in this tutorial
Click here!
helm repo add k8ssandra https://helm.k8ssandra.io/stable
helm repo update
cd ~/github
git clone https://github.com/k8ssandra/k8ssandra-operator.git
cd k8ssandra-operator
scripts/setup-kind-multicluster.sh --clusters 1 --kind-worker-nodes 4
helm repo add jetstack https://charts.jetstack.io
helm repo update
kubectl config use-context kind-k8ssandra-0
helm install cert-manager jetstack/cert-manager \
--namespace cert-manager --create-namespace --set installCRDs=true
helm install k8ssandra-operator k8ssandra/k8ssandra-operator -n k8ssandra-operator --create-namespace
cat <<EOF | kubectl -n k8ssandra-operator apply -f -
apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
name: demo
spec:
cassandra:
serverVersion: "4.0.1"
datacenters:
- metadata:
name: dc1
size: 3
storageConfig:
cassandraDataVolumeClaimSpec:
storageClassName: standard
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
config:
jvmOptions:
heapSize: 512M
stargate:
size: 1
heapSize: 256M
EOF
CASS_USERNAME=$(kubectl get secret demo-superuser -n k8ssandra-operator -o=jsonpath='{.data.username}' | base64 --decode)
echo $CASS_USERNAME
CASS_PASSWORD=$(kubectl get secret demo-superuser -n k8ssandra-operator -o=jsonpath='{.data.password}' | base64 --decode)
echo $CASS_PASSWORD
kubectl exec -it demo-dc1-default-sts-0 -n k8ssandra-operator -c cassandra -- nodetool -u $CASS_USERNAME -pw $CASS_PASSWORD status
kubectl exec -it demo-dc1-default-sts-0 -n k8ssandra-operator -c cassandra -- cqlsh -u $CASS_USERNAME -p $CASS_PASSWORD -e "CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 3};"
kubectl exec -it demo-dc1-default-sts-0 -n k8ssandra-operator -c cassandra -- cqlsh -u $CASS_USERNAME -p $CASS_PASSWORD -e "insert into test.users (email, name, state) values ('john#gamil.com', 'John Smith', 'NC');"
kubectl exec -it demo-dc1-default-sts-0 -n k8ssandra-operator -c cassandra -- cqlsh -u $CASS_USERNAME -p $CASS_PASSWORD -e "insert into test.users (email, name, state) values ('joe#gamil.com', 'Joe Jones', 'VA');"
kubectl exec -it demo-dc1-default-sts-0 -n k8ssandra-operator -c cassandra -- cqlsh -u $CASS_USERNAME -p $CASS_PASSWORD -e "insert into test.users (email, name, state) values ('sue#help.com', 'Sue Sas', 'CA');"
kubectl exec -it demo-dc1-default-sts-0 -n k8ssandra-operator -c cassandra -- cqlsh -u $CASS_USERNAME -p $CASS_PASSWORD -e "insert into test.users (email, name, state) values ('tom#yes.com', 'Tom and Jerry', 'NV');"
The files truststore.jks and keystore.jks are stored locally on the pc under the ./mnt/keystore/... and ./mnt/truststore/... directory.
Here i create the secrets keystoreSecret and truststoreSecret.
kubectl create secret generic keystore --from-file=./mnt/keystore/keystore.jks -n k8ssandra-operator
kubectl create secret generic truststore --from-file=./mnt/truststore/truststore.jks -n k8ssandra-operator
Now I have run the above examples.
This is the value.yaml
cassandra:
version: 4.0.1
cassandraYamlConfigMap: cassandra-config
encryption:
keystoreSecret: keystore
keystoreMountPath: /mnt/keystore
truststoreSecret: truststore
truststoreMountPath: /mnt/truststore
heap:
size: 512M
datacenters:
- name: dc1
size: 1
I'm trying to run this example as follows.
helm upgrade k8ssandra-operator k8ssandra/k8ssandra-operator -n k8ssandra-operator -f value.yaml
Now I'm trying to run the configmap file.
config-file.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: cassandra-config
data:
cassandra.yaml: |-
server_encryption_options:
internode_encryption: all
keystore: /mnt/keystore/keystore.jks
keystore_password: cassandra
truststore: /mnt/truststore/truststore.jks
truststore_password: cassandra
kubectl apply -f config-file.yaml -n k8ssandra-operator
i can switch to cassandra bash without any problems
kubectl exec -it demo-dc1-default-sts-0 -n k8ssandra-operator -- /bin/bash
In cassandra bash environment i can't find truststore and keystore files under /mnt/truststore/truststore.jks and /mnt/keystore/keystore.jks directories.
I'm trying to log into cassandra with ssl but I can't.
cassandra#demo-dc1-default-sts-0:/$ cqlsh --ssl -u demo-superuser -p JKv59QPynp3s0qGSf1DZ demo-dc1-stargate-service
Validation is enabled; SSL transport factory requires a valid certfile to be specified. Please provide path to the certfile in [ssl] section as 'certfile' option in /home/cassandra/.cassandra/cqlshrc (or use [certfiles] section) or set SSL_CERTFILE environment variable.
Have you checked this doc?
It applies to k8ssandra-operator (considered K8ssandra v2), while the links you found apply to K8ssandra v1 (where everything was helm charts).
Let me know if that works for you.
I wish to run a Drone CI/CD pipeline on a Raspberry Pi, including a stage to update a Kubernetes Deployment. Unfortunately, all the pre-built solutions that I've found for doing so (e.g. 1, e.g. ) are not built for arm64 architecture, so I believe I need to build my own.
I am attempting to adapt the commands from here (see also README.md, which describes the authorization required), but my attempt to contact the cluster still fails with authorization problems:
$ cat service-account-definition.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: drone-demo-service-account
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: drone-demo-service-account-clusterrolebinding
subjects:
- kind: ServiceAccount
name: drone-demo-service-account
namespace: default
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
$ kubectl apply -f service-account-definition.yaml
serviceaccount/drone-demo-service-account created
clusterrolebinding.rbac.authorization.k8s.io/drone-demo-service-account-clusterrolebinding created
$ kubectl get serviceaccount drone-demo-service-account
NAME SECRETS AGE
drone-demo-service-account 1 10s
$ kubectl get secret $(kubectl get secrets | grep 'drone-demo-service-account-token' | cut -f1 -d' ') -o jsonpath='{.data.ca\.crt}' > secrets/cert
$ head -c 10 secrets/cert
LS0tLS1CRU%
$ kubectl get secret $(kubectl get secrets | grep 'drone-demo-service-account-token' | cut -f1 -d' ') -o jsonpath='{.data.token}' | base64 > secrets/token
$ head -c 10 secrets/token
WlhsS2FHSk%
$ cat Dockerfile
FROM busybox
COPY . .
CMD ["./script.sh"]
$ cat script.sh
#!/bin/sh
server=$(cat secrets/server) # Pre-filled
cert=$(cat secrets/cert)
# Added this `tr` call, which is not present in the source I'm working from, after noticing that
# the file-content contains newlines
token=$(cat secrets/token | tr -d '\n')
echo "DEBUG: server is $server, cert is $(echo $cert | head -c 10)..., token is $(echo $token | head -c 10)..."
# Cannot depend on the binami/kubectl image (https://hub.docker.com/r/bitnami/kubectl), because
# it's not available for arm64 - https://github.com/bitnami/charts/issues/7305
wget https://storage.googleapis.com/kubernetes-release/release/v1.19.2/bin/linux/arm64/kubectl
chmod +x kubectl
./kubectl config set-credentials default --token=$token
echo $cert | base64 -d > ca.crt
./kubectl config set-cluster default --server=$server --certificate-authority=ca.crt
./kubectl config set-context default --cluster=default --user=default
./kubectl config use-context default
echo "Done with setup, now cat-ing .kube/config"
echo
cat $HOME/.kube/config
echo "Attempting to get pods"
echo
./kubectl get pods
$ docker build -t stack-overflow-testing . && docker run stack-overflow-testing
Sending build context to Docker daemon 10.75kB
Step 1/3 : FROM busybox
---> 3c277069c6ae
Step 2/3 : COPY . .
---> 74c6a132d255
Step 3/3 : CMD ["./script.sh"]
---> Running in dc55f33f74bb
Removing intermediate container dc55f33f74bb
---> dc68a5d6ba9b
Successfully built dc68a5d6ba9b
Successfully tagged stack-overflow-testing:latest
DEBUG: server is https://rassigma.avril:6443, cert is LS0tLS1CRU..., token is WlhsS2FHSk...
Connecting to storage.googleapis.com (142.250.188.16:443)
wget: note: TLS certificate validation not implemented
saving to 'kubectl'
kubectl 18% |***** | 7118k 0:00:04 ETA
kubectl 43% |************* | 16.5M 0:00:02 ETA
kubectl 68% |********************** | 26.2M 0:00:01 ETA
kubectl 94% |****************************** | 35.8M 0:00:00 ETA
kubectl 100% |********************************| 38.0M 0:00:00 ETA
'kubectl' saved
User "default" set.
Cluster "default" set.
Context "default" created.
Switched to context "default".
Done with setup, now cat-ing .kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority: /ca.crt
server: https://rassigma.avril:6443
name: default
contexts:
- context:
cluster: default
user: default
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
user:
token: WlhsS2FHSkhZM[...REDACTED]
Attempting to get pods
error: You must be logged in to the server (Unauthorized)
If I copy the ~/.kube/config from my laptop to the docker container, kubectl commands succeed as expected - so, this isn't a networking issue, just an authorization one. I do note that my laptop-based ~/.kube/config lists client-certificate-data and client-key-data rather than token under users: user:, but I suspect that's because my base config is recording a non-service-account.
How can I set up kubectl to authorize as a service account?
Some reading I have done that didn't answer the question for me:
kubenetes documentation on AuthN/AuthZ
Google Kubernetes Engine article on service accounts
Configure Service Accounts for Pods (this described how to create and associate the accounts, but not how to act as them)
Two blog posts (1, 2) that refer to Service Accounts
It appears you have used | base64 instead of | base64 --decode
How can I tell with kubectl how much ephemeral storage a pod is currently using?
In a Kubernetes pod spec, I can specify resource requests and limits for CPU, memory, and ephemeral storage:
resources:
requests:
memory: "60Mi"
cpu: "70m"
ephemeral-storage: "2Gi"
limits:
memory: "65Mi"
cpu: "75m"
ephemeral-storage: "4Gi"
However, to set good requests and limits on ephemeral storage, I need to know what this value actually is for a running pod, which I can't figure out. I can get CPU and memory usage using kubectl top pod, but, from what I can tell, ephemeral storage usage is only actually calculated when making an actual eviction decision.
The pure raw approach for this is to use the disk usage (du) Unix command line.
Shell into your pod:
$ kubectl exec -it <pod-id> sh
Change dirs to the mount point of your ephemeral-storage (if you are using volume mounts):
$ mount # check mount points if you'd like
$ cd /mnt/of/ephemeral
$ du .
If you are not using volume mounts:
$ du .
There are other tools that you can use to get metrics:
cAdvisor also embedded into the kubelet code, exposed under the /stats/summary or /metrics endpoint. More info here. An example output:
$ curl -k -H 'Authorization: Bearer <Redacted>' \
https://node-hostname:10250/stats/summary
{
"node": {
"nodeName": "node-hostname",
"systemContainers": [
{
"name": "kubelet",
...
"volume": [
{
"time": "2018-11-08T23:52:03Z",
"availableBytes": 1969168384,
"capacityBytes": 1969180672,
"usedBytes": 12288,
"inodesFree": 480748,
"inodes": 480757,
"inodesUsed": 9,
"name": "kube-proxy-token-pprwb"
}
],
"ephemeral-storage": {
"time": "2018-11-09T00:05:10Z",
"availableBytes": 31057477632,
"capacityBytes": 41567858688,
"inodesFree": 4873887,
"inodes": 5120000
}
...
}
Similarly:
$ curl -k -H 'Authorization: Bearer <Redacted>' \
https://node-hostname:10250/stats/summary
# HELP apiserver_audit_event_total Counter of audit events generated and sent to the audit backend.
# TYPE apiserver_audit_event_total counter
apiserver_audit_event_total 0
# HELP apiserver_client_certificate_expiration_seconds Distribution of the remaining lifetime on the certificate used to authenticate a request.
# TYPE apiserver_client_certificate_expiration_seconds histogram
apiserver_client_certificate_expiration_seconds_bucket{le="0"} 0
apiserver_client_certificate_expiration_seconds_bucket{le="21600"} 0
apiserver_client_certificate_expiration_seconds_bucket{le="43200"} 0
...
More info on kubelet authentication/authorization.
Prometheus
More info on K8s metrics here.
You can do this through the raw command.
kubectl get --raw "/api/v1/nodes/(your-node-name)/proxy/stats/summary"
There is also this
kubectl get --raw "/api/v1/nodes/(your-node-name)/proxy/metrics/cadvisor""
EDIT:
I've created a Prometheus exporter for this now.
https://github.com/jmcgrath207/k8s-ephemeral-storage-metrics
I am using the following command to create a configMap.
kubectl create configmap test --from-file=./application.properties --from-file=./mongo.properties --from-file=./logback.xml
Now, I have modified a value for a key from mongo.properties which i need to update in kubernetes.
Option1 :-
kubectl edit test
Here, it opens the entire file. But, I want to just update mongo.properties and hence want to see only the mongo.properties. Is there any other way?
Note :- I dont want to have mongo.properties in a separate configMap.
Thanks
Now you can. Just throw: kubectl edit configmap <name of the configmap> on your command line. Then you can edit your configuration.
Another option is actually you can use this command:
kubectl create configmap some-config \
--from-file=some-key=some-config.yaml \
-n some-namespace \
-o yaml \
--dry-run | kubectl apply -f -
Refer to Github issue: Support updating config map and secret with --from-file
kubectl edit configmap -n <namespace> <configMapName> -o yaml
This opens up a vim editor with the configmap in yaml format. Now simply edit it and save it.
Here's a neat way to do an in-place update from a script.
The idea is;
export the configmap to YAML (kubectl get cm -o yaml)
use sed to do a command-line replace of an old value with a new value (sed "s|from|to")
push it back to the cluster using kubectl apply
In this worked example, I'm updating a log level variable from 'info' level logging to 'warn' level logging.
So, step 1, read the current config;
$ kubectl get cm common-config -o yaml
apiVersion: v1
data:
CR_COMMON_LOG_LEVEL: info
kind: ConfigMap
Step 2, you modify it locally with a regular expression search-and-replace, using sed:
$ kubectl get cm common-config -o yaml | \
sed -e 's|CR_COMMON_LOG_LEVEL: info|CR_COMMON_LOG_LEVEL: warn|'
apiVersion: v1
data:
CR_COMMON_LOG_LEVEL: warn
kind: ConfigMap
You can see the value has changed. Let's push it back up to the cluster;
Step 3; use kubectl apply -f -, which tells kubectl to read from stdin and apply it to the cluster;
$ kubectl get cm common-config -o yaml | \
sed -e 's|CR_COMMON_LOG_LEVEL: info|CR_COMMON_LOG_LEVEL: warn|' | \
kubectl apply -f -
configmap/common-config configured
No, you can't.
Replace in kubernetes will simply replace everything in that configmap. You can't just update one file or one single property in it.
However, if you check with the client Api, you will find if you create a configmap with lots of files. Then, those files will be stored as a HashMap, where key is file name by default, value is the file content encoded as a string. So you can write your own function based on existing key-value pair in HashMap.
This is what I found so far, if you find there is already existing method to deal with this issue, please let me know :)
FYI, if you want to update just one or few properties, it is possible if you use patch. However, it is a little bit hard to implement.
this and this may help
Here is how you can add/modify/remove files in a configmap with some help from jq:
export configmap to a JSON file:
CM_FILE=$(mktemp -d)/config-map.json
oc get cm <configmap name> -o json > $CM_FILE
DATA_FILES_DIR=$(mktemp -d)
files=$(cat $CM_FILE | jq '.data' | jq -r 'keys[]')
for k in $files; do
name=".data[\"$k\"]"
cat $CM_FILE | jq -r $name > $DATA_FILES_DIR/$k;
done
add/modify a file:
echo '<paste file contents here>' > $DATA_FILES_DIR/<file name>.conf
remove a file:
rm <file name>.conf
when done, update the configmap:
kubectl create configmap <configmap name> --from-file $DATA_FILES_DIR -o yaml --dry-run | kubectl apply -f -
delete temporary files and folders:
rm -rf CM_FILE
rm -rf DATA_FILES_DIR
Here is a complete shell script to add new file to configmap (or replace existing one) based on #Bruce S. answer https://stackoverflow.com/a/54876249/2862663
#!/bin/bash
# Requires jq to be installed
if [ -z "$1" ]
then
echo "usage: update-config-map.sh <config map name> <config file to add>"
return
fi
if [ -z "$2" ]
then
echo "usage: update-config-map.sh <config map name> <config file to add>"
return
fi
CM_FILE=$(mktemp -d)/config-map.json
kubectl get cm $1 -o json > $CM_FILE
DATA_FILES_DIR=$(mktemp -d)
files=$(cat $CM_FILE | jq '.data' | jq -r 'keys[]')
for k in $files; do
name=".data[\"$k\"]"
cat $CM_FILE | jq -r $name > $DATA_FILES_DIR/$k;
done
echo cunfigmap: $CM_FILE tempdir: $DATA_FILES_DIR
echo will add file $2 to config
cp $2 $DATA_FILES_DIR
kubectl create configmap $1 --from-file $DATA_FILES_DIR -o yaml --dry-run | kubectl apply -f -
echo Done
echo removing temp dirs
rm -rf $CM_FILE
rm -rf $DATA_FILES_DIR
Suggestion
I would highly consider using a CLI editor like k9s (which is more like a K8S CLI managment tool).
As you can see below (ignore all white placeholders), when your cluster's context is set on terminal you just type k9s and you will reach a nice terminal where you can inspect all cluster resources.
Just type ":" and enter the resource name (configmaps in our case) which will appear in the middle of screen (green rectangle).
Then you can choose the relevant configmap with the up and down arrows and type e to edit it (see green arrow).
For all Configmaps in all namespaces you choose 0, for a specific namespace you choose the number from the upper left menu - for example 1 for kube-system:
I managed to update a setting ("large-client-header-buffers") in the nginx pod's /etc/nginx/nginx.conf via configmap. Here are the steps I have followed..
Find the configmap name in the nginx ingress controller pod describition
kubectl -n utility describe pods/test-nginx-ingress-controller-584dd58494-d8fqr |grep configmap
--configmap=test-namespace/test-nginx-ingress-controller
Note: In my case, the namespace is "test-namespace" and the configmap name is "test-nginx-ingress-controller"
Create a configmap yaml
cat << EOF > test-nginx-ingress-controller-configmap.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: test-nginx-ingress-controller
namespace: test-namespace
data:
large-client-header-buffers: "4 16k"
EOF
Note: Please replace the namespace and configmap name as per finding in the step 1
Deploy the configmap yaml
kubectl apply -f test-nginx-ingress-controller-configmap.yaml
Then you will see the change is updated to nginx controller pod after mins
i.g.
kubectl -n test-namespace exec -it test-nginx-ingress-controller-584dd58494-d8fqr -- cat /etc/nginx/nginx.conf|grep large
large_client_header_buffers 4 16k;
Thanks to the sharing by NeverEndingQueue in How to use ConfigMap configuration with Helm NginX Ingress controller - Kubernetes
I have misunderstanding with how to execute $() commands in exec. i'm creating a job in kubernetes with this params:
command:
- ./kubectl
- -n
- $MONGODB_NAMESPACE
- exec
- -ti
- $(kubectl
- -n
- $MONGODB_NAMESPACE
- get
- pods
- --selector=app=$MONGODB_CONTAINER_NAME
- -o
- jsonpath='{.items[*].metadata.name}')
- --
- /opt/mongodb-maintenance.sh
but the part with $(kubectl -n ... --selector ...) is treated as a string and don't execute. Please tell me how to do it properly. Thanks!
As far as I know this is not achievable by putting each section as an array element. Instead you can do something like the following:
command:
- /bin/sh
- -c
- |
./kubectl -n $MONGODB_NAMESPACE exec -ti $(kubectl -n $MONGODB_NAMESPACE get pods --selector=app=$MONGODB_CONTAINER_NAME -o jsonpath='{.items[*].metadata.name}') -- /opt/mongodb-maintenance.sh
From the output of the kubectl exec, I noticed that you can use -- to separate your arguments
# List contents of /usr from the first container of pod 123456-7890 and sort by modification time.
# If the command you want to execute in the pod has any flags in common (e.g. -i),
# you must use two dashes (--) to separate your command's flags/arguments.
# Also note, do not surround your command and its flags/arguments with quotes
# unless that is how you would execute it normally (i.e., do ls -t /usr, not "ls -t /usr").
kubectl exec 123456-7890 -i -t -- ls -t /usr