I have 2 services. Service A and Service B. They correspond to deployments dA and dB.
I set up my cluster and start both services/deployments. Service A is reachable from the external world. External World --> Service A <--> Service B.
How can I scale dB (change replicaCount and run kubectl apply OR kubectl scale) from within Service A that is responding to a user request.
For example, if a user that is being served by Service A wants some extra resource in my app, I would like to provide that by adding an extra pod to dB. How do I do this programatically?
Every Pod, unless it opts out, has a ServiceAccount token injected into it, which enables it to interact with the kubernetes API according to the Role associated with the ServiceAccount
Thus, one can use any number of kubernetes libraries -- most of which are "in cluster" aware, meaning they don't need any further configuration to know about that injected ServiceAccount token and how to use it -- to issue scale events against any resource the ServiceAccount's Role is authorized to use
You can make it as simple or as complex as you'd like, but the tl;dr is akin to:
curl --cacert /var/run/secrets/kubernetes.io/ca.crt \
--header "Accept: application/json" \
--header "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/token)" \
https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}/api/v1/namespaces
Related
Requirement: We need to access the Kubernetes REST end points from our java code. Our basic operations using the REST end points are to Create/Update/Delete/Get the deployments.
We have downloaded the kubectl and configured the kubeconfig file of the cluster in our Linux machine.
We can perform operations in that cluster using the kubectl. We got the bearer token of that cluster running the command 'kubectl get pods -v=8'. We are using this bearer token in our REST end points to perform our required operations.
Questions:
What is the better way to get the bearer token?
Will the bearer token gets change during the lifecycle of the cluster?
follow this simple way
kubectl proxy --port=8080 &
curl http://localhost:8080/api/
from java code use the below approach
# Check all possible clusters, as you .KUBECONFIG may have multiple contexts:
kubectl config view -o jsonpath='{"Cluster name\tServer\n"}{range .clusters[*]}{.name}{"\t"}{.cluster.server}{"\n"}{end}'
# Select name of cluster you want to interact with from above output:
export CLUSTER_NAME="some_server_name"
# Point to the API server refering the cluster name
APISERVER=$(kubectl config view -o jsonpath="{.clusters[?(#.name==\"$CLUSTER_NAME\")].cluster.server}")
# Gets the token value
TOKEN=$(kubectl get secrets -o jsonpath="{.items[?(#.metadata.annotations['kubernetes\.io/service-account\.name']=='default')].data.token}"|base64 -d)
# Explore the API with TOKEN
curl -X GET $APISERVER/api --header "Authorization: Bearer $TOKEN" --insecure
Q: What is the better way to get the bearer token?
A: Since you have configured access to the cluster, you might use
kubectl describe secrets
Q: Will the bearer token gets change during the lifecycle of the cluster?
A: Static tokens do not expire.
Please see Accessing Clusters and Authenticating for more details.
If your token comes from a ServiceAccount, best way should be to add this Service Account to the deployment/statefull or daemonSet where your java runs.This will lead to tokne being written in your pod in "/var/run/secrets/kubernetes.io/serviceaccount/token". Config is explained here
Most of kubernetes client library knows how to discover a service account linked to a running context and "automaticaly" use it. Not an expert be it seems yes for Java client see : github
This token will not be automaticaly refreshed by kubernetes unless someone force a rolling.
Background
I have a K8S cluster with a number of different pods that have their own specific service accounts, cluster roles, and cluster role bindings, so that they can execute various read/write requests directly with the K8S REST API. There are some complicated requests that can be issued, and I'd like to make a function to wrap the complex logic. However, the various services in the cluster are written in multiple (i.e. 6+) programming languages, and there does not (yet) seem to be a trivial way to allow all these services to directly re-use this code.
I'm considering creating a "proxy" micro-service, that exposes its own REST API, and issues the necessary requests and handles the "complex logic" on behalf of the client.
Problem
The only problem is that, with the current deployment model, a client could request that the proxy micro-service execute an HTTP request that the client itself isn't authorized to make.
Question
Is there a trivial/straightforward way for one pod, for example, to identify the client pod, and execute some kind of query/result-of-policy operation (i.e. by delegating the authentication to the K8S cluster authentication mechanism itself) to determine if it should honor the request from the client pod?
Kubernetes Authentication model represents a way how the particular user or service account can be entitled in k8s cluster, however Authorization methods determine whether initial request from the cluster visitor, aimed to do some action on cluster resources/objects, has sufficient permissions to make that possible.
Due to the fact that you've used specific service accounts per each Pod entire the cluster and granting them specific RBAC rules, it might be possible to use SelfSubjectAccessReview API in order to inspect requests to k8s REST API and determine whether the client's Pod service account has appropriate permission to perform any action on target's Pod namespace.
That can be achievable using kubectl auth can-i subcommand by submitting essential information for user impersonation.
I assume that you might also be able to query k8s authorization API group within HTTP request schema and then parse structured data from JSON/YAML format, like in the example below:
Regular kubectl auth can-i command to check whether default SA can retrieve data about Pods in default namespace:
kubectl auth can-i get pod --as system:serviceaccount:default:default
Equivalent method via HTTP call to k8s REST API using JSON type of content within Bearer Token authentication:
curl -k \
-X POST \
-d #- \
-H "Authorization: Bearer $MY_TOKEN" \
-H 'Accept: application/json' \
-H "Impersonate-User: system:serviceaccount:default:default" \
-H 'Content-Type: application/json' \
https://<API-Server>/apis/authorization.k8s.io/v1/selfsubjectaccessreviews <<'EOF'
{
"kind": "SelfSubjectAccessReview",
"apiVersion": "authorization.k8s.io/v1",
"spec":{"resourceAttributes":{"namespace":"default","verb":"get","resource":"pods"}}
}
EOF
Output:
.... "status": {
"allowed": true,
"reason": "RBAC: allowed by RoleBinding ....
I was looking at the kubernetes API endpoints listed here. Im trying to create a deployment which can be run from the terminal using kubectl ru CLUSTER-NAME IMAGE-NAME PORT. However I cant seem to find any endpoint for this command in the link I posted above. I can create a node using curl POST /api/v1/namespaces/{namespace}/pods and then delete using the curl -X DELETE http://localhost:8080/api/v1/namespaces/default/pods/node-name where node name HAS to be a single node (if there are 100 nodes, each should be done individually). Is there an api endpoint for creating and deleting deployments??
To make it easier to eliminate fields or restructure resource representations, Kubernetes supports multiple API versions, each at a different API path, such as /api/v1 or /apis/extensions/v1beta1 and to extend the Kubernetes API, API groups is implemented.
Currently there are several API groups in use:
the core (oftentimes called legacy, due to not having explicit group name) group, which is at REST path /api/v1 and is not specified as part of the apiVersion field, e.g. apiVersion: v1.
the named groups are at REST path /apis/$GROUP_NAME/$VERSION, and use apiVersion: $GROUP_NAME/$VERSION (e.g. apiVersion: batch/v1). Full list of supported API groups can be seen in Kubernetes API reference.
To manage extensions resources such as Ingress, Deployments, and ReplicaSets refer to Extensions API reference.
As described in the reference, to create a Deployment:
POST /apis/extensions/v1beta1/namespaces/{namespace}/deployments
I debugged this by running kubectl with verbose logging: kubectl --v=9 update -f dev_inventory.yaml.
It showed the use of an API call like this one:
curl -i http://localhost:8001/apis/extensions/v1beta1/namespaces/default/deployments
Note that the first path element is apis, not the normal api. I don't know why it's like this, but the command above works.
I might be too late to help in this question, but here is what I tried on v1.9 to deploy a StatefulSet:
curl -kL -XPOST -H "Accept: application/json" -H "Content-Type: application/json" \
-H "Authorization: Bearer <*token*>" --data #statefulset.json \
https://<*ip*>:6443/apis/apps/v1/namespaces/eng-pst/statefulsets
I converted the statefulset.yaml to json cause I saw the data format when api was doing the POST was in json.
I ran this command to find out the API call i need to make for my k8s object:
kubectl --v=10 apply -f statefulset.yaml
(might not need a v=10 level but I wanted to as much info as I could)
The Kubernetes Rest Api documentation is quite sophisticated but unfortunately the deployment documentation is missing.
Since the Rest schema is identical to other resources you can figure out the rest calls:
GET retrieve a deployment by name:
curl -H "Authorization: Bearer ${KEY}" ${API_URL}/apis/extensions/v1beta1/namespaces/${namespace}/deployments/${NAME}
POST create a new deployment
curl -X POST -d #deployment-definition.json -H "Content-Type: application/json" -H "Authorization: Bearer ${KEY}" ${API_URL}/apis/extensions/v1beta1/namespaces/${namespace}/deployments
You should be able to use the calls right away when you provide the placeholder for your
API key ${KEY}
API url ${API_URL}
Deployment name ${NAME}
Namespace ${namespace}
Have you tried the analogous URL?
http://localhost:8080/api/v1/namespaces/default/deployment/deployment-name
I have a set of pods providing nsqlookupd service.
Now I need each nsqd container to have a list of nsqlookupd servers to connect to (while service will point to different every time) simultaneously. Something similar I get with
kubectl describe service nsqlookupd
...
Endpoints: ....
but I want to have it in a variable within my deployment definition or somehow from within nsqd container
Sounds like you would need an extra service running either in your nsqd container or in a separate container in the same pod. The role of that service would be to pole the API regularly in order to fetch the list of endpoints.
Assuming that you enabled Service Accounts (enabled by default), here is a proof of concept on the shell using curl and jq from inside a pod:
# Read token and CA cert from Service Account
CACERT="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
# Replace the namespace ("kube-system") and service name ("kube-dns")
ENDPOINTS=$(curl -s --cacert "$CACERT" -H "Authorization: Bearer $TOKEN" \
https://kubernetes.default.svc/api/v1/namespaces/kube-system/endpoints/kube-dns \
)
# Filter the JSON output
echo "$ENDPOINTS" | jq -r .subsets[].addresses[].ip
# output:
# 10.100.42.3
# 10.100.67.3
Take a look at the source code of Kube2sky for a good implementation of that kind of service in Go.
Could be done with a StatefuSet. Stable names + stable storage
I would like to access to OpenShift and Kubernetes API from inside a pod to query and modify objects in the application the pod belongs to.
In the documentation (https://docs.openshift.org/latest/dev_guide/service_accounts.html) I found this description on how to access the api:
$ TOKEN="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"
$ curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
"https://openshift.default.svc.cluster.local/oapi/v1/users/~" \
-H "Authorization: Bearer $TOKEN"
The problem is when I for example want to access a pod, I need to know the namespace I'm in:
https://openshift.default.svc.cluster.local/oapi/v1/namespaces/${namespace}/pods
The only way I found so far is to submit the namespace as an environment variable, but I would like to not requiring the user to enter that information.
At least in kubernetes 1.5.3 I can also see the namespace in /var/run/secrets/kubernetes.io/serviceaccount/namespace.
You can get the namespace of your pod automatically populated as an environment variable using the downward API.
I found a solution by tracing what the web console does.
I am allowed to ask for the project list without having cluster-admin rigths on the following url:
https://openshift.default.svc.cluster.local/oapi/v1/projects
Only the projects which I have rights to are listed, and then it is possible to determine the current project which name is also the namespace.
Would love if there was an easier solution, but this works.