How to programmatically modify a running k8s pod status conditions? - kubernetes

I'm trying to modify the running state of my pod, managed by a deployment controller both from command line via kubectl patch and from the k8s python client API. Neither of them seem to work
From the command line, I tried both strategic merge match and JSON merge patch, but neither of them works. For e.g. I'm trying to patch the pod conditions to make the status field to False
kubectl -n foo-ns patch pod foo-pod-18112 -p '{
"status": {
"conditions": [
{
"type": "PodScheduled",
"status": "False"
},
{
"type": "Ready",
"status": "False"
},
{
"type": "ContainersReady",
"status": "False"
},
{
"type": "Initialized",
"status": "False"
}
],
"phase": "Running"
}
}' --type merge
From the python API
# definition of various pod states
ready_true = { "type": "Ready", "status": "True" }
ready_false = { "type": "Ready", "status": "False" }
scheduled_true = { "type": "PodScheduled", "status": "True" }
cont_ready_true = { "type": "ContainersReady", "status": "True" }
cont_ready_false = { "type": "ContainersReady", "status": "False" }
initialized_true = { "type": "Initialized", "status": "True" }
initialized_false = { "type": "Initialized", "status": "False" }
patch = {"status": { "conditions": [ready_false, initialized_false, cont_ready_false, scheduled_true ], "phase" : "Running" }}
p_status = v1.patch_namespaced_pod_status(podname, "default", body=patch)
While running the above snippet, I don't see any errors and the response p_status has all the pod conditions modified as applied in the patch, but I don't see any events from API server related to this pod status change.
May be the deployment controller is rolling back the changes to a working config? I'm looking for ways to patch the pod conditions and test if my custom controller (not related to the question) is able to see those new pod conditions.

You should not.
Clients write the desired state in the spec: and controllers write the status:-part.

Related

For unsuccessful actions, there should not be outputs

I'm trying to test my workflow with HTTP request action... And when I select Status "Failure" - I can't add Output. But when I send a request without testing - I can see Output from the failed action (i.e. Status code, Body, Headers).
So, how can I test this one with Output parameters? Actually, I have to handle Status code in the subsequent actions.
My workflow seems like this:
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Bad": {
"inputs": {
"body": "Bad: #{outputs('HTTP')['statusCode']}",
"statusCode": 503
},
"kind": "http",
"runAfter": {
"HTTP": [
"TIMEDOUT",
"FAILED"
]
},
"type": "Response"
},
"HTTP": {
"inputs": {
"method": "GET",
"uri": "#{appsetting('externalServiceUrl')}/api/entities/"
},
"runAfter": {},
"type": "Http"
},
"Success": {
"inputs": {
"body": "Success",
"statusCode": 200
},
"kind": "http",
"runAfter": {
"HTTP": [
"Succeeded"
]
},
"type": "Response"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"triggers": {
"manual": {
"inputs": {},
"kind": "Http",
"type": "Request"
}
}
},
"kind": "Stateful"
}
Actually, I have to handle Status code in the subsequent actions.
You just need to change the status as Succeeded and in Status Code when you scroll down you can find a bunch of status codes that you can set.
Additionally, To Handle Status Code further you can also use the Condition action of Control check if the HTTP connector has satisfied the conditions that we are looking for. Here is my logic app workflow.
So the Success executes for 200 and 202 status and rest all codes comes under bad action when executed.
My solution in Acceptance tests to test this workflow - I just mocked API using WireMock.
But lack of possibility to test with the default functionality (without Kludge) seems like a bug.

Extract status of Kubernetes CR created via ansible-operator

I am new to json query. Facing trouble extracting the status.conditions[ansibleResult].type
I have a CRD defined and created CR against the same, which is picked up by operator-sdk running ansible in the background. I am updating the CRD to provide relevant status once CR is accepted and processed by operator-sdk.
CR output in json appears like below.
{
"apiVersion": "vault.cpe.oraclecloud.com/v1alpha1",
"kind": "OciVaultKeys",
"metadata": {
"annotations": {
"kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"vault.cpe.oraclecloud.com/v1alpha1\",\"kind\":\"OciVaultKeys\",\"metadata\":{\"annotations\":{},\"name\":\"operator-key-broken\",\"namespace\":\"tms\"},\"spec\":{\"freeformTags\":[{\"key\":\"Type\",\"value\":\"Optional-Values-Added\"}],\"ociVaultKeyName\":\"operator-key-broken\",\"ociVaultKeyShapeAlgorithm\":\"RSA\",\"ociVaultKeyShapeLength\":32,\"ociVaultName\":\"ocivault-sample-12\"}}\n"
},
"creationTimestamp": "2022-03-18T07:43:03Z",
"finalizers": [
"vault.cpe.oraclecloud.com/finalizer"
],
"generation": 1,
"name": "operator-key-broken",
"namespace": "tms",
"resourceVersion": "717880023",
"selfLink": "/apis/vault.cpe.oraclecloud.com/v1alpha1/namespaces/tms/ocivaultkeys/operator-key-broken",
"uid": "0d634e72-f592-48e0-be9b-ebfa017b2dfe"
},
"spec": {
"freeformTags": [
{
"key": "Type",
"value": "Optional-Values-Added"
}
],
"ociVaultKeyName": "operator-key-broken",
"ociVaultKeyShapeAlgorithm": "RSA",
"ociVaultKeyShapeLength": 32,
"ociVaultName": "ocivault-sample-12"
},
"status": {
"conditions": [
{
"lastTransitionTime": "2022-03-18T07:43:27Z",
"message": "",
"reason": "",
"status": "False",
"type": "Successful"
},
{
"lastTransitionTime": "2022-03-18T08:26:08Z",
"message": "Running reconciliation",
"reason": "Running",
"status": "False",
"type": "Running"
},
{
"ansibleResult": {
"changed": 0,
"completion": "2022-03-18T08:26:24.217728",
"failures": 1,
"ok": 14,
"skipped": 1
},
"lastTransitionTime": "2022-03-18T08:26:25Z",
"message": "The task includes an option with an undefined variable. The error was: No first item, sequence was empty.\n\nThe error appears to be in '/home/opc/cpe-workstation/mr_folder/workspace-2/osvc-kubernetes-operators/oci-services/roles/ocivaultkeys/tasks/fetch_vault_details_oci.yml': line 12, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: DEBUG | Fetch Vault Details | Extract Vault OCID n service_endpoint in source region\n ^ here\n",
"reason": "Failed",
"status": "True",
"type": "Failure"
}
]
}
}
I wish to reliably extract the status.conditions[].type (for the element ansibleResult) in CRD.
CRD definition extract is as below
- name: v1alpha1
served: true
storage: true
additionalPrinterColumns:
- description: 'Status of the OCI Vault Key'
jsonPath: .status.conditions[-1].type
name: STATUS
type: string
priority: 0
CRD is looking for a jsonPath expression to extract.
Thanks
Please try following :
kubectl get ocivaultkeys operator-key-broken -o jsonpath='{.status.conditions[?(#.ansibleResult)].type}'
Expected output : Failure
jsonpath help

POD is being terminated and created again due to scale up and it's running twice

I have an application that runs a code and at the end it sends an email with a report of the data. When I deploy pods on GKE , certain pods get terminated and a new pod is created due to Auto Scale, but the problem is that the termination is done after my code is finished and the email is sent twice for the same data.
Here is the JSON file of the deploy API:
{
"apiVersion": "batch/v1",
"kind": "Job",
"metadata": {
"name": "$name",
"namespace": "$namespace"
},
"spec": {
"template": {
"metadata": {
"name": "********"
},
"spec": {
"priorityClassName": "high-priority",
"containers": [
{
"name": "******",
"image": "$dockerScancatalogueImageRepo",
"imagePullPolicy": "IfNotPresent",
"env": $env,
"resources": {
"requests": {
"memory": "2000Mi",
"cpu": "2000m"
},
"limits":{
"memory":"2650Mi",
"cpu":"2650m"
}
}
}
],
"imagePullSecrets": [
{
"name": "docker-secret"
}
],
"restartPolicy": "Never"
}
}
}
}
and here is a screen-shot of the pod events:
Any idea how to fix that?
Thank you in advance.
"Perhaps you are affected by this "Note that even if you specify .spec.parallelism = 1 and .spec.completions = 1 and .spec.template.spec.restartPolicy = "Never", the same program may sometimes be started twice." from doc. What happens if you increase terminationgraceperiodseconds in your yaml file? – "
#danyL
my problem was that I had another jobs that deploy pods on my nodes with more priority , so it was trying to terminate my running pods but the job was already done and the email was already sent , so i fixed the problem by fixing the request and the limit resources on all my json files , i don't know if it's the perfect solution but for now it solved my problem.
Thank you all for you help

What is my Custom Resource Definition URL in Kubernetes

I am trying to hit my custom resource definition endpoint in Kubernetes but cannot find an exact example for how Kubernetes exposes my custom resource definition in the Kubernetes API. If I hit the custom services API with this:
https://localhost:6443/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions
I get back this response
"items": [
{
"metadata": {
"name": "accounts.stable.ibm.com",
"selfLink": "/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/accounts.stable.ibm.com",
"uid": "eda9d695-d3d4-11e9-900f-025000000001",
"resourceVersion": "167252",
"generation": 1,
"creationTimestamp": "2019-09-10T14:11:48Z",
"deletionTimestamp": "2019-09-12T22:26:20Z",
"finalizers": [
"customresourcecleanup.apiextensions.k8s.io"
]
},
"spec": {
"group": "stable.ibm.com",
"version": "v1",
"names": {
"plural": "accounts",
"singular": "account",
"shortNames": [
"acc"
],
"kind": "Account",
"listKind": "AccountList"
},
"scope": "Namespaced",
"versions": [
{
"name": "v1",
"served": true,
"storage": true
}
],
"conversion": {
"strategy": "None"
}
},
"status": {
"conditions": [
{
"type": "NamesAccepted",
"status": "True",
"lastTransitionTime": "2019-09-10T14:11:48Z",
"reason": "NoConflicts",
"message": "no conflicts found"
},
{
"type": "Established",
"status": "True",
"lastTransitionTime": null,
"reason": "InitialNamesAccepted",
"message": "the initial names have been accepted"
},
{
"type": "Terminating",
"status": "True",
"lastTransitionTime": "2019-09-12T22:26:20Z",
"reason": "InstanceDeletionCheck",
"message": "could not confirm zero CustomResources remaining: timed out waiting for the condition"
}
],
"acceptedNames": {
"plural": "accounts",
"singular": "account",
"shortNames": [
"acc"
],
"kind": "Account",
"listKind": "AccountList"
},
"storedVersions": [
"v1"
]
}
}
]
}
This leads me to believe I have correctly created the custom resource accounts. There are a number of examples that don't seem to be quite right and I cannot find my resource in the Kubernetes REST api. I can use with my custom resource from kubectl but I need to expose it with RESTful APIs.
https://localhost:6443/apis/stable.example.com/v1/namespaces/default/accounts
returns
404 page not found
Where as:
https://localhost:6443/apis/apiextensions.k8s.io/v1beta1/apis/stable.ibm.com/namespaces/default/accounts
returns
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "the server could not find the requested resource",
"reason": "NotFound",
"details": {},
"code": 404
}
I have looked at https://docs.okd.io/latest/admin_guide/custom_resource_definitions.html and https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/
The exact URL would be appreciated.
This is a quite decent way retrieving K8s REST API resource executing kubectl get command on some top debugging levels, like #Suresh Vishnoi mentioned in the comment:
kubectl get <api-resource> -v=8
Apparently, eventually checked by #Amit Kumar Gupta, the correct URL accessing custom resource as per your CRD json output is the following:
https://<API_server>:port/apis/stable.ibm.com/v1/namespaces/default/accounts
Depending on the authentication method you may choose: X509 Client Certs, Static Token File, Bearer Token or HTTP API proxy in order to authenticate user requests against Kubernetes API.

Kubernetes - Job scheduling API

I am trying to schedule Jobs in Kubernetes.
https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/
The job can be created using the below command.
$ kubectl create -f ./cronjob.yaml
cronjob "hello" created
Is there any REST API using which the Job can be created from a Java client?
Thanks
The respective REST endpoint is described in the official API reference. You will find the CronJob resource in the batch/v1beta1 API group. To create a new CronJob resource, you'll need a POST call to the /apis/batch/v1beta1/namespaces/{namespace}/cronjobs URL.
A respective HTTP request might look something like this:
POST /apis/batch/v1beta1/namespaces/default/cronjobs HTTP/1.1
Content-Type: application/json
Content-Length: ...
Authorization: ...
[other headers]
{
"metadata": {
"name": "some-cron"
},
"spec": {
...
}
}
There are also older versions of the same resource, for example in the batch/v2alpha1 API group. As a rule of thumb, I'd recommend using the newest API version available to you. Especially, do not rely on alpha APIs in production; they tend to deprecate pretty quickly between releases.
To create a batch/v1beta1 CronJob using the Java client, have a look at the createNamespacedCronJob method of the io.kubernetes.client.openapi.apis.BatchV1beta1Api class.
HTTP Request
POST /apis/batch/v1beta1/namespaces/{namespace}/cronjobs
You can take a look here for API overview: cronjob-v1beta1-batch
Add CronJob object in request Body.
{
"apiVersion": "batch/v1beta1",
"kind": "CronJob",
"metadata": {
"name": "hello"
},
"spec": {
"schedule": "*/1 * * * *",
"jobTemplate": {
}
}
}
Check here its spec: writing-a-cron-job-spec
Part of swagger.json
"post": {
"description": "create a CronJob",
"consumes": [
"*/*"
],
"produces": [
"application/json",
"application/yaml",
"application/vnd.kubernetes.protobuf"
],
"schemes": [
"https"
],
"tags": [
"batch_v1beta1"
],
"operationId": "createBatchV1beta1NamespacedCronJob",
"parameters": [
{
"name": "body",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/io.k8s.api.batch.v1beta1.CronJob"
}
}
],
"responses": {
"200": {
"description": "OK",
"schema": {
"$ref": "#/definitions/io.k8s.api.batch.v1beta1.CronJob"
}
},
"201": {
"description": "Created",
"schema": {
"$ref": "#/definitions/io.k8s.api.batch.v1beta1.CronJob"
}
},
"202": {
"description": "Accepted",
"schema": {
"$ref": "#/definitions/io.k8s.api.batch.v1beta1.CronJob"
}
},
"401": {
"description": "Unauthorized"
}
},
"x-kubernetes-action": "post",
"x-kubernetes-group-version-kind": {
"group": "batch",
"kind": "CronJob",
"version": "v1beta1"
}
}
See full swagger.json