Bellow is the how I watch kubernetes to get deployments status(include added\modified\delete ......)
[boomer#bzha kubernetes]$ curl http://10.110.200.24:8080/apis/extensions/v1beta1/watch/namespaces/kube-system/deployments
{"type":"ADDED","object":{"kind":"Deployment","apiVersion":"extensions/v1beta1","metadata":{"name":"influxdb","namespace":"kube-system","selfLink":"/apis/extensions/v1beta1/namespaces/kube-system/deployments/influxdb","uid":"37a07ded-6a25-11e8-b67e-0050568ddfc2","resourceVersion":"6896550","generation":2,"creationTimestamp":"2018-06-07T07:34:31Z","labels":{"k8s-app":"influxdb","task":"monitoring"},"annotations":{"deployment.kubernetes.io/revision":"2"}},"spec":{"replicas":1,"selector":{"matchLabels":{"k8s-app":"influxdb","task":"monitoring"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"k8s-app":"influxdb","task":"monitoring"}},"spec":{"volumes":[{"name":"tz-config","hostPath":{"path":"/usr/share/zoneinfo/Asia/Shanghai","type":""}}],"containers":[{"name":"influxdb","image":"hub.skyinno.com/google_containers/heapster-influxdb-amd64:v1.3.3","resources":{"limits":{"cpu":"4","memory":"4Gi"},"requests":{"cpu":"100m","memory":"128Mi"}},"volumeMounts":[{"name":"tz-config","mountPath":"/etc/localtime"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":1,"maxSurge":1}}},"status":{"observedGeneration":2,"replicas":1,"updatedReplicas":1,"readyReplicas":1,"availableReplicas":1,"conditions":[{"type":"Available","status":"True","lastUpdateTime":"2018-06-07T07:34:31Z","lastTransitionTime":"2018-06-07T07:34:31Z","reason":"MinimumReplicasAvailable","message":"Deployment has minimum availability."}]}}}
{"type":"ADDED","object":{"kind":"Deployment","apiVersion":"extensions/v1beta1","metadata":{"name":"prometheus-core","namespace":"kube-system","selfLink":"/apis/extensions/v1beta1/namespaces/kube-system/deployments/prometheus-core","uid":"fa7b06da-6a2a-11e8-9521-0050568ddfc2","resourceVersion":"8261846","generation":6,"creationTimestamp":"2018-06-07T08:15:45Z","labels":{"app":"prometheus","component":"core"},"annotations":{"deployment.kubernetes.io/revision":"6"}},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"prometheus","component":"core"}},"template":{"metadata":{"name":"prometheus-main","creationTimes
......
when I first start curl to watch the deployment api, I notice that it will return all deployments list first (which type is ADDED), my question is that:
Is it always list deployments first when I watch the api?
When I watch something else, like service or ingress, does it
show the same behavior?
Where could I find document or code about this?
Yes, watching without a resourceVersion specified first sends ADDED events for all existing objects.
It is more common to perform a list operation first to obtain all existing objects, then start a watch passing the resourceVersion from the returned list result to watch for changes from that point.
Related
I created a custom resource definition (CRD) and its controller in my cluster, now I can create custom resources, but how do I validate update requests to the CR? e.g., only certain fields can be updated.
The Kubernetes docs on Custom Resources has a section on Advanced features and flexibility (never mind that validating requests should be considered a pretty basic feature 😉). For validation of CRDs, it says:
Most validation can be specified in the CRD using OpenAPI v3.0 validation. Any other validations supported by addition of a Validating Webhook.
The OpenAPI v3.0 validation won't help you accomplish what you're looking for, namely ensuring immutability of certain fields on your custom resource, it's only helpful for stateless validations where you're looking at one instance of an object and determining if it's valid or not, you can't compare it to a previous version of the resource and validate that nothing has changed.
You could use Validating Webhooks. It feels like a heavyweight solution, as you will need to implement a server that conforms to the Validating Webhook contract (responding to specific kinds of requests with specific kinds of responses), but you will have the required data at least to make the desired determination, e.g. knowing that it's an UPDATE request and knowing what the old object looked like. For more details, see here. I have not actually tried Validating Webhooks, but it feels like it could work.
An alternative approach I've used is to store the user-provided data within the Status subresource of the custom resource the first time it's created, and then always look at the data there. Any changes to the Spec are ignored, though your controller can notice discrepancies between what's in the Spec and what's in the Status, and embed a warning in the Status telling the user that they've mutated the object in an invalid way and their specified values are being ignored. You can see an example of that approach here and here. As per the relevant README section of that linked repo, this results in the following behaviour:
The AVAILABLE column will show false if the UAA client for the team has not been successfully created. The WARNING column will display a warning if you have mutated the Team spec after initial creation. The DIRECTOR column displays the originally provided value for spec.director and this is the value that this team will continue to use. If you do attempt to mutate the Team resource, you can see your (ignored) user-provided value with the -o wide flag:
$ kubectl get team --all-namespaces -owide
NAMESPACE NAME DIRECTOR AVAILABLE WARNING USER-PROVIDED DIRECTOR
test test vbox-admin true vbox-admin
If we attempt to mutate the spec.director property, here's what we will see:
$ kubectl get team --all-namespaces -owide
NAMESPACE NAME DIRECTOR AVAILABLE WARNING USER-PROVIDED DIRECTOR
test test vbox-admin true API resource has been mutated; all changes ignored bad-new-director-name
Is there a way to easily query Kubernetes resources in an intuitive way? Basically I want to run queries to extract info about objects which match my criteria. Currently I face an issue where my match labels isn't quite working and I would like to run the match labels query manually to try and debug my issue.
Basically in a pseudo code way:
Select * from pv where labels in [red,blue,green]
Any third party tools who do something like this? Currently all I have to work with is the search box on the dashboard which isn't quite robust enough.
You could use kubectl with JSONPath (https://kubernetes.io/docs/reference/kubectl/jsonpath/). More information on JSONPath: https://github.com/json-path/JsonPath
It allows you to query any resource property, example:
kubectl get pods -o=jsonpath='{$.items[?(#.metadata.namespace=="default")].metadata.name}'
This would list all pod names in namespace "default". Your pseudo code would be something along the lines:
kubectl get pv -o=jsonpath='{$.items[?(#.metadata.label in ["red","blue","green"])]}'
Let assume that I have a collection with /playrequests endpoint. It is a collection (list) for those players who want to find another player to start a match.
The server will check this collection periodically and if it finds two unassigned players, it will create another resource in another collection with /quickmatchs endpoint and also change (for example) a field in the PlayRequests collection for both players to shows that they are assigned to a quickMatch.
At this point, players can send a PUT or PATCH request to set the (for example) "ready" field of their related quickMach resource to true. so the server and each of them can find out that if both of them is ready and the match can be started.
(The Issue Part Is Below Part...)
Also, before a the playRequests assigned to a match and also after they assigned to it, they can send a DELETE request to /playrequests endpoint to tell the server that they want to give up the request. So if the match doesn't create yet, It is ok. the resource related to the player will remove from playRequests collection. but if player assigned to a match, the server must delete the related playRequest and also it must delete the related quickMatch resource from the quickMatchs collection. ( and also we should modify the playRequest related to another player to indicate that it's unassigned now. or we can check and change it later when he to check the status of his related resources in both collection. It is not the main issue for now. )
So, my question is that is it ok to change a resource that is related to the given end point and also change another resource accordingly, If it is necessary? ( I mean is it ok to manipulate different resources with different endpoints in one request? I don't want to send multiple requests.) or I need to mix those two collections to avoid such an action?
I know that many things ( different strategies ) are possible but I want to know that (from the viewpoint of RESTFUL) what is standard/appropriate and what is not? (consider that I am kinda new to restful)
it ok to change a resource that is related to the given end point and also change another resource accordingly
Yes, and there are consequences.
Suppose we have two resources, /fizz and /buzz, and the representations of those resources are related via some implementation details on the server. On the client, we have accessed both of these resources, so we have cached copies of each of them.
The potential issue here is that the server changes the representations of these resources together, but as far as the client is concerned, they are separate.
For instance, if the client sends an unsafe request to change /fizz, a successful response message from the server will invalidate the locally cached copy of that representation, but the stale representation of /buzz does not get evicted. In effect, the client now has a view of the world with version 0 /buzz and version 1 /fizz.
Is that OK? "It depends" -- will expensive things happen to you if your clients are in a state of unmatched representations? Can you patch over the "problem" in other ways (for instance, by telling the client to check resources for updates more often)?
I have couple of questions regarding the configMap versioning.
Is it possible to use a specific version of a configMap in the deployment file?
I dont see any API's to get list of versions. How to get the list of versions?
Is it possible to compare configMap b/w versions?
How to control the number of versions?
Thanks
Is it possible to use a specific version of a configMap in the deployment file?
Not really.
The closest notion of a "version" is resourceVersion, but that is not for the user to directly act upon.
See API conventions: concurrency control and consistency:
Kubernetes leverages the concept of resource versions to achieve optimistic concurrency. All Kubernetes resources have a "resourceVersion" field as part of their metadata. This resourceVersion is a string that identifies the internal version of an object that can be used by clients to determine when objects have changed.
When a record is about to be updated, it's version is checked against a pre-saved value, and if it doesn't match, the update fails with a StatusConflict (HTTP status code 409).
The resourceVersion is changed by the server every time an object is modified. If resourceVersion is included with the PUT operation the system will verify that there have not been other successful mutations to the resource during a read/modify/write cycle, by verifying that the current value of resourceVersion matches the specified value.
The resourceVersion is currently backed by etcd's modifiedIndex.
However, it's important to note that the application should not rely on the implementation details of the versioning system maintained by Kubernetes. We may change the implementation of resourceVersion in the future, such as to change it to a timestamp or per-object counter.
The only way for a client to know the expected value of resourceVersion is to have received it from the server in response to a prior operation, typically a GET. This value MUST be treated as opaque by clients and passed unmodified back to the server.
Clients should not assume that the resource version has meaning across namespaces, different kinds of resources, or different servers.
Currently, the value of resourceVersion is set to match etcd's sequencer. You could think of it as a logical clock the API server can use to order requests.
However, we expect the implementation of resourceVersion to change in the future, such as in the case we shard the state by kind and/or namespace, or port to another storage system.
In the case of a conflict, the correct client action at this point is to GET the resource again, apply the changes afresh, and try submitting again.
This mechanism can be used to prevent races like the following:
Client #1 Client #2
GET Foo GET Foo
Set Foo.Bar = "one" Set Foo.Baz = "two"
PUT Foo PUT Foo
When these sequences occur in parallel, either the change to Foo.Bar or the change to Foo.Baz can be lost.
On the other hand, when specifying the resourceVersion, one of the PUTs will fail, since whichever write succeeds changes the resourceVersion for Foo.
resourceVersion may be used as a precondition for other operations (e.g., GET, DELETE) in the future, such as for read-after-write consistency in the presence of caching.
"Watch" operations specify resourceVersion using a query parameter. It is used to specify the point at which to begin watching the specified resources.
This may be used to ensure that no mutations are missed between a GET of a resource (or list of resources) and a subsequent Watch, even if the current version of the resource is more recent.
This is currently the main reason that list operations (GET on a collection) return resourceVersion.
When I do a kubectl describe <pod>, the bottom section has an "Events" section, displaying Events related to that pod. For example, an event with Reason "failedScheduling", with the message "Failed for reason PodFitsResources and possibly others"
How can I query the API to return that list of events?
If I call /api/v1/namespaces/<ns>/pods/<pod_name>, it doesn't return any Events. If I try the /api/v1/events endpoint, I can specify a labelSelector parameter, but the name of the pod isn't a label of the Event, though it is in the object.involvedObject.name field.
I could request the entire Event stream and filter out the few Events that interest me client-side, but that seems like overkill. kubectl is able to do it, so I figure there must be some way that I'm missing.
Thanks.
I think events support a fieldSelector for the involved object kind and name
You can also turn the verbosity level on kubectl up to 8 to see network traces to see what it is doing
If you are still wondering how kubectl gets the events along with the describe command, then have a look at the following:
https://github.com/kubernetes/kubernetes/blob/b6a0718858876bbf8cedaeeb47e6de7e650a6c5b/pkg/kubectl/describe/versioned/describe.go#L242
If you see what's happening is that they first get the details of the resource requested (see https://github.com/kubernetes/kubernetes/blob/b6a0718858876bbf8cedaeeb47e6de7e650a6c5b/pkg/kubectl/describe/versioned/describe.go#L235 ) and then they get all the events from that namespace and filter out the events for the requested resource. See Line 242 in the same link.
So they are not using some other undocumented API or other ways, What you thought as overkill is what they are doing.