Once I register the crd into the k8s cluster, I can use .yaml to create it, without operator running. Then What happends to these created resouces?
I have seen the Reconciler of operator, but it's more like an async status transfer. When we create a pod, we can directly get the pod ip from the create result. But it seems that I didn't find a place to write my OnCreate hook. (I just see some validate webhook, but never see a hook that be called when creation request made, defines how to create the resourse, and return the created resourse info to the caller ).
If my story is that for one kind of resource, in a time window, all coming creation will multiplex only one pod. Can you give me some advice?
That's a big story for kubernetes crd/controller life cycle, I try to make a simple representation.
After register a new CRD, and create CR, kube-api-server do not care if there is a related controller existed or not. see the process:
That's means the resource(your CR) will be store to etcd, has no business of your controller
ok, let talk about your controller. your controller will setup a list/watch(actually a long live http link) to the api-server and register hook(what you ask, right?) for different event: onCreate, onUpdate and onDelete. Actually you will handle all event in your controller's reconcile (remember kubernetes reconcile's responsibility: move current state to desired state). see the diagram:
For the list/watch link in your controller, you need set different link for different kind of resource. for example: if you care about event for pod, you need set pod list/watch or care about deployment, and set a deployment list/watch...
Related
I recently got started with building a Kubernetes operator. I'm using the Fabric8 Java Kubernetes Client but I think my question is more general and also applies to other programming languages and libraries.
When reading through blog posts, documentation or textbooks explaining the operator pattern, I found there seem to be two options to design an operator:
Using an infinite reconcile loop, in which all corresponding Kubernetes objects are retrieved from the API and then some action is performed.
Using informers, which are called whenever an observed Kubernetes resource changes.
However, I don't find any source discussion which option should be used in which case. Are there any best practices?
You should use both.
When using informers, it's possible that the handler gets the events out of order or even not at all. The former means the handler needs to define and reconcile state - this approach is referred to as level-based, as opposed to edge-based. The latter means reconciliation needs to be triggered on a regular interval to account for that possibility.
The way controller-runtime does things, reconciliation is triggered by cluster events (using informers behind the scenes) related to the resources watched by the controller and on a timer. Also, by design, the event is not passed to the reconciler so that it is forced to define and act on a state.
We are having a Kubernetes service whose pods take some time to warm up with first requests. Basically first incoming requests will read some cached values from Redis and these requests might take a bit longer to process. When these newly created pods become ready and receive full traffic, they might become not very responsive for up to 30 seconds, before everything is correctly loaded from Redis and cached.
I know, we should definitely restructure the application to prevent this, unfortunately that is not feasible in a near future (we are working on it).
It would be great if it was possible to reduce the weight of the newly created pods, so they would receive 1/10 of the traffic in the beggining with the weight increasing as the time would pass. This would be also great for newly deployed versions of our application to see if it behaves correctly.
Why you need the cache loading in first call instead of having in heartbeat which is hooked to readiness probe? One other option is to make use of init containers in kubernetes
Until the application can be restructured to do this "priming" internally...
For when running on Kubernetes, look into Container Lifecycle Hooks and specifically into the PostStart hook. Documentation here and example here.
It seems that the behavior of "...The Container's status is not set to RUNNING until the postStart handler completes" is what can help you.
There's are few gotchas like "... there is no guarantee that the hook will execute before the container ENTRYPOINT" because "...The postStart handler runs asynchronously relative to the Container’s code", and "...No parameters are passed to the handler".
Perhaps a custom script can simulate that first request with some retry logic to wait for the application to be started?
I'm interested in watching a stream of Events from Kubernetes, to determine whether a deployment was successful, or if any of the Pods were unable to be scheduled.
I could call the endpoint /api/v1/watch/events, or I could call /api/v1/events?watch=true. Is there a difference between those two? I'm confused about the purpose of them.
Thanks.
We're making watch a query param and removing it from the path (legacy form). You should call /api/v1/events?watch=true. See more discussions here if you're interested.
Say I am using a service A which is imported in another service B. While B is running normally(ofcourse A is Active), what will happen is service A is uninstalled while service B is still running?
Service A -> Service B
What will be the different scenarios in case I am using ServiceReference, ServiceTracker & DS?
When a service is unpublished in OSGi, an event is sent to all bundles currently using that service to tell them that they should stop using it.
If you are using DS, your unbind method will be called. When it is called, you should make best efforts to stop using the service as soon as possible. But ultimately OSGi is a cooperative system, it cannot force you to release the service. However if you don't then you can cause problems, for example the service publisher will not be fully garbage-collected. You end up sabotaging the dynamics of the OSGi platform, possibly creating memory leaks and so on.
If you are using ServiceTracker then the removedService method will be called, and you need to respond in the same way. But didn't I tell you in the other question not to use ServiceTracker?? ;-)
If you are using ServiceReference then you need to explicitly register a ServiceListener in order to receive these events. This is why you really really shouldn't use this low-level API until you have gained a lot more experience (and once you do have that experience, you won't want to use it anyway!).
First of all: one of the advantages of OSGi is that the behaviour of the framework and standard services are clearly specified. Those specifications can be downloaded from the OSGi Alliance web site, or, if you don't like reading PDFs, ordered for print. The question you are asking is perfectly answered in those specifications.
That said, in summary: when a service is unregistered:
The ServiceReference object remains as it is. However, a call to ServiceReference.getService() will return null. Note that when using ServiceReferences you should release any references to the actual service object as retrieved via getService(), this normally requires some kind of tracking of the service.
For ServiceTracker ServiceTracker.remove is called. This normally results in a call to removedService() on the ServiceTracker or the defined ServiceTrackerCustomizer.
For DS, the defined unbind method for the referenced service is called (if specified). Furthermore, if the cardinality for the used service indicates that the service is mandatory, the using service may also be unregistered, even possibly deactivated or a new instance activated depending on the availability of alternative services and the policy defined for the service.
I'm currently learning the WF framework, so bear with me; mostly I'm looking for where to start looking, not necessarily a direct answer. I just can't seem to figure out how to begin researching what I'd like in The Google.
Let's say I have a simple one-step workflow (much more complicated than that, but for simplicity's sake). This workflow needs to watch a certain record in the database to see when it changes. I don't have the capability to "push" via a trigger from the database when the row changes, so I need to poll for it every so often.
This workflow needs to be persisted to the database to be durable against restarts and whatnot as this is a long-running workflow. I'm trying to figure out the best way to get it to check every 3 minutes or so and also persist to the database. Do the persistence capabilities of the framework allow for that? It seems to be time-based. And since the workflow won't be reawakened by an external event, how does it reload from the database and check the same step it did previously again? Does it attempt the last unfulfilled activity automatically upon reloading?
Do "while" activities with a delay attached to it work at all, or can it be handled solely through the persistence services?
I'm not sure what you mean by "handled soley through persistence services"? Persistence refers only to the storing of an idle workflow.
You could have a Delay and a Code activity in a Sequence in a While loop. When in the Delay the workflow will go idle and may be persisted if necessary. However depending on how much state is needed when persisting the workflow and/or how many such workflows you would have running at any one time may mean that a leaner approach is necessary.
A leaner approach would be to externalise the DB watching and have some "DB watching" workflow service raise an event when the desired change has occured. This service would be added to Workflow runtime.
To that end you need a service contract which is defined by an Inteface with the [ExternalDataExchange] attribute. This interface in turn defines an event that the service will raise when the desired DB change is detected. It also defines a method that a Workflow can call to specify what what change this service should be looking for. The method should accept an instance GUID so that the requesting instance can be found when the DB change is detected.
In the workflow you use a CallExternalMethodActivity to call this services method. You then flow to a HandleExternalEventActivity which listen for the event. At this point the workflow will go idle and can be persisted. It will remain there until the service raises the event.