In the use of Argo Workflows, we have a scenario to call the existing microservices. We can make an HTTP call through a script, but this method creates a new pod each time, which takes a long time. Wanted to know if it's good practice to do that and what would be the best way to do that?
An Argo Workflow will always create at least one pod. You're right that this is probably excessive if the only purpose of your Workflow is to make an HTTP call.
For something lighter-weight, check out Argo Events. You can configure an EventSource, choosing from a wide variety of input sources hopefully building something very similar to whatever you're using to kick off the Argo Workflow. Then you can configure a Trigger to make an HTTP call.
Related
I'm trying to implement automatic backfills in Argo workflows, and one of the last pieces of the puzzle I'm missing is how to access the lastScheduledTime field from my workflow template.
I see that it's part of the template, and I see it getting updated each time a workflow is scheduled, but I can't find a way to access it from my template to calculate how many executions I might have missed since the last time the scheduler was online.
Is this possible? Or maybe, is this the best way to implement this functionality on Argo?
I'm writing a controller for a k8s CRD.
The job the controller has to do will usually be quick, but could on occasion take a really long time - let's say as much as an hour.
Is that ok for a Reconcile? Or should I move that work out of the controller into a separate pod, and have the controller monitor the process of that pod?
I see no reason why the reconcile loop couldn't take as long as you need.
Technically speaking a reconcile is just getting a copy of a resource i.e. an HTTP Get or an event if you're using the Watch API, followed by a change to the resource e.g updating the resource Status fields i.e an HTTP PUT/POST.
The only caveat is making sure the resource version you have is still the latest one when trying to change it. Including resource versions in your request should solve this problem.
More info here: https://kubernetes.io/docs/reference/using-api/api-concepts/#resource-versions
What I would like to do
I would like to create a Kubernetes workflow where users could POST jobs whenever they wanted, and they might do it at any time, not necessarily scheduling anything (CronJobs), or specifying parallelism or completion requirements, i.e., users could create Jobs on demand.
How I would do it right now
The way I'm thinking about accomplishing this is by simply applying the Jobs to the Kubernetes cluster (I also have to make sure the job doesn't have the same name of a current one because otherwise Kubernetes will think it's a mistake and won't create another one). However, this feels improper because the Jobs will be kind of scattered on the cluster and I would lose control over them (though Kubernetes would supposedly automatically manage them optimally).
Is there a better, proper a way?
I imagine a more proper way of configuring all this is to create some sort of Deployment and Service on top of the Jobs, but is that an existing feature on Kubernetes? Huge companies probably have had this problem in the past so I wonder: what are the best practices for this Kubernetes Jobs On Demand use case?
Not a full answer but you might be interested in this project: https://github.com/ivoscc/kubernetes-task-runner.
It provides an API to launch one-time tasks as Jobs on a Kubernetes cluster, handles input/output files via GCS and periodically cleans up finished Jobs.
Description
I have a CR associated with a POD with a container that is exposing an API, eg:
/available
returning for example
{"available":"true"}
Is there a way to create a controller watcher on that API call that whenever the response changes will trigger the reconcile function?
I believe it could be possible using a channel with the controller watcher, but I don't see any similar examples out there
Using
Kubernetes operator-sdk version v0.5.0+git
I'm afraid it's not as easy as you hope. The Kubernetes controller objects react to add/edit/delete operations of some kind of resource in the cluster. A value exposed inside an API of something running inside the Pod is not visible in the resource database. There is not event going off, notifying about the change either.
I see two things you can consider doing:
Create a regular controller that would have the reconcile function triggered fairly often, but would check the availability value inside it's reconcile function. If it changed, it would do something, if it didn't it would do nothing.
Just create a separate task, outside the controller domain, that would monitor this API value and do something.
Controllers work using notifications, they don't actively watch for changes, they are notified about them. That's different than what you want to do - periodically check the API response.
I am working on WCF WorkFlow Service Application. i have two ReceiveAndSendReply Activity in sequence. i have set CanCreateInstance to true in both the activities Now i am not able to access the second ReceiveAndSendReply Activity. i know this because its needs to be executed sequentially right ? then how can i create WCF service methods in such way that i can call any of the method anytime.i think that can be achieved by creating State Machine Workflow but how to create in WCF WorkFlow Please suggest me some best ways i can do this.
You can put ReceiveAndSendReply in a pick branch. You can have several branches with several ReceiveAndSendReply activities. But if you have more service calls to your workflow then the recommended way is to use state machine.
Create a parallel activity and add one branch for every receiveandSend activity and set its CanCreateInstance to true