My requirement is:
Workflow should run daily at 2pm. Workflow has been scheduled to run at 2pm
We have lookup on master tables. Records with IDs that are not present in the master tables will get rejected.
These new IDs have to be loaded into the master tables manually and then the workflow has to be re-run.
Daily the same thing happens.
My question is -
Is it possible to schedule a workflow to run twice every day(one for the first run, the other to run after the master table is updated)?
If No, can I manually start a scheduled workflow? Will it make the workflow unscheduled?
Please, Can any one help me with this?
Informatica's scheduler is a weak spot. I guess using two copies of the same workflow with different schedules would be the easiest solution.
Got a solution for my problem.
Once a workflow is scheduled, even if a particular session has to be re-run manually, whole workflow has be run from the workflow manager.
If that particular session is run manually, scheduling will be gone.
So always run the workflow instead of a session, so that scheduling will remain.
Related
Currently we are using Github Actions for CI for infrastructure.
Infrastructure is using terraform and a code change on a module triggers plan and deploy for changed module only (hence only updates related modules, e.g 1 pod container)
Since auto-update can be triggered by another github repository push they can come relatively on same time frame, e.g Pod A Image is updated and Pod B Image is updated.
Without any concurrency in place, since terraform holds lock, one of the actions will fail due to lock timeout.
After implementing concurreny it is ok for just 2 on same time pushes to deploy as second one can wait for first one to finish.
Yet if there are more coming, Githubs concurreny only takes into account last push for queue and cancels waiting ones (in progress one can still continue). This is logical from single app domain perspective but since our Infra code is using difference checks, by passing deployments on canceled job actually bypasses and deployment!.
Is there a mechanism where we can queue workflows (or even maybe give a queue wait timeout) on Github Actions ?
Eventually we wrote our own script in workflow to wait for previous runs
Get information on current run
Collect previous non completed runs
and wait until completed (in a loop)
If exited waiting loop continue
on workflow
Tutorial on checking status of workflow jobs
https://www.softwaretester.blog/detecting-github-workflow-job-run-status-changes
Hi I'm running batch jobs via SCDF in openshift environment. All the jobs have been scheduled through the scheduling option in SCDF. Is there way to pause or Hold those jobs from executing instead of destroying the schedules ? Since the number of jobs are more, everytime we have to recreated the schedules for all of them.
Thanks.
We have an open issue: spring-cloud/spring-cloud-dataflow#3276 to add support for it.
Feel free to update the issue with your use-case requirements and the acceptance criteria. Better yet, it'd be great if you can contribute adding support for it in a PR; we would love to collaborate and release it.
I have a job in Rundeck with many tasks within, but when some task fails I have to duplicate de Job, remove all the other tasks, save it and then run this new reduced copy of my original job.
Is there a way to run only specific tasks without having to do all this workaround?
Thanks in advance.
AFAIK there is no way to do that.
As a workaround, you can simply add options for every step in your Rundeck job, so for instances, if you have 3 script steps in your job, you can add 3 options named: skip_step_1, skip_step_2 and skip_test_3 and then assign true to the ones that have finished successfully and false to the one that has failed in the first execution. And for every script step, you can add a condition whether to run it or not.
A smiliar feature request is already proposed to the rundeck team :
Optionally execute workflow step based on job options
I am eagerly awaiting the release of Kubernetes v1.3 in mid to late June, so that I can access cron scheduling for jobs. In the meantime, what I plan to do is the following:
Deploy a job on my Kubernetes cluster
Use jenkins as a cron tool to trigger the job in defined intervals (e.g. 1 hour).
I have two questions:
How do I update a job? For replication controllers, I would simply do a rolling update, but in the jobs API spec (http://kubernetes.io/docs/user-guide/jobs/) there are no details about how to do this. For example, lets say that I want to use my jenkins deploy system to update the job whenever I do a git commit.
Is it possible to use the kubernetes API to trigger jobs? For example, I have a job that runs and then the pod is terminated on completion. Then, 1 hour later, I want to use jenkins to trigger the job again.
Thanks so much!
I am not sure if there is any fancy way to trigger a completed job, but one way to do it can be to delete and recreate the job.
Re: rolling-update: that is required for long running pods, which is what RCs control.
For jobs: You can update the podTemplateSpec in jobSpec and that will ensure that any new pod created by the job after the update will have the updated podTemplateSpec (note: already running pods will not be affected).
Hope this helps!
Is there any way to do the following:
I have 2 jobs. One job on offline node has to trigger the second one. Are there any plugins in Jenkins that can do this. I know that TeamCity has a way of achieving this, but I think that Jenkins is more constrictive
When you configure your node, you can set Availability to Take this slave on-line when in demand and off-line when idle.
Set Usage as Leave this machine for tied jobs only
Finally, configure the job to be executed only on that node.
This way, when the job goes to queue and cannot execute (because the node is offline), Jenkins will try to bring this node online. After the job is finished, the node will go back to offline.
This of course relies on the fact that Jenkins is configured to be able to start this node.
One instance will always be turn on, on which the main job can be run. And have created the job which will look in DB and if in the DB no running instances, it will prepare one node. And the third job after running tests will clean up my environment.