I've been searching for days on how to layout a rundeck workflow with job dependencies. what I need to do is to have 3 jobs: job-1 and job-2 are scheduled to run in parallel while job-3 will only be triggered after the completion of both job-1, and job-2. assuming that job-1 and job-2 have different execution times.
I tried using job state conditionals to do that but it seems that the condition if not met will halt or fail only. My idea is to halt the execution until all the parent jobs completes and then resume the workflow.
You can achieve this by compiling a master job which includes 2 steps:
step: job-1 and job-2 as a sub-job which includes both (run in parallel if node oriented execution is selected)
step: job-3
But not all 3 in in the same flow.
Right now you can use Job State Conditional feature for that: https://docs.rundeck.com/2.9.4/plugins-user-guide/bundled-plugins.html#job-state-plugin
Rundeck cannot do this for you automatically. You can set a scheduler for job-3 to run after the max timestamp of job1 or job2. Enable "retry" for job3 incase the dependencies would be fail.
Related
My requirement is, i have to run azure pipeline after 15days of the trigger. I have 2 pipelines where first pipeline will execute and after that pipeline is started, the second pipeline should trigger after 15days only. How can i do that? Can we tweak cron to do this?
I tried using cron but there is no way to set this up.
There is no out-of-the-box type of trigger to achieve it according to Microsoft document at present.
But you can make the job depends on an agentless job that includes delay task to pause execution of the pipeline for a fixed delay time. Please refer to doc:Delay task
for example:
1 add a delay task in an agentless job and configure delay time 21600 minutes(15 days)
2 make the job depend on agentless job
3 set build trigger
Note :set build job timeout to 0
I have a complex devops build script in yaml. Is there some way that if a given step takes too much time the process is killed (or some task is executes which kills certain processed).
This is would be useful in our case where we have large tests suites in several DLLs. I am seeing often that some tests fail and after devops hangs. I would like to kill the testrunner and other processes which may be hanging with (and also without) a timeout.
Is this possible on devops?
You can specify timeoutInMinutes and cancelTimeoutInMinutes for the job:
jobs:
- job: Test
timeoutInMinutes: 10 # how long to run the job before automatically cancelling
cancelTimeoutInMinutes: 2 # how much time to give 'run always even if cancelled tasks' before stopping them
More information: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml#timeouts
I have a DAG workflow as below
taskA -> after taskA completes taskB and taskC runs in parallel -> once task B and C completes taskD starts. In case taskC fails due to some external issue which needs a manual intervention for correction. After correction can we manually restart (from UI or CLI) the workflow so that it resumes directly from the failed taskC and goes to taskD and completes the workflow.
Yes, clicking 'retry' from the workflow in the GUI will do just that.
Using Azure DevOps, I am creating a release pipeline in which I would like to have one stage having 1 job with 5 steps. First three steps are same task types but with different variables, and I would like to make them parallel so some flow would go like:
Job
parallel: step1, step2, step3
then: step4 (after all 3 parallel steps succeeded/failed)
then: step5 (after step4 is done)
This is the current job setup
I am not sure how to set up Control Options - run this task for all of these steps. I would need somehow to set the first three to run immediately (maybe Custom condition "always()") and step 4 and step 5 to run sequentially after previous steps are done.
Step 5 can be: Even if the previous task has failed unless the deployment was cancelled, but I am not sure if I set the same setting for step 4 will it consider only step 3 as previous or all three previous (step1 - step3) tasks.
Also for Execution plan Parallelism I guess it's ok to set multi-agent to three since I would have max 3 steps executing in parallel overall.
parallel: step1, step2, step3
If you have 5 task in one agent job, and just want to run previous three tasks parallel first, I'm afraid to say that this does not supported in Azure Devops.
While you put several tasks in one agent job, that's means the task will and must running in order. And also, as you mentioned in the second pic, the specified in Multi-agent is used for run agent job parallel, rather than run task parallel.
Fortunately, until now, there has been such suggestion raised by other user in our official Developer Community. And there are many users has the same demand with you. You can vote and comment there. Our Product Group team will take your suggestions kindly. If it has enough votes which means its high priority, and the team will consider it seriously.
Why rundeck not launching scheduled spark jobs even if the previous job is still executing?
Rundeck is skipping the jobs set to launch during the execution of the previous job, then after the completion of its execution launch new job based on the schedule.
But I want to launch a scheduled job even if the previous job is executing.
Check your workflow strategy, here you have an explanation about that:
https://www.rundeck.com/blog/howto-controlling-how-and-where-rundeck-jobs-execute
You can design a workflow strategy based on "Parallel" to launch the jobs simultaneously on your node.
Example using the parallel strategy with a parent job.
Example jobs:
Job one, Job two and Parent Job (using parallel strategy).