I have a blueprint copy(C2) created from the source copy (C1) that has the default rollout config.
Also there exist a launcher that triggers the workflow (W1) on any modification done on any page hierarchy of C2. The workflow purpose is to do some processing and then activate the page.
So the workflow W1 was being triggered whenever there is rollout from C1 to C2
Issue
Whenever any page is deactivated in C2 , the page is going to activate state because of the launcher( since cq:lastReplicationAction being changed from Activate to Deactivate)
For the above issue I placed a condition in launcher as cq:lastReplicationAction != Deactivate. The deactivate issue gets resolved but now the workflow W1 is not being triggered whenever there is a rollout for a page in C1 to a deactivated page in C2.
Although, I tried with possible solutions like adding the rollout config action - targetActivate & workflow on copy C2 but no luck. Is there any way we can trigger the workflow W1 on rollout ? Please suggest.
Related
Currently we are using Github Actions for CI for infrastructure.
Infrastructure is using terraform and a code change on a module triggers plan and deploy for changed module only (hence only updates related modules, e.g 1 pod container)
Since auto-update can be triggered by another github repository push they can come relatively on same time frame, e.g Pod A Image is updated and Pod B Image is updated.
Without any concurrency in place, since terraform holds lock, one of the actions will fail due to lock timeout.
After implementing concurreny it is ok for just 2 on same time pushes to deploy as second one can wait for first one to finish.
Yet if there are more coming, Githubs concurreny only takes into account last push for queue and cancels waiting ones (in progress one can still continue). This is logical from single app domain perspective but since our Infra code is using difference checks, by passing deployments on canceled job actually bypasses and deployment!.
Is there a mechanism where we can queue workflows (or even maybe give a queue wait timeout) on Github Actions ?
Eventually we wrote our own script in workflow to wait for previous runs
Get information on current run
Collect previous non completed runs
and wait until completed (in a loop)
If exited waiting loop continue
on workflow
Tutorial on checking status of workflow jobs
https://www.softwaretester.blog/detecting-github-workflow-job-run-status-changes
I'm learning about semaphores in the Argo project workflows to avoid concurrent workflows using the same resource.
My use case is that I have several external resources which only one workflow can use one at a time. So far so good, but sometimes the resource needs some maintenance and during that period I don't want to Argo to start any workflow.
I guess I have two options:
I tested manually setting the semaphore value in the configMap to the value 0, but Argo started one workflow anyway.
I can start a workflow that runs forever, until it is deleted, claiming the synchronization lock, but that requires some extra overhead to have workflows running that don't do anything.
So I wonder how it is supposed to work if I set the semaphore value to 0, I think it should not start the workflow then since it says 0. Anyone have any info about this?
This is the steps I carried out:
First I apply my configmap with kubectl -f.
I then submit some workflows and since they all use the same semaphore Argo will start one and the rest will be executed in order one at a time.
I then change value of the semapore with kubectl edit configMap
Submit new job which then Argo will execute.
Perhaps Argo will not reload the configMap when I update the configMap through kubectl edit? I would like to update the configmap programatically in the future but used kubectl edit now for testing.
Quick fix: after applying the ConfigMap change, cycle the workflow-controller pod. That will force it to reload semaphore state.
I couldn't reproduce your exact issue. After using kubectl edit to set the semaphore to 0, any newly submitted workflows remained Pending.
I did encounter an issue where using kubectl edit to bump up the semaphore limit did not automatically kick off any of the Pending workflows. Cycling the workflow controller pod allowed the workflows to start running again.
Besides using the quick fix, I'd recommend submitting an issue. Synchronization is a newer feature, and it's possible it's not 100% robust yet.
I'm looking for a definitive answer for k8s' response to a job being updated - specifically, if I update the container spec (image / args).
If the containers are starting up, will it stop & restart them?
If the job's pod is all running, will it stop & restart?
If it's Completed, will it run it again with the new setup?
If it failed, will it run it again with the new setup?
I've not been able to find documentation on this point, but if there is some I'd be very happy to get some signposting.
The .spec.template field can not be updated in a Job, the field is immutable. The Job would need to be deleted and recreated which covers all of your questions.
The reasoning behind the changes aren't available in the github commit or pr, but these changes were soon after Jobs were originally added. Your stated questions are likely part of that reasoning as making it immutable removes ambiguity.
My requirement is:
Workflow should run daily at 2pm. Workflow has been scheduled to run at 2pm
We have lookup on master tables. Records with IDs that are not present in the master tables will get rejected.
These new IDs have to be loaded into the master tables manually and then the workflow has to be re-run.
Daily the same thing happens.
My question is -
Is it possible to schedule a workflow to run twice every day(one for the first run, the other to run after the master table is updated)?
If No, can I manually start a scheduled workflow? Will it make the workflow unscheduled?
Please, Can any one help me with this?
Informatica's scheduler is a weak spot. I guess using two copies of the same workflow with different schedules would be the easiest solution.
Got a solution for my problem.
Once a workflow is scheduled, even if a particular session has to be re-run manually, whole workflow has be run from the workflow manager.
If that particular session is run manually, scheduling will be gone.
So always run the workflow instead of a session, so that scheduling will remain.
Question: Is there a way to block a build stream if another build stream is in the process of building?
Here is what I'm trying to do. I have two build streams that contain multiple jobs. I can block the jobs up or down stream a given stream. I cannot however block a build stream if the other build stream is building.
Here is my setup:
Build Stream 1: A1 -> B1 -> C1
Build Stream 2: A2 -> B2 -> C2
Job A1 and A2 are setup using the SCM plugin to poll separate locations in our repo. So a build can be trigger at any time it finds a change. The build streams should be treated independently but block if the other have jobs building.
I've tried Build Blocker Plug-in but problem is that it only blocks if a job or jobs are currently building. It doesn't seem to care about the other builds waiting in the build queue.
I'm limited on the number of resources that are provided to me so I'm having to double up on our builds.
The Exclusion Plugin may be able to help you here.
What you would do is set up two other jobs to manage the build trains, M1 job will have build steps that do the following:
Grab the semaphore
Trigger A1
Wait for C1 to finish
Release the semaphore
M2 job will be similar
Grab the semaphore
Trigger A2
Wait for C2 to finish
Release the semaphore
You would need timeouts, but that is essentially the idea