I want to run a Veracode security scan on our big application. The problem is that the scan takes about 5 days and I do not want to keep the pipeline agent pinned down during all this time.
My current solution is this:
Have build 1 to upload the binaries and initiate the pre-scan.
Have build 2 run automatically when build 1 is over and poll the status of the upload. If the pre-scan is over - start the scan. If the scan is over - mark as success and clear the schedule. If the pre-scan or scan is still running - schedule the build 2 to run every 30 minutes from the current point in time by modifying the schedule in the build definition and mark the build as partially succeeded.
This scheme works, but I wonder if it can be simplified.
What if the build 2 was a Release tied to the build 1?
But is it possible to repeat the logic without scheduling a new Release every time and without pinning down the build agent for the entire duration of the scan (5 days) and by using YAML (i.e. unified pipeline instead of Build + Release) ?
After all, a single Release is able to deploy the same stage mutliple times - these are the different deployment attempts. The problem is that I do not know how to request an attempt in 30 minutes without just blocking the agent for the next 30 minutes, which is unacceptable.
So this brings up the question - can we use a YAML pipeline to start a build and then poll a third party service until it returns what we need (the payload must be parsed and examined, HTTP Status code is not enough) so that the next stage could run?
You're looking for the concept of "gates" or "approvals and checks" depending on whether you're using YAML or classic pipelines.
You can add an "invoke REST API" check that periodically queries a REST endpoint and only proceeds when it satisfies the necessary conditions. This is a "serverless" operation -- it does not tie up an agent.
Related
We're using SonarQube for tests, and there's one token it uses, as long as one pipeline is running, it goes fine, but if I run different pipelines (all of them have E2E tests as final jobs), they all fail, because they keep calling a token that expires as soon as its used by one pipeline (job). Would it be possible to have -all- pipelines pause at job "x" if they detect some pipeline running job "x" already? The jobs have same names across all pipelines. Yes, I know this is solved by just running one pipeline at a time, but that's not what my devs wanna do.
The best way to make jobs run one by one is set demands for your agent job to run on a specific self-hosted agent. Just as below, set a user-defined capabilities for the self-hosted agent and then require run on the agent by setting demands in agent job.
In this way, agent jobs will only run on this agent. Build will run one by one until the previous one complete.
Besides, you could control if a job should run by defining approvals and checks. By using Invoke REST API check to make a call to a REST API such as Gets a list of builds, and define the success criteria as build count is zero, then, next build starts.
Currently we are using Github Actions for CI for infrastructure.
Infrastructure is using terraform and a code change on a module triggers plan and deploy for changed module only (hence only updates related modules, e.g 1 pod container)
Since auto-update can be triggered by another github repository push they can come relatively on same time frame, e.g Pod A Image is updated and Pod B Image is updated.
Without any concurrency in place, since terraform holds lock, one of the actions will fail due to lock timeout.
After implementing concurreny it is ok for just 2 on same time pushes to deploy as second one can wait for first one to finish.
Yet if there are more coming, Githubs concurreny only takes into account last push for queue and cancels waiting ones (in progress one can still continue). This is logical from single app domain perspective but since our Infra code is using difference checks, by passing deployments on canceled job actually bypasses and deployment!.
Is there a mechanism where we can queue workflows (or even maybe give a queue wait timeout) on Github Actions ?
Eventually we wrote our own script in workflow to wait for previous runs
Get information on current run
Collect previous non completed runs
and wait until completed (in a loop)
If exited waiting loop continue
on workflow
Tutorial on checking status of workflow jobs
https://www.softwaretester.blog/detecting-github-workflow-job-run-status-changes
I have a Azure Devops pipeline (yml) with a stage that deploys an application to an environment and then runs a bunch of tests against it. This pipeline is triggered when a PR is created. We sometimes have multiple runs of the same pipelines happening at once resulting in two deployments to the same environment happening at the same time.
Is it possible to configure the pipeline in such a way that the deploy stage can only be executed one at a time?
Simple example of what I'm trying to do:
Pipeline (yml) with stages: 1) Build -> 2) Deploy/Test -> 3) Release
Run 1: Build: complete -> Deploy/Test In progress -> Release waiting for stage 2
Run 2: Build: complete -> Deploy/Test waiting for Run 1 stage 2
If you use YAML then make you release in deployment job where you use environment with enabled exclusive lock. However, this has some drawback:
On developer community you can find a feature request for better handling this. And there is one workaround which involved calling REST API. But, as someone already mentioned:
Workarounds involving polling end up with race conditions where two queued builds can both end up starting.
So there is no ideal solution, but if you can please upvote above mentioned request.
Using Azure DevOps, I am creating a release pipeline in which I would like to have one stage having 1 job with 5 steps. First three steps are same task types but with different variables, and I would like to make them parallel so some flow would go like:
Job
parallel: step1, step2, step3
then: step4 (after all 3 parallel steps succeeded/failed)
then: step5 (after step4 is done)
This is the current job setup
I am not sure how to set up Control Options - run this task for all of these steps. I would need somehow to set the first three to run immediately (maybe Custom condition "always()") and step 4 and step 5 to run sequentially after previous steps are done.
Step 5 can be: Even if the previous task has failed unless the deployment was cancelled, but I am not sure if I set the same setting for step 4 will it consider only step 3 as previous or all three previous (step1 - step3) tasks.
Also for Execution plan Parallelism I guess it's ok to set multi-agent to three since I would have max 3 steps executing in parallel overall.
parallel: step1, step2, step3
If you have 5 task in one agent job, and just want to run previous three tasks parallel first, I'm afraid to say that this does not supported in Azure Devops.
While you put several tasks in one agent job, that's means the task will and must running in order. And also, as you mentioned in the second pic, the specified in Multi-agent is used for run agent job parallel, rather than run task parallel.
Fortunately, until now, there has been such suggestion raised by other user in our official Developer Community. And there are many users has the same demand with you. You can vote and comment there. Our Product Group team will take your suggestions kindly. If it has enough votes which means its high priority, and the team will consider it seriously.
Lets say that after every deployment I want to execute some system tests (putting a message and check if it reaches its destination).
What are my options?
I looked at post-deployment gates and the idea would be to invoke an azure function that would trigger the success, wait some time and then "assert". Is this the right way? What about timeouts since I'm going to wait (possibly for a minute or two).
In your release pipeline, you can create a first stage for deployment, and a second one for system tests.
In the system tests stage, you can choose the pre-deployment conditions to suit your requirements: e.g. after stage (and select the first deployment stage)
Optionally you can add pre-deployment approvals so somebody needs to manually approve this.