I have an Azure Pipeline Build. The *.yaml file executes correctly a Python script (PythonScript#0). This script itself creates (if does not exist), executes and publishes Azure ML pipeline. It runs well when the Build is executed manually or is triggered by commits.
But I want to schedule the automated execution of the ML pipeline (Python script) on a daily basis.
I tried the following approach:
pipeline_id = published_pipeline.id
recurrence = ScheduleRecurrence(frequency="Day", interval=1)
recurring_schedule = Schedule.create(ws,
name=<schedule_name>,
description="Title",
pipeline_id=pipeline_id,
experiment_name=<experiment_name>,
recurrence=recurrence)
In this case the pipeline runs during 3-4 seconds and terminates successfully. However, the Python script is not executed.
Also, I tried to schedule the execution of a pipeline using Build, but I assume that it is a wrong approach. It rebuilds a pipeline, but I need to execute a previously published pipeline.
schedules:
- cron: "0 0 * * *"
displayName: Daily build
always: true
How can I execute my published pipeline daily? Should I use Release (which agents, which tasks?)?
Also, I tried to schedule the execution of a pipeline using Build, but
I assume that it is a wrong approach. It rebuilds a pipeline, but I
need to execute a previously published pipeline.
Assuming your python-related task runs after many other tasks, then it's not recommended to simply schedule the whole build pipeline, it will rerun the pipeline(other tasks+python script).
Only the pipeline can be scheduled the instead of tasks, so I suggest you can create a new build pipeline to run the python script. Also, a private agent is more suitable for this scenario.
Now we get two pipelines: Original A and B which used to run the python script.
Set B's build completion to be A, so that if A builds successfully the first time, B will run after that.
Add a command-line task or PS task as pipeline A's last task. This task(modify the yml and then push the change) will be responsible for updating the B's corresponding xx.yml file to schedule B.
In this way, if A(other tasks) builds successfully, B(pipeline to run python script) will execute. And B will run daily after that successful build.
Hope it helps and if I misunderstand anything, feel free to correct me.
Related
I have a pipeline that runs an agentless job. This job produces a result that I would like to pass to the next or next job. The problem is that all the examples I've found set variables on agent jobs, not agentless. See here, all the examples use script commands, which need to be run on an agent.
Is there a way to set an output variable from an agentless job? How else can I pass the result from an agentless job to the next?
Setting output variables from agentless jobs isn't supported
powershell runs Windows PowerShell and will only work on a Windows agent.
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/powershell?view=azure-devops
Depending on what your use-case is, you may be able to use dependsOn and condition in your jobs to achieve your goal. E.g. for retries for builds that are not idempotent. Otherwise an agent-based configuration may be needed.
I'm new to test plan. I'm trying to run a powershell script from there.
The only way I see it to run the ps script is from outside the testplan as a Job for the agent but that's no use, I want to get results from that execution.
A test plan does not support any actions. This is a container for test suites, test cases and test executions. If you want to get test runs, you can use rest api: Runs - List
the only way I see it to run the ps script is from outside the
testplan as a Job for the agent but that's no use
As Shamrai Aleksander said, test plan is just a container of test suites and test cases. So It's not supported to run PS from test plan itself directly.
I'm not sure why you want to get results in that way using PS script, as I know we can check test results in web portal(See pic in step5). Also, since we need to configure build and release in Test Plan settings, if you want to run PS script, you can add a Power Shell task before or after the VSTest task in release pipeline.
Good afternoon, I need help with this case, I want to generate a release description in my pipeline builds, I tried to set the variable in the build and I used a group variable but I was not successful, the idea is to generate a build that contains a description of what contains and when generating the new release already has the value in the variable Release.ReleaseDescription, I have a slack task that sends approvals, it would be good to have this description so that the people who approve see what it contains. This manual procedure is currently performed when the release is generated and a description is placed.
Variable group can only share variables with static values from build pipeline to release pipeline. However there is an extension task tool Variable Tools for Azure DevOps Services that can accomplish this. You can follow below steps:
1, You need first to search for Variable Tools for Azure DevOps Services and install it to your organization.
This task include two subtasks as the task describles:.
Variable Save Task - During your build you can save the variables to a json file stored with your other build assets
Variable Load Task - During your Release you can load the saved variables and gain access to them.
2, Then you need to define a variable (BuildDescription)in your build pipeline
3, Add a powershell task to assign a value to variable BuildDescription.
4, add Variable Save Task to save variable BuildDescription to a json file and store it to the build artifacts folder which will be published to azure devops server as a part of aritfacts.
5, in the release pipeline, add task Variable Load Task, and then you can use the variable (BuildDescription) in your release pipeline.
Update:
As above tasks cannot be run on linux system. We can write bash scripts to do the variable save task and variable load task.
To save the variable, you only need to replace above variable save task with bash task to run below command.
echo '{"des":"description"}' > variable-meta.json
To load the variable. Add bash task to replace variable load task
val= ($(jq '.description' variable-meta.json))
echo "##vso[task.setvariable variable=BuildDescription]$val"
I want to run a Powershell file from the Jenkins Pipeline, using the powershell command. Should be easy, right?
node()
{
stage ('Hello World') {
powershell returnStatus: true, script: 'C:\\HelloWorld.ps1'
}
}
C:\HelloWorld.ps1 is a one-liner:
Write-Host "Hello World"
But running the job causes the process to hang. Here is the console output:
Started by user Administrator
[Pipeline] node
Running on master in C:\Jenkins\workspace\HelloWorld
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Hello World)
[Pipeline] powershell
[HelloWorld] Running PowerShell script
And then it just hangs until I cancel the job.
What to do?
I had the same issue and managed to solve it with downgrading DurableTask plugin from 1.17 to 1.15. It seems to be part of JENKINS-46496 bug. The older plugin version is there 1.15.
You can update the plugin to a needed version in Manage Plugins->Advanced Tab->Upload Plugin button.
I had the same issue and the problem was that the job had spaces in its name.
This issue should be fixed in the next release: https://github.com/jenkinsci/durable-task-plugin/pull/51
I have the same problem in about 70 % of my cases. I have a pipeline script that kicks off about 20 parallel jobs spread out over different agents. Each of these tries to run a Powershell script initially. About 30 % of the jobs succeed to run the script, the rest hangs just like it does for Ola.
The build agents are configured exactly the way (clones in a VM cluster). Powershell v4 is used. Upon several tries one single build agent sometimes succeeds in running Powershell, sometimes don't.
I've been running this pipeline for 5-6 weeks and it's just the last week this behavior has been seen. Fortunately the script isn't in production yet :-)
I haven't had the time to do a full investigation. Could it be that some of the pipeline plugins have been updated and introduced this bug?
I'm using TFS 2015 Update 2 along with the new Build/Release system. I have a powershell script I'm executing via the Powershell Task. This task executes a powershell script that needs access to the $distributedExecutionContext magic variable I see in many different VSTS Task code samples.
This script in question is not technically a task, but instead is being executed by the Powershell task that comes delivered with TFS.
No matter what I do, I can't see to obtain the $distributedExecutionContext variable. It's always null. Here is an example:
Import-Module "Microsoft.TeamFoundation.DistributedTask.Task.Internal"
if($distributedTaskContext)
{
#... this never happens
}
Is this variable only available if the powershell being run is being run inside an actual task?
The default powershell task that you are using runs the script entirely as a different process and the $distributedTaskContext variable is not available to the script.
It is only available only to the task's powershell script.
If you are going to write a custom task, I would like you to use the new vsts-task-lib SDK which improves a lot over old SDK.