I have a pipeline that runs an agentless job. This job produces a result that I would like to pass to the next or next job. The problem is that all the examples I've found set variables on agent jobs, not agentless. See here, all the examples use script commands, which need to be run on an agent.
Is there a way to set an output variable from an agentless job? How else can I pass the result from an agentless job to the next?
Setting output variables from agentless jobs isn't supported
powershell runs Windows PowerShell and will only work on a Windows agent.
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/powershell?view=azure-devops
Depending on what your use-case is, you may be able to use dependsOn and condition in your jobs to achieve your goal. E.g. for retries for builds that are not idempotent. Otherwise an agent-based configuration may be needed.
Related
I have an Azure Pipeline Build. The *.yaml file executes correctly a Python script (PythonScript#0). This script itself creates (if does not exist), executes and publishes Azure ML pipeline. It runs well when the Build is executed manually or is triggered by commits.
But I want to schedule the automated execution of the ML pipeline (Python script) on a daily basis.
I tried the following approach:
pipeline_id = published_pipeline.id
recurrence = ScheduleRecurrence(frequency="Day", interval=1)
recurring_schedule = Schedule.create(ws,
name=<schedule_name>,
description="Title",
pipeline_id=pipeline_id,
experiment_name=<experiment_name>,
recurrence=recurrence)
In this case the pipeline runs during 3-4 seconds and terminates successfully. However, the Python script is not executed.
Also, I tried to schedule the execution of a pipeline using Build, but I assume that it is a wrong approach. It rebuilds a pipeline, but I need to execute a previously published pipeline.
schedules:
- cron: "0 0 * * *"
displayName: Daily build
always: true
How can I execute my published pipeline daily? Should I use Release (which agents, which tasks?)?
Also, I tried to schedule the execution of a pipeline using Build, but
I assume that it is a wrong approach. It rebuilds a pipeline, but I
need to execute a previously published pipeline.
Assuming your python-related task runs after many other tasks, then it's not recommended to simply schedule the whole build pipeline, it will rerun the pipeline(other tasks+python script).
Only the pipeline can be scheduled the instead of tasks, so I suggest you can create a new build pipeline to run the python script. Also, a private agent is more suitable for this scenario.
Now we get two pipelines: Original A and B which used to run the python script.
Set B's build completion to be A, so that if A builds successfully the first time, B will run after that.
Add a command-line task or PS task as pipeline A's last task. This task(modify the yml and then push the change) will be responsible for updating the B's corresponding xx.yml file to schedule B.
In this way, if A(other tasks) builds successfully, B(pipeline to run python script) will execute. And B will run daily after that successful build.
Hope it helps and if I misunderstand anything, feel free to correct me.
I have a situation where I want to make the execution of my scripts smarter. I have a set of scripts that execute at a given time, but because sometimes the input files are not posted at the correct times the scripts run into errors and get unexpected results. so one of the solutions I was thinking of is to make the execution of the scripts dependent of each other. Here is what I mean:
script 1 runs at 6 pm
validates that the file is there
if it's there, set a flag
the flag is active so execute script 2 at 9 pm
if it's NOT there, the flag is not set
the flag is not set so script 2 is not executed
Right now script 1 and script 2 are set with the Task Scheduler at those times, I checked the Scheduler for those type of conditions, but didn't find anything.
You can set triggers in Task Scheduler, like when an event happens for basically everything you can see in eventviewer.
I would suggest Write-Eventlog from the script which works on the file, and depending on the result the sched task would get triggerd.
I suggest you to have single script running every N-minutes on single scheduled task via Task Scheduler.
The master script will analyze activities and have all logical conditions those determine when and which external script to run. You can also have flag files.
You know how, with a Rundeck job, when you define an launch command script, you can reference job run option variables with things like #option_var_1#. Is there a symbol like this that evaluates to the number of the job run? I'd like to pass that value to my executable as a command-line variable.
I assume you are looking for the execution id of the job.
job.execid: ID of the current Execution
I'm using TFS 2015 Update 2 along with the new Build/Release system. I have a powershell script I'm executing via the Powershell Task. This task executes a powershell script that needs access to the $distributedExecutionContext magic variable I see in many different VSTS Task code samples.
This script in question is not technically a task, but instead is being executed by the Powershell task that comes delivered with TFS.
No matter what I do, I can't see to obtain the $distributedExecutionContext variable. It's always null. Here is an example:
Import-Module "Microsoft.TeamFoundation.DistributedTask.Task.Internal"
if($distributedTaskContext)
{
#... this never happens
}
Is this variable only available if the powershell being run is being run inside an actual task?
The default powershell task that you are using runs the script entirely as a different process and the $distributedTaskContext variable is not available to the script.
It is only available only to the task's powershell script.
If you are going to write a custom task, I would like you to use the new vsts-task-lib SDK which improves a lot over old SDK.
I have a scheduled job on rundeck (2.6.2).
This jobs run a script that needs an node environment variable available (like $HOME, $USER or $PWD. A custom one. ) for all user in the node/nodes.
I could use jobs options to solve this if I wanted trigger the job from API ( Or manually. Rundeck ask me for the option ) but is a scheduled job. I can't use Options -> Default Value because the jobs could run in nodes with different values for this environment variable.
There is any way to offers all / some node environment variables to rundeck to be used inside the scheduled jobs?
(I have thought in use Options -> Allowed Values -> Remote URL but is a mess. Too complicated to me requirement)
Thanks.
The easy way in my case has been to customize /etc/rundeck/profile adding into it all the stuff I wanted.
Seems a pretty good solution to me.
I succeeded to perform this by adding the following lines:
set -a
. /etc/environment
. /etc/profile
1) put those lines into the file: /etc/rundeck/profile
2) put those lines into a script step
Remark: I'm using only script steps in my rundeck and I'm always put this lines in the first line of the script step:
#!/usr/bin/env bash