Azure pipelines - How to manage versions of the tasks? - azure-devops

I would like to understand how the tasks work in Azure pipeline, and where can I find the documentation for it.
I have a self-hosted agent where I have installed Terraform, Azure CLI and Azure Powershell.
There are some tasks defined for them, I actually have difficulties to find them in a documentation,for example with terraform, I have to go to azure release and simulate an action of adding task, then convert that in yaml and there I have the task name:
steps:
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV3#3
displayName: 'Terraform : azurerm'
inputs:
command: custom
customCommand: import
environmentServiceNameAzureRM: 'something'
Also, some tasks asks to install some resources in order to use them, will it affect the resources of my self-hosted agent? for example if we have to install the task before using it ,will it affect somehow the version of my self-hosted terraform?
I have azure powershell installed in the self-hosted agent with an old version, but there is a task that in the version parameter has the latest version, but when executing the task it uses my old version.
How versions for tasks work? Can I use the latest version or az cli without installing it in the self-hosted agent but only with the tasks for the deployments only?
- task: AzurePowerShell#5
displayName: 'DevCode'
inputs:
#azureSubscription: 'something'
azureSubscription: 'something'
ScriptPath: 'something'
azurePowerShellVersion: LatestVersion
The same for az cli.

For the commonly used pipeline tasks published by Microsoft (such as Azure CLI task and Azure PowerShell task), you can find the related documents here.
All these tasks are open-source on the GitHub repository "microsoft/azure-pipelines-tasks".
For the Terraform task, you can find it on Marketplace here and this GitHub repository.
Normally, the tasks would not change the tools have installed on your self-hosted agent machine, also not automatically install the required tools. They just call the required tools have installed on the agent machine.
For example, when you run the Azure CLI task on your self-hosted agent, if you have installed the Azure CLI tool on the agent machine, the task will call the tool to run the specified scripts. If the Azure CLI tool has not installed on the machine, normally the task will go to failure since it cannot find the required Azure CLI tool on the machine.
You do not need to manage the versions of the tasks, you just use the tasks in your pipelines and select the available released versions of the tasks (latest versions are recommended). If a new version of a task is released, you can go to edit your pipelines to use the new version.

Related

How to copy files from a linux agent to windows agent in Azure DevOps release pipeline?

I have to run a Linux docker image for performing testing in Azure DevOps release pipeline. Hence, I have used;
Agent Pool : Azure Pipelines and Agent Specification: ubuntu-20.04
for that task. Now, result of this test has to be send as an email. For that 'Send Email' extension in Azure DevOps marketplace can be used. But this is configured only for Windows agent.
So, there is a requirement to run two agent jobs, one for Linux and other Windows. How can I access the report file generated in first job, to send email in second.
Had tried publishing pipeline artifacts, but it didn't worked as artifacts are not supported in release.

Error when I run the Visual studio run task in CI

I have a .net application and I am getting below error while running Visual studio task in CI pipeline.
##[error]No agent found in pool TEST APP which satisfies the following demand: visualstudio. All demands: msbuild, visualstudio, Agent.Version -gtVersion 2.115.0
Please let me know how to resolve this.
Your pool does not contain build agents with Visual Studio. You can check your polls and agents on the administration page: Agent pools.
You have to install and register a build agent on a server or virtual machine with visual studio.
Additionally, if your project on Azure DevOps Services, you can use MS Hosted build agents. Update your yaml to:
pool:
vmImage: windows-latest
or if you use classic build, update here:

Azure Devops Pipeline: Possible to cache task container?

I'm setting up a multi-stage Azure Devops yaml pipeline for a .Net Framework application.
Part of the pipeline will involve using the AWSPowerShellModuleScript task to configure load balancer rules in AWS.
My Task looks like so...
- task: AWSPowerShellModuleScript#1.7.0
name: SetupLoadBalancerRules
inputs:
awsCredentials: 'My AWS Service Connection'
regionName: 'ap-southeast-2'
scriptType: 'filepath'
filePath: 'pipeline-scripts/manage-aws-load-balancer-rules.ps1'
Everything is working correctly. However the AWSPowerShellModuleScript tasks are quite slow to initialise. The powershell itself is very fast, but the task requires approximately 1.5 minutes to setup.
I'm running 2 of these tasks in different stages of my pipeline, so this adds 3 minutes to the total time. This may not seem like a lot, but the application itself is quite small, so the setup for these tasks is actually the most time consuming part of the pipeline.
As far as I can tell, it seems that the pipeline is starting a generic container, and then installing the AWS Powershell tools, every time it needs to run one of these tasks.
This seems to be very wasteful and inefficient, so I was wondering if there might be some better way to handle it, for example, caching the built container after the powershell tools are installed, or use an existing image with the tools already installed etc.
I'm very new to using the yaml pipelines, so I'm not sure what's possible.
I like my pipelines to be as efficient as possible, so it just bothers me that this is re-running this repetetive install process every time I need to run a simple powershell script.
Also I should mention that I'm using a hosted Devops Agent... vmImage: 'windows-2019'
Just in case it helps. This is from the task log output...
Checking install status for AWS Tools for Windows PowerShell module.
AWS Tools for Windows PowerShell module not found.
Installing AWS Tools for Windows PowerShell module to current user scope
Name Version Source Summary
---- ------- ------ -------
nuget 2.8.5.208 https://onege... NuGet provider for the OneGet meta-package manager
So it determines that the AWS Tools are not installed, and then possibly uses nuget to install it??
I thought perhaps I could use a cache task to cache the install, but even if I could find where the tools are installed to, it seems unlikely that simply restoring the folder would be sufficient.
Using a Microsoft-hosted agent, each time you run a pipeline, you get a fresh virtual machine. So the tool needs to be installed in each pipeline.
A stage is one or more jobs, which are units of work assignable to the same machine. Using Microsoft-hosted agent, each stage uses a separate agent generally. So the tool will be installed in each stage.
In a word, Microsoft-hosted agent is not be able to cache tools. In order to pre-install the tool or not install tool every time, you could deploy Self-hosted Windows agents, and install the tool on every machine running agent service.

How to authenticate azure devops pipeline , so that pipeline can restore yarn packages successfully

My application uses an yarn package which is installed in local after authentication.Registry url is specified in .npmrc file. Now my build pipeline fails, and says "Package restore failed" (Request failed "401 Unauthorized""). How to authenticate the azure devops build pipeline?
I tried adding a task in my YML file as below , specified the npmrc path in it,so that it silently authenticate from the pipeline process, but this doesn't work
task: npmAuthenticate#0
inputs:
workingFile: "Portal/.npmrc"
customEndpoint: msasg.pkgs.visualstudio.com, 1DS-SDK
displayName: "Authenticate NPM"
Is there a different way to achieve this?
You can try to use Yarn Task from the Marketplace as it is more Azure DevOps aware and can handle this credentials bit for you.
In addition, the yarn package you use is installed locally, do you use a self-hosted agent to run the pipeline ?

In VSO release Powershell task is failing while using 'Azure Pipelines' agent and same task is passing while using 'Hosted MSIT Fortify 2017' agent

Normal VSO release should work with the 'Azure Pipelines' agent. But it is failing in Azure Powershell task saying 'Could not find the modules: 'Az.Accounts'.. While same task is running with the 'Fortify' agent
Tried all possible agents other than 'Fortify'. Didn't work.
I had the same problem with version 4 of the task. I switched to 3 and it got fixed.
I came across a similar error using task Azure Powershell "Could not find the modules: 'Az.Accounts' with Version: '5.1.1' If the module was recently installed, retry after restarting the Azure Pipelines task agent".
My fix to this error is to use the latest version of azurePowerShellVersion,
azurePowerShellVersion: 'latestVersion'
For classic view:
select the latest version 4.* of task Azure Powershell
check latest installed verison for azure powershell version
If above fix does not work for you. For better troubleshooting, please post the full error message and the detailed settings of Azure Powershell task.