UI Selenium tests on Azure Devops 2019 - azure-devops

What is the correct way to run automated UI tests by a self-hosted agent?
I tried to add a tests step in the release pipeline but it's not working because the agent cannot find the DLLs with tests (they are in a few separate projects)
##[warning]No test assemblies found matching the pattern: **\Test.UI.dll,!**\*TestAdapter.dll,!**\obj\**.
Currently, the release pipeline is simple: one artifact from the build pipeline and one stage with the following steps:
1. IIS Web App Deploy
2. IIS Web App Manage
3. VsTest: tests are selected using Test assemblies option

No test assemblies found matching the pattern.
It seems that the .dll files don't exist in the artifact or under the path of the release agent. You may need to share more information about the Pipeline. (e.g. build agent, pipeline definition and vstest task definition)
Before this, you can refer to the following steps for troubleshooting.
First of all, you need to make sure that the .dll files exist in the build artifacts.
You could check this in Build Summary -> Artifacts. You could download it and check the files in the artifact.
If the files don't exist, you could add a Copy files task before the Publish Artifacts task.
For example:
- task: CopyFiles#2
displayName: 'Copy Files to: $(build.artifactstagingdirectory)'
inputs:
SourceFolder: '$(agent.builddirectory)'
TargetFolder: '$(build.artifactstagingdirectory)'
enabled: false
Then the "VsTest" task will find the files in $(System.DefaultWorkingDirectory) by default(No customization). In release pipeline, the path is equals to C:\agent\_work\rx\a (e.g. C:\agent\_work\r1\a)
Since you are using the self-hosted agent , you could directly check the files in the target path on your local machine.
If you couldn't find the target files, you may need to modify the search folder or Test files path.
You also could check the file path in the Release Pipeline log -> Download Artifacts Step.

Related

Share CMake build folder between jobs in Azure DevOps

I'm building a pipeline in Azure DevOps that first builds a project using CMake and then, conditionally, creates an installer out of it. The creation of the installer is also done using CMake (and CPack). Since the installer is only supposed to be created and published under certain conditions (a tag is created), I'd like to share the build folder from the first job and reuse it in the second job. I do so by uploading the artifacts as the last step from the first job
- task: PublishBuildArtifacts#1
inputs:
pathToPublish: $(System.DefaultWorkingDirectory)/build
artifactName: build_release
(pipeline artifacts are currently not supported by our environment)
and downloading and moving it in the second step
- task: DownloadBuildArtifacts#0
inputs:
downloadType: 'single'
artifactName: build_release
downloadPath: $(System.DefaultWorkingDirectory)/tmp
- task: CopyFiles#2
displayName: Copy Artifacts into Build Directory
inputs:
SourceFolder: $(System.DefaultWorkingDirectory)/tmp/build_release
TargetFolder: $(System.DefaultWorkingDirectory)/build
Unfortunately, I'm running into an issue with cached absolute paths now:
CMake is re-running because D:/.../_work/171/s/build/vs-project/CMakeFiles/generate.stamp dependency file is missing.
CMake Error: The current CMakeCache.txt directory D:/.../_work/212/s/build/vs-project/CMakeCache.txt is different than the directory d:/.../_work/171/s/build/vs-project where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
CMake Error: The source directory "D:/.../_work/171/s/build" does not exist.
I understand that several paths are cached in CMakeCache.txt and do not match since the build is taking place in a separate folder. Is there an elegant way to have ADO rewrite the paths in CMakeCache.txt automatically? Or to make sure the folder names stay constant for the two jobs?
Any help appreciated here!

Azure pipeline yml: publish or file copy?

At the end of my pipeline I want to copy the bin directly to a network share from which it can be used to deploy.
I see there are two possible task typed that could do this:
copy files that makes a task: CopyPublishBuildArtifacts#1; or
_ copy and publish build artefacts_ that makes a CopyPublishBuildArtifacts#1.
And this publish is different to dotnet publish?
Which one should I pick and why?
They appear to have identical parameters.
What is the # for?
Azure pipeline yml: publish or file copy?
The task CopyPublishBuildArtifacts is deprecated. If you're using Team Foundation Server 2017 or newer, we recommend that you use Pipeline Artifacts.
And if you want to know the different between publishbuildartifacts vs publishpipelineartifact, you could check below thread for the details:
What is the difference between Build Artifact and Pipeline Artifact tasks?
And this publish is different to dotnet publish?
The answer is yes. The dotnet publish task is to serve the specific project. Its function is similar to that we choose the Publish option for net core project in Visual Studio.
But we could specify the folder or files to publish for the publishbuildartifacts or publishpipelineartifact task.
Which one should I pick and why?
We recommend to use the publishpipelineartifact task if you just want to copy the bin directly to a network share.
You could check the reason from here:
For build artifacts, it's common to copy files to $(Build.ArtifactStagingDirectory) and then use the Publish Build Artifacts task to publish this folder. With the Publish Pipeline Artifact task, you can just publish directly from the path containing the files.
By default, the Download Pipeline Artifact task downloads files to $(Pipeline.Workspace). This is the default and recommended path for all types of artifacts.
File matching patterns for the Download Build Artifacts task are expected to start with (or match) the artifact name, regardless if a specific artifact was specified or not. In the Download Pipeline Artifact task, patterns should not include the artifact name when an artifact name has already been specified. For more information, see single artifact selection.
What is the # for?
The role of # is to specify the version of the task. For example, #1 is to use the version 1.0 of the task.

Download all files from all artifacts (artifact names) except from one in AzureDevOps YAML pipelines

I have a pipeline with with some stages like
Build -> Dev -> Test -> Prod
The build stage generate some nugets for different parts of the system. As this happen in different jobs, a couple of artifact names is published like:
server\server.nupkg
client\client.nupkg
auth\auth.nupkg
Those are then consumed in the following stages that dowload, apply variable substitution to some configs and publish a new nuget configured to an environment as a drop.
For example the test stage has one job that transform the config in the needed nuget and publish that as an pipeline artifact. So at that point the artifacts for the hole pipeline looks like:
server\server.nupkg
client\client.nupkg
auth\auth.nupkg
Test
server.nupkg
client.nupkg
auth.nupkg
When it comes to the prod stage i want to download all *.nupkg from all artifact names except from the Test artifact name.
I've tried to use an exclude patterns in the 'DownloadPipelineArtifact' task like this but no success:
- task: DownloadPipelineArtifact#2
displayName: Download nugets
inputs:
buildType: 'current'
itemPattern: |
'*/*.nupkg'
'!Test'
'!Test/*.nupkg'
targetPath: '$(MyDirectory)/nugets'
Any ideas?
If server, client, auth and Test are different artifacts then you should be able to either download the first three separately or do it in a single task invocation by using "exclude" file matching pattern with first segment being the artifact name. See Publish and download artifacts in Azure Pipelines | Multiple artifacts.
Example #1, download artifacts separately:
- download: current
artifact: server
- download: current
artifact: client
- download: current
artifact: auth
Although, based on the current docs, the following example is supposed to work, it doesn't. It looks like the docs are incorrect. See this issue for details:
https://github.com/microsoft/azure-pipelines-agent/issues/3416
Example #2, download artifacts using file matching patterns:~~
Note double asterisk for recursive wildcard and the absence of the quotes that would be included literally in the multiline YAML string otherwise.
- download: current
patterns: |
**/*.nupkg
!Test/**

How to change Azure DevOps environment agent folder structure

To achieve continuous deployment, we used a (classic) release pipeline on Azure DevOps to deploy a webservice to a VM in our intranet. To benefit of a yaml deployment pipeline, I replaced our former deployment pool agent on that VM with an environment agent.
For the actual deployment, we use the IIS Web App Deploy task. The source directory for this tasks defaults to $(System.DefaultWorkingDirectory)\**\*.zip. The $(System.DefaultWorkingDirectory) translates to the a subdirectory of the concrete release.
Unfortunately for me, the environment agent downloads the artifact next to the a-folder instead of into it like the environment pool agent. Thus the default settings of the deploy task cannot find it. I am aware that I can easily work around this issue by using $(System.DefaultWorkingDirectory)\..\**\*.zip. I was just wondering why Microsoft introduced such a development speed bump into the environment agents.
Is there any way I can make the environment agent download the artifact into $(System.DefaultWorkingDirectory) aka. a instead of next to it?
If you are using deployment job in yaml pipeline. The artifacts will be automatically downloaded to $(Pipeline.Workspace)/(folder next to $(System.DefaultWorkingDirectory)) in deployment jobs. See below extract from here:
Artifacts from the current pipeline are downloaded to $(Pipeline.Workspace)/.
Artifacts from the associated pipeline resource are downloaded to $(Pipeline.Workspace)/{pipeline resource identifier}/.
All available artifacts from the current pipeline and from the associated pipeline resources are automatically downloaded in deployment jobs and made available for your deployment. To prevent downloads, specify download: none.
To make the environment agent download the artifact into $(System.DefaultWorkingDirectory).
You can specify download: none. And use Download Pipeline Artifacts task and specify the path parameter to download your artifacts to $(System.DefaultWorkingDirectory). See below:
- deployment:
environment: Dev
strategy:
runOnce:
deploy:
steps:
- download: none #prevent automatically download
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'current'
targetPath: '$(System.DefaultWorkingDirectory)' # download to default folder.
Anther workaround is to change the source directory for IIS Web App Deploy task defaults to $(Pipeline.Workspace)\**\*.zip
When changing from Classic to YAML pipelines for deployments, you will probably need to change $(System.DefaultWorkingDirectory) to $(Pipeline.Workspace). You can verify this with the steps:
- pwsh: Get-ChildItem $(System.DefaultWorkingDirectory) -Recurse
displayName: Check System.DefaultWorkingDirectory
- pwsh: Get-ChildItem $(Pipeline.Workspace) -Recurse
displayName: Check Pipeline.Workspace
(Yes, this is an awfully verbose way of doing it. It will, though, give you more complete insight into the file structure supporting your job.)

Why is this Azure DevOps pipeline release failing?

This is an ASP.NET Core 3.0 project that builds with no errors, but when it triggers the pipeline to release to Azure App Service, it fails with the following error:
2019-11-10T23:09:23.8008460Z ##[error]Error: No package found with specified pattern: D:\a\r1\a***.zip
What needs to be done to fix the release pipeline? The pipeline release is pulling the latest build as its artifact.
Assumptions
The following info assumes that you are appropriately publishing your build artifact from your Build pipeline, and that you have added the correct build artifact into you release pipeline.
In your release pipeline you have specified a build artifact in the Artifacts area
When adding your build artifact to your release pipeline, you chose to give it an alias of Build Artifact. This means that at the very lease (with default settings) your .zip file will be in some sub-directory of $(system.DefaultWorkingDirectory)/Build Artifact/
A new unique folder in the agent is created for every release pipeline when you initiate a release, and the artifacts are downloaded into that folder. The $(System.DefaultWorkingDirectory) variable maps to this folder.
To ensure the uniqueness of every artifact download, each artifact source linked to a release pipeline is automatically provided with a specific download location known as the source alias. This location can be accessed through the variable:
$(System.DefaultWorkingDirectory)\[source alias]
This uniqueness also ensures that, if you later rename a linked artifact source in its original location (for example, rename a build pipeline in Azure Pipelines or a project in Jenkins), you don't need to edit the task properties because the download location defined in the agent does not change.
The source alias is, by default, the name of the source selected when you linked the artifact source, prefixed with an underscore; depending on the type of the artifact source this will be the name of the build pipeline, job, project, or repository. You can edit the source alias from the artifacts tab of a release pipeline; for example, when you change the name of the build pipeline and you want to use a source alias that reflects the name of the build pipeline.
(from some of the abundant documentation
Instead of searching for your package using ***.zip (which isn't proper wildcard syntax) use Build Artifact/**/*.zip
** is for recursively searching into directories
(I don't know what folder)
* is for searching a part of a given level of the path
any file/folder that
starts with (SomeFile.*)
ends with (*File.zip)
and i think contains (*meFi*)
The pipeline YAML was missing the following tasks. Not sure why this isn't included in the ASP.NET Core template, very confusing for developers new to Azure DevOps.
- task: DotNetCoreCLI#2
inputs:
command: 'publish'
publishWebProjects: true
- task: CopyFiles#2
inputs:
targetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
When adding an Artifact of Source type "Build", select "Default version" as "Latest from the build pipeline default branch with tags", as follows: