Within Azure Devops I have a build pipeline which builds and publishes artifacts and a release pipeline which downloads those artifacts, defines some infrastructure configuration, and batch uploads the artifacts to a web container.
After the configuration definition I want to add a task to fetch the clientId of an AD registered app, dumps it into a json file and copies the file in the same folder as the build artifacts. The json has to be uploaded to the web container to provide runtime configuration for a spa app.
What I have tried:
generate a json in a release task and copy it into said folder
commit an empty json in the code, have it published as build artifact and update its content in a release task
use the file transform task which only seem to allow updating a key / value, not generating a new one
The contents of the folder which gets uploaded seem to be locked.
Is that correct ? What can I do to achieve my goal ?
Releases don't publish artifacts. Releases consume published artifacts. A release can be run multiple times for the same build. A release can have multiple environments. What you want to do would fall apart immediately in any of those scenarios.
What you should do is write a custom BASH or Powershell script (depending on your preferences and OS) that does exactly what you describe:
Generate an appropriate JSON file
Upload the JSON file to the "web container"
You haven't provided any details about what a "web container" is or what your deployment environment is (i.e. AWS, Azure, containers running in Kubernetes), so that's the most thorough answer that can be provided.
Related
I have created a c++ pipeline where the output of the build pipeline is published to drop container. The structure is the following
drop/v1.0.0/Release/MyService.dll
drop/v1.1.0/Release/MyService.dll
drop/v1.1.0/Release/MyService.dll
My engineers will need to view drop folder and according to the version that needs to be manually deployed to a client the will download the dll file.
As far as I understand there is not any way to view them under Artifacts (what a shame). I go to the project settings under Storage but I cannot view them either there. Only place that I am able to find them is under the pipeline run and then I have to find in which version of the pipeline run a specific service version was produced. This is a maze. We have dozens of c++ projects and we have to keep track of which pipeline version run of each project matches the service version.
Is there any way to be able to access them like in a folder structure?
You could use Builds - List via rest API to get all the builds for a pipeline, then use : Artifacts - List rest API to get all the artifacts for a build. It will list all the download URL for artifacts, then you could download them together or choose the one you want to download.
Besides, you could use the publishLocation argument in publish build artifacts task to copy the artifacts to a file share (FilePath). And the file share must be accessible from the agent running the pipeline. In this way you could publish all your artifacts to the file share you want for better management.
In addition, you could also use Universal Package task to publish your artifacts to your feed for better review.
I run UI tests for my app in a release definition in Azure DevOps. I generate test report. I decided that it is convenient to save it in the build directory(wrong assumption?). The directory where the report is is:
browserName + DateTime.Now.ToString("ddMMyyyyHHmmss", CultureInfo.InvariantCulture) + #"\";
so the directory regex would be for instance : Chrome\d+
I build the test project in the release pipeline, run the tests, then I try to publish my report. The goal is to make it available in the Azure DevOps, or send a link to download it or any other way that makes it accessible.
To do so I added a step Publish Build Artifact
but then I get an error:
but then I don't have a fileshare available(I am able to create Azure Storage Account for instance) additionaly Publish Build Artifact doesn't support wildcards so I can't use regex Chrome\d+ to pin down the report directory.
Question: How can I make a file that is generated during release pipeline accessible?
EDIT: I found out in the meantime that I have Sharepoint available with enough storage.
Unfortunately publishing from release pipeline is not allowed.
Can we publish artifacts in release pipeline - Azure devOps?
One way to get around this is to try to publish Universal package in release pipeline but it has limitations. Create feed and publish there your files so you can share URL to others. It is not best option but if your test results files are not large you can publish to the feed and clean it sometimes(manually, because REST API provides way to delete package but does not provide function to get list of all packages published).
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/package/universal-packages?view=azure-devops
The disadvantage of this option is that for free users it has limit up to 2GB. You can delete old packages when required but it takes around 24 hours to free space. But you can forget to free space and your pipeline will fail with not enough storage error and by next 24 hours you will have to disable this task to let pipeline pass.
"I decided that it is convenient to save it in the build directory"
Remember that agent working directory is cleaned depending on option you choose.
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/pipeline-options-for-git?view=azure-devops#clean-the-local-repo-on-the-agent
Im setting up a new build and deploy pipeline in Azure Devops. It is an older Web application with some transformation files for the web.config. In the old days we would build the same code x times depending on how many environments. This this is no longer necesary as I read from here https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/transforms-variable-substitution?view=vsts#xmltransform
it looks like the deploy pipeline can pick up the changes from my transform file.
But the problem is that my other transform files does not get included in the package so I get these warning message:
[warning]Unable to apply transformation for the given package. Verify the following.
[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the <DependentUpon> tag for each config in the csproj file and rebuild.
[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
And yes when i download the artifact the Web.[stage].config files are not there as suggested.
Is there some setting somewhere that let me include these files? Or stop them from being transformed?
For Web applications
MSBuild knows how to transform the web.config files based on the following settings/properties/parameters (in order)
Build Configuration dotnet publish --configuration Release
Publish profile dotnet publish --configuration Release /p:PublishProfile=FolderProfile
Environment dotnet publish --configuration Release /p:EnvironmentName=Production
Custom file transform dotnet publish --configuration Release /p:CustomTransformFileName=custom.transform
I think it's typical for developers to make this happen based on build configuration only, and I believe MSBuild (and dotnet) know how to do this based on the <DependentUpon>Web.config</DependentUpon> element in the Web.[configuration].config item in the project or build script file.
Azure DevOps Release Pipelines is a little different.
The pipeline wants to transform your web.config after the project has been built/published and doesn't know how to do that if MSBuild (or dotnet) has already made an attempt at it. Thus:
[warning]Unable to apply transformation for the given package. Verify the following.
[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the tag for each config in the csproj file and rebuild.
[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
The warning text states:
Remove the <DependentUpon> tag for each config in the csproj
Thus: you need to remove the tag from the csproj to prevent MSBuild from transforming the files
Or: you need to use the /p:TransformWebConfigEnabled=False argument to MSBuild. (note: I believe it is correct that this can be used w/o removing the dependent upon tag, but I could be wrong)
Make sure the transform source and target files are in the same folder inside the package.
There may be several ways to do this. I've chosen to mark the transform source config files as content to force MSBuild to include them in the published package.
Now you need to organize your release pipeline in accordance with the File Transforms and Value Substitutions documentation.
[section]Starting: IIS Web App Deploy
====================================
Task : IIS Web App Deploy
Description : Deploy a website or web application using Web Deploy
Version : 0.0.51
Author : Microsoft Corporation
Help : More information
====================================
...[command]C:...\ctt\ctt.exe s:C:...\Web.config t:C:...\Web.Release.config d:C:...\Web.config pw i
[command]C:...\ctt\ctt.exe s:C:...\Web.config t:C:...\Web.Development.config d:C:...\Web.config pw i
XML Transformations applied successfully
...
For Non-Web Applications Needing .config Transformation
Getting your .config files to the release pipeline can happen several ways. Here are two.
Your release should have "access" to the repository as a part of the artifact, which will ensure that the deploy agent downloads the source (not desirable IMHO).
You will need to include the web.[stage].config files as part of your build artifact with a copy task, or a minimatch that picks them up.
Once you have the .config files available to the release pipeline
You can use the File Transform Task or XDT Transform Task to perform the transformation operations.
Option 2 is the route I've gone.
Here is an image of what that task looks like for me.
That task puts the config in the artifact from the build that I can then use in the release pipeline without rebuilding xx times.
Cleanup
If you're in a position where you care to not have the transform files persisting on the agent after the release is complete, then you'll need to add that to your pipeline.
I cannot use the Publish Build Artifacts Task in a release-definition, I'm getting the following error:
ErrorHostTypeNotSupported: This task must run in a build to publish artifacts to Visual Studio Team Services/TFS.
So my question is:
How can I save a file from a release-definition?
Why I want to achieve this:
I'm using a Deploy to Kubernetes Task in my release definition (which deploys a microservice with a yaml file to azure aks), this yaml file is tokenized during the release (I'm replacing the build-id). So at the end, I want to be able to download this file for each release and to use it for a manual deployment (if necessary).
Thanks for your help.
There is no container for artifacts in Release Management, so you cannot use that task in Release.
As a workaround, you can use ##vso[task.uploadfile]local file path command in VSTS Task Logging Command to upload the file.
Upload user interested file as additional log information to the
current timeline record. The file shall be available for download
along with task logs.
To do this, add a powershell task in your release definition like below:
When you want to use the file, download the release logs as zip, the file will be included in it.
I am trying to get Visual Studio Team Services (VSTS) to perform a web deploy of my app. According to the VSTS UI the web deploy is successful, but when I browse the deployed web site it shows the default IIS screen ("Welcome - IIS").
This is my publish step in the build pipeline:
This is my release definition:
This is the resulting deploy folder:
Build.log (with replaced values) - https://drive.google.com/file/d/1y6q2Cjr1gxBVMcHeh6n_r7qu-JpJFSyC/view
Do I need to add an additional step to the release pipeline to get the .zip files extracted?
You do not need to add an additional step to extract your zip, but you need to tell the IIS Web App Deploy task to deploy your zip file and not a folder. You have specified a folder for the Package or Folder input, so the task is doing exactly as you told it to do, deploy the specified folder.
If you only have one zip file in your build artifacts, the default value for this field, $(System.DefaultWorkingDirectory)\**\*.zip, will pickup your zip file and correctly deploy it. If you have multiple zip files in your build artifacts, you will need to specify the full path to the zip file in order for it to deploy.