How can I customize the creation of my Service Manifest file? - azure-service-fabric

When I add a new Actor to my Service Fabric project, the Service is automatically added to my ApplicationManifest.xml and ServiceManifest.xml files because we have UpdateServiceFabricManifestEnabled set to true. For certain projects, we need to require services to have PlacementConstraints so they're deployed to the proper node.
How do I hook into this process so that I can specify the PlacementConstraints without having to remember to edit any of the manifest files?

The service manifest file gets automatically populated with the actor service types as part of the build. There's an MSBuild target that gets run after the built-in "Build" target which does this. You can tack on your own logic that gets run after this. In that logic, you can make any necessary changes to the service manifest file. Here's an example that ensures that placement constraints are added to all the service types in the service manifest file. It uses an inline MSBuild task but you could rewrite this to be contained in a compiled assembly or whatever you wanna do.
This sample should be placed at the end of the file in your Actor service project:
<UsingTask TaskName="EnsurePlacementConstraints" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.Core.dll">
<ParameterGroup>
<ServiceManifestPath ParameterType="System.String" Required="true" />
</ParameterGroup>
<Task>
<Reference Include="System.Xml" />
<Reference Include="System.Xml.Linq" />
<Using Namespace="System.Xml.Linq" />
<Code Type="Fragment" Language="cs">
<![CDATA[
const string FabricNamespace = "http://schemas.microsoft.com/2011/01/fabric";
XDocument serviceManifest = XDocument.Load(ServiceManifestPath);
IEnumerable<XElement> serviceTypes = serviceManifest.Root.Element(XName.Get("ServiceTypes", FabricNamespace)).Elements();
bool changesMade = false;
foreach (XElement serviceType in serviceTypes)
{
XName placementConstraintsName = XName.Get("PlacementConstraints", FabricNamespace);
if (serviceType.Element(placementConstraintsName) == null)
{
XElement placementConstraints = new XElement(placementConstraintsName);
placementConstraints.Value = "(add your contraints here)";
serviceType.AddFirst(placementConstraints);
changesMade = true;
}
}
if (changesMade)
{
serviceManifest.Save(ServiceManifestPath);
}
]]>
</Code>
</Task>
</UsingTask>
<Target Name="EnsurePlacementConstraints" AfterTargets="Build">
<EnsurePlacementConstraints ServiceManifestPath="$(MSBuildThisFileDirectory)\PackageRoot\ServiceManifest.xml" />
</Target>

Related

Update Build number in App config xml file on build pipeline

I have a build pipeline in Azure DevOps, I need to update the build number in my apconfig exe file that will be $(Build.BuildNumber).
I just tried this way:
Adding a variable name = BuildNumber value = $(Build.BuildNumber).
And in my apconfig.exe file have a key same like <add key="BuildNumber" value="1812201901" />.
Why I have tried like this way: thinking like it will update in the config file if variable name match with the key.
But it is not working. can anyone please help? I have just started in CI/CD.
Update Build number in App config xml file on build pipeline
Just like the Shayki said, using the Replace Tokens extension should be the directly way to resolve this issue.
But since you need to request to get this extension, as workaround, you could also use power shell scripts to resolve this issue, you can check below my test powershell scripts:
$currentDirectory = [IO.Path]::GetDirectoryName($MyInvocation.MyCommand.Path)
$appConfigFile = [IO.Path]::Combine($currentDirectory, 'App.config')
$appConfig = New-Object XML
$appConfig.Load($appConfigFile)
foreach($BuildNumber in $appConfig.configuration.add)
{
'name: ' + $BuildNumber.name
'BuildNumber: ' + $BuildNumber.value
$BuildNumber.value = '123456789'
}
$appConfig.Save($appConfigFile)
As result, the app.config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
</startup>
<add key="BuildNumber" value="123456789" />
</configuration>
Note: Set the powershell scripts at the same folder of the app.config file.
Hope this helps.
You can use the Replace Tokens extension and in the apconfig.exe file put this:
<add key="BuildNumber" value="__BuildNumber__" />
Configure the task to search variables with __ prefix and suffix:
Now the value will be replaced with the value of the BuildNumber variable you configured (equal to Build.BuildNumber).

Securely Signing ClickOnce Applications in Azure DevOps Pipeline

I'm trying to do CI/CD in Azure DevOps with a ClickOnce application. How can I securely make my code signing certificate available during the build when using a hosted agent?
Note I'm aware you can use a script as suggested at Visual studio team services deploymen/buildt certificate error. However this approach is not secure. The certificate would be loaded into the certificate store of the account the hosted agent is running under. This would allow the agent, and hence other Azure DevOps accounts, to potentially access and use the certificate.
The solution to the issue is to override the built in task SignFile. Interestingly enough the task SignFile uses a built in function in Microsoft.Build.Tasks.Deployment.ManifestUtilities.SecurityUtilities.SignFile which has two overloads, one that takes a thumbprint, and one that takes a file and password.
The solution is then to create a new Task that can reference the other overload. Since we cannot change the calling SignFile we need to maintain the same signature, and place the appropriate variables in the environment variables. In this case "CertificateFile" and "CertificatePassword".
Then reference those two in the overwritten SignFile. What I did was to create a new targets file (filesign.targets) and place the code there. Checked that in to my repository and referenced it from the main project file(s).
<Import Project="filesign.targets" />
This way we can also hold our key files in an Azure Key Vault, load them at built and give them a unique password just for that build.
The targets file holds the new FileSign task:
<?xml version="1.0" encoding="Windows-1252"?>
<!--
***********************************************************************************************
Microsoft.VisualStudio.Tools.Office.targets
WARNING: DO NOT MODIFY this file unless you are knowledgeable about MSBuild and have
created a backup copy. Incorrect changes to this file will make it
impossible to load or build your projects from the command-line or the IDE.
This file defines the steps in the standard build process specific for Visual Studio Tools for
Office projects.
Copyright (C) Microsoft Corporation. All rights reserved.
***********************************************************************************************
-->
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<UsingTask TaskName="SignFile" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.Core.dll">
<ParameterGroup>
<SigningTarget Required="true" ParameterType="Microsoft.Build.Framework.ITaskItem" />
<CertificateThumbprint ParameterType="System.String" />
<TargetFrameworkVersion ParameterType="System.String" />
<TimestampUrl ParameterType="System.String" />
<CertificateFile ParameterType="System.String" />
<CertificatePassword ParameterType="System.String" />
</ParameterGroup>
<Task>
<Reference Include="mscorlib" />
<Reference Include="Microsoft.Build.Tasks.Core" />
<Using Namespace="System" />
<Code Type="Fragment" Language="cs">
<![CDATA[
var EnvCertFile = System.Environment.GetEnvironmentVariable("CertificateFile");
Log.LogMessage("CertFile:!!" + EnvCertFile);
if (string.IsNullOrWhiteSpace(CertificateFile) && string.IsNullOrWhiteSpace(EnvCertFile)) {
var signFile = new Microsoft.Build.Tasks.SignFile();
signFile.CertificateThumbprint = CertificateThumbprint;
signFile.SigningTarget = SigningTarget;
signFile.TargetFrameworkVersion = TargetFrameworkVersion;
signFile.TimestampUrl = TimestampUrl;
return signFile.Execute();
} else {
var certificate = string.IsNullOrWhiteSpace(CertificateFile) ? EnvCertFile : CertificateFile;
var EnvCertPassword = System.Environment.GetEnvironmentVariable("CertificatePassword");
var certificatePassword = string.IsNullOrWhiteSpace(CertificatePassword) ? EnvCertPassword : CertificatePassword;
var testString = new System.Security.SecureString();
// Use the AppendChar method to add each char value to the secure string.
if (!string.IsNullOrWhiteSpace(certificatePassword))
foreach (char ch in certificatePassword)
testString.AppendChar(ch);
Microsoft.Build.Tasks.Deployment.ManifestUtilities.SecurityUtilities.SignFile(certificate, testString,
TimestampUrl == null ? null : new Uri(TimestampUrl),
SigningTarget.ItemSpec);
return true;
}
]]>
</Code>
</Task>
</UsingTask>
</Project>
Code based on:
https://gist.github.com/KirillOsenkov/4cd32c40bffd3045f77e
References:
https://github.com/Microsoft/msbuild/blob/fc10ea8ce260b764bb9fa5033b327af9fefcaabe/src/Tasks/ManifestUtil/SecurityUtil.cs
https://github.com/Microsoft/msbuild/blob/master/src/Tasks/SignFile.cs

Controlling the sequence of events in a Wixtoolset (.msi) installer

I am creating a Microsoft Installer (.msi file) using the Wixtoolset (Windows Installer XML). This installer must automate the installation of an existing .exe program (named installer.exe below) and copy a custom configuration file (named settings.conf below) to the target directory. In addition the installer must modify the configuration file using the InstallFiles command below. But the timing of events is critical. If the executable installer runs too early, it fails or exhibits strange behavior. And if the executable installer run too late in the install sequence, it overwrites my modified configuration file with the generic values. I believe this can be done by assigning a string to the Before or After property value. What Before or After property assignment will allow the executable to run properly but not overwrite the configuration file I moved by the CopyFile element? Here is my Wixtoolset XML code.
<Property Id="CONFIGFOLDER" Value="C:\acme\config" >
<Feature
Id="ConfigurationFile"
Title="Configuration File"
Level="1"
<ComponentRef Id="CMP_ACME_Config_File" />
</Feature>
<DirectoryRef Id="TARGETDIR">
<Component Id="CMP_ACME_Config_File" Guid="">
<File
Id="ACME_Config"
Source="MySettings.conf"
KeyPath="yes"
<CopyFile Id="Copy_ACME_Config"
DestinationProperty="CONFIGFOLDER"
DestinationName="settings.conf" />
</File>
</Component>
</DirectoryRef>
<Binary
Id="InstallerEXE"
SourceFile="installer.exe" />
<CustomAction
Id="Launch_Installer"
BinaryKey="InstallerEXE"
Impersonate="yes"
Execute="deferred"
ExeCommand=""
Return="check" />
<InstallExecuteSequence>
<Custom Action="Launch_Installer"
Before="InstallFiles">
</Custom>
</InstallExecuteSequence>
</Property>
I can't explain exactly why this works but assigning "InstallFiles" to the "After" property in the "Custom" element seems to do the trick.
<InstallExecuteSequence>
<Custom Action="Launch_Installer"
After="InstallFiles">
</Custom>
</InstallExecuteSequence>

How to return a value from a phing target?

I would like to have a foreach task like this, which iterates over all the files/directories in a directory "A" -
<foreach param="dirname" absparam="absname" target="subtask">
<fileset dir="${dir.destination}/${dir.subdir}/">
<type type="file" />
</fileset>
</foreach>
The target "subtask" should check if the counterpart of the file/folder exists in another directory "B" (I am comparing directory A and B basically), and return either of the following if it does not -
a flag.
name of the file.
Following is some code for reference -
<target name="subtask">
<if>
<filesmatch file1="${file1}" file2="${file2}"/>
<then>
Return false. But how?
</then>
<else>
Return true of name of the file. How?
</else>
</if>
</target>
Note - It is okay if this can be done without calling a target. I am not sure if the logic can be fit inside the foreachtask itself. Could not find any such thing in the phing documentation.
Basically, I should be having the list of file names which are not present in the directory B, by the end of the loop.
You may also read this question of mine, if you can give some pointers to solve the issue in some other way.
Update
Rephrasing this question, since I feel that the problem description is not clear. The phing documentation says, a target has no return value -
Targets are collections of project components (but not other targets)
that are assigned a unique name within their project. A target
generally performs a specific task -- or calls other targets that
perform specific tasks -- and therefore a target is a bit like a
function (but a target has no return value).
I don't understand why is it designed so. With this bounty, I would like to know if there is some workaround for me other than having to define my own custom tasks in PHP, and then set properties -
$this->getProject()->setNewProperty('modifiedElements', implode("\n\n",$modifiedElementsArray));
which can be accessed in the build file
I have a target which checks whether my production code base has any differences from the expected git revision -
<target name="compare_prod_with_expected_revision">
<input propertyname="box.git_version">
Enter git version of the production codebase:
</input>
<exec command="git reset --hard ${box.git_version}" dir="${dir.scratchpad}" />
<!-- Scratchpad brought to revision ${box.git_version} -->
<echo>Verifying whether production is at revision ${box.git_version}..</echo>
<exec command="diff -arq --exclude='.git' ${dir.scratchpad}/${dir.subdir} ${dir.destination}/${dir.subdir}" outputProperty="diffList"/><!-- #TODO ignore.swp files in this step. Diff says .swp files present in production code. But doing ls -a there does not show the same. -->
<php function="strlen" returnProperty="productionDeviationFromExpectedBranch"><!-- #TODO - find how to not show this step during build process. Put it in a target and set hidden="true" -->
<param value="${diffList}"/>
</php>
<if>
<equals arg1="${productionDeviationFromExpectedBranch}" arg2="0" />
<then>
<echo>Verified production is at revision ${box.git_version}</echo>
</then>
<else>
<echo>Differences - </echo>
<echo>${diffList}</echo>
</else>
</if>
</target>
Now, I want to phingcall this target and would like to access some property set by it.
I think I understood your purposes, and at the same time I feel like you chosen not the optimal tool of doing this.
As you mentioned in your question, official documentation on phing is clear about tasks (targets):
Targets are collections of project components (but not other targets) that are assigned a unique name within their project. A target generally performs a specific task -- or calls other targets that perform specific tasks -- and therefore a target is a bit like a function (but a target has no return value).
Targets should be components of your application, which execute specific task, atomic task. It could be initialization task, configuration fetching, compilation step, assets preparation and dumping, deployment task, clean-up task, etc. There's no "output", returned by target in the standard sense, but the result of target execution is the success of execution itself: success or failure.
One should not try to put way too much of logic into such project targets, as it is not intended to do complicated calculations, do heavy logical decisions, etc. I mean, Phing can do it, such things are possible, but this setup would be bulky, unreadable, and hard to scale/re-factor.
With Phing you can easily define conditional execution and branching of logical flow, you may define the sequence of execution of tasks (dependencies) - this is what makes it laconic and elegant. Keep targets as simple as possible, split the project into small, finished logical tasks.
Based on the projects I've been working with, the biggest targets, probably, were initialization stage and configs fetching. Here's some example, to understand what it might contain, I took it from real project:
<target name="init_configuration">
<echo msg="Define initial configuration for the deployment..." />
<if>
<not>
<isset property="host" />
</not>
<then>
<property name="host" value="dev" override="true" />
<echo message="The value of hostname has been set to ${host}" />
</then>
<else>
<echo message="The value of hostname is ${host}" />
</else>
</if>
<if>
<not>
<isset property="version" />
</not>
<then>
<property name="version" value="1.0.0" override="true" />
<echo message="The value of version has been set to ${version}" />
</then>
<else>
<echo message="The value of version is ${version}" />
</else>
</if>
<property name="host_credital_file" value="config/hosts/${host}.properties" />
<property file="${host_credital_file}" />
<available file="${host_credital_file}" property="hostfilefound" value="true"/>
<fail unless="hostfilefound" message="Missing Hostfile configuration file (${host_credital_file})!" />
<echo msg="Configuration is done" />
</target>
Other targets were extremely simplistic, they are normally – 1-5 lines long, and do only small purpose, small task. This would be, probably, the best recommendation when working with Phing.
The logic which you are trying to put on shoulders of Phing is possible, but would be extremely bulky.
Consider the point: how much quicker, easier, and more readable the same thing could be done with simple bash script in your example. Or even to write small CLI utility in PHP, which will do the job elegantly and quick. After that in Phing you'll leave parametric target which will execute this "revision diff script" from CLI.
Phing is a great tool for what it is designed for, but it can't be an optimal choice for every purpose. Just do not put way to much responsibility and logic into it.
As a workaround, for more complicated things it's better to combine Phing with with something specialized: bash scripting, PHP CLI, nodeJS (+ Grunt, Gulp, etc)... and just to add calls of a Phing targets later.
This is the way I managed to have targets which behave like functions:
<target name="-console-cmd-return-property" hidden="true">
<exec command="${command}" checkreturn="${checkreturn}" logoutput="${logoutput}" outputProperty="${outputProperty}"/>
</target>
It gets invoked like this:
<phingcall target="--console-return-property">
<property name="command" value="ps auxwww"/>
<property name="checkreturn" value="true"/>
<property name="logoutput" value="false"/>
<property name="outputProperty" value="ps_output"/>
</phingcall>
Of course it works because it relies on existing exec, and it is not generic...
The target "subtask" should check if the counterpart of the file/folder exists in another directory "B" (I am comparing directory A and B basically), and return either of the following if it does not -
a flag.
name of the file.
You could compare two directories without using a foreach task like this:
<project name="Phing Build Test" default="print-missing" basedir=".">
<resolvepath propertyName="dir.a" path="path/to/dir/a"/>
<resolvepath propertyName="dir.b" path="path/to/dir/b"/>
<target name="print-missing">
<apply executable="echo" failonerror="false" returnProperty="files.found" outputProperty="missing">
<srcfile/>
<fileset id="srcfiles" dir="${dir.a}" includes="*">
<present present="srconly" targetdir="${dir.b}"/>
</fileset>
</apply>
<if>
<equals arg1="${files.found}" arg2="0"/>
<then>
<echo msg="${missing}"/>
</then>
</if>
</target>
</project>

First run notepad with my.cfg and only then start the service

I install along with my application:
1) a service that starts and stops my application as needed
2) a conf file that contains actually the user data and that will be shown to the user to modify as needed (I give the user the chance to change it by running notepad.exe with my conf file during installing)
The problem is that in my code the service I install starts before the user had the chance to modify the conf file. What I would like is:
1) first the user gets the chance to change the conf file (run notepad.exe with the conf file)
2) only afterward start the service
<Component Id="MyService.exe" Guid="GUID">
<File Id="MyService.exe" Source="MyService.exe" Name="MyService.exe" KeyPath="yes" Checksum="yes" />
<ServiceInstall Id='ServiceInstall' DisplayName='MyService' Name='MyService' ErrorControl='normal' Start='auto' Type='ownProcess' Vital='yes'/>
<ServiceControl Id='ServiceControl' Name='MyService' Start='install' Stop='both' Remove='uninstall'/>
</Component>
<Component Id="my.conf" Guid="" NeverOverwrite="yes">
<File Id="my.cfg" Source="my.cfg_template" Name="my.cfg" KeyPath="yes" />
</Component>
[...]
<Property Id="NOTEPAD">Notepad.exe</Property>
<CustomAction Id="LaunchConfFile" Property="NOTEPAD" ExeCommand="[INSTALLDIR]my.cfg" Return="ignore" Impersonate="no" Execute="deferred"/>
<!--Run only on installs-->
<InstallExecuteSequence>
<Custom Action='LaunchConfFile' Before='InstallFinalize'>(NOT Installed) AND (NOT UPGRADINGPRODUCTCODE)</Custom>
</InstallExecuteSequence>
What am I doing wrong in the above code and how could I change it in order to achieve what I need? (first run notepad with my conf file and then start the service).
I would extend the MSI UI to ask for the parts the user needs to modify and then update the text file using XmlFile and XmlConfig elements. Then Windows installer can come by and start the service.