We have a need that periodically, we will run a build configuration that among other things, recreates tokens/logins etc. We want to save these back to Team City as Environment variables. Builds that we subsequently do will want to look at this Environment Variable store and do a string replace within our configurations as required.
I've taken a look at :
##teamcity[setParameter name='env.TEST' value='test']
But from reading the documentation, this is only used to pass variables between build steps within the same build. It doesn't actually save the variable back to Team City.
Is there any way (Preferably from a powershell script), to call Team City and tell it to add a Environment Variable (Or any other variable).
In order to persist a value back to a parameter you have to call the REST API.
I use a PowerShell script that acts as a wrapper around the Invoke-RestMethod cmdlets in PowerShell 3+ that can be reused in a build step to achieve what you want.
Step 1.
Save the script to a PowerShell file and add it to your source control rest-api-wrapper.ps1
Step 2.
Create a PowerShell build step referencing the script and pass in the following arguments, tailored for your situation
%teamcity.serverUrl%/httpAuth/app/rest/projects/project_id/parameters/parameter_name
"Username"
"Password"
"PUT"
"TheValueToSave"
More details can be found here - TeamCity Documentation
Hope this helps
Related
In my tests I'm using environmental variable. Due to security reasons, when setting it in azure devops pipeline I need to mark it as secret. Is it possible to pass it as argument, within vstest task?
Apparently lacking enough reputation to comment on the answer by #daniel-mann, I need to follow-up through an answer myself.
Regarding the use of a runsettings file;
Yes, I'm sure you can do it like this, but there's a much simpler way. In the task definition, you have the option to override test run parameters.
For a non-YAML pipeline, you do this in the "Override test run parameters" option (the tooltip says "Override parameters defined in the TestRunParameters section of runsettings file or Properties section of testsettings file. For example: -key1 value1 -key2 value2."), so like this then:
-SomeSecret $(SomeSecret)
For a YAML-pipeline, you can do this:
overrideTestrunParameters: '-SomeSecret $(SomeSecret)'
The nice thing is that the test run parameter that you'd like to override DOES NOT need to exist in your runsettings file. In fact, there's not even need for a runsettings file at all!
The bad thing about this approach in general is that you'll have to access your secrets through the "TestContext.Properties" collection (for NUnit), which really sucks if you want a transparent solution that works equally well both for local development (using "user secrets") and in an Azure pipeline.
Another potentially bad thing about this is that these "overridden" test run parameters are just that - parameters for THAT test run only. If you happen to have a mixture of .NET Framework and .NET Core tests, you would want to execute those in two different test runs (for it to work at all...), and then you'd need to duplicate those overrides (given that some of the same secrets are needed, that is).
Regarding adding an additional task in your pipeline to set the appropriate environment variables;
Absolutely. Using the "Batch script" task is well suited for this, where you just pass your secret-based variables as parameters to the script and pick them up inside the script file.
For this to work as expected, though, you will need to allow the task to modify the environment.
For a non-YAML pipeline, this is done by ticking off the "Modify Environment" checkbox.
For a YAML pipeline, you could do it like this:
- task: BatchScript#1
displayName: 'Export key vault vars as env. vars (for tests)'
inputs:
filename: 'ExportKeyVaultEnvironmentVariables.cmd'
arguments: '$(SomeSecret)'
modifyEnvironment: true
Where in the "ExportKeyVaultEnvironmentVariables.cmd" script, you just do this:
set SomeSecret=%1
Note: If your secret by chance has some funny characters, especially having a trailing "=" character, you might experience that what you get when collecting the parameter inside the script is NOT what you sent in.
You can avoid this problem by enclosing the parameters in double quotes, like this:
arguments: '"$(SomeSecret)"'
And then collect the parameter by removing those surrounding quotes by using the "~" parameter modifier, like this:
set SomeSecret=%~1
A nice bonus effect of this approach is that your shiny new full-blown environment variables persist for the remainder of your pipeline. Referencing back to my "bad thing" about having to potentially duplicate the test run parameter overrides, that would not be needed here.
Regarding the additional option mentioned by the OP;
Absolutely, in which case you wouldn't need a "Batch script" task (that needs to call a script file), but just a "Command line" task.
BUT, be aware of a possible gotcha! Yes, this will create an environment variable for your test code to pick up, if you access it through "Environment.GetEnvironmentVariable".
In my case, I was building an "IConfigurationRoot" instance, like this:
/// <summary>
/// Gets a configuration instance, based on user secrets (for local test execution) and environment variables (for Azure pipeline execution).
/// </summary>
/// <typeparam name="T">The type of the class that represents the "runtime" (and thus is able to get hold of any configuration).</typeparam>
/// <returns>An <see cref="IConfigurationRoot"/> instance.</returns>
public static IConfigurationRoot GetConfigurationRoot<T>()
where T : class =>
new ConfigurationBuilder()
//// Note: The "AddUserSecrets" method requires the "Microsoft.Extensions.Configuration.UserSecrets" package
.AddUserSecrets<T>()
//// Note: The "AddEnvironmentVariables" method requires the "Microsoft.Extensions.Configuration.EnvironmentVariables" package
.AddEnvironmentVariables()
.Build();
This works by adding various "configuration providers", which eventually allows you to access them all seamlessly through "configuration["SomeSecret"]".
What I found, though, if I'm not seriously mistaken, is that "SomeSecret" was still not available in the "EnvironmentVariablesConfigurationProvider" that's added, even though I could perfectly fine access it directly with the above mentioned method. Go figure (but I might be mistaken...).
Possible alternative approach (but for YAML pipelines only?);
It seems that for a YAML pipeline, you can explicitly set environment variables for a task, like this:
- task: VSTest#2
env:
SomeSecret: $(SomeSecret)
[...]
I haven't tested this myself, but seen a colleague do it (myself, I currently don't have a YAML pipeline). At least this variable can be picked up with "Environment.GetEnvironmentVariable", but I don't know if this can be picked up through an "IConfigurationRoot" instance.
I haven't seen any option to achieve the same in a non-YAML pipeline.
But again, this also suffers from the same possible "having to duplicate the environment variables across several test runs" problem.
See also my solution to my own question over at the Azure DevOps guys
I've been through this. There's no way to pass variables to VSTest on the command line, which means you have to jump through a few hoops.
You have a few options:
Use a runsettings file with a TestRunParameters section, then access it via TestContext.Properties["variableName"] within the tests themselves. You can use standard token replacement patterns to transform the XML file.
Use an app.config or appsettings.json (depending on your platform). This works pretty much the same as above, except, of course, you use the standard configuration classes to retrieve the values.
Add a step to your pipeline that sets the appropriate environment variables. Secrets don't get automatically mapped to environment variables for security purposes, but there's nothing that's stopping you from doing it yourself.
Move the secret values into a keyvault or some other sort of external secret storage and configure the test to pull the secrets at runtime.
In my Octopus project, the first step launches a bunch of nested stacks implemented with cloudformation.
I need to share the outputs of the master stack launched from Octopus, how can I do that?
Thanks.
The output variables from the CloudFormation template will be available to later steps the same as any other Octopus output variable, this is mentioned in the first paragraph of the documentation page.
Output variables can be accessed a number of different ways, depending on where you are accessing them, for example, in Powershell they can be accessed via the parameters dictionary $OctopusParameters["Octopus.Action[Step Name].Output.VariableName"].
You can also access them using the Variable Binding syntax, #{Octopus.Action[Step Name].Output.VariableName}
More information about output variables is available in the docs.
I'm moving my configuration from using web.config transforms to being based on VSTS variables. I get process variables, you define a variable, pick an environment, and you're good to go. I also see "Variable Groups", these seem great, have KeyVault integration, and overall seem like a much better option.
But...I don't see a way to bind a Variable Group to a specific environment in my VSTS release process. I can't honestly see how these would be any use to me without this feature.
I've experimented with one workaround, but it didn't work. I tried:
Naming my variable group & variables with an environment prefix e.g.
Variable Group Name="Production ConnectionStrings"
Variable name="Production_LoggingConnectionString"
I thought once I linked the "Production_ConnectionStrings" variable, I could reference $(Production_LoggingConnectionString) from within a standard Process variable, but this didn't work.
I think I could come up with some powershell that would do something like the above and set variables, but this seems a bit too custom for me.
Does anyone else have an idea that I can use variable groups per environment, easily, without waiting around for VSTS to build this feature (if ever). Btw, if you want this feature, there is a suggestion here you can upvote: Make it possible to link a variable group to a specific environment in a release definition
This has now been implemented in VSTS variable groups as scopes. Go to your release definition -> Variables -> Variable Groups -> Link variable group, and you get the link window as below, where you can choose the scope to be either release or one or more of your environments!
I did not manage to find any release information on this feature, I just stumbled upon it as I was tweaking my releases.
I ended up using a powershell script to define my process variable based on the variable groups, it works great.
Let's say I want a variable named "LoggingConnectionString" and this has different values per environment
Steps:
Define a Variable group, e.g. "SharedLoggingVariables"
Inside this Variable group, define a variable/value for each environment, e.g. "LoggingConnectionStringDev", "LoggingConnectionStringProduction"
Back in your Process Variables for the Build/Release, make SURE you don't have a variable named "LoggingConnectionString", otherwise this will overwrite the value coming from the variable group
In your Release process, create a Powershell inline script at the beginning of the Agent with the following code
Param(
[string]$LoggingConnectionString
)
Write-Host "##vso[task.setvariable variable=LoggingConnectionString]$LoggingConnectionString"
Pass your variable group as an argument to this inline powershell, e.g.
-LoggingConnectionString "$(LoggingConnectionStringDev)"
The final powershell step should look something like this:
During release, the powershell will set your process variable from the variable groups. If powershell isn't an option for you, there are other options
No, there is no way to use variable Groups per environment.
As the user voice you linked, you can vote and follow up for the suggested feature.
The work around for now is using environment variables to overwrite the variables in variable Group.
Assume the variable LoggingConnectionString with the value Server=myDB in variable group need to be used both for Dev environment and staging environment. But for staging environment, it needs to use another value (such as Server=stageDB) from the variable LoggingConnectionString. So you can add the an environment variable LoggingConnectionString with the value Server=stageDB for staging environment.
When the variable $(LoggingConnectionString) is used in Dev environment, it will use the value (Server=myDB) defined in variable group.
When the variable $(LoggingConnectionString) is used in staging environment, since the variables both defined in environment variable and variable group, it will use the value (Server=stageDB) defined in environment variable.
Would be grateful to any assistance on how I can install an MSI and modify its default property values through an xml file that has the new values to be inserted during install time. The new property values will then be passed into MSIEXEC as a parameter by referencing the xml file and will therefore look like this:
msiexec /I MyMSIFle.msi PROPERTIESFILE=ProdProperties.xml
The need for this is because we have a number of environments. For sake of argument let's say DEV, TEST and PROD. The MSI property values differ for each environment and will be held in discrete XML properties files, e.g. DEV-Properties.xml, TEST-Properties.xml and PROD-Properties.xml.
The MSI is a single, generic MSI that we intend to install across all three environments successfully, simply by passing in the correct property values, all embedded within the individual XML files.
I'll be particularly happy to accept solutions using Powershell, Windows Batch scripts, VBScript, etc, but no third-party software as we have strict restrictions on using any such products within my company.
Thank you
I suggest you create an custom action for your MSI. Here is an example how:
http://blogs.technet.com/b/alexshev/archive/2009/05/15/from-msi-to-wix-part-22-dll-custom-actions-introduction.aspx
You could pass the .xml file name as an property and deserialize the file from the given path to an object. From the object you could then override some more properties.
Anyhow I think it is not sensible to contain such logic in an installer. I think a better way would be to write a simple flag to registry telling what environment is in question and the let the installed program deduce rest.
So I am working on some IIS management scripts for a specific IIS Site setup exclusive to a product to complete tasks such as:
- Create Site
- Create App Pool
- Create Virtual directories
The problem, is I would like to keep separate scripts for each concern and reference them in a parent script. The parent script could be ran to do a full deployment/setup. Or you could run the individual scripts for a specific task. The problem is that they are interactive, so they will request for the user information relevant to completing the task.
I am not sure how to approach the problem where each script will have a script body that will acquire information from the user, yet if it is loaded into the parent script avoid that specific's scripts script body from prompting the user.
NOTE: I know I could put them into modules, and fire off the individual "Exported to the environment" functions, but this script is going to be moved around to the environment that needs setup, and having to manually put modules (psm1) files into the proper PowerShell module folders just to run the scripts is a route I am not particularly fond of.
I am new to scripting with Powershell, any thoughts or recommendations?
Possible Answer*
This might be (a) solution: but I found I could Import-Modules from the working directory and from there have access to those exported functions.
I am interested in any other suggestions as well.
They way I would address it would be to implement a param block at the top of each sub script that would collect the information it needs to run. If a sub script is run individually the param block would prompt the user for the data needed to run that individual script. This also allows the parent script to pass the data needed to run the subscripts as the parent script calls the sub script. The data needed can be hard coded in the parent script or prompted for or some mixture thereof. That way you can make the sub scripts run either silently or with user interaction. You get the user interaction for free from Powershell's parameter handling mechanism. In the subscripts add a parameter attribute to indicate that Powershell will request those particular parameter values from the user if they are not already provided by the calling script.
At the top of your sub scripts, use a parameter block to collected needed data.
param
(
[parameter(Mandatory=$true, HelpMessage="This is required, please enter a value.")]
[string] $SomeParameter
)
You can have a deploy.ps1 script which dot sources the individual scripts and then calls the necessary functions within them:
. $scriptDir\create-site.ps1
. $scriptDir\create-apppool.ps1
. $scriptDir\create-virtualdirectories.ps1
Prompt-Values
Create-Site -site test
Create-AppPool -pool test
Create-VirtualDirectories -vd test
In the individual functions, you can see if the values needed are passed in from the caller ( deploy.ps1 or the command line)
For example, create-site.ps1 will be like:
function Create-Site($site){
if(-not $site){
Prompt-Values
}
}
The ideal is to make the module take care of maintaining storing it's own settings, depending on distribution concerns of that module, and to make commands to help work with the settings.
I.e.
Write a Set-SiteInfo -Name -Pool -VirtualDirectory and have that store values in the registry or in the local directory of the module ($psScriptRoot), then have other commands in the module use this.
If the module is being put in a location where there's no file write access for low-rights users (i.e. a web site directory, or $psHome), then it's a notch better to store the values in the registry.
Hope this Helps