I want to check test cases passed(highlighted in below snap) value and compare with some threshold and if test case passed value is more than threshold then run next task from build
There isn’t the built-in variable that can get the details of test result.
There are some ways can do it during the build:
Analysis the test result file (e.g. trx file in TestResults folder) through PowerShell or other script
Retrieve the test run through Test Run REST API with buildUri (format like vstfs:///Build/Build/{build id} filter, then get the necessary information (e.g. totoalTests, passedTests)
After that, you can set the variable value through Logging Command (##vso[task.setvariable]value)
Related
TL;DR: Search and replace placeholders in a text file with the decrypted values of secrets in a variable group.
I would like to use a PowerShell script to receive a variable group in a release pipeline and then iterate through the list, performing search-and-replace on a file being released.
The variables in the variable list are secrets so I want to overwrite the placeholders in the file with the decrypted value of the variables.
The values of the variables are environment specific, so I do not want to provide the values at build time and do not want to include the decrypted values in a stored artifact.
The file being search-replaced will be used in an execution at release time but will not be deployed to a host, so will be destroyed upon completion of the pipeline execution.
The Tokenization task from the Visual Studio Marketplace does this job well. You'll need to install it into your Azure DevOps organisation, it's available at https://marketplace.visualstudio.com/items?itemName=TotalALM.totalalm-tokenization
By default the Tokenization task uses double underscores to identify the placeholders. It will replace any text that that matches the name of a variable in your release definition as long as it's surrounded by double underscores.
So if you want to write the value of a variable called MySecretVariable into your file you'll need to add a place holder like __MySecretVariable__ into the file where ever you want that value to be written.
The Tokenization task will write any encrypted values into the file in plain text but in the release logs they will be obfuscated.
If you're storing your variables in a variable group just link that to the release definition and set the scope to the appropriate environment.
So, I've found that, as long as I have a Base64 token configured for Variable Groups (click 'Show all scopes' when creating a new PAT) then I can call GET https://dev.azure.com/{organization}/{project}/_apis/distributedtask/variablegroups/{groupId}?api-version=5.1-preview.1 to get the variable group I need.
The above, however, will not return a value for secrets, although there is a hack/workaround for this (involving multiple pipeline steps).
The advisable route is to create a Key Vault in Azure and perform the processing either in Powershell or code.
As I need the decrypted key values passed into my application via a repeated find-replace, I have implemented a Powershell script in one Release pipeline step and consume the output in the next step.
I am helping a friend with their VSTS build and we are using both build identifiers
$(Build.BuildId)
$(Build.BuildNumber)
This is specific to a Xamarin.iOS build where we are using the new VSTS Build tasks for updating the Info.Plist. We need to use the $(Build.BuildId) for the version code which is just a number that increments. Then we want to use the $(Build.BuildNumber) for the actual Version Name. If was to format this into the build string it would look like this:
$(Build.BuildNumber) = 1.0.0
$(Build.BuildId) = 148
Result = 1.0.0 (148)
The problem is we have already submitted our app to the app store with a larger build ID then what our current build is at. Instead of running the build X number of times to sync it up we would like to manually update the BuildId so we can start using this for our full workflow.
No, it’s impossible.
The predefined variable Build.BuildId value is unique to record echo build in the VSTS account.
Even you can change the value by logging command, but it can only work for the current build, when you queue next the build, the build ID will continue increased without any effect.
Such as current Build.BuildId is 148, even if you use the value to 100 by Write-Host "##vso[task.setvariable variable=Build.BuildId]2148" in a build, but when another build is queued, the Build.BuildId will be 149.
The work around is using a user defined variable instead Build.BuildId:
Add a user defined variable such as custom.BuildId and set the value with ($(Build.BuildID)+GapNumber).
Such as if the current Build.BuildID is 148, while the build ID you submitted in app store is 2148, so you can use the custom.BuildId with the value ($(Build.BuildID)+2000). Then use the variables $(custom.BuildId) and $(Build.BuildNumber) for your app.
For next build. The custom.BuildId value will be 1149.
I'm using the counter expression:
MyAppVersion $[counter('my-counter-prefix', 500)]
With MyAppVersion being 500, 501, 502...
This function can only be used in an expression that defines a variable. It cannot be used as part of a condition for a step, job, or stage.
Evaluates a number that is incremented with each run of a pipeline.
Parameters: 2. prefix and seed.
Prefix is a string expression. A separate value of counter is tracked for each unique value of prefix
Seed is the starting value of the counter
You can create a counter that is automatically incremented by one in each execution of your pipeline. When you define a counter, you provide a prefix and a seed.
This is not possible to update Build.BuildId but you can set Build.BuildNumber as it is shown here using logging commands:
Write-Host "##vso[build.updatebuildnumber]my-new-build-number"
I have a series of test steps in my SOAPUI test suite, I want to generate a random value and pass it to the tests?. I've tried to use ${=(int)(Math.random()*50000)} but it only passes different random values to my tests requests.
Thank you
If I understand correctly you are trying to pass the same random value to all Test Steps in a given Test Suite.
Create a Setup Script, it's located at the bottom of your Testsuite window:
runner.testSuite.setPropertyValue( "rnd",((int)Math.random()*50000).toString());
Use the newly created Property in your Teststeps:
=${#TestSuite#rnd}
Now whenever you run your Testsuite another random value will be created and all Teststeps in all Testcases use the same value.
As the first step in a build configuration I am trying to dynamically change a parameter and use it in the subsequent steps. Reading online, it seems that the way to do this is to call ##teamcity[setParameter. But this doesn't seem to be working. It doesn't even change the value in the same step.
For example, I have created a test parameter and set it's default value to '1'. Inside a powershell script, I tried to change it to 2, as shown below.
But the output remains unchanged as can be seen below
I am currently using TeamCity 8.0.3 (build 27540). What am I doing wrong?
EDIT: I think the problem might be the command you're using to set the parameter. Try:
Write-Host "##teamcity[setParameter name='TestParameter' value='2']"
--
We've experienced the same behavior. The key here is 'subsequent steps.' You must modify the parameter in a separate build step that is run before the step in which you want to use the new parameter.
It's my understanding that all parameters in a build step are evaluated immediately before the execution of that step. The tokens will be replaced with the unmodified values of those parameters. Thus, what actually gets executed by the build agent is:
Write-Host "TestParameter value is 1"
Write-Host "##teamcity[setParameter name='TestParameter' value='2']"
Write-Host "TestParameter value is 1"
This is a follow up to my earlier questions on setting up tags: Can I use tags in SpecFlow to determine the right environment to use? and setting up variables from those tags: How to set up a URL variable to be used in NUnit/SpecFlow framework
I've set up some variables to aid in populating my NUnit Tests, but I find that when the NUnit runner finds the test that fits the first tag the test runs it with the settings of the second tag. Since the tags are important to me to not only know what test to run, but what variables to use, this is causing me problems.
So if I have the following tags:
#first
#first #second
#second
If I run #second everything is fine. If I run #first I get any scenario that has only #first fine, but when it comes to scenarios where I have both #first #second the scenario is run, because #first is there, however, it uses the parameters for #second. Since I am running the DLL through the NUnit-Console and the Tests are written through SpecFlow I am not sure where the issue may lie.
Does anyone have advice on setting up tests to run like this?
You've not been very specific, but it sounds like you have a feature file like this:
#first
Scenario: A - Something Specific happens under the first settings
Given ...etc...
#second
Scenario: B - Something Specific happens under the second settings
Given ...etc...
#first #second
Scenario: C - Something general happens under the first and second settings
Given ...etc...
It looks like you are selecting tests to run in NUnit by running all the tests in the "first" category.
If you set up event definitions like this:
[BeforeFeature("first")]
public static string FirstSettings()
{ ... }
[BeforeFeature("second")]
public static string SecondSettings()
{ ... }
If you execute scenario C then FirstSettings() and SecondSettings() will be executed before it. This is regardless of whether you used the #second category to select the test to run under NUnit.
This is almost certainly the reason that you are seeing the second settings applied to your test with both tags - I expect the second settings overwrite the first ones, right?
My only advice for setting up tests like this, is that binding events and so on to specific tags can be useful but should be used as little as possible. Instead make your individual step definitions reusable, and set up your test environment, where possible, with Given steps.