I have a automation script which built in Java/selenium. I want to run my test cases from VSTS by triggering my script and also want to get the result back in VSTS. Can anybody give me the path how can I make that happen. Also where should I keep by project?
I was doing research on it. But doesn't make sense to me
https://learn.microsoft.com/en-us/azure/devops/pipelines/test/continuous-test-selenium?view=azure-devops
run my test cases from VSTS by triggering my script and also want to
get the result back in VSTS
For this issue ,there is a blog that gives a path to do this , please refer to it to see if it helps .
In this path , you need to do stuff both to your project, and to your vsts build definition. The jest-junit package is needed in the project and then create a build with three tasks(“npm”,“Publish Test Results”, “Publish Code Coverage Results” ).
Related
I am trying to create a release pipeline in Azure DevOps. We already have a functioning build pipeline that works well, it is able to package the build with VSBuild and publish it as an artifact. Then in the release pipeline I am using an IIS Deployment job (which includes IIS Manage and IIS Deploy tasks) and it gets that artifact to deploy.
The problem is that we already have a publish profile (.pubxml) that should take care of pretty much everything the IIS Deployment is doing (at least as far I as I understand it). So to me it seems I have two options that don't require me to refactor the project configuration itself.
I can try to mimic the settings on the IIS Deployment job to match our .pubxml as closely as possible and manually applying any changes that aren't doable through the task settings. Obviously this is not ideal as that would require us to update both when ever we make changes and it introduces a large chance of the pipeline breaking down over time.
I can scrap the idea of using IIS Deployment and just use a VSBuild task that uses arguments /p:DeployOnBuild=true /p:PublishProfile=Staging. This doesn't seem like best practices because it means my release pipeline isn't passing a build package to deploy, it is just creating a new one at each stage.
So is there a better option that would allow me to utilize the package I created with VSBuild and the .pubxml configuration together in a deploy? If that isn't possible then are either of my options the "correct" way to handle my situation or am I just missing another method of deployment I could use?
Thank you for any help or insight you can provide. Please let me know if there is any more information I can give that would be useful.
You can try using publish settings file (*.publishsettings) for your IIS deployment.
A publish settings file (.publishsettings) is different than a publishing profile (.pubxml) created in Visual Studio. A publish settings file is created by IIS or Azure App Service, or it can be manually created, and then it can be imported into Visual Studio.
To view more details, you can see:
Publish an application to IIS by importing publish settings in Visual Studio
Deploy your app to a folder, IIS, Azure, or another destination
So unfortunately there doesn't seem to be a way I can achieve everything I wanted in this. The publish profiles are required for when we build the project so without making changes to how we configure those I need to build the project whenever I want to deploy. Ultimately I went with option #2. I essentially just copied most of the build tasks used in the testing pipeline and placed those in the release pipeline with a few modified commands to actually deploy the build once finished. It all seems to work just fine but still doesn't feel like best practices. If I am missing something please let me know and I will make updates as appropriate.
I'm currently using cTest to publish test results from google test into azure.
Is there any way i can decorate my test so it comes up already linked to a requirement. without manually having to link it in azure?
Edit:
I'm using gtest recordproperty to add data to the xml output of the test case. I have tried with #1234( as in a commit), 1234 and several combinations. I cannot find anything in the documentation.
Currently, you could refer to this doc: Run automated tests from test plans to automate test cases in your test plans and run them directly from Azure Test Plans. And then open the Test Plans page and select the test plan to see the status of your tests if tests are updated after test execution is complete.
In addition, if you turn on below “Automatically link new work in this build” option in build pipeline page, and then you can link work item to Git commit, the new build queued by this commit will link to this work item when it completes successfully.
Also, you could enable “Create work item on failure” option to create a new work item when this build is failed.
BTW, you could link work items to deployments refer to this doc.
I have a pipeline that runs on pull requests and executes tests , it also produces an xml execution report (it is a node project and tests run via jest) , and coverage report that I use for sonarcloud. However , the developers want to be able to see the tests results on each pr and see what tests failed in particular, so my question is whether it is possible to somehow export and display those results in github or sonarcloud
Please refer to the following two methods to get the test cases metric in sonarcloud:
You can add sonar.testExecutionReportPaths in Additional Properties option in Prepare Analysis Configuration task. About sonar.coverageReportPaths
For SonarQube to pick up and analyse the content of the report file, you need to inform the sonar scanner. About the location of the XML file. To do so, edit the sonarqube.properties file. Please refer to this.
coverage reporting:
sonar.testExecutionReportPaths=reports/xxx/xxx.xml
In addition, you can also refer to this similar case.
How to get the path to a script (not procedure) in Azure DevOps build?
I'm trying to get the path to either my solution or project file to use in Visual Studio database project PostDeployment.
Works locally
In the Script.PostDeployment.sql file in a Visual Studio Database project I have the following code
SELECT #solutionDir = REPLACE('$(SolutionPath)','MySoulution.sln','');
SET #File = #solutionDir + 'myScript.sql'
-- and here I can use the #File
When I do a local publish/or build I can use the variable/macro $(SolutionPath) where I get the full local path to the solution.
With that I can point to the script I need to access.
Fails in AzureDevops
But Azure DevOps build doesn´t have the $(SolutionPath) macro and I get the following error
SCRIPT.POSTDEPLOYMENT.SQL(17,32,17,32): Build error SQL72008: Variable SolutionPath is not defined.
So I need a way (in AzureDevops) in the build step to get the path to the scripts.
What I have tried
I have tried all kinds of macros that just don't seem to work.
I have tried to work with predefined variables
I tried to use SqlCommandVariableOverride but it seems to have gone missing from the database project some time ago.
Questions
How can I add $(SolutionPath) (or something else) to my build step so this works both locally and in Azure DevOps build?
Is there another way to get a solution or project url to the postdeployment SQLCMD file?
Is MSBuild Extension Pack something I should look at? I will if it is my only option.
p.s
I also created a ticket for this in Azure Pipelines Tasks since they don´t seem to be that active in answering and I can't wait.
e.s
I asked the same question at developercommunity.visualstudio.com. Lets hope somebody there will be able to answer the question. If so I will update this question with an answer.
So first double check that your build is publishing the artifacts. there should be a blue button in the top right corner of your build that says "artifacts". To have access to the artifact you need to do an archive task after you do the build and point it at
$(Build.ArtifactStagingDirectory)
Then I would recommend not using SQL to make direct references to the filesystem for the build server. If this is absolutely necessary then you should use environment variables.
I would recommend setting up a release pipeline for deployment of the SQL script. Release pipelines are really designed for deployment, whereas builds are for compilation.
To configure the release, point it to the artifact and then you can access that artifact by $(System.DefaultWorkingDirectory)/**/*.zip then run whatever commands you want to do to process that.
I am using VSTS for my OPA5 Tests, so all works for one project. For this I created a Build for these Projects i wanted to test.
But if I want to test all projects, do I need to create a build for all Project or is there a solution to build all projects with one build definition?
The build should do always the same things, saved in a YAML File.
I have seen thats is possible to do builds with difficult branches but not with difficult repositories.
So has anyone a solution for this or is it impossible at the moment?
Yes, it's possible.
You just need to clone another git repositories at the beginning of the build.
So you can add a PowerShell task as the first task and execute git clone command.
And If you are using YAML file, just add the script to execute the PowerShell task.
Besides, you can also refer the post VSTS build from multiple repositories.