I'm trying to get all the test cases from a test suite in VSTS, even test cases that have no run data, marked as "Active." I know how to get the case ID, but not the name/title. Right now it goes through the run data and results and gets all the information from the result, but it won't work if there is no run data.
I'm working in C# using RestSharp for RestRequests. This is one of my RestRequests: /_apis/test/runs?planId=" + planID.ToString() + "&includeRunDetails=true&api-version=5.0-preview.2 This works to get all the information I need for test cases with run data. I have been using Microsoft docs on VSTS API for help: https://learn.microsoft.com/en-us/rest/api/vsts/?view=vsts-rest-5.0
Getting the test cases from the suite would be ideal, but I'll accept anything that works.
You can combine Get all test cases in a suite api and Returns a list of work items api. First, get the test case ids from the api below:
GET https://{accountName}.visualstudio.com/{project}/_apis/test/Plans/{planId}/suites/{suiteId}/testcases?api-version=5.0-preview.3
Then, get more information of the test cases from the following api:
GET https://{accountName}.visualstudio.com/{project}/_apis/wit/workitems?ids={ids}&api-version=5.0-preview.3
Related
The API call:
https://dev.azure.com/{organization}/{project}/_apis/test/Plans/{planId}/Suites/{suiteId}/points?api-version=7.0
Returns JSON data with objects "LastTestRun" and "LastTestResult" that contain their Ids. I am trying to find a proper way of getting older (not last) data of TestResults of the given TestPoint using API calls.
I tried the following API call:
https://dev.azure.com/{organization}/{project}/_apis/test/runs?planId=123
That returns me a list of all TestRuns in the given TestPlan. I can loop through the list and make API call for every single TestRun:
https://dev.azure.com/{organization}/{project}/_apis/test/Runs/{runId}/results
And, then retrieve the TestPointId. The problem is I'm working with large amounts of data, so I'm looping through 7000 API calls and it takes too much time (parallel of course).
I tried calling the last API call with $select so I can retrieve only the TestPointId but it is not supported.
A version of the AzureDevops I am working with: Version Azure DevOps Server 2020 Update 1.2.
Based on your expectation, I have tested as well to get test results with only test point value and tested get the runs of a specific test case in a test suite. Neither succeeded.
Currently there seemed no official API support the test point filter. For this, you may access this URL: https://aka.ms/AzDevOpsIdeas to submit any comments and proposals for future releases and implementations.
I need to get a table of test case results using a query. I have many test cases which are children to many parent test suites. Each test case is tagged, so i am able to get them all in one query, regardless of their parent suite.
I have two issues -
- I cannot return the outcome/result of each test case
- I cannot include a column which identifies the parent suite name to which this test case belongs.
In summary i need to be able to perform tag driven test case outcome reporting, and have full control over the columns to be pulled
e.g. parent suite name, outcome, date run, build version (custom fields).
Am i missing something as this does not seem to be a feature of Azure Devops which seems like something that should be fundamentally available. Quick and easy to configure human readable dynamic test results reporting bolted onto the Test Plan functionality Azure Devops already provides.
I am afraid that your idea of using the query to get the test outcomes is impossible.
The content you can query through the azure devops query is defined in the Column options. But, there is no test outcomes defined in the Column, so it is impossible to query this result.
To get test case results,you could try to use this rest api.
GET https://dev.azure.com/{organization}/{project}/_apis/test/Runs/{runId}/results?api-version=5.0
I use this api in the post man and successfully get a list of test results.
TL;DR: How do I relate a release to the tests visible in the Tests tab via the API?
I am running a Release Pipeline which executes tests against the website being deployed. The tests generate files (within my test code) and I want to use the DevOps REST API to attach those files to the Test report associated with the Release.
Note: I am running these tests as smoke tests after release as part of the release pipeline so that I can test the deployed website. This would not be possible as part of the build pipeline.
I can get the release information, but I'm having trouble identifying how to find the related test(s) and attach a file.
Additional: The link in this SO post appears to have been changed and no longer points to the intended page, but now points to the overall documentation for the REST API.
Currently, I believe the process should be:
Get release details
Get test run id
Get test case result id
Create test result attachment
The primary problem is getting the test run id and test case result id from the release information.
I guess I'm a bit late with this, but still.
You are right about the process. Getting Test run ID is possible via accessing logs of corresponding task inside the release. Here's the needed endpoint:
https://{organizationName}.vsrm.visualstudio.com/{projectName}/_apis/release/releases/{releaseID}/environments/{stageID}/deployPhases/{deployPhaseID}/tasks/{taskId}/logs
Please note that test run ID and test IDs will only be available after the test run task is finished.
I'd recommend getting release ID, stage ID and deployment phase ID directly from the release by accessing default variables provided by azure devops (e.g. $(Release.ReleaseId), other ones are easily googled) rather than fetching via API and matching by name, cause it takes time and does not return all releases, only 100 first or smth like that.
After this, you need to get TaskID. I found it possible and applicable looking for that by Task Name. Just fetch the release by it's ID via API and look for task which matches needed name using this endpoint:
https://{organizationName}.vsrm.visualstudio.com/{projectName}/_apis/release/releases/{releaseId}?api-version=5.0
Going through all the deploySteps, deployPhases, deployJobs to get the task name is a bit cumbersome, but possible (I used LINQ).
After getting these 4 base IDs, you are now able to get logs of the Run tests task. In received log I have used a regex to extract test run ID. Then, having test run ID, you can grab the list of test run results:
https://{organizationName}.visualstudio.com/{projectName}/_apis/test/runs/{testRunId}/results
To attach some files to a test, you will need this test's AzDo ID. How can you get it? Well, I can think of mapping the tests by their names (you will need to store a list of those beforehand) and then looping through fetched test results and the stored list. After a match is found, get the respective id, and attach anything you want:
https://{organizationName}.visualstudio.com/{projectName}/_apistest/Runs/{testRunId}/Results/{testId}/attachments?api-version=6.1-preview.1
The developers at my company are in the process of incorporating VSTS into our testing. I am developing unit tests for our code, using the VSTS Rest API to post the results of the tests, grouped in test runs.
My problem is that I am unable to update the test run to show the number of failed tests and the correct pass rate. My demonstration code uses four unit tests, with 3 passing results and 1 failing result. On the page of test runs, it shows 0 Failed tests and a 0% Pass Rate.
Internet searches haven't yielded any information on how those fields are set or calculated. I've done some searching in the documentation for the REST API in the hopes that I would just need to set a certain field when calling the endpoints. Although the Failed Tests and Pass Rate fields are returned as part of the update call, it doesn't seem like you can set those fields directly for a test run. I haven't found any alternate endpoints that affect those fields.
https://learn.microsoft.com/en-us/rest/api/vsts/test/runs/update?view=vsts-rest-5.0
So, the question is, in a nutshell: How do I update the Failed Tests and Pass Rate fields for a VSTS Test Run using the REST API?
I am programming in C#, using HttpClients to call the REST API endpoints, and passing the relevant data via JSON. Everything is created and updated properly in VSTS; it is just these two fields that don't seem to be working.
I am having this issue. I am creating the run, adding test results then closing the test run. On the front end my runs look like this:
Test run stats
However all my results show passes within the run like this
Test results
Can't understand why this is the case as the graphs on the test runs are correctly picking up the passes and fails.
I'm experimenting with the relatively new REST API for the TFS.
I can get single work items by ID, all their details (e.g. other linked work items by link) ... but I couldn't find any hint or example how to get all test results (test runs?) for a given test case.
Or do I have to "download" (get) all test results of all tests in all test suites of my plan to filter out manually the two or three tests results of my wanted test case?
Test resulta are not associated with Test Cases directly. A test case could pass and fail On multiple Test plane as part of multiple Test suites.
You would need to first look up all of the plane and did that contain that test case, then guess at the context before getting the results.
If you use a suite ID then you can easily look up the results.
var testResults =Connection.teamProject.TestResults.ByTestId(testcase.Id);
var res = testResults.Where(testResult => testPointsIds.Contains(testResult.TestPointId)).ToArray<ITestCaseResult>();