Is there any guidelines or recommended practices of test plans and multiple environments (test, pre, prod)? So I make a test plan for a given solution. And then I run it for dev,then for pre and then for prod. It sounds like a little overhead to have 3 totally same test plans, how do you guys run this? Is there anyway I can run a test plan, and provide a parameter that now its for a specific environment?
Related
Our organization wants to move all the test cases from Azure DevOps to TestRail. How can I do that? When I checked azure DevOps rest APIs here , there is no such api to pull all the test cases for a project.
Is there any tool that I can use to migrate test cases from Azure DevOps to TestRail?
Have you considered migrating to a test management tool by using a CSV file import? Azure DevOps allows you to export your test cases to a CSV file. You can learn more here:
Export TestCases to Excel/CSV in Azure DevOps
You can then import test cases as a CSV file into various test management tools. For example, in the test management tool Testmo you can import test cases from a CSV file and easily map columns to fields:
Importing CSV/Excel
Full disclosure: I'm one the developers behind Testmo, so I'm using it as an example. But this approach should work with various other test management tools as well.
Yes, OpsHub is a commercial third-party migration tool that facilitates smooth migration of Test Management work items including Test cases from Azure DevOps to TestRail. OpsHub has experience in undertaking complex migration projects, ensuring ZERO downtime, no data loss, systematic cut-over, and a factory approach, which should be an excellent choice for your use case.
OpsHub also takes care of the following use cases:
Non-disruptive Migration: Migration can cause massive disruptions to operations by not letting users use TestRail while the migration is going on. Often the cost of disruption dominates all other explicit and hidden costs. OpsHub minimizes disruption to all aspects of current operations.
Supported Entities: OpsHub supports the migration of the entities, Test Plan, Test Result, Test Run, Test Suite, Test Cases from Azure DevOps to Tests, Test Cases, Test Plans, Test Results, Test Runs, Test Suites, Sections, Milestones, etc., to TestRail. Projects and other instance-level configurations like fields, users, etc. will have to be manually taken over or with the help of the TestRail team.
Please reach out to OpsHub’s Migration Experts for the steps on migration planning from Azure DevOps to TestRail.
Full disclosure: I'm work for OpsHub.
How to run testrigor tests on different environments like QA, Preprod? I need to data driven the environment specific urls
There are different ways to achieve that in testRigor.
Using different test suites, one per environment. One of the suites can contain the test cases and the others can be children of it inheriting the test cases.
On the same suite by using "branch testing". When you call testRigor via API you can trigger branch testing specifying with URL will be the target of the test. Detail of this can be found on CI/CD integration menu option within a test suite.
I will try to explain the setup the procedure and hopefully someone can tell me what is the best approach to achieve my question at the end.
Setup:
We have the following environments for our application: QA UAT and Prod.
QA is our internal test environment whereas UAT and Prod are on customer side.
Procedure:
We prepare test cases in DevOps, and then run tests in the QA environment.
If all is ok and we are ready for UAT, application is deployed to UAT in order to have a test session (UAT session) with the customer, The test session will run test cases which are a subset of the original test cases which were used during the QA phase
Similar is repeated for Prod. except that usually the customer is not involved.
Question:
What is the best practice to have these different test cases sets (QA, UAT and Prod), in order to keep a record of the test runs on each of the environments.
I can think of:
Creating 3 test plans which reference the main set?
or creating 3 test suites?
or creating 3 configurations?
Your help is appreciated.
P.S. Mostly we do manual testing
different sets of test cases for different environments in Azure DevOps
We recommend you create 3 test plan for different environments, for each environments, you can create a test plan and import the existing test cases into that plan. You can also, if you wish, divide the test cases into separate test suites within the plan to enable easier management and monitoring of these separate sets of test cases.
Note: If you copy or clone the test cases. A copy creates a new baseline. Changes to these new test cases don't affect your previous test plans.
I'm currently running xUnit unit tests in by build pipeline to prevent releasing builds with failing tests. I see that there's an option to associated automated tests with test plans within Azure DevOps. Is this a good practice for unit tests, or should this only be done with functional and/or integration tests?
Run unit, function and integration test suites in Azure Pipelines. Azure Test plans is supposed to improve overall code quality by using manual and exploratory testing services for your applications.
Are there any built-in options to perform Regression testing and Performance testing of APIs in Azure API Management? If not, are there any options to do that within the Azure ecosystem (like VSTS Performance tests tool etc.)? Thanks!
I think there are no built-in options to perform Regression testing that available in Azure. But for performance testing, there are some.
Regression Testing
For regression.I'd recommend going with manual testing like creating Test Plan, Test Cases execute/test them manually using VSTS
Creating a Test Plan in VSTS
Creating a Test Case in VSTS
Run Manual Test in VSTS
Load/Performance Testing
There are 3 ways you can easily do this
With Visual Studio
With VSTS
Azure Portal
I prefer going with VSTS