I already integrated results of automated test with XRAY (from GitHub Actions)
But I would like to know if anyone knows how to manually trigger automated test from XRAY, pressing the "RUN" button in a test plan.
Thank you
You can run trigger manually form xray using Automation Rulu execution in JIRA .
follow this guides: https://www.atlassian.com/devops/testing-tutorials/jira-xray-integration-trigger-automated-tests#:~:text=In%20Jira%20settings%2C%20under%20Automation,from%20the%20Test%20Plan%20issue.
Related
I'm currently using cTest to publish test results from google test into azure.
Is there any way i can decorate my test so it comes up already linked to a requirement. without manually having to link it in azure?
Edit:
I'm using gtest recordproperty to add data to the xml output of the test case. I have tried with #1234( as in a commit), 1234 and several combinations. I cannot find anything in the documentation.
Currently, you could refer to this doc: Run automated tests from test plans to automate test cases in your test plans and run them directly from Azure Test Plans. And then open the Test Plans page and select the test plan to see the status of your tests if tests are updated after test execution is complete.
In addition, if you turn on below “Automatically link new work in this build” option in build pipeline page, and then you can link work item to Git commit, the new build queued by this commit will link to this work item when it completes successfully.
Also, you could enable “Create work item on failure” option to create a new work item when this build is failed.
BTW, you could link work items to deployments refer to this doc.
I've created CI&CD on Azure repo, and it is doing code deployment on App service.
Now created Automation that opens the browsers and do actions and generates reports for test cases. (C#+Selenium+report package).
Now I need to the setup Test plan, I know, needs to run headless testing, my the questions are
need to do without buying a Test plan package, then how we should do? (I have a subscription but not with test plans)
How to do using Azure Test plans (any recent blog pls, and no MS docs links).
any help appreciated
As far as I know, you can execute Selenium tests directly in the pipeline(Without test plan).
Here are the steps:
Add Visual Studio Test Task to run the Selenium tests and the test result will be output to the Test tab(Build Pipeline or Release Pipeline).
You also could use Publish Test Results task to publish test result.
Here is a blog about the detailed steps: Automating Selenium Tests in Azure Pipelines.
For headless testing,
You could refer to UI testing considerations.
By the way, since you have already created CI/CD, you can also consider integrating UI testing steps into CI/CD.
If you still want to configure test plan, you could refer to this doc.
Regards,
Your help will be appreciated.
I have created a pipeline in VSTS\Azure-DevOps. It gets its sources from a repository in Bitbucket. Queueing a build works fine. It builds and the tests succeed.
Now I want a build to run on every commit to the repository on Bitbucket. However, when I edit the pipeline and in the Triggers tab enable 'Continuous Integration' and click 'Save' I get the following error:
Unable to configure a service on the selected Bitbucket repository. Bitbucket returned the error 'Forbidden: '.
I am confused that I get 'Forbidden', while getting the source-code already works.
What is it that I am doing wrong? Is there something I must configure in VSTS\Azure-DevOps or in Bitbucket?
Answering my own question:
It appeared that in Bitbucket I only had the rights of 'Writer' for the Repository. When we changed it to 'Administrator' enabling Continuous Integration worked and we verified that committing a code change triggered the build.
Good news / bad news.
It looks like - for now - you can configure a pipeline without being a BitBucket admin on the repo... but not using the templates.
So you can build an empty pipeline based on a BitBucket repo (no admin access), and manually add each of the tasks.
Based on further tests: what you cannot do is set the Continuous Integration trigger, because that requires admin access to set up the webhooks
I know, this is not what you want... but at least there is a way to end up with a working pipeline.
Regards,
Jose
I am using Github as my repo as well as my Kanban/Scrum board. We use Visual Studio Team Services for our automated builds. We really like the way VSTS works and it works well with Github as the repo.
However, I want to be able to create a new Github issue/bug if and when our Continuous Integration build fails. I know you can create a VSTS Work Item but I would rather keep all issues centralized.
Is there any way to hook up VSTS to create a Github repo whenever a build fails? Or perhaps create a Github issue whenever a new VSTS Work Item is created?
We are running our own build server so possibly something can be done on that end?
Yes, you can create a github issue when VSTS build failed with two options.
Option1:
In VSTS build definition, add a powershell task in the end of the build process. Functions in the powershell should include:
Detect above build tasks in the build definition. Use REST API timeline to get build detail, you can find each task result in result parameter.
Determine to create a github issue or not. If all above build tasks are pass to build, don’t create github issue. Else, create a github issue by github API.
Option2:
Create your own website, and in VSTS use web hooks to tigger build fail information for your own website. After your own website receive the build information, it can create a github issue.
Recently, I've met several bots in Google TensorFlow git repos.
When I submit a PR, one TF bot finds some related groups for the PR and says, "Can one of admins verify this patch?"
Then, one of a verifier bot says, I need to signup something, and I left a comment, Done. Then, the bot checked and marked it was done.
Then, later one admin commented, "jenkins, test please." I guess this triggers Jenkins and reports the test and other checker results.
How could I setup the (bot)environment and github workflow like this?
https://github.com/tensorflow/tensorflow/pull/1838
You have to learn a continuous integration tool like jenkins. Jenkins has triggers to find such events. After events like below triggered. You can use variety of plugins to accomplish very different tasks.
Source Code commit
Source Code push
Poll bug tracker periodically.
For example
Jenkins twitter plugin
Jenkins github plugin
There are a lot of different plugins for jenkins. See them here.
They use this plugin in jenkins :
https://wiki.jenkins-ci.org/display/JENKINS/GitHub+pull+request+builder+plugin
All how to configure is given on the page