I am developing a unit test project where I create an item in a test, then create sub items for it in the following test.
These tests are parameterized tests, and these parameters are collected in the runtime, so when the project starts it starts. It fails to retrieve the parent item from the database because they are not created yet "as I haven't run the first test yet".
Is there a workaround for this?
The first function:
[Test, Sequential]
public void AddInitiative([ValueSourceAttribute("Get_AddInitiatives_Data_FromExcel")]AddInitiative Initiative_Object)
{
string URL = "http://" + Server_name + Port_number + "/IntegrationHub/IntegrationHub.svc/RestUnsecure/AddInitiative";
string Token = Get_Security_token("gpmdev\\administrator", "Xyz7890", TenantID_Input);
var Response = POST_Request(Initiative_Object, URL, Token);
Guid Returned_GUID = GenericSerializer<Guid>.DeserializeFromJSON(Response);
DataBase_Queries DB = new DataBase_Queries();
List<StrategyItem> StrategyItemsFromDB=DB.GetStrategyItemByID(Returned_GUID.ToString());
Assert.AreEqual(Initiative_Object.Initiative.Name_En, StrategyItemsFromDB[0].Name_En);
}
The second function that fails:
[Test, Sequential]
public void AddInitiativeMilestones([ValueSourceAttribute("Get_AddInitiativeMilestones_Data_FromExcel")]AddMilestone Milestone_Object)
{
string URL = "http://" + Server_name + Port_number + "/IntegrationHub/IntegrationHub.svc/RestUnsecure/AddInitiativeMilestones";
string Token = Get_Security_token("gpmdev\\administrator", "Xyz7890", TenantID_Input);
var Response = POST_Request(Milestone_Object, URL, Token);
List<Milestone> Returned_Milestone = GenericSerializer<List<Milestone>>.DeserializeFromJSON(Response);
DataBase_Queries DB = new DataBase_Queries();
List<StrategyItem> StrategyItemsFromDB = DB.GetStrategyItemByID(Returned_Milestone[0].ID.ToString());
Assert.AreEqual(Milestone_Object.Milestones[0].Name_En, Returned_Milestone[0].Name_En);
Assert.AreEqual(Milestone_Object.Milestones[0].Name_En,StrategyItemsFromDB[0].Name_En);
}
Update: When I clicked from the GUI on Clear fixture the test data was reloaded, but it there a way to do that without the GUI?
It's generally bad practice in unit tests to have one test depend on (i.e. use output from) another test. In this case, with NUnit, it's actually impossible.
It's impossible because NUnit creates tests long before they are executed. NUnit will call your TestCaseSource methods at what we call "load time" when NUnit decides what tests exists and populates a GUI if you use one.
The code in your tests executes at "run time" for the tests. In a gui, this may happen multiple times for each load - every time you click Run, for example.
Note that I'm explaining this in terms of a GUI because it's an easy way to conceptualize it. NUnit works the same way whether you are running in batch or interactively.
If you want something to happen only once, before any tests are run, you can use OneTimeSetUp (TestFixtureSetUp in NUnit V2) to set it up. You can use a member of the class to save whatever you need from that execution and access it from your tests. However, this will still happen at "run time", decades (in computer terms) after your tests have been loaded.
Related
We are working on building an approach that automatically update test suite's execution metrics onto the VSTS server. After going through the REST API document for VSTS, we were able to do the following using those automated APIs
Create a Test RUN with desired list of existing test cases
Update the Test RESULTS (outcome and status) for the above created Test RUN
Now it is possible to update whether the Test Case is Pass, Fail or any other available outcomes. But we are looking for an automated approach with which we can update the status of each Test Step inside each Test Case to Pass, Fail or any other available outcomes.
Hope I have explained our pain area in more understandable way.
Please reply your suggestions.
Thanks in advance.
Test Steps listed in VSTS Test Cases still belongs to test result.
If you get a test result with parameter `detailsToInclude=Iterations', you will see there is "actionResults" to determine test steps outcome:
Get https://xxx.visualstudio.com/TestCase/_apis/test/runs/xx/results/xx?api-version=3.0-preview&detailsToInclude=Iterations
But I've tried update "actionResults" with REST api Update test results for a test run, and found it doesn't support update "actionResults". Your requirement can not be achieved with rest api.
Instead of REST api, you can use the client api as this case mentioned: How to add/update individual result to each test step in testcase of VSTS/TFS programatically
Simple sample:
int testpointid = 176;
var u = new Uri("https://[account].visualstudio.com");
VssCredentials c = new VssCredentials(new Microsoft.VisualStudio.Services.Common.VssBasicCredential(string.Empty, "[pat]"));
TfsTeamProjectCollection _tfs = new TfsTeamProjectCollection(u, c);
ITestManagementService test_service = (ITestManagementService)_tfs.GetService(typeof(ITestManagementService));
ITestManagementTeamProject _testproject = test_service.GetTeamProject("scrum2015");
ITestPlan _plan = _testproject.TestPlans.Find(115);
ITestRun testRun = _plan.CreateTestRun(false);
testRun.Title = "apiTest";
ITestPoint point = _plan.FindTestPoint(testpointid);
testRun.AddTestPoint(point, test_service.AuthorizedIdentity);
testRun.Save();
testRun.Refresh();
ITestCaseResultCollection results = testRun.QueryResults();
ITestIterationResult iterationResult;
foreach (ITestCaseResult result in results)
{
iterationResult = result.CreateIteration(1);
foreach (Microsoft.TeamFoundation.TestManagement.Client.ITestStep testStep in result.GetTestCase().Actions)
{
ITestStepResult stepResult = iterationResult.CreateStepResult(testStep.Id);
stepResult.Outcome = Microsoft.TeamFoundation.TestManagement.Client.TestOutcome.Passed; //you can assign different states here
iterationResult.Actions.Add(stepResult);
}
iterationResult.Outcome = Microsoft.TeamFoundation.TestManagement.Client.TestOutcome.Passed;
result.Iterations.Add(iterationResult);
result.Outcome = Microsoft.TeamFoundation.TestManagement.Client.TestOutcome.Passed;
result.State = TestResultState.Completed;
result.Save(true);
}
testRun.State = Microsoft.TeamFoundation.TestManagement.Client.TestRunState.Completed;
results.Save(true);
Currently I'm using Eclipse with Nokia/Red plugin which allows me to write robot framework test suites. Support is Python 3.6 and Selenium for it.
My project is called "Automation" and Test suites are in .robot files.
Test suites have test cases which are called "Keywords".
Test Cases
Create New Vehicle
Create new vehicle with next ${registrationno} and ${description}
Navigate to data section
Those "Keywords" are imported from python library and look like:
#keyword("Create new vehicle with next ${registrationno} and ${description}")
def create_new_vehicle_Simple(self,registrationno, description):
headerPage = HeaderPage(TestCaseKeywords.driver)
sideBarPage = headerPage.selectDaten()
basicVehicleCreation = sideBarPage.createNewVehicle()
basicVehicleCreation.setKennzeichen(registrationno)
basicVehicleCreation.setBeschreibung(description)
TestCaseKeywords.carnumber = basicVehicleCreation.save()
The problem is that when I run test cases, in log I only get result of this whole python function, pass or failed. I can't see at which step it failed- is it at first or second step of this function.
Is there any plugin or other solution for this case to be able to see which exact python function pass or fail? (of course, workaround is to use in TC for every function a keyword but that is not what I prefer)
If you need to "step into" a python defined keyword you need to use python debugger together with RED.
This can be done with any python debugger,if you like to have everything in one application, PyDev can be used with RED.
Follow below help document, if you will face any problems leave a comment here.
RED Debug with PyDev
If you are wanting to know which statement in the python-based keyword failed, you simply need to have it throw an appropriate error. Robot won't do this for you, however. From a reporting standpoint, a python based keyword is a black box. You will have to explicitly add logging messages, and return useful errors.
For example, the call to sideBarPage.createNewVehicle() should throw an exception such as "unable to create new vehicle". Likewise, the call to basicVehicleCreation.setKennzeichen(registrationno) should raise an error like "failed to register the vehicle".
If you don't have control over those methods, you can do the error handling from within your keyword:
#keyword("Create new vehicle with next ${registrationno} and ${description}")
def create_new_vehicle_Simple(self,registrationno, description):
headerPage = HeaderPage(TestCaseKeywords.driver)
sideBarPage = headerPage.selectDaten()
try:
basicVehicleCreation = sideBarPage.createNewVehicle()
except:
raise Exception("unable to create new vehicle")
try:
basicVehicleCreation.setKennzeichen(registrationno)
except:
raise exception("unable to register new vehicle")
...
I try to use eclipse P2 to enable a tool of mine for auto-updating itself on eclipse start up. While doing so, I want to use an UpdateOperation which is only suited to "my" feature with id "my.feature.id". Whenever this query gets issued in an eclipse installation it has an empty result and thus nothing to update.
So: How do I use the QueryUtil right to get a collection containing only my feature for update as input for an UpdateOperation?
The following method is called when wanting to start the update on eclipse start up:
public class P2Util {
public static IStatus checkForUpdates(IProvisioningAgent agent, IProgressMonitor monitor) {
ProvisioningSession session = new ProvisioningSession(agent);
IQuery<IInstallableUnit> query = QueryUtil.createLatestQuery(QueryUtil.createIUQuery("my.feature.id"));
UpdateCheckActivator.info("Update Query Expression: " + query.getExpression());
IProfileRegistry registry= (IProfileRegistry)agent.getService(IProfileRegistry.SERVICE_NAME);
IProfile profile= registry.getProfile(IProfileRegistry.SELF);
IQueryResult<IInstallableUnit> result = profile.query(query, monitor);
Set<IInstallableUnit> unitsForUpdate = result.toUnmodifiableSet();
UpdateOperation operation = new UpdateOperation(session, unitsForUpdate);
}
}
The solution is very simple but confusing at first. The query trying to find features with "my.feature.id" was querying for the wrong id.
In the feature.xml of my feature it is "my.feature.id" but the installable unit gets for unobvious reasons the suffix "feature.group". If one adds this to the id in the query, you get the correct result and the update operation succeeds as expected.
I have created java method to send mail and that mail attaching TestNG emailable-report. Mail and report is sending fine to specified email address.
My issue is I am calling sendmail method at last means when all other tests complete but thing is I am always getting previous last report in mail. So is it like TestNG update report after all class execution?
I want to get latest emailable-report in mail rather than previous last emailable-report.. How can I do that?
I want to get latest emailable-report in mail rather than previous
last emailable-report.. How can I do that?
I think..you're sending mail in your #aftersuite.You're getting previous test emailable-report because the test is currently running and only when it finishes reports will be generated.
I will suggest you to use a continuous Integration server like jenkins because it gives sending emails as a post build option or build tool like Maven or ant to run your tests, then have a post test event to email the results.Maven also provides many plugins to automatically send mails after test execution like Postman mail plugin
Another solution : If you are not willing to use a continuous Integration server(jenkins) or maven or ant
TestNG IReporter listener
Create your own CustomReport by implementing your class to IReporter Interface.This interface has only one method to implement generateReport. This method has all the information of a complete test execution in the List and we can generate report using it.
public class CustomReporter implements IReporter{
#Override
public void generateReport(List<XmlSuite> arg0, List<ISuite> arg1,
String outputDirectory) {
// Second parameter of this method ISuite will contain all the suite executed.
for (ISuite iSuite : arg1) {
//Get a map of result of a single suite at a time
Map<String,ISuiteResult> results = iSuite.getResults();
//Get the key of the result map
Set<String> keys = results.keySet();
//Go to each map value one by one
for (String key : keys) {
//The Context object of current result
ITestContext context = results.get(key).getTestContext();
//results of all the test case will be stored in the context object
//Ex: context.getFailedTests(); will give all failed tests and similarly you can get passed and skipped test results make your own html report using the above data
}
}
}
}
Hope this helps you...Kindly get back if you need any further help
For Windows OS, I've created (for TestNG) in #AfterSuite at the end a method to send mail with attachment and batch file with 2 steps. First step launch mvn clean test to execute all test without previous results. Second step launch test without parameter clean and operates on non existing group :)
Batch is in folder with tests.
start /wait cmd /k "mvn clean test && exit" /secondary /minimized
REM to send email with results in file, whitch was composed after test suite (mvn without: clean)
start /wait cmd /k "mvn test -Dgroups=PhantomGroupNonExist && exit" /secondary /minimized
I am using NUnit to write a test for a class that knows how to serialize itself to XML. The class has lots of properties so the XML fragment produced by the function I'm testing might be very long even with the default state of a new object.
When I run the test in the NUnit test runner and I've purposefully broken the expected returned XML, the test runner only shows a truncated version of the expected and actual string returned from the function that serializes the object to XML. Such as:
MyProject.MyTests.CanCreateObjectAndEdit:
Expected string length 525 but was 1485. Strings differ at index 509.
Expected: "...ffer="False" IsThing="False" /></MyObject>"
But was: "...ffer="False" IsThing="False" /><MySubObject ItemID="60..."
--------------------------------------------^
Is there any way to get NUnit to return the entire expected and actual string? I have NUnit 2.6.3 (the latest release) and I am using the NUnit x86 GUI test runner.
My current workaround is to create a console app, copy the code out of the test, run it and print the output to a debug window, and then paste that output back into my test.
Almost every Assertion method (i.e. Assert.AreEquals) takes a "message" parameter as a third argument.
It is printed only on test failures and it is intended to provide useful information to diagnose a test failure. I think it is exactly what you need.
Hope it helps.
(With apologies for any transcription errors in the code below: I was testing this on a remote computer without copy/paste)
I tested the message parameter as suggested by Manuel.
[Test]
public void LongTest()
{
string s1 = new string('.', 1000);
string s1 = new string('.', 500) + "+" + new string('.', 500);
Assert.That(s1, Is.EqualTo(s2));
}
and got the equivalent result to the one in the question:
Changing the Assert to
Assert.That(s1, Is.EqualTo(s2), s1 + "\r\n\r\n" + s2);
changed the result to
which is possibly less than helpful, especially when the tooltip comes up showing you the entire thing. However, you can right-click on that area in the GUI runner and Copy it, and you do indeed get the whole text copied to the clipboard.