SonarQube cannot parse TEST-report.xml which contains any failures - swift

How to send the XML report to SonarQube? I mean, while the TEST-report.xml file contains any failures, the import operation fails. I got an error:
Running SonarQube using SonarQube Runner.17:18:18.452 ERROR: Error during SonarScanner execution
java.lang.NullPointerException
at java.text.DecimalFormat.parse(DecimalFormat.java:2030)
at java.text.NumberFormat.parse(NumberFormat.java:383)
...
The TEST-report.xml file (JUnit) contains something like:
<?xml version='1.0' encoding='UTF-8'?>
<testsuites name='X UITests.xctest' tests='12' failures='3'>
<testsuite name='X.MoreViewTests' tests='1' failures='1'>
<testcase classname='X.MoreViewTests' name='testSucceed_allAction'>
<failure message='XCTAssertTrue failed'>X UITests/Cases/Bottom Navigation/MoreViewTests.swift:212</failure>
</testcase>
</testsuite>
<testsuite name='X.OrderViewTests' tests='1' failures='0'>
<testcase classname='X.OrderViewTests' name='testSucceed_allAction' time='68.570'/>
</testsuite>
</testsuites>
But when I removed the failure lines, becomes:
<?xml version='1.0' encoding='UTF-8'?>
<testsuites name='X UITests.xctest' tests='12' failures='3'>
<testsuite name='X.OrderViewTests' tests='1' failures='0'>
<testcase classname='X.OrderViewTests' name='testSucceed_allAction' time='68.570'/>
</testsuite>
</testsuites>
It works. Any idea? Thanks.

Your XML report is invalid (doesn't meet the Surefire XML format requirements).
testsuite required parameters:
name - Full class name of the test for non-aggregated testsuite documents. Class name without the package for aggregated testsuites documents
tests - The total number of tests in the suite
failures - The total number of tests in the suite that failed. A failure is a test which the code has explicitly failed by using the mechanisms for that purpose. e.g., via an assertEquals
errors - The total number of tests in the suite that errored. An errored test is one that had an unanticipated problem. e.g., an unchecked throwable; or a problem with the implementation of the test
skipped - The total number of ignored or skipped tests in the suite
The exception starts with DecimalFormat.parse
java.lang.NullPointerException
at java.text.DecimalFormat.parse(DecimalFormat.java:2030)
at java.text.NumberFormat.parse(NumberFormat.java:383)
which means that the XML report parser couldn't parse a number. You have two missing parameters: errors and skipped. I don't know how you generate these reports (which JUnit version etc.), but probably the used tool is outdated or miss-configured.

Related

Azure devops not showing the testsuite name under a testsuite in the report(xml) section of the pipeline

I am generating xml reports via postman in the azure devops pipeline. My Postman collection structure is below:
Collection name->Folder name->Requests(Containing test cases)
When the xml report gets generated then it looks like below:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Suite1" tests="2" time="1.317">
<testsuite name="Folder1 / Request1" id="abc" timestamp="time/date" tests="1" failures="1" errors="0" time="0.658">
<testcase name="Verify that the success status code is 200" time="0.179" classname="ABC">
<failure type="AssertionFailure" message="expected response to have status code 200 but got 401">
I expect from Azure devops to show the generated & categorize xml report as:
Suite1->Folder1->Test cases inside Request1
but it actually shows like below:
Suite1->Test cases inside Request1
I expect all the test cases to be inside "Folder1" instead of Suite1. SomeHow it is skipping the "Folder1" inside "Suite1". Let me know if needed more explanation.

XSLT streaming not streaming

I'm using Saxon-EE for the purpose of streaming XSLT transformation of large XML. The transformation works fine but it seems it's not really streaming since the java.exe process is inflating: for a 100 MB XML, process memory increases ~1GB. This is the XSLT:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:fo="http://www.w3.org/1999/XSL/Format"
xmlns:bb="urn:xx-zz-1.1"
xmlns:aa="urn:xx-yy-1.1">
<xsl:mode streamable="yes"/>
<xsl:output method="text" omit-xml-declaration="yes" indent="no"/>
<xsl:template match="/">
<xsl:for-each select="aa:LevelOne/aa:LevelTwo">
<xsl:iterate select="bb:LevelThree! copy-of(.)">
<xsl:value-of select="concat(bb:fieldOne,',',bb:fieldTwo,'
')"/>
</xsl:iterate>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>
This is the XML:
<?xml version="1.0" encoding="utf-8"?>
<aa:LevelOne xmlns="urn:xx-zz-1.1" xmlns:aa="urn:xx-yy-1.1">
<aa:LevelTwo xmlns="urn:xx-zz-1.1" xmlns:aa="urn:xx-yy-1.1">
<LevelThree xmlns="urn:xx-zz-1.1">
<fieldOne>f1</fieldOne>
<fieldTwo>f2</fieldTwo>
</LevelThree>
<!-- Level three is repeated many times -->
</aa:LevelTwo>
</aa:LevelOne>
I would like to know if there (& what) is a problem with the XSLT above.
The code I use:
net.sf.saxon.s9api.Processor processor = new net.sf.saxon.s9api.Processor(true);
processor.setConfigurationProperty(Feature.STREAMABILITY, "standard");
XsltCompiler compiler = processor.newXsltCompiler();
XsltExecutable stylesheet = compiler.compile(new StreamSource(stylesheetFile));
Serializer out = processor.newSerializer(outputCsvFile);
Xslt30Transformer transformer = stylesheet.load30();
transformer.applyTemplates(new StreamSource(xmlFile), out);
EDIT: Fixed the XSLT so it compiles & added XML example.
Remark: using command java -cp "<path>\test;<path>\saxon9ee.jar" com.example.test.Test -t does not ouput additional info (only the printlns in the code). java -cp "<path>\test;<path>\saxon9ee.jar" -t com.example.test.Test outputs: Unrecognized option: -Xt Error: Could not create the Java Virtual Machine. If I change the XSLT to non-streamable rule e.g. remove the iterate line, program outputs Template rule is not streamable, also without -t option. In this case if I remove the streamability requirement from code/xslt, the error goes away.
Thanks.
Probably the most likely reason Saxon would fall back to non-streaming mode is that it hasn't located a Saxon-EE license. The easiest way to test that is (unintuitively!) by calling processor.isSchemaAware() - that will only be true if you're running Saxon-EE code with a recognized license, which is exactly the same condition to enable streaming.
If it hasn't found a license, the Saxon documentation includes a section on troubleshooting license problems at http://www.saxonica.com/documentation/index.html#!about/license
Also, try it from the command line with option -t; that will give you more information (a) about streaming, and (b) about loading of license files.
I think, if the data is as simple and as regular as shown in the question, then you can avoid the use of copy-of() and simply use
<xsl:iterate select="bb:LevelThree">
<xsl:value-of select="bb:*" separator=","/>
<xsl:text>
</xsl:text>
</xsl:iterate>
That in a quick test shows a reduced memory consumption compared to your posted approach.
As for your posted approach not using streaming with Saxon EE 9.9, I have tested the posted XSLT and the input sample with Saxon 9.9 EE from the command line with the -t option and it shows the input is streamed.
I also think the Java code shown is fine to process the file with streaming with Saxon EE.
For a detailed analysis of the memory consumption and any problems you encounter with that it might be better to raise an issue with all details on the Saxonica support site. I am not sure how the memory info Saxon outputs relates exactly to the one you say you see for java.exe.

How do I have a meaningful error message from powershell script running in TeamCity build?

I have a TeamCity build and one of the steps is an MSBuild invokation of a .proj file:
Runner type: MSBuild
Build file path: TestProject.proj
Targets: Test
inside the .proj I have targets:
<Target Name="DeployTestService">
<Message Text="Deploying test service" />
<Exec Command="powershell -Command "& { myUsefulPsScript.ps }"" />
</Target>
<Target Name="Test">
// other stuff, then
<CallTarget Targets="DeployTestService" />
</Target>
and it works good most of the time. Yet if the powershell script fails (an unhandled exception is thrown and I can see its text in the full log) and exits with non-zero code I see the following in Teamcity build results:
Tests passed: (some number); exit code 1
and the build tree just says:
Build failure condition (1)
Process exited with code 1
[Time]Process exited with code 1
and until I get the full log I don't really know which step failed and how exactly. It's not about fixing the initial problem it's about locating the faulty step faster.
Is there a way to make Teamcity say something like "runner in step N failed with this error message [message which I see in the full log]"?
You need to use the Teamcity output format. For example, every time a new test starts, print:
##teamcity[testStarted name='foo|'s test']
This way Teamcity will know what was the last step or the last test before the app crashed and it will be able to format the output accordingly.

How to generate test overview in Frisby

As mentioned in frisby official document (http://frisbyjs.com/) , I am using --junitreport something like following
jasmine-node ./demo/validation_spec.js/ --junitreport --output
C:\Users\Administrator\Documents\script/Reports
which is generating 15 xml files. As I am calling 15 time post using for loop. File contains data like following
<?xml version="1.0" encoding="UTF-8" ?>
<testsuites>
<testsuite name="Frisby Test:blank action " errors="0" tests="1" failures="0" time="0.284" timestamp="2017-03-21T13:55:15">
<testcase classname="Frisby Test: blank action " name="
[ POST https://localhost:8443/api/v2/settings/pointwise ]" time="0.282"></testcase>
</testsuite>
</testsuites>
so my difficulty is that I need to read each and every file to check if tests is pass or not. I want some kind of overview page like http://maven.apache.org/surefire/maven-surefire-report-plugin/surefire-report.html
which contain information how many test passed, fail etc. Is it possible to get that.

How to get ScalaTest to populate test runtimes in the xml report?

We use ScalaTest to run tests and display the results in Jenkins. It works great, but for some reason ScalaTest` does not populate the test runtime.
I'm adding test options like this:
Tests.Argument("-oD", "-u", "target/test-reports")
But the XML file does not have timings (see time is always 0):
<testcase classname="beekeeper.warehouse.jdbc.JDBCWarehouseTest" name="testListTables" time="0.0">
</testcase>
<testcase classname="beekeeper.warehouse.jdbc.JDBCWarehouseTest" name="testGetDatabases" time="0.0">
</testcase>
<testcase classname="beekeeper.warehouse.jdbc.JDBCWarehouseTest" name="testSchema" time="0.0">
</testcase>
Do I need to do something else to have it report this, or is it not supported?
I can't reproduce this issue. Using sbt 1.1.6 and ScalaTest 3.0.5 I see non-zero time field:
<testcase classname="example.HelloSpec" name="The Hello object should say hello" time="0.016">