We are using Karate heavily for various projects and though the report generated using karate Reports are more than anyone would need. I am still interested in getting Allure integrated in the mix.
Added allure-junit4 dependency and added allure listener
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<argLine>
-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
<!-- -Dcucumber.options="--plugin io.qameta.allure.cucumberjvm.AllureCucumberJvm"-->
</argLine>
<properties>
<property>
<name>listener</name>
<value>io.qameta.allure.junit4.AllureJunit4</value>
</property>
</properties>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>${aspectj.version}</version>
</dependency>
</dependencies>
</plugin>
Now allure-results is getting created and I can see report but it's almost blank.
How can I get allure report generated on karate based project?
If Allure supports the Cucumber JSON output it should work. Else suggest you take this up with the Allure team.
You can refer to this thread (for Extent): https://github.com/intuit/karate/issues/619
EDIT: Since I refer anyone asking about extending / custom reports to this answer, read on.
In Karate 1.0 onwards, the Results object can be used to get all data about the test results. Also multiple JSON files will be output to the <build>/karate-reports. You can even re-try some tests and merge the results: https://github.com/intuit/karate/wiki/1.0-upgrade-guide#retry-framework-experimental
Also please be aware of changes to the Java hooks, it is called RuntimeHook now: https://github.com/intuit/karate/wiki/1.0-upgrade-guide#hooks
Related
I'm trying setting up mapstruct with my project, I'm used to lombok that does it via a simple jvm agent so I really can't understand how to make mapstruct work.
Here's my pom:
<properties>
<m2e.apt.activation>jdt_apt</m2e.apt.activation>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.version}</version>
<configuration>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
<annotationProcessorPaths>
<path>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<version>${springboot.version}</version>
</path>
<path>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>${lombok.version}</version>
</path>
<path>
<groupId>org.mapstruct</groupId>
<artifactId>mapstruct-processor</artifactId>
<version>${org.mapstruct.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>${springboot.version}</version>
</plugin>
</plugins>
</build>
Please mind that, before mapstruct entered the game, I didn't need this whole maven compile plugin block: everything was working fine. I could build my springboot fat jar with no problems at all, no need to explicitely specify spring and lombok annotation processing...they were very good times.
Now I'm not even sure that I didn't introduce some regressions with the above code but, anyhow, I noticed that mapstruct classes are only generated when I do "mvn package".
I would have expected, like for lombok, for them to be generated automatically each time I saved an object but this does not happen.
Do you have any idea? And can you assure me that that specific build block does not change anything in my spring boot project?
For the code generation to work automatically make sure that you have the m2e-apt plugin installed.
As for having or not having the maven-compiler-plugin block. It is a flavour that the MapStruct teams like to recommend.
MapStruct is an annotation processor, which means that you don't need it during runtime. Therefore, you need 2 dependencies mapstruct where the annotations are located and the mapstruct-processor where the processor is located. The processor has some extra dependencies and contains shaded Freemarker for the code generation.
The use of annotationProcessorPaths allows you to have annotation processor paths and not have conflicts on your classpath, i.e. not accidentally using some internal MapStruct classes in your production code. On top of that, you don't need Spring Boot to package the mapstruct-processor provided dependency for your runtime application.
From the maven-compiler documentation the annotationProcessorPaths means the following:
Classpath elements to supply as annotation processor path. If specified, the compiler will detect annotation processors only in those classpath elements. If omitted, the default classpath is used to detect annotation processors. The detection itself depends on the configuration of annotationProcessors.
So if you really want you can add the mapstruct-processor to your normal dependencies.
I am trying to build a Scala-Maven project on IntelliJ IDEA. Right after creating the project, it says build successful. This is how my pom.xml looks like:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.dbloads.pgms</groupId>
<artifactId>Arts</artifactId>
<version>1.0-SNAPSHOT</version>
<inceptionYear>2008</inceptionYear>
<properties>
<scala.version>2.7.0</scala.version>
</properties>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.specs</groupId>
<artifactId>specs</artifactId>
<version>1.2.5</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.4.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
<args>
<arg>-target:jvm-1.5</arg>
</args>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<configuration>
<downloadSources>true</downloadSources>
<buildcommands>
<buildcommand>ch.epfl.lamp.sdt.core.scalabuilder</buildcommand>
</buildcommands>
<additionalProjectnatures>
<projectnature>ch.epfl.lamp.sdt.core.scalanature</projectnature>
</additionalProjectnatures>
<classpathContainers>
<classpathContainer>org.eclipse.jdt.launching.JRE_CONTAINER</classpathContainer>
<classpathContainer>ch.epfl.lamp.sdt.launching.SCALA_CONTAINER</classpathContainer>
</classpathContainers>
</configuration>
</plugin>
</plugins>
</build>
<reporting>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.4.0</version>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
</plugins>
</reporting>
</project>
Next I tried to add the compiler options in project like below:
Once added and when I click the RUN button, I get the below error message:
[ERROR] Plugin net.alchim31.maven:scala-maven-plugin:3.4.0 or one of its dependencies could not be resolved: Failed to read artifact descriptor for net.alchim31.maven:scala-maven-plugin:jar:3.4.0: Could not transfer artifact net.alchim31.maven:scala-maven-plugin:pom:3.4.0 from/to central (https://repo.maven.apache.org/maven2): PITC-Zscaler-EMEA-Amsterdam3PR.proxy.corporate.ge.com: Unknown host PITC-Zscaler-EMEA-Amsterdam3PR.proxy.corporate.ge.com -> [Help 1]
I am using the jdk version: 1.8.0_172 and I have added the Scala plugin directly from the plugins. Hence it is the latest version of the Scala.
Could anyone let me know how can I fix this problem.
It looks like you need to configure Maven and IntelliJ to use network proxy settings, since it looks like you might be behind a corporate firewall.
Maven has the ability to configure a proxy through its settings (in ~/.m2/settings.xml on Unix-like systems, or %HOME%\.m2\settings.xml on Windows), as follows:
<settings ...>
.
.
<proxies>
<!-- You can have one of these for each possible proxy. -->
<proxy>
<active>true</active>
<!-- Pick some ID for your proxy here. -->
<id>corp-proxy</id>
<!-- Choose your protocol here. E.g. "http", "socks4" or "socks5" -->
<protocol>http</protocol>
<!-- Specify the proxy server name (or IP address) and port of your proxy here. -->
<host>proxy.example.com</host>
<port>8080</port>
<!-- Identify any hosts here that you can access directly. It's unlikely that you'll
need this unless you have a proxy repository (such as Nexus, Artifactory, etc.) on
your corporate network. -->
<nonProxyHosts>www.google.com|*.example.com</nonProxyHosts>
<!-- The following fields are only necessary if required by your proxy. If you need to
enter your own username and password, make sure you do not add this file to version
control! -->
<username>proxyuser</username>
<password>somepassword</password>
</proxy>
</proxies>
.
.
</settings>
Meanwhile, IntelliJ is configured to use proxies through its settings. Refer to this answer for further details. (Note that setting proxy information via the JAVA_OPTS environment variable will work for running any Java/Scala/JVM application that needs to access the Internet via a proxy.)
Alternatively, if your proxy settings are configured correctly or are not required, it might be a temporary network connection issue, so make sure you have an Internet connection and try again. (The exception is a failure by Maven to download the plugin from the Maven central repository.)
BTW, the version of Scala you have specified—2.7.0—is ancient and almost certainly will not work with JDK 8. Either use an older JDK or a more recent version of Scala (the current release is 2.12.6).
Note that if you need to work with the current version of Apache Spark, you must currently use Scala 2.11.x - the most recent release being 2.11.12.
UPDATE:
From your comments, it seems there is some confusion about the roles played by Maven, the scala-maven-plugin, IntelliJ and the IntelliJ Scala plugin, so I'll quickly summarize them here. Please bear with me if I cover topics you're already familiar with...
Maven is a system for building and publishing software. (It actually does a lot more than just that, which is why Maven describes itself as project management software.) It allows developers to specify, in a single place, all of their software's dependencies (third-party libraries used by the software), which Maven downloads as required from the Central Maven Repository—mostly open-source—or from other, private repositories, as required. Further settings control how compilers are configured, tests are run, reports generated, etc.
Maven was developed primarily for development of Java-language projects. The scala-maven-plugin provides support for the Scala language and compiler within Maven. It is this plugin that downloads the version of the Scala compiler specified by your project and uses it to compile and build your sources.
If you look at your Maven project's pom.xml file, you will notice the following lines in the build section:
<project ...>
...
<build>
...
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.4.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
<args>
<arg>-target:jvm-1.5</arg>
</args>
</configuration>
</plugin>
...
</build>
...
</project>
and again in the reporting section:
<project ...>
...
<reporting>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.4.0</version>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
</plugins>
</reporting>
</project>
In both cases, there is a line that reads <scalaVersion>${scala.version}</scalaVersion>, which tells Maven which version of Scala you want to use. The plugin then uses this version of the Scala compiler, and has Maven download it to a cached, local repository on your machine (typically, in C:\Users\{your account}\.m2 on a Windows machine). Note that both Maven and the scala-maven-plugin will ignore any versions of Scala you have installed on your machine.
So which version of Scala is the plugin going to download for you? The value ${scala.version} states that the version number is stored as the value of a property named scala.version. Your pom.xml file also has lines, near the top, that create this property:
<project ...>
...
<properties>
<scala.version>2.7.0</scala.version>
</properties>
...
</project>
So, you can see that you will use version 2.7.0 of the Scala compiler. If you want to use the latest Scala version, you should change this to:
<project ...>
...
<properties>
<scala.version>2.12.6</scala.version>
</properties>
...
</project>
OK, so now you will be using the latest version of the Scala compiler. Now let's move on to IntelliJ...
IntelliJ IDEA is an Integrated Development Environment (IDE), primarily aimed at development with the Java language. It provides syntax highlighting, smart code completion, and other features to simplify the process of writing code. In order to provide those features for the Scala programming language, you need to install its Scala plugin.
You can configure IntelliJ to use any version of Scala that you have installed on your machine. IntelliJ will then know how to compile, build and run your software and can work without using your Maven project object model (POM) file's build definition.
However, one of the reasons for using Maven is to ensure a consistent build environment for developing a project, so that it is not at the whim of whatever each developer may or may not have installed on their machine. For this reason, if a project uses Maven, it's a good idea to tell IntelliJ. That way, IntelliJ can use Maven's pom.xml file to specify the version of the compiler, download dependencies, configure the compiler settings, etc.
So, the above information should help you to get up-and-running with your project, working with your corporation's network proxy and using the latest version of Scala, using Maven and IntelliJ.
This question already has answers here:
Why does format("kafka") fail with "Failed to find data source: kafka." (even with uber-jar)?
(8 answers)
Closed 4 years ago.
I am trying to connect Spark Structured Streaming with kafka and it throws the below error:
Exception in thread "main" java.lang.ClassNotFoundException: Failed to find data source: kafka. Please find packages at ...
Based on the documentation I have added the required dependencies
and my kafka and zookeeper servers are running.
Not sure what the issue is.
Also, I am using it this way
import spark.implicits._
val feedback =spark.readStream.format("kafka").option("kafka.bootstrap.servers", "localhost:2181").option("subscribe", "kafka_input_topic")
.load().as[InputMessage].filter(_.lang.equals("en"))
Any help is appreciated. Thank you
The problem, as you mentioned in your comments, is this:
<scope>provided</scope>
Remove the provided scope for sql-kafka, as it is not provided by the Spark installation.
you could use the kafka data source by the fully-qualified name (not the alias) as follows:
spark.readStream.format("org.apache.spark.sql.kafka010.KafkaSourceProvider").load
The issue is that the necessary jar is not included in CLASSPATH at runtime (not build time).
Based on the documentation you linked to you added the required dependencies to your build definition file (pom.xml or build.sbt or build.gradle), but the exception happens while you try to run the application which is after it is built, doesn't it?
What you miss is that part of the documentation about deployment, i.e. Deploying:
As with any Spark applications, spark-submit is used to launch your application. spark-sql-kafka-0-10_2.11 and its dependencies can be directly added to spark-submit using --packages, such as,
./bin/spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.2.0 ..
You have to add this --packages or you'd have to create an uber-jar that would make the dependency part of your jar file.
If using maven then the following way of building jar with dependencies might solve your issue.
Add the spark dependencies like below:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.2.1</version>
<scope>${spark.scope}</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql-kafka-0-10_2.11</artifactId>
<version>2.2.1</version>
</dependency>
Then configure your maven profiles as below:
<profiles>
<profile>
<id>default</id>
<properties>
<profile.id>dev</profile.id>
<spark.scope>compile</spark.scope>
</properties>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
<profile>
<id>test</id>
<properties>
<profile.id>test</profile.id>
<spark.scope>provided</spark.scope>
</properties>
</profile>
<profile>
<id>online</id>
<properties>
<profile.id>online</profile.id>
<spark.scope>provided</spark.scope>
</properties>
</profile>
</profiles>
Add the followign plugin:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
Then buld your jar using mvn clean install -Ponline -DskipTests. This should solve your issue
My entire pom.xml is below. With this pom I get this error in Eclipse "Plugin execution not covered by lifecycle configuration: org.apache.cxf:cxf-java2ws-plugin:3.1.8:java2ws (execution: process-classes, phase: process-classes)".
Nevertheless, it does work properly. I mean, if I "mvn clean package install" I get the output wsdl file desired.
If I added pluginManagement, the error in Eclipse desapears but I don't get the wsdl file desired neither I get an error in my console. The two closest discussions I found about it was "Publishing wsdl java M2E plugin execution not covered" and "How to solve "Plugin execution not covered by lifecycle configuration" for Spring Data Maven Builds" but I didn't understand them. As far as I can see, the idea is to change to take advantage of
"<lifecycleMappingMetadata>...<action><execute/>".
My straight question is: why does my below pom works when I take away pluginManagement? I guess, not sure, that I am missing a basic knowledgement about the relantionship between pluginManagement and execution. The most relevant part from my question is not what is worng with Eclipse (I found few people saying to ignore it).
I have been using pluginManagement for while but I have never wondering exactly what extra features it adds to my pom. Since now it is failing with java2ws, I am really interested to understand if there is any extra configuration I should add in my pom in order to get it up and running with pluginManagement and goal>java2ws.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>grp</groupId>
<artifactId>art</artifactId>
<packaging>war</packaging>
<version>0.0.1-SNAPSHOT</version>
<name>art Maven Webapp</name>
<url>http://maven.apache.org</url>
<properties>
<jdk.version>1.8</jdk.version>
<cxf.version>3.1.8</cxf.version>
<spring.version>4.3.4.RELEASE</spring.version>
<!-- <maven.compiler.source>1.8</maven.compiler.source> <maven.compiler.target>1.8</maven.compiler.target> -->
</properties>
<dependencies>
<!-- Spring dependencies -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
<version>${spring.version}</version>
</dependency>
<!-- Apache cxf dependencies -->
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-frontend-jaxws</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http</artifactId>
<version>${cxf.version}</version>
</dependency>
<!-- servlet & jsp -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.0.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.servlet.jsp</groupId>
<artifactId>javax.servlet.jsp-api</artifactId>
<version>2.3.1</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
<version>1.1.2</version>
</dependency>
</dependencies>
<build>
<finalName>art</finalName>
<!-- <pluginManagement> -->
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.2</version>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-java2ws-plugin</artifactId>
<version>${cxf.version}</version>
<executions>
<execution>
<id>process-classes</id>
<phase>process-classes</phase>
<configuration>
<className>art.VmxService</className>
<outputFile>${project.basedir}/src/main/resources/VmxService.wsdl</outputFile>
<genWsdl>true</genWsdl>
<verbose>true</verbose>
<address>http://localhost:9080/art/VmxService</address>
</configuration>
<goals>
<goal>java2ws</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<!-- </pluginManagement> -->
</build>
</project>
The pluginManagement section serves a similar purpose like the dependencyManagement section. It defines plugins and their version and configuration defaults, without actually adding them to the maven build lifecycle.
Once the plugin is added in a module it will pick up the configuration from the pluginManagement section.
Also see: Maven: What is pluginManagement?
So if a similar configuration of the same plugin is used in multiple modules you can collect them together in one place. If the plugin is only used in one module I prefer to just put it in there directly in the build. But both ways work.
Remember you also need to add the plugin to the build.plugins - simply having them in pluginManagement does nothing.
The warning in eclipse relates more to the life-cycle of your IDE. It differs a bit from the maven lifecycle and in some cases it cannot detect (or could not?) at what moment a plugin is supposed to run. Some plugins also cannot execute without a maven project. So I'm never sure what that lifecycle-mapping plugin tries to solve :/
Anyways: if you generate the classes using a maven build and this works for you (not having that done when telling eclipse to 'build' the project without maven) you're good.
I thought that information (the lifecycle mapping) is nowadays baked into the plugins directly and read by the m2eclipse plugin. I've seen such xml files in some plugins. So the lifecycle-mapping plugin might not be required anymore at all.
My goal is to generate a swagger definition like this from my Jersey ressources.
To get this, my pom.xml looks like :
<plugin>
<groupId>com.github.kongchen</groupId>
<artifactId>swagger-maven-plugin</artifactId>
<version>2.3.1</version>
<configuration>
<apiSources>
<apiSource>
<locations>com.rest.resources</locations>
<apiVersion>1.0</apiVersion>
<swaggerDirectory>${basedir}/src/main/webapp/docs</swaggerDirectory>
</apiSource>
</apiSources>
</configuration>
</plugin>
I had also the swagger/jersey dependency:
<dependency>
<groupId>com.wordnik</groupId>
<artifactId>swagger-jaxrs_2.10</artifactId>
<version>1.3.10</version>
<scope>compile</scope>
</dependency>
The json is well generated, BUT i'm getting one json file for each of my resources instead of only one
What i'm missing
You're missing nothing, it's working as expected.
Even in the link you provided, there's a file per resource.
Check out these links:
http://petstore.swagger.wordnik.com/api/api-docs/pet
http://petstore.swagger.wordnik.com/api/api-docs/user
http://petstore.swagger.wordnik.com/api/api-docs/store