#Karate Gatling is not generating report when i hit the endpoint once - scala

My Gatling Simulation class,
class <MyClass> extends Simulation {
before {
println("Simulation is about to start!")
}
val smapleTest = scenario("test").exec(karateFeature("classpath:demo/get-user.feature"))
setUp(
smapleTest.inject(rampUsers(1) over (10 seconds))).maxDuration(1 minutes)
//).assertions(global.responseTime.mean.lt(35))
after {
println("Simulation is finished!")
}
}
My get-user.feature file,
Scenario Outline: Hit wskadmin url
Given http://172.17.0.1:5984/whisk_local_subjects/guest
And header Authorization = AdminAuth
And header Content-Type = 'application/json'
When method get
Then status <stat>
* print result
Examples:
| stat |
| 200 |
When i run the simulation class, below console logs i am getting:
Simulation com.karate.openwhisk.performance.SmokePerformanceTest started...
13:20:48.877 [GatlingSystem-akka.actor.default-dispatcher-5] INFO i.gatling.core.controller.Controller - InjectionStopped expectedCount=1
13:20:49.473 [GatlingSystem-akka.actor.default-dispatcher-4] INFO com.intuit.karate - karate.env system property was: null
13:20:49.525 [GatlingSystem-akka.actor.default-dispatcher-7] INFO com.intuit.karate - [print] I am here in get-user
13:20:49.706 [GatlingSystem-akka.actor.default-dispatcher-4] DEBUG com.intuit.karate - request:
1 > GET http://172.17.0.1:5984/whisk_local_subjects/guest
1 > Accept-Encoding: gzip,deflate
1 > Authorization: Basic d2hpc2tfYWRtaW46c29tZV9wYXNzdzByZA==
1 > Connection: Keep-Alive
1 > Content-Type: application/json
1 > Host: 172.17.0.1:5984
1 > User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_144)
13:20:49.741 [GatlingSystem-akka.actor.default-dispatcher-4] DEBUG com.intuit.karate - response time in milliseconds: 34
1 < 200
Note: Here i am getting the response in 34 mili seconds, but gating is unable to generate the report. Below is the error message i am getting
Error:
Generating reports...
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.gatling.mojo.MainWithArgsInFile.runMain(MainWithArgsInFile.java:50)
at io.gatling.mojo.MainWithArgsInFile.main(MainWithArgsInFile.java:33)
Caused by: java.lang.UnsupportedOperationException: There were no requests sent during the simulation, reports won't be generated
at io.gatling.charts.report.ReportsGenerator.generateFor(ReportsGenerator.scala:48)
at io.gatling.app.RunResultProcessor.generateReports(RunResultProcessor.scala:76)
at io.gatling.app.RunResultProcessor.processRunResult(RunResultProcessor.scala:55)
at io.gatling.app.Gatling$.start(Gatling.scala:68)
at io.gatling.app.Gatling$.fromArgs(Gatling.scala:45)
at io.gatling.app.Gatling$.main(Gatling.scala:37)
at io.gatling.app.Gatling.main(Gatling.scala)
... 6 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.199 s
[INFO] Finished at: 2018-07-24T13:20:50+05:30
[INFO] Final Memory: 30M/332M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.gatling:gatling-maven-plugin:2.2.4:test (default-cli) on project
openwhisk: Gatling failed.: Process exited with an error: 255 (Exit
value: 255) -> [Help 1]
But if i run the same simulation file simple change in feature file as below
Scenario Outline: Hit wskadmin url
Given http://172.17.0.1:5984/whisk_local_subjects/guest
And header Authorization = AdminAuth
And header Content-Type = 'application/json'
When method get
Then status <stat>
* print result
Examples:
| stat |
| 200 |
| 200 |
Then gatling generates the report.
Please help me someone what is the root cause.

Thank you for your interest in karate-gatling and the very detailed report.
This is a bug, which we have fixed and just made a release for.
Can you upgrade your karate-gatling version to 0.8.0.1 and let me know how it goes ?

Related

spark elastic search throwing 403 forbidden error

When I tried to connect the elasticsearch from spark using basic authentication to create a new index got below error.
Error from elastic search is not giving full error information to debug more
org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: [HEAD] on [devl_test_index] failed; server[https://<elasticServerHost>:9200] returned [403|Forbidden:]
at org.elasticsearch.hadoop.rest.RestClient.checkResponse(RestClient.java:477)
at org.elasticsearch.hadoop.rest.RestClient.executeNotFoundAllowed(RestClient.java:447)
at org.elasticsearch.hadoop.rest.RestClient.exists(RestClient.java:539)
at org.elasticsearch.hadoop.rest.RestClient.indexExists(RestClient.java:534)
at org.elasticsearch.hadoop.rest.RestClient.touch(RestClient.java:545)
at org.elasticsearch.hadoop.rest.RestRepository.touch(RestRepository.java:364)
at org.elasticsearch.hadoop.rest.RestService.initSingleIndex(RestService.java:660)
at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:636)
at org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:65)
at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:101)
at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:101)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:109)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Code used to connect:
Sbt dependency: "org.elasticsearch" % "elasticsearch-hadoop" % "7.5.0"
import org.elasticsearch.spark.sql._
val spark = SparkSession.builder().appName("SparkJDBC")
.enableHiveSupport()
.config("spark.es.port","9200")
.config("spark.es.nodes", "<elasticServerHost>")
.config("spark.es.nodes.wan.only","true")
.config("spark.es.net.ssl","true")
.config("spark.es.net.http.auth.user","USERNAME")
.config("spark.es.net.http.auth.pass","PASSWRD")
.master("local[*]")
.getOrCreate()
val df = spark.sql("select * from employee")
df.saveToEs("devl_test_index")
This error will be thrown from elastic search server when the user tries to access the index instead of assigned group.
In my case, my user group has access to the index which starts with employee* but I tried to access the index starts with devl*
If you run the spark in debug mode you will get the actual error stack trace with more info like below:
20/02/19 10:38:57 DEBUG wire.header: << "HTTP/1.1 403 Forbidden[\r][\n]"
20/02/19 10:38:57 DEBUG wire.header: << "HTTP/1.1 403 Forbidden[\r][\n]"
20/02/19 10:38:57 DEBUG wire.header: << "content-type: application/json; charset=UTF-8[\r][\n]"
20/02/19 10:38:57 DEBUG wire.header: << "content-length: 259[\r][\n]"
20/02/19 10:38:57 DEBUG wire.header: << "[\r][\n]"
20/02/19 10:38:57 DEBUG wire.content: << "{"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:admin/aliases/get] is unauthorized for user [rdsuser]"}],"type":"security_exception","reason":"action [indices:admin/aliases/get] is unauthorized for user [USERNAME]"},"status":403}"
20/02/19 10:38:57 DEBUG sql.EsDataFrameWriter: Provided index name [devl_test_index] is not an alias. Reason: [org.elasticsearch.hadoop.rest.EsHadoopRemoteException: security_exception: action [indices:admin/aliases/get] is unauthorized for user [USERNAME]
null]
Internally spark will check for the existence of index name before it creates a new one. In this process, it will hit the alias API /_all/_alias/devl_test_index
Error response:
{
"error": {
"root_cause": [
{
"type": "security_exception",
"reason": "action [indices:admin/aliases/get] is unauthorized for user [USERNAME]"
}
],
"type": "security_exception",
"reason": "action [indices:admin/aliases/get] is unauthorized for user [USERNAME]"
},
"status": 403
}

akka-http no stack trace or details on error

I got a structure which can basically be summarized as:
outside user makes a rest request to akka-http server
akka-http makes a request(query?) to a (some)data source using asynchttpclient
akka-http transforms the result from asynchttpclient and serves it back to user
At some point I am getting an error from akka which tells me almost nothing. This error happens right after the asynchttpclient returns me some results. (I can infact at this point print the results on the log, they are there parsed from json etc.. but akka had already errored out)
Even in debug logging level I got no decipherable error message from akka or a stacktrace.
only message I got is:
2017-03-24 17:22:55 INFO CompanyRepository:111 - search company with name:"somecompanyname"
2017-03-24 17:22:55 INFO CompanyRepository:73 - [QUERY TIME]: 527ms
[ERROR] [03/24/2017 17:22:55.951] [company-api-system-akka.actor.default-dispatcher-3] [akka.actor.ActorSystemImpl(company-api-system)] Error during processing of request: 'requirement failed'. Completing with 500 Internal Server Error response.
This error message is the only thing I get. Relevant parts of my config:
akka {
loglevel = "DEBUG"
# edit -- tested with sl4jlogger with no change
#loggers = ["akka.event.slf4j.Slf4jLogger"]
#logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
parsing {
max-content-length = 800m
max-chunk-size = 100m
}
server {
server-header = akka-http/${akka.http.version}
idle-timeout = 120 s
request-timeout = 120 s
bind-timeout = 10s
max-connections = 1024
pipelining-limit = 32
verbose-error-messages = on
}
client {
user-agent-header = akka-http/${akka.http.version}
}
host-connection-pool {
max-connections = 4
}
}
akka.http.routing {
verbose-error-messages = on
}
Anyone knows if I can make akka to spit out more details about what/where the error is occurring?
Edit: I realized I do NOT get this same error on resultsets which are smaller in size. <- ignore
Edit 2:
Added akka.loglevel = DEBUG, spits out a lot more noise but still not detail about the actual error.
Converted asynchttpclient to akka quickly to rule out AHC
I already had a wrapper around my query to time it, added some logging there trying to pinpoint when exactly the error is happening.
def queryTimer[ R <: Future[ Any ] ]( block: => R ): R = {
val t0 = System.currentTimeMillis()
val result = block
result.onComplete { maybeResult =>
val t1 = System.currentTimeMillis()
logger.info( "[QUERY TIME]: " + ( t1 - t0 ) + "ms" )
maybeResult match {
case Success(some) =>
logger.info( "successful feature:")
logger.info( FormattedString.prettyPrint(some))
case Failure(someFailure) =>
logger.info( "failed feature:")
logger.debug( FormattedString.prettyPrint(someFailure))
}
}
result
}
resulting log:
2017-03-28 13:19:10 INFO CompanyRepository:111 - search company with name:"some company"
[DEBUG] [03/28/2017 13:19:10.497] [company-api-system-akka.actor.default-dispatcher-2] [EventStream(akka://xca-api-actor-system)] logger log1-Logging$DefaultLogger started
[DEBUG] [03/28/2017 13:19:10.497] [company-api-system-akka.actor.default-dispatcher-2] [EventStream(akka://xca-api-actor-system)] Default Loggers started
[DEBUG] [03/28/2017 13:19:10.613] [company-api-system-akka.actor.default-dispatcher-2] [AkkaSSLConfig(akka://xca-api-actor-system)] Initializing AkkaSSLConfig extension...
[DEBUG] [03/28/2017 13:19:10.613] [company-api-system-akka.actor.default-dispatcher-2] [AkkaSSLConfig(akka://xca-api-actor-system)] buildHostnameVerifier: created hostname verifier: com.typesafe.sslconfig.ssl.DefaultHostnameVerifier#779e2339
[DEBUG] [03/28/2017 13:19:10.633] [xca-api-actor-system-akka.actor.default-dispatcher-3] [akka://xca-api-actor-system/user/pool-master/PoolInterfaceActor-0] (Re-)starting host connection pool to localhost:27474
[DEBUG] [03/28/2017 13:19:10.727] [xca-api-actor-system-akka.actor.default-dispatcher-3] [akka://xca-api-actor-system/system/IO-TCP/selectors/$a/0] Resolving localhost before connecting
[DEBUG] [03/28/2017 13:19:10.740] [xca-api-actor-system-akka.actor.default-dispatcher-4] [akka://xca-api-actor-system/system/IO-DNS] Resolution request for localhost from Actor[akka://xca-api-actor-system/system/IO-TCP/selectors/$a/0#-815754478]
[DEBUG] [03/28/2017 13:19:10.749] [xca-api-actor-system-akka.actor.default-dispatcher-4] [akka://xca-api-actor-system/system/IO-TCP/selectors/$a/0] Attempting connection to [localhost/127.0.0.1:27474]
[DEBUG] [03/28/2017 13:19:10.751] [xca-api-actor-system-akka.actor.default-dispatcher-4] [akka://xca-api-actor-system/system/IO-TCP/selectors/$a/0] Connection established to [localhost:27474]
2017-03-28 13:19:10 INFO CompanyRepository:73 - [QUERY TIME]: 376ms
2017-03-28 13:19:10 INFO CompanyRepository:77 - successful feature:
[ERROR] [03/28/2017 13:19:10.896] [company-api-system-akka.actor.default-dispatcher-7] [akka.actor.ActorSystemImpl(company-api-system)] Error during processing of request: 'requirement failed'. Completing with 500 Internal Server Error response.
2017-03-28 13:19:10 INFO CompanyRepository:78 - SearchResult(List(
( prettyprint output here!!! lots and lots of legit result, json parsed succcesfully into a bunch of case classes)
as you can see my logging format and akkas' are different, the ERROR is coming from akka with do details, while everything looks like working.
Edit 3: logs with sleep in between calls
new query timer function with sleeps
def queryTimer[ R <: Future[ Any ] ]( block: => R ): R = {
val t0 = System.currentTimeMillis()
val result = block
result.onComplete { maybeResult =>
val t1 = System.currentTimeMillis()
logger.info( "[QUERY TIME]: " + ( t1 - t0 ) + "ms" )
maybeResult match {
case Success(some) =>
Thread.sleep(500)
logger.info( "successful feature:")
Thread.sleep(500)
logger.info( FormattedString.prettyPrint(some))
Thread.sleep(500)
logger.info("we are there!")
case Failure(someFailure) =>
logger.info( "failed feature:")
logger.debug( FormattedString.prettyPrint(someFailure))
}
}
result
}
logs with sleeps
[DEBUG] [03/30/2017 11:11:58.629] [xca-api-actor-system-akka.actor.default-dispatcher-7] [akka://xca-api-actor-system/system/IO-TCP/selectors/$a/0] Attempting connection to [localhost/127.0.0.1:27474]
[DEBUG] [03/30/2017 11:11:58.631] [xca-api-actor-system-akka.actor.default-dispatcher-7] [akka://xca-api-actor-system/system/IO-TCP/selectors/$a/0] Connection established to [localhost:27474]
11:11:59.442 [pool-2-thread-1] DEBUG o.a.netty.channel.DefaultChannelPool - Closed 0 connections out of 0 in 0 ms
11:11:59.496 [pool-1-thread-1] DEBUG o.a.netty.channel.DefaultChannelPool - Closed 0 connections out of 0 in 0 ms
11:12:00.250 [ForkJoinPool-2-worker-15] INFO c.s.s.r.neo4j.CompanyRepository - [QUERY TIME]: 1880ms
[ERROR] [03/30/2017 11:12:00.265] [company-api-system-akka.actor.default-dispatcher-3] [akka.actor.ActorSystemImpl(company-api-system)] Error during processing of request: 'requirement failed'. Completing with 500 Internal Server Error response.
11:12:00.543 [pool-2-thread-1] DEBUG o.a.netty.channel.DefaultChannelPool - Closed 0 connections out of 0 in 0 ms
11:12:00.597 [pool-1-thread-1] DEBUG o.a.netty.channel.DefaultChannelPool - Closed 0 connections out of 0 in 0 ms
11:12:00.752 [ForkJoinPool-2-worker-15] INFO c.s.s.r.neo4j.CompanyRepository - successful feature:
11:12:01.645 [pool-2-thread-1] DEBUG o.a.netty.channel.DefaultChannelPool - Closed 0 connections out of 0 in 0 ms
11:12:01.697 [pool-1-thread-1] DEBUG o.a.netty.channel.DefaultChannelPool - Closed 0 connections out of 0 in 0 ms
11:12:01.750 [ForkJoinPool-2-worker-15] INFO c.s.s.r.neo4j.CompanyRepository - SearchResult(List( "lots of legit result here"
11:12:02.281 [ForkJoinPool-2-worker-15] INFO c.s.s.r.neo4j.CompanyRepository - we are there!
Edit 4 and solution!
Apparently the default exception handler does not print a stack trace! overriding the exception handler with a very basic catch all:
implicit def myExceptionHandler: ExceptionHandler =
ExceptionHandler {
case e: Exception => {
logger.info("---------------- exception log start")
logger.error(e.getMessage, e)
logger.error("cause" , e.getCause)
logger.error("cause" , e.getStackTraceString )
logger.info( FormattedString.prettyPrint(e))
logger.info("---------------- exception log end")
Directives.complete("server made a boo boo")
}
}
results in a stack trace that befuddles the sh*t out of me!!
11:42:04.634 [company-api-system-akka.actor.default-dispatcher-2] INFO c.stepweb.scarifgate.CompanyApiApp$ - ---------------- exception log start
11:42:04.640 [company-api-system-akka.actor.default-dispatcher-2] ERROR c.stepweb.scarifgate.CompanyApiApp$ - requirement failed
java.lang.IllegalArgumentException: requirement failed
at scala.Predef$.require(Predef.scala:212) ~[scala-library-2.11.8.jar:na]
at spray.json.BasicFormats$StringJsonFormat$.write(BasicFormats.scala:121) ~[spray-json_2.11-1.3.2.jar:na]
at spray.json.BasicFormats$StringJsonFormat$.write(BasicFormats.scala:119) ~[spray-json_2.11-1.3.2.jar:na]
at spray.json.ProductFormats$class.productElement2Field(ProductFormats.scala:46) ~[spray-json_2.11-1.3.2.jar:na]
at com.stepweb.scarifgate.services.CompanyService.productElement2Field(CompanyService.scala:14) ~[classes/:na]
at spray.json.ProductFormatsInstances$$anon$3.write(ProductFormatsInstances.scala:73) ~[spray-json_2.11-1.3.2.jar:na]
at spray.json.ProductFormatsInstances$$anon$3.write(ProductFormatsInstances.scala:68) ~[spray-json_2.11-1.3.2.jar:na]
at spray.json.PimpedAny.toJson(package.scala:39) ~[spray-json_2.11-1.3.2.jar:na]
at spray.json.CollectionFormats$$anon$1$$anonfun$write$1.apply(CollectionFormats.scala:26) ~[spray-json_2.11-1.3.2.jar:na]
at spray.json.CollectionFormats$$anon$1$$anonfun$write$1.apply(CollectionFormats.scala:26) ~[spray-json_2.11-1.3.2.jar:na]
at scala.collection.immutable.List.map(List.scala:273) ~[scala-library-2.11.8.jar:na]
at spray.json.CollectionFormats$$anon$1.write(CollectionFormats.scala:26) ~[spray-json_2.11-1.3.2.jar:na]
at spray.json.CollectionFormats$$anon$1.write(CollectionFormats.scala:25) ~[spray-json_2.11-1.3.2.jar:na]
at spray.json.ProductFormats$class.productElement2Field(ProductFormats.scala:46) ~[spray-json_2.11-1.3.2.jar:na]
at com.stepweb.scarifgate.services.CompanyService.productElement2Field(CompanyService.scala:14) ~[classes/:na]
at spray.json.ProductFormatsInstances$$anon$1.write(ProductFormatsInstances.scala:30) ~[spray-json_2.11-1.3.2.jar:na]
at spray.json.ProductFormatsInstances$$anon$1.write(ProductFormatsInstances.scala:26) ~[spray-json_2.11-1.3.2.jar:na]
at akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport$$anonfun$sprayJsonMarshaller$1.apply(SprayJsonSupport.scala:62) ~[akka-http-spray-json_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport$$anonfun$sprayJsonMarshaller$1.apply(SprayJsonSupport.scala:62) ~[akka-http-spray-json_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.Marshaller$$anonfun$compose$1$$anonfun$apply$15.apply(Marshaller.scala:73) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.Marshaller$$anonfun$compose$1$$anonfun$apply$15.apply(Marshaller.scala:73) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.Marshaller$$anon$1.apply(Marshaller.scala:92) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.GenericMarshallers$$anonfun$optionMarshaller$1$$anonfun$apply$1.apply(GenericMarshallers.scala:19) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.GenericMarshallers$$anonfun$optionMarshaller$1$$anonfun$apply$1.apply(GenericMarshallers.scala:18) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.Marshaller$$anon$1.apply(Marshaller.scala:92) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.PredefinedToResponseMarshallers$$anonfun$fromStatusCodeAndHeadersAndValue$1$$anonfun$apply$5.apply(PredefinedToResponseMarshallers.scala:58) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.PredefinedToResponseMarshallers$$anonfun$fromStatusCodeAndHeadersAndValue$1$$anonfun$apply$5.apply(PredefinedToResponseMarshallers.scala:57) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.Marshaller$$anon$1.apply(Marshaller.scala:92) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.Marshaller$$anonfun$compose$1$$anonfun$apply$15.apply(Marshaller.scala:73) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.Marshaller$$anonfun$compose$1$$anonfun$apply$15.apply(Marshaller.scala:73) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.Marshaller$$anon$1.apply(Marshaller.scala:92) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.ToResponseMarshallable$$anonfun$1$$anonfun$apply$1.apply(ToResponseMarshallable.scala:29) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.ToResponseMarshallable$$anonfun$1$$anonfun$apply$1.apply(ToResponseMarshallable.scala:29) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.Marshaller$$anon$1.apply(Marshaller.scala:92) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.GenericMarshallers$$anonfun$futureMarshaller$1$$anonfun$apply$3$$anonfun$apply$4.apply(GenericMarshallers.scala:33) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.marshalling.GenericMarshallers$$anonfun$futureMarshaller$1$$anonfun$apply$3$$anonfun$apply$4.apply(GenericMarshallers.scala:33) ~[akka-http_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.util.FastFuture$.akka$http$scaladsl$util$FastFuture$$strictTransform$1(FastFuture.scala:41) ~[akka-http-core_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.util.FastFuture$$anonfun$transformWith$extension1$1.apply(FastFuture.scala:51) [akka-http-core_2.11-10.0.0.jar:10.0.0]
at akka.http.scaladsl.util.FastFuture$$anonfun$transformWith$extension1$1.apply(FastFuture.scala:50) [akka-http-core_2.11-10.0.0.jar:10.0.0]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) [scala-library-2.11.8.jar:na]
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55) [akka-actor_2.11-2.4.16.jar:na]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91) [akka-actor_2.11-2.4.16.jar:na]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91) [akka-actor_2.11-2.4.16.jar:na]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91) [akka-actor_2.11-2.4.16.jar:na]
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72) [scala-library-2.11.8.jar:na]
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90) [akka-actor_2.11-2.4.16.jar:na]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39) [akka-actor_2.11-2.4.16.jar:na]
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415) [akka-actor_2.11-2.4.16.jar:na]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library-2.11.8.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library-2.11.8.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library-2.11.8.jar:na]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library-2.11.8.jar:na]
11:42:04.640 [company-api-system-akka.actor.default-dispatcher-2] ERROR c.stepweb.scarifgate.CompanyApiApp$ - cause
11:42:04.641 [company-api-system-akka.actor.default-dispatcher-2] ERROR c.stepweb.scarifgate.CompanyApiApp$ - cause
11:42:04.644 [company-api-system-akka.actor.default-dispatcher-2] INFO c.stepweb.scarifgate.CompanyApiApp$ - java.lang.IllegalArgumentException: requirement failed
11:42:04.644 [company-api-system-akka.actor.default-dispatcher-2] INFO c.stepweb.scarifgate.CompanyApiApp$ - ---------------- exception log end
so... the exception is caused here in spray.json.BasicFormats
implicit object StringJsonFormat extends JsonFormat[String] {
def write(x: String) = {
require(x ne null) // <-----------------------------------
JsString(x)
}
def read(value: JsValue) = value match {
case JsString(x) => x
case x => deserializationError("Expected String as JsString, but got " + x)
}
}
which sort of means one of the strings in this thousands of lines of response is null. Special thanks goes to the laziness of using that "require" without a message. Debugging which string is empty where will be a nightmare but I still think akka should fail in a better way.
akka-http no stack trace or details on error
Well, default akka-http ExceptionHandler doesn't print stack trace and prints only error message or its class name if the message is empty but you can provide custom exception handler that will print anything you want (i.e. stack trace in your example).
Some examples of how to make a custom exception handler are provided at GitHub ExceptionHandlerExamplesSpec.spec
The simplest way in your case seems to be to define your own custom implicit exception handler
import akka.http.scaladsl.model._
import akka.http.scaladsl.server._
import StatusCodes._
import Directives._
implicit def myExceptionHandler: ExceptionHandler =
ExceptionHandler {
case NonFatal(e) =>
logger.error(s"Exception $e at\n${e.getStackTraceString}")
complete(HttpResponse(InternalServerError, entity = "Internal Server Error"))
}
}
Try setting the loggers as well - from your configuration it seems they're not set. Something like:
akka {
loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = "DEBUG"
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
}
Also, consider using akka-slf4j along with their recommended logging backend logback.
This should make akka spit more details.

StackOverflowError with Maven 3.3.9, Java 8 and Scala 2.12

I get an java.lang.StackOverflowError when I try to compile and package my code with mvn package and Java 1.8.0_112.
My code compiles well with Java 7.
I tried to set JAVA_OPTS=-Xss512m and tried with 1G as well but always get the same error.
The code is written in Scala and I am using these versions in my pom.xml:
<scala.version>2.10.4</scala.version>
<spark.version>1.6.0-cdh5.7.1</spark.version>
<hbase.version>1.2.0-cdh5.7.1</hbase.version>
<kafka.version>0.9.0.0</kafka.version>
<jackson.version>2.7.2</jackson.version>
<iodadm.version>05.4.1</iodadm.version>
I tried also with Scala version 2.12.0 because 2.10.x is not compatible with Java 8.
part of the error stack:
INFO] java.lang.reflect.InvocationTargetException
[INFO] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[INFO] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[INFO] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[INFO] at java.lang.reflect.Method.invoke(Method.java:498)
[INFO] at scala_maven_executions.MainHelper.runMain(MainHelper.java:164)
[INFO] at scala_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
[ERROR] Caused by: java.lang.StackOverflowError
[INFO] at java.security.AccessController.doPrivileged(Native Method)
[INFO] at java.io.PrintWriter.<init>(PrintWriter.java:116)
[INFO] at java.io.PrintWriter.<init>(PrintWriter.java:100)
[INFO] at scala.reflect.api.Printers$class.render(Printers.scala:168)
[INFO] at scala.reflect.api.Universe.render(Universe.scala:59)
[INFO] at scala.reflect.api.Printers$class.show(Printers.scala:190)
One more thing, stackoverflow error is usually a runtime execution error, how is it possible to get it during compilation ?
[Added 1] I found that this error is caused by scala classes with at least 150 properties
[Added 2] The function throwing this error is equals function, i developed it like this:
def equals(other:MyClass):Boolean = {
this.P1 == other.P1 &&
this.P2 == other.P2 &&
... ~ 180 similar line ...
this.P180 == other.P180
}
Thank you.
I solved the problem by splitting the logical operation in many
def equals(other:MyClass):Boolean = {
Boolean bool1=this.P1 == other.P1 &&
this.P2 == other.P2 &&
...
this.P100 == other.P100
Boolean bool2=this.P101 == other.P101 &&
this.P102 == other.P102 &&
...
this.P200 == other.P200
bool1 && bool2
}
The approach to solve this type of problems is to compile the code gradually till finding the code bloc that throws the error
I hope this will help someone

org.scalajs.jsenv.ExternalJSEnv$NonZeroExitException: PhantomJS2 exited with code 2

I am using scalatest as my testing framework, here is my test code :
import org.scalatest.FunSuite
class SampleTest extends FunSuite{
test("hello") {
println(s"sdfsf")
}
}
build.sbt settings
val scalatestJS = libraryDependencies += "org.scalatest" %%% "scalatest" % Version.scalatest % Test
val scalatestJSSettings = Seq(
scalatestJS,
scalaJSStage in Global := FastOptStage,
// scalaJSStage in Global := FullOptStage,
// jsDependencies += RuntimeDOM,
// jsDependencies += ProvidedJS / "test-bundle.js" % Test,
jsEnv in Test := new PhantomJS2Env(scalaJSPhantomJSClassLoader.value, addArgs = Seq("--web-security=no"))
// jsEnv in Test := new NodeJSEnv()
)
Debug out :
[debug] Scala compilation took 0.961938628 s
[debug] Invalidating client/SampleTest.scala...
[debug] Invalidating (transitively) by inheritance from client/SampleTest.scala...
[debug] Initial set of included nodes: Set(client/SampleTest.scala)
[debug] Invalidated by transitive inheritance dependency: Set(client/SampleTest.scala)
[debug] The client/SampleTest.scala source file has the following implicit definitions changed:
[debug] unconstrainedEquality, convertToEqualizer.
[debug] All member reference dependencies will be considered within this context.
[debug] New invalidations:
[debug] Set()
[debug] Initial set of included nodes: Set()
[debug] Previously invalidated, but (transitively) depend on new invalidations:
[debug] Set()
[debug] All newly invalidated sources after taking into account (previously) recompiled sources:Set()
[debug] Copy resource mappings:
[debug]
[info] Fast optimizing client/assets/client-test-fastopt.js
[debug] Linker: Compute reachability: 710892 us
[debug] Linker: Assemble LinkedClasses: 445560 us
[debug] Basic Linking: 1168893 us
[debug] Inc. optimizer: Batch mode: true
[debug] Inc. optimizer: Incremental part: 127681 us
[debug] Inc. optimizer: Optimizing 11793 methods.
[debug] Inc. optimizer: Optimizer part: 5987357 us
[debug] Inc. optimizer: 6140114 us
[debug] Refiner: Compute reachability: 243271 us
[debug] Refiner: Assemble LinkedClasses: 27389 us
[debug] Refiner: 272567 us
[debug] Emitter: Class tree cache stats: reused: 0 -- invalidated: 1375
[debug] Emitter: Method tree cache stats: resued: 0 -- invalidated: 6643
[debug] Emitter (write output): 1353125 us
[debug] Global IR cache stats: reused: 0 -- invalidated: 130 -- trees read: 1748
[debug] Loading JSEnv with linked file client/assets/client-test-fastopt.js
org.scalajs.jsenv.ExternalJSEnv$NonZeroExitException: PhantomJS2 exited with code 2
at org.scalajs.jsenv.ExternalJSEnv$AbstractExtRunner.waitForVM(ExternalJSEnv.scala:107)
at org.scalajs.jsenv.ExternalJSEnv$ExtRunner.run(ExternalJSEnv.scala:156)
at org.scalajs.sbtplugin.FrameworkDetector.detect(FrameworkDetector.scala:66)
at org.scalajs.sbtplugin.ScalaJSPluginInternal$$anonfun$60.apply(ScalaJSPluginInternal.scala:737)
at org.scalajs.sbtplugin.ScalaJSPluginInternal$$anonfun$60.apply(ScalaJSPluginInternal.scala:720)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:235)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[error] (client/test:loadedTestFrameworks) org.scalajs.jsenv.ExternalJSEnv$NonZeroExitException: PhantomJS2 exited with code 2
[error] Total time: 15 s, completed May 20, 2016 12:20:29 AM
Note : This is used to work fine but not anymore :s
Edit :
ScalaTest : 3.0.0-M15
ScalaJS : 0.6.9
Scala : 2.11.8
Phantom JS : 2.0.0
In my code i have a class which extends js class
#ScalaJSDefined
class MyLab(container : dom.Node) extends JSLab(container) {... }
even though i am not using this class in my tests,looks like source code generated for this line is some how looking for JSLab in global! which causes crash of phantomjs. Interestingly tests are executing fine on Rhino :s. anyhow i added JSLab as test dependency and error gone.
I don't know whom to blame here , whether its stupid developer(me) who went in different direction while solving problem or scala.js for not helpful in why phantomjs crashed :s

Why is my spring boot stateless filter being called twice?

I'm trying to implement stateless token-based authentication on a rest api I've developed using Spring Boot. The idea is that the client includes a JWT token with any request, and a filter extracts this from the request, and sets up the SecurityContext with a relevant Authentication object based on the contents of the token. The request is then routed as normal, and secured using #PreAuthorize on the mapped method.
My security config is as follows :
#Configuration
#EnableWebSecurity
#EnableGlobalMethodSecurity(prePostEnabled = true)
public class SecurityConfig extends WebSecurityConfigurerAdapter {
#Autowired
private JWTTokenAuthenticationService authenticationService;
#Override
protected void configure(HttpSecurity http) throws Exception
{
http
.csrf().disable()
.headers().addHeaderWriter(new XFrameOptionsHeaderWriter(XFrameOptionsMode.SAMEORIGIN))
.and()
.authorizeRequests()
.antMatchers("/auth/**").permitAll()
.antMatchers("/api/**").authenticated()
.and()
.addFilterBefore(new StatelessAuthenticationFilter(authenticationService), UsernamePasswordAuthenticationFilter.class);
}
With the stateless filter that extends GenericFilterBean defined as follows :
public class StatelessAuthenticationFilter extends GenericFilterBean {
private static Logger logger = Logger.getLogger(StatelessAuthenticationFilter.class);
private JWTTokenAuthenticationService authenticationservice;
public StatelessAuthenticationFilter(JWTTokenAuthenticationService authenticationService)
{
this.authenticationservice = authenticationService;
}
#Override
public void doFilter( ServletRequest request,
ServletResponse response,
FilterChain chain) throws IOException, ServletException {
HttpServletRequest httpRequest = (HttpServletRequest) request;
Authentication authentication = authenticationservice.getAuthentication(httpRequest);
SecurityContextHolder.getContext().setAuthentication(authentication);
logger.info("===== Security Context before request =====");
logger.info("Request for: " + httpRequest.getRequestURI());
logger.info(SecurityContextHolder.getContext().getAuthentication());
logger.info("===========================================");
chain.doFilter(request, response);
SecurityContextHolder.getContext().setAuthentication(null);
logger.info("===== Security Context after request =====");
logger.info("Request for: " + httpRequest.getRequestURI());
logger.info(SecurityContextHolder.getContext().getAuthentication());
logger.info("===========================================");
}
}
And the endpoint defined like this :
#PreAuthorize("hasAuthority('user')")
#RequestMapping ( value="/api/attachments/{attachmentId}/{fileName:.+}",
method = RequestMethod.GET)
public ResponseEntity<byte[]> getAttachedDocumentEndpoint(#PathVariable String attachmentId, #PathVariable String fileName)
{
logger.info("GET called for /attachments/" + attachmentId + "/" + fileName);
// do something to get the file, and return ResponseEntity<byte[]> object
}
When doing a GET on /api/attachments/someattachment/somefilename, including the token, I can see that the filter is being invoked twice, once apparently with the token, and once without. But the restcontroller mapped to the request is only called once.
[INFO] [06-04-2015 12:26:44,465] [JWTTokenAuthenticationService] getAuthentication - Getting authentication based on token supplied in HTTP Header
[INFO] [06-04-2015 12:26:44,473] [StatelessAuthenticationFilter] doFilter - ===== Security Context before request =====
[INFO] [06-04-2015 12:26:44,473] [StatelessAuthenticationFilter] doFilter - Request for: /api/attachments/1674b08b6bbd54a6efaff4a780001a9e/jpg.png
[INFO] [06-04-2015 12:26:44,474] [StatelessAuthenticationFilter] doFilter - Name:iser, Principal:user, isAuthenticated:true, grantedAuthorites:[user]
[INFO] [06-04-2015 12:26:44,474] [StatelessAuthenticationFilter] doFilter - ===========================================
[INFO] [06-04-2015 12:26:44,476] [AttachmentRESTController] getAttachedDocumentEndpoint - GET called for /api/attachments/1674b08b6bbd54a6efaff4a780001a9e/jpg.png
[INFO] [06-04-2015 12:26:44,477] [AttachmentDBController] getAttachment - getAttachment method called with attachmentId:1674b08b6bbd54a6efaff4a780001a9e , and fileName:jpg.png
[INFO] [06-04-2015 12:26:44,483] [StatelessAuthenticationFilter] doFilter - ===== Security Context after request =====
[INFO] [06-04-2015 12:26:44,484] [StatelessAuthenticationFilter] doFilter - Request for: /api/attachments/1674b08b6bbd54a6efaff4a780001a9e/jpg.png
[INFO] [06-04-2015 12:26:44,484] [StatelessAuthenticationFilter] doFilter -
[INFO] [06-04-2015 12:26:44,484] [StatelessAuthenticationFilter] doFilter - ===========================================
[INFO] [06-04-2015 12:26:44,507] [JWTTokenAuthenticationService] getAuthentication - No token supplied in HTTP Header
[INFO] [06-04-2015 12:26:44,507] [StatelessAuthenticationFilter] doFilter - ===== Security Context before request =====
[INFO] [06-04-2015 12:26:44,507] [StatelessAuthenticationFilter] doFilter - Request for: /api/attachments/1674b08b6bbd54a6efaff4a780001a9e/jpg.png
[INFO] [06-04-2015 12:26:44,507] [StatelessAuthenticationFilter] doFilter -
[INFO] [06-04-2015 12:26:44,508] [StatelessAuthenticationFilter] doFilter - ===========================================
[INFO] [06-04-2015 12:26:44,508] [StatelessAuthenticationFilter] doFilter - ===== Security Context after request =====
[INFO] [06-04-2015 12:26:44,508] [StatelessAuthenticationFilter] doFilter - Request for: /api/attachments/1674b08b6bbd54a6efaff4a780001a9e/jpg.png
[INFO] [06-04-2015 12:26:44,508] [StatelessAuthenticationFilter] doFilter -
[INFO] [06-04-2015 12:26:44,508] [StatelessAuthenticationFilter] doFilter - ===========================================
What's going on here ?
Edit :
It's even stranger than I first thought - implementing a simple endpoint that just returns a simple message displays the expected behaviour - it's seems like only when I try to return data as a ResponseEntity as above does this problem occur.
Endpoint :
#PreAuthorize("hasAuthority('user')")
#RequestMapping("/api/userHelloWorld")
public String userHelloWorld()
{
return "Hello Secure User World";
}
Output, showing single call to filter (with extra debug on):
[INFO] [06-04-2015 19:43:25,831] [JWTTokenAuthenticationService] getAuthentication - Getting authentication based on token supplied in HTTP Header
[INFO] [06-04-2015 19:43:25,844] [StatelessAuthenticationFilter] doFilterInternal - ===== Security Context before request =====
[INFO] [06-04-2015 19:43:25,844] [StatelessAuthenticationFilter] doFilterInternal - Request for: /api/userHelloWorld
[INFO] [06-04-2015 19:43:25,844] [StatelessAuthenticationFilter] doFilterInternal - Response = null 200
[INFO] [06-04-2015 19:43:25,844] [StatelessAuthenticationFilter] doFilterInternal - Name:user, Principal:user, isAuthenticated:true, grantedAuthorites:[user]
[INFO] [06-04-2015 19:43:25,845] [StatelessAuthenticationFilter] doFilterInternal - ===========================================
[DEBUG] [06-04-2015 19:43:25,845] [DispatcherServlet] doService - DispatcherServlet with name 'dispatcherServlet' processing GET request for [/api/userHelloWorld]
[DEBUG] [06-04-2015 19:43:25,847] [AbstractHandlerMethodMapping] getHandlerInternal - Looking up handler method for path /api/userHelloWorld
[DEBUG] [06-04-2015 19:43:25,848] [AbstractHandlerMethodMapping] getHandlerInternal - Returning handler method [public java.lang.String RESTController.userHelloWorld()]
[DEBUG] [06-04-2015 19:43:25,849] [DispatcherServlet] doDispatch - Last-Modified value for [/api/userHelloWorld] is: -1
[DEBUG] [06-04-2015 19:43:25,851] [AbstractMessageConverterMethodProcessor] writeWithMessageConverters - Written [Hello Secure User World] as "text/plain;charset=UTF-8" using [org.springframework.http.converter.StringHttpMessageConverter#3eaf6fe7]
[DEBUG] [06-04-2015 19:43:25,852] [DispatcherServlet] processDispatchResult - Null ModelAndView returned to DispatcherServlet with name 'dispatcherServlet': assuming HandlerAdapter completed request handling
[DEBUG] [06-04-2015 19:43:25,852] [FrameworkServlet] processRequest - Successfully completed request
[INFO] [06-04-2015 19:43:25,852] [StatelessAuthenticationFilter] doFilterInternal - ===== Security Context after request =====
[INFO] [06-04-2015 19:43:25,853] [StatelessAuthenticationFilter] doFilterInternal - Request for: /api/userHelloWorld
[INFO] [06-04-2015 19:43:25,853] [StatelessAuthenticationFilter] doFilterInternal - Response = text/plain;charset=UTF-8 200
[INFO] [06-04-2015 19:43:25,853] [StatelessAuthenticationFilter] doFilterInternal -
[INFO] [06-04-2015 19:43:25,853] [StatelessAuthenticationFilter] doFilterInternal - ===========================================
It is a part of filters that has still some black magic for me (*), but I know that this is a common problem, and Spring has a subclass of GenericFilterBean specially for dealing with it : just use OncePerRequestFilter as base class and your filter should be called only once.
(*) I've read it could be caused by the request being dispatched multiple times via the request dispatcher
Okay - so this is pretty ridiculous, but it seems like it's an issue with the way I was invoking the request (via the POSTMAN Chrome Extension)
Postman seems to fire in 2 requests, one with headers, one without. There's an open bug report describing this here :
https://github.com/a85/POSTMan-Chrome-Extension/issues/615
The behaviour is not seen if the request is invoked using curl, or just straight from the browser.