Gatling - GC overhead limit exceeded - scala

I am getting the following error when I tried to Initiate my test.
The test includes a large feeder which loops for 620,000 and has a map of 120 items with in the feeder.
Uncaught exception when running trip_dte: java.lang.OutOfMemoryError: GC overhead limit exceeded
[error] sbt.ForkMain$ForkError: java.lang.OutOfMemoryError: GC overhead limit exceeded
[error] at scala.collection.immutable.HashMap$HashTrieMap.updated0(HashMap.scala:358)
[error] at scala.collection.immutable.HashMap.$plus(HashMap.scala:62)
[error] at scala.collection.immutable.HashMap.$plus(HashMap.scala:36)
[error] at scala.collection.mutable.MapBuilder.$plus$eq(MapBuilder.scala:29)
[error] at scala.collection.mutable.MapBuilder.$plus$eq(MapBuilder.scala:25)
[error] at scala.collection.generic.Growable.$anonfun$$plus$plus$eq$1(Growable.scala:59)
[error] at scala.collection.generic.Growable$$Lambda$44/1950093417.apply(Unknown Source)
[error] at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32)
[error] at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29)
[error] at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
[error] at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:59)
[error] at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:50)
[error] at scala.collection.mutable.MapBuilder.$plus$plus$eq(MapBuilder.scala:25)
[error] at scala.collection.generic.GenMapFactory.apply(GenMapFactory.scala:48)

You don't provide enough information to be able to figure out your issue.
Possible issues:
you're using an eager feeder (large CSV like feeders are lazy in modern Gatling version, but no idea which Gatling version or which kind of feeder you use).
your system under load or your bandwidth can't handle the load you're trying to generate and your virtual users pile up in memory (can't complete their journey and no virtual users keep on arriving).

Related

Scala Play Framework: Header doesn't exists

I am trying to run play application and it runs correctly. However, while serving to few APIs, I get following error:
[error] play - Cannot invoke the action, eventually got an error: java.lang.RuntimeException: Header doesn't exist
[error] application -
! #714mk098l - Internal server error, for (POST) [/ivi/customer/login/] ->
play.api.Application$$anon$1: Execution exception[[RuntimeException: Header doesn't exist]]
at play.api.Application$class.handleError(Application.scala:296) ~[play_2.10-2.3.9.jar:2.3.9]
at play.api.DefaultApplication.handleError(Application.scala:402) [play_2.10-2.3.9.jar:2.3.9]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$4.apply(PlayDefaultUpstreamHandler.scala:320) [play_2.10-2.3.9.jar:2.3.9]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$4.apply(PlayDefaultUpstreamHandler.scala:320) [play_2.10-2.3.9.jar:2.3.9]
at scala.Option.map(Option.scala:145) [scala-library-2.10.5.jar:na]
Caused by: java.lang.RuntimeException: Header doesn't exist
at scala.sys.package$.error(package.scala:27) ~[scala-library-2.10.5.jar:na]
at play.api.mvc.Headers$$anonfun$apply$10.apply(Http.scala:338) ~[play_2.10-2.3.9.jar:2.3.9]
at play.api.mvc.Headers$$anonfun$apply$10.apply(Http.scala:338) ~[play_2.10-2.3.9.jar:2.3.9]
at scala.Option.getOrElse(Option.scala:120) [scala-library-2.10.5.jar:na]
at play.api.mvc.Headers$class.apply(Http.scala:338) ~[play_2.10-2.3.9.jar:2.3.9]
I feel like this should related to configuration rather than code issue. Can someone please guide here?

Cannot use gatling conf values in scala class.

I have a gatling project where I set up cert locations using gatling.conf for 2-way ssl connection. I need to use these certificates in a helper function in the same project. I am unable to specify conf variables gatling.http.ssl.trustStore.file and gatling.http.ssl.keyStore.file in my scala class as it errors with the following error. Note that I can use application.conf variables in my code with out any issues
Code that errors:
val trustStoreLoc = config.getString("gatling.http.ssl.trustStore.file")
Error is:
[error] Exception in thread "main" com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'gatling'
[error] at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:124)
[error] at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:147)
[error] at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:159)
[error] at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:164)
[error] at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:206)
[error] at oncue.gatling.authbe.benchmarks.auth.userOpsSimulation.<init>(userOpsSimulation.scala:21)
[error] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[error] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
[error] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[error] at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
[error] at java.lang.Class.newInstance(Class.java:379)
[error] at io.gatling.core.runner.Runner.run(Runner.scala:37)
[error] at io.gatling.app.Gatling.start(Gatling.scala:235)
[error] at io.gatling.app.Gatling$.fromMap(Gatling.scala:54)
[error] at io.gatling.app.Gatling$.runGatling(Gatling.scala:79)
[error] at io.gatling.app.Gatling$.runGatling(Gatling.scala:58)
[error] at io.gatling.app.Gatling$.main(Gatling.scala:50)
[error] at io.gatling.app.Gatling.main(Gatling.scala)
Gatling uses Typesafe Config's withFallback mechanism to allow to have :
system properties overriding...
...gatling.conf values overriding...
... gatling-defaults.conf values
This mechanism is implemented here : https://github.com/gatling/gatling/blob/master/gatling-core/src/main/scala/io/gatling/core/config/GatlingConfiguration.scala#L73-77
My guess is that you only use gatling.conf and in this case, if the line is commented, the value is simply not defined.
Either uncomment that property in your gatling.conf or reuse GatlingConfiguration logic to take gatling.conf values into account.

Akka Configuration Exception: Logger specified can't be loaded

I am trying to load test my spray http client based on an actor model using the apache bench tool. The performance is not great. All the more, when I increase the concurrency level to around 200-300, I see several errors related to loggers.
Code Snippet
val result = for {
response <- GlobalObject.io.ask(HttpRequest(GET,URL)).mapTo[spray.http.HttpResponse]
} yield response
result Oncomplete{
case Success(response) => doSomething
}
Full Stack Trace:
[info] [WARN] [03/31/2015 14:31:52.580] [squbs-akka.actor.default-dispatcher-26] [EventStream(akka://hcp-client)] Logger log1-Logging$DefaultLogger did not respond within Timeout(5000 milliseconds) to InitializeLogger(bus)
[error] error while starting up loggers
[error] akka.ConfigurationException: Logger specified in config can't be loaded [akka.event.Logging$DefaultLogger] due to [akka.event.Logging$LoggerInitializationException: Logger log1-Logging$DefaultLogger did not respond with LoggerInitialized, sent instead [TIMEOUT]]
[error] at akka.event.LoggingBus$$anonfun$4$$anonfun$apply$1.applyOrElse(Logging.scala:116)
[error] at akka.event.LoggingBus$$anonfun$4$$anonfun$apply$1.applyOrElse(Logging.scala:115)
[error] at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
[error] at scala.util.Failure$$anonfun$recover$1.apply(Try.scala:215)
[error] at scala.util.Try$.apply(Try.scala:191)
[error] at scala.util.Failure.recover(Try.scala:215)
[error] at akka.event.LoggingBus$$anonfun$4.apply(Logging.scala:115)
[error] at akka.event.LoggingBus$$anonfun$4.apply(Logging.scala:110)
[error] at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:728)
[error] at scala.collection.Iterator$class.foreach(Iterator.scala:750)
[error] at scala.collection.AbstractIterator.foreach(Iterator.scala:1202)
[error] at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
[error] at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
[error] at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:727)
[error] at akka.event.LoggingBus$class.startDefaultLoggers(Logging.scala:110)
[error] at akka.event.EventStream.startDefaultLoggers(EventStream.scala:26)
[error] at akka.actor.LocalActorRefProvider.init(ActorRefProvider.scala:622)
[error] at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:619)
[error] at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:616)
[error] at akka.actor.ActorSystemImpl._start(ActorSystem.scala:616)
[error] at akka.actor.ActorSystemImpl.start(ActorSystem.scala:633)
[error] at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
[error] at akka.actor.ActorSystem$.apply(ActorSystem.scala:109)
[error] at com.paypal.vyasa.persistors.KafkaStorage.<init>(KafkaStorage.scala:40)
[error] at sun.reflect.GeneratedConstructorAccessor22.newInstance(Unknown Source)
[error] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[error] at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
[error] at akka.util.Reflect$.instantiate(Reflect.scala:66)
[error] at akka.actor.ArgsReflectConstructor.produce(Props.scala:352)
[error] at akka.actor.Props.newActor(Props.scala:252)
[error] at akka.actor.ActorCell.newActor(ActorCell.scala:552)
[error] at akka.actor.ActorCell.create(ActorCell.scala:578)
[error] at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
[error] at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
[error] at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:279)
[error] at akka.dispatch.Mailbox.run(Mailbox.scala:220)
[error] at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
[error] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
[error] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
[error] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
[error] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[error] Caused by: akka.event.Logging$LoggerInitializationException: Logger log1-Logging$DefaultLogger did not respond with LoggerInitialized, sent instead [TIMEOUT]
[error] at akka.event.LoggingBus$class.akka$event$LoggingBus$$addLogger(Logging.scala:185)
[error] at akka.event.LoggingBus$$anonfun$4$$anonfun$apply$4.apply(Logging.scala:114)
[error] at akka.event.LoggingBus$$anonfun$4$$anonfun$apply$4.apply(Logging.scala:113)
I searched the web, but seems the error is unique. Any leads anyone ? Also any leads on how the performance can be inreased ? Total number of requests seems to throttles around 15 per second.
This error has been already mentioned in the web:
https://groups.google.com/forum/#!topic/akka-user/nng-bb2IZFA
It means that one of actors (log1-Logging) did not respond in time. You can increase the timeout using logger-startup-timeout setting:
# Loggers are created and registered synchronously during ActorSystem
# start-up, and since they are actors, this timeout is used to bound the
# waiting time
logger-startup-timeout = 5s
The possible reason why it may happen is executor service (pool) configuration:
maybe you chose too low (or high) pool capacity
maybe you're using pool, which may be influenced by thread starvation problem - any pool which doesn't support managed blocking (including regular built-in fixed and dynamic pools). This can be resolved by using fork-join-pool (which used by default in akka)
This is probably due to LOGBACK-1221, which was fixed in Logback 1.1.10. If you're using an earlier version of Logback, the solution is simply to update to 1.1.10 or later.

typesafe activator dependency error

I just download the last activator version (version 1.2.12 - offline distribution) and I have found some errors in dependency when I follow the documentation in order to create a new scala project (https://www.playframework.com/documentation/2.3.x/NewApplication).
How can I get the unresolved dependency?
Can I install like mvn:install the missing dependency?
[info] Updating {file:/home/myhome/workspaceScala/play-scala/project/}play-scala-build...
[info] Resolving com.typesafe.play#routes-compiler_2.10;2.3.8 ...
[error] Server access Error: Connection timed out url=http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.play/routes-compiler_2.10/2.3.8/ivys/ivy.xml
[error] Server access Error: Connection timed out url=http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.play/routes-compiler_2.10/2.3.8/jars/routes-compiler_2.10.jar
[info] Resolving commons-io#commons-io;2.0.1 ...
[error] Server access Error: Connection timed out url=http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/commons-io/commons-io/2.0.1/ivys/ivy.xml
[info] Resolving com.typesafe.play#sbt-run-support_2.10;2.3.8 ...
[error] Server access Error: Connection timed out url=http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.play/sbt-run-support_2.10/2.3.8/ivys/ivy.xml
[error] Server access Error: Connection timed out url=http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.play/sbt-run-support_2.10/2.3.8/jars/sbt-run-support_2.10.jar
Thank you singhakash and Sarvesh, it is a network problem.
I think the best solution is to set environment variable ACTIVATOR_OPTS.
I add this line to my .bashrc file (~/.bashrc):
export ACTIVATOR_OPTS="-Dhttp.proxyUser=MyProxyAccount -Dhttp.proxyPassword=MyProxyPwd -Dhttp.proxyHost=MyProxyIp -Dhttp.proxyPort=MyProxyPort"
Thanks a lot

GWTP - An error occurred while trying to write the image bundle

I am getting a lot of these errors while trying to build gwtp+mgwt app using ant-
Preparing method tabBarMoreImage
[java] [ERROR] An error occurred while trying to write the image bundle.
[java] javax.imageio.IIOException: Can't create output stream!
[java] at javax.imageio.ImageIO.write(ImageIO.java:1560)
...
Preparing method tabBarMostRecentImage
[java] [ERROR] An error occurred while trying to write the image bundle.
[java] javax.imageio.IIOException: Can't create output stream!
...
[java] Caused by: javax.imageio.IIOException: Can't create cache file!
[java] at javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:397)
[java] at javax.imageio.ImageIO.write(ImageIO.java:1558)
[java] ... 33 more
[java] Caused by: java.io.IOException: Too many open files
[java] at java.io.UnixFileSystem.createFileExclusively(Native Method)
[java] at java.io.File.checkAndCreate(File.java:1704)
[java] at java.io.File.createTempFile(File.java:1792)
[java] at javax.imageio.stream.FileCacheImageOutputStream.(FileCacheImageOutputStream.java:71)
[java] at com.sun.imageio.spi.OutputStreamImageOutputStreamSpi.createOutputStreamInstance(OutputStreamImageOutputStreamSpi.java:50)
[java] at javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:393)
[java] ... 34 more
[java] Preparing method getButtonBarArrowDownImage
[java] [ERROR] An error occurred while trying to write the image bundle.
[java] javax.imageio.IIOException: Can't create output stream!
[java] at javax.imageio.ImageIO.write(ImageIO.java:1560)
[java] at com.google.gwt.resources.rg.ImageBundleBuilder.createImageBytes(ImageBundleBuilder.java:558)
[java] at com.google.gwt.resources.rg.ImageBundleBuilder.toPng(ImageBundleBuilder.java:544)
...
The application runs fine in dev mode. Any ideas?
Maybe GWT compiler is leaking file descriptors ( hence the "Too many open files" exception). Maybe you're on environment that has limited the number of open files per user (is this your own machine)?
Check all user limits with ulimit -a.
You can try to increase the open file limit by using ulimit -Hn <number>. This should at least help you diagnose.
Had exactly the same problem (and error message).
Found the solution in this thread on google groups (GWT)
I am using Linux (I don't know if that's your case)
In /etc/security/limits.conf I added
* soft nofile 65535
* hard nofile 65535
To apply the new settings, log out and in again.