I'm playing a bit with scala and play framework, but when i try to run my test project using activator, sbt on a mac i keep getting:
[info] Caused by: java.security.PrivilegedActionException: null
[info] at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_60]
[info] at play.runsupport.Reloader$.play$runsupport$Reloader$$withReloaderContextClassLoader(Reloader.scala:39) ~[na:na]
[info] at play.runsupport.Reloader.reload(Reloader.scala:321) ~[na:na]
[info] at play.core.server.DevServerStart$$anonfun$mainDev$1$$anon$1$$anonfun$get$1.apply(DevServerStart.scala:113) ~[play-server_2.11-2.4.3.jar:2.4.3]
[info] at play.core.server.DevServerStart$$anonfun$mainDev$1$$anon$1$$anonfun$get$1.apply(DevServerStart.scala:111) ~[play-server_2.11-2.4.3.jar:2.4.3]
[info] at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) ~[scala-library-2.11.7.jar:na]
[info] at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) ~[scala-library-2.11.7.jar:na]
[info] at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) ~[akka-actor_2.11-2.3.13.jar:na]
[info] at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397) ~[akka-actor_2.11-2.3.13.jar:na]
[info] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) ~[scala-library-2.11.7.jar:na]
[info] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) ~[scala-library-2.11.7.jar:na]
[info] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) ~[scala-library-2.11.7.jar:na]
[info] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) ~[scala-library-2.11.7.jar:na]
[info] Caused by: java.util.concurrent.TimeoutException: Futures timed out after [300000 milliseconds]
[info] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) ~[scala-library-2.11.7.jar:na]
[info] at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) ~[scala-library-2.11.7.jar:na]
[info] at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190) ~[scala-library-2.11.7.jar:na]
[info] at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) ~[scala-library-2.11.7.jar:na]
[info] at scala.concurrent.Await$.result(package.scala:190) ~[scala-library-2.11.7.jar:na]
[info] at play.forkrun.ForkRun$$anonfun$askForReload$1.apply(ForkRun.scala:127) ~[na:na]
[info] at play.forkrun.ForkRun$$anonfun$askForReload$1.apply(ForkRun.scala:125) ~[na:na]
[info] at play.runsupport.Reloader$$anonfun$reload$1.apply(Reloader.scala:323) ~[na:na]
[info] at play.runsupport.Reloader$$anon$3.run(Reloader.scala:43) ~[na:na]
I'm not really sure what am i doing wrong.
I had the same problem and I've found the solution here https://support.pivotal.io/hc/en-us/articles/201821186-Namenode-fails-to-start-with-error-jurisdiction-policy-files-are-not-signed-by-a-trusted-signer-
I already had installed Java 8 JDK on my machine so I had to download the policy files from this website:
http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
Then I just pasted the files to the location which is provided in the "readme.txt".
The loction depends on the OS that you are using.
And everything works fine :)
Related
I am trying to complete the sbt testing tutorial
I complete all steps and run sbt test. However I receive a nullpointer exception I do not know how to resolve.
C:\Users\Sayth\OneDrive\Projects\scalatest\scalatesttutorial>sbt test
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option MaxPermSize; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option MaxPermSize; support was removed in 8.0
[info] Loading project definition from C:\Users\Sayth\OneDrive\Projects\scalatest\scalatesttutorial\project
[info] Updating {file:/C:/Users/Sayth/OneDrive/Projects/scalatest/scalatesttutorial/project/}scalatesttutorial-build...
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by sbt.ivyint.ErrorMessageAuthenticator$ (file:/C:/Users/Sayth/.sbt/boot/scala-2.10.7/org.scala-sbt/sbt/0.13.17/ivy-0.13.17.jar) to field java.net.Authenticator.theAuthenticator
WARNING: Please consider reporting this to the maintainers of sbt.ivyint.ErrorMessageAuthenticator$
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
java.lang.NullPointerException
at scala.reflect.io.JavaToolsPlatformArchive.iterator(ZipArchive.scala:242)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.reflect.io.AbstractFile.foreach(AbstractFile.scala:92)
at scala.tools.nsc.util.DirectoryClassPath.traverse(ClassPath.scala:308)
at scala.tools.nsc.util.DirectoryClassPath.x$16$lzycompute(ClassPath.scala:317)
at scala.tools.nsc.util.DirectoryClassPath.x$16(ClassPath.scala:317)
at scala.tools.nsc.util.DirectoryClassPath.packages$lzycompute(ClassPath.scala:317)
at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:317)
at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:297)
at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:375)
at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:375)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.tools.nsc.util.MergedClassPath.packages$lzycompute(ClassPath.scala:375)
at scala.tools.nsc.util.MergedClassPath.packages(ClassPath.scala:370)
at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader.doComplete(SymbolLoaders.scala:243)
at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:194)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1231)
at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:240)
at scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:59)
at scala.tools.nsc.Global.rootMirror(Global.scala:57)
at scala.tools.nsc.Global.rootMirror(Global.scala:37)
at scala.reflect.internal.Definitions$DefinitionsClass.<init>(Definitions.scala:166)
at scala.reflect.internal.Definitions$definitions$.<init>(Definitions.scala:20)
at scala.reflect.internal.SymbolTable.definitions$lzycompute(SymbolTable.scala:13)
at scala.reflect.internal.SymbolTable.definitions(SymbolTable.scala:13)
at scala.tools.nsc.Global$Run.<init>(Global.scala:1290)
at sbt.compiler.Eval$$anon$1.<init>(Eval.scala:141)
at sbt.compiler.Eval.run$lzycompute$1(Eval.scala:141)
at sbt.compiler.Eval.run$1(Eval.scala:141)
at sbt.compiler.Eval.unlinkAll$1(Eval.scala:144)
at sbt.compiler.Eval.evalCommon(Eval.scala:153)
at sbt.compiler.Eval.evalDefinitions(Eval.scala:122)
at sbt.EvaluateConfigurations$.evaluateDefinitions(EvaluateConfigurations.scala:271)
at sbt.EvaluateConfigurations$.evaluateSbtFile(EvaluateConfigurations.scala:109)
at sbt.Load$.sbt$Load$$loadSettingsFile$1(Load.scala:775)
at sbt.Load$$anonfun$sbt$Load$$memoLoadSettingsFile$1$1.apply(Load.scala:781)
at sbt.Load$$anonfun$sbt$Load$$memoLoadSettingsFile$1$1.apply(Load.scala:780)
at scala.collection.MapLike$class.getOrElse(MapLike.scala:128)
at scala.collection.AbstractMap.getOrElse(Map.scala:58)
at sbt.Load$.sbt$Load$$memoLoadSettingsFile$1(Load.scala:780)
at sbt.Load$$anonfun$loadFiles$1$2.apply(Load.scala:788)
at sbt.Load$$anonfun$loadFiles$1$2.apply(Load.scala:788)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at sbt.Load$.loadFiles$1(Load.scala:788)
at sbt.Load$.discoverProjects(Load.scala:799)
at sbt.Load$.discover$1(Load.scala:585)
at sbt.Load$.sbt$Load$$loadTransitive(Load.scala:633)
at sbt.Load$$anonfun$loadUnit$1.sbt$Load$$anonfun$$loadProjects$1(Load.scala:482)
at sbt.Load$$anonfun$loadUnit$1$$anonfun$40.apply(Load.scala:485)
at sbt.Load$$anonfun$loadUnit$1$$anonfun$40.apply(Load.scala:485)
at sbt.Load$.timed(Load.scala:1025)
at sbt.Load$$anonfun$loadUnit$1.apply(Load.scala:485)
at sbt.Load$$anonfun$loadUnit$1.apply(Load.scala:459)
at sbt.Load$.timed(Load.scala:1025)
at sbt.Load$.loadUnit(Load.scala:459)
at sbt.Load$$anonfun$25$$anonfun$apply$14.apply(Load.scala:311)
at sbt.Load$$anonfun$25$$anonfun$apply$14.apply(Load.scala:310)
at sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:91)
at sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:90)
at sbt.BuildLoader.apply(BuildLoader.scala:140)
at sbt.Load$.loadAll(Load.scala:365)
at sbt.Load$.loadURI(Load.scala:320)
at sbt.Load$.load(Load.scala:316)
at sbt.Load$.load(Load.scala:305)
at sbt.Load$$anonfun$4.apply(Load.scala:146)
at sbt.Load$$anonfun$4.apply(Load.scala:146)
at sbt.Load$.timed(Load.scala:1025)
at sbt.Load$.apply(Load.scala:146)
at sbt.Load$.defaultLoad(Load.scala:39)
at sbt.BuiltinCommands$.liftedTree1$1(Main.scala:503)
at sbt.BuiltinCommands$.doLoadProject(Main.scala:503)
at sbt.BuiltinCommands$$anonfun$loadProjectImpl$2.apply(Main.scala:495)
at sbt.BuiltinCommands$$anonfun$loadProjectImpl$2.apply(Main.scala:495)
at sbt.Command$$anonfun$applyEffect$1$$anonfun$apply$2.apply(Command.scala:59)
at sbt.Command$$anonfun$applyEffect$1$$anonfun$apply$2.apply(Command.scala:59)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Command$.process(Command.scala:93)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.State$$anon$1.runCmd$1(State.scala:183)
at sbt.State$$anon$1.process(State.scala:187)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.MainLoop$.next(MainLoop.scala:96)
at sbt.MainLoop$.run(MainLoop.scala:89)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:68)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:63)
at sbt.Using.apply(Using.scala:24)
at sbt.MainLoop$.runWithNewLog(MainLoop.scala:63)
at sbt.MainLoop$.runAndClearLast(MainLoop.scala:46)
at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:30)
at sbt.MainLoop$.runLogged(MainLoop.scala:22)
at sbt.StandardMain$.runManaged(Main.scala:61)
at sbt.xMain.run(Main.scala:35)
at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
at xsbt.boot.Launch$.run(Launch.scala:109)
at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
at xsbt.boot.Launch$.launch(Launch.scala:117)
at xsbt.boot.Launch$.apply(Launch.scala:18)
at xsbt.boot.Boot$.runImpl(Boot.scala:56)
at xsbt.boot.Boot$.main(Boot.scala:18)
at xsbt.boot.Boot.main(Boot.scala)
[error] java.lang.NullPointerException
[error] Use 'last' for the full log.
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?
There are some compatibility issues between scala and the latest version of java. If you uninstall java 10, and then install java 8 instead, those errors will go away.
It's clearly indicated on the same website getting-started-with-scala-and-sbt-on-the-command-line
Make sure you have the Java 8 JDK (also known as 1.8)
I want to Save and Read Spark DataFrame from AWS S3. I googled it a lot but found nothing much of use.
The code that I have written is like this:
val spark = SparkSession.builder().master("local").appName("test").getOrCreate()
spark.sparkContext.hadoopConfiguration.set("fs.s3n.awsAccessKeyId", "**********")
spark.sparkContext.hadoopConfiguration.set("fs.s3n.awsSecretAccessKey", "********************")
import spark.implicits._
spark.read.textFile("s3n://myBucket/testFile").show(false)
List(1,2,3,4).toDF.write.parquet("s3n://myBucket/test/abc.parquet")
But when run it I get following error:
org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException: S3 HEAD request failed for '/myBucket/testFile' - ResponseCode=403, ResponseMessage=Forbidden
[info] at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:245)
[info] at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:119)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] at java.lang.reflect.Method.invoke(Method.java:498)
[info] at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
[info] at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
[info] at org.apache.hadoop.fs.s3native.$Proxy15.retrieveMetadata(Unknown Source)
[info] at org.apache.hadoop.fs.s3native.NativeS3FileSystem.getFileStatus(NativeS3FileSystem.java:414)
[info] ...
[info] Cause: org.jets3t.service.S3ServiceException: S3 HEAD request failed for '/myBucket/testFile' - ResponseCode=403, ResponseMessage=Forbidden
[info] at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:477)
[info] at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRestHead(RestS3Service.java:718)
[info] at org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:1599)
[info] at org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectDetailsImpl(RestS3Service.java:1535)
[info] at org.jets3t.service.S3Service.getObjectDetails(S3Service.java:1987)
[info] at org.jets3t.service.S3Service.getObjectDetails(S3Service.java:1332)
[info] at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:111)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] ...
[info] Cause: org.jets3t.service.impl.rest.HttpException:
[info] at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:475)
[info] at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRestHead(RestS3Service.java:718)
[info] at org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:1599)
[info] at org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectDetailsImpl(RestS3Service.java:1535)
[info] at org.jets3t.service.S3Service.getObjectDetails(S3Service.java:1987)
[info] at org.jets3t.service.S3Service.getObjectDetails(S3Service.java:1332)
[info] at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:111)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] ...
I am using
Spark: 2.1.0
Scala: 2.11.2
AWS Java SDK: 1.11.126
Any help is appreciated !
I have tried following things on spark version 2.1.1 and its worked fine for me.
Step 1: Download following jars:
-- hadoop-aws-2.7.3.jar
-- aws-java-sdk-1.7.4.jar
Note:
If you not able to find the following jars, then you can get the jars from hadoop-2.7.3
Step 2: Place the above jars into $SPARK_HOME/jars/
Step 3: code:
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
val conf = new SparkConf().setMaster("local").setAppName("My App")
val sc = new SparkContext(conf)
sc.getOrCreate.hadoopConfiguration.set("fs.s3a.access.key"", "***********")
sc.getOrCreate.hadoopConfiguration.set("fs.s3a.secret.key", "******************")
val input = sc.textFile("s3a://mybucket/*.txt")
List(1,2,3,4).toDF.write.parquet("s3a://mybucket/abc.parquet")
Set the secrets in the spark conf itself with an option like "spark.hadoop.fs.s3n...", so that spark propagates it around with the work
After switching Spark framework from 1.6 to 2.1, running my tests results in many occurrences of the following error:
[info] java.lang.AssertionError: assertion failed: copyAndReset must return a zero value copy
[info] at scala.Predef$.assert(Predef.scala:170)
[info] at org.apache.spark.util.AccumulatorV2.writeReplace(AccumulatorV2.scala:163)
[info] at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
[info] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] at java.lang.reflect.Method.invoke(Method.java:497)
[info] at java.io.ObjectStreamClass.invokeWriteReplace(ObjectStreamClass.java:1075)
[info] at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1136)
[info] at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
[info] at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
[info] at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
[info] ...
Any ideas what's happening and how I can fix this?
In the play 2.4.4 framework I get at a seemingly random time the error 'key not found: SOURCE' at deployment time. After this happens once there is no way to use that development environment again. I have to go back to a previously saved version of the project and try again. If I make the same code changes (for example something simple like extend a table in a Play HTML page) the same might happen or not. (I use Intellij Idea version 15 Ultimate)
After some research this error message seems to have to do with the generation of the *.template.scala file for the html pages of the play framework.
Suggested older remedies talk about using the the 'play clean update' command but nowadays there seems to be only the Activate and I have not found a way to let id do cleaning and updating.
Any idea of why this is happening almost every 2 or 3 deployments ? What can I do to reset the situation ? Any suggestions are greatly appreciated.
Stack dump follows for information:
--- (Running the application, auto-reloading is enabled) ---
[info] p.c.s.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
Server started, use Alt+D to stop
[info] Compiling 39 Scala sources and 1 Java source to E:\source\scalaIntelliJ\auctioneer\target\scala-2.11\classes...
java.util.NoSuchElementException: key not found: SOURCE
at scala.collection.MapLike$class.default(MapLike.scala:228)
at scala.collection.AbstractMap.default(Map.scala:58)
at scala.collection.MapLike$class.apply(MapLike.scala:141)
at scala.collection.AbstractMap.apply(Map.scala:58)
at play.twirl.compiler.GeneratedSource.source(TwirlCompiler.scala:129)
at play.twirl.compiler.GeneratedSource.sync(TwirlCompiler.scala:138)
at play.twirl.sbt.TemplateCompiler$$anonfun$syncGenerated$2.apply(TemplateCompiler.scala:38)
at play.twirl.sbt.TemplateCompiler$$anonfun$syncGenerated$2.apply(TemplateCompiler.scala:38)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at play.twirl.sbt.TemplateCompiler$.syncGenerated(TemplateCompiler.scala:38)
at play.twirl.sbt.TemplateCompiler$.compile(TemplateCompiler.scala:23)
at play.twirl.sbt.SbtTwirl$$anonfun$compileTemplatesTask$1.apply(SbtTwirl.scala:87)
at play.twirl.sbt.SbtTwirl$$anonfun$compileTemplatesTask$1.apply(SbtTwirl.scala:86)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:235)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[error] (compile:twirlCompileTemplates) java.util.NoSuchElementException: key not found: SOURCE
[error] application -
! #6po0l44eg - Internal server error, for (GET) [/] ->
play.sbt.PlayExceptions$UnexpectedException: Unexpected exception[NoSuchElementException: key not found: SOURCE]
at play.sbt.run.PlayReload$$anonfun$taskFailureHandler$1.apply(PlayReload.scala:51) ~[na:na]
at play.sbt.run.PlayReload$$anonfun$taskFailureHandler$1.apply(PlayReload.scala:44) ~[na:na]
at scala.Option.map(Option.scala:145) ~[scala-library-2.11.7.jar:na]
at play.sbt.run.PlayReload$.taskFailureHandler(PlayReload.scala:44) ~[na:na]
at play.sbt.run.PlayReload$.compileFailure(PlayReload.scala:40) ~[na:na]
at play.sbt.run.PlayReload$$anonfun$compile$2$$anonfun$apply$3.apply(PlayReload.scala:20) ~[na:na]
at play.sbt.run.PlayReload$$anonfun$compile$2$$anonfun$apply$3.apply(PlayReload.scala:20) ~[na:na]
at scala.util.Either$LeftProjection.map(Either.scala:377) ~[scala-library-2.11.7.jar:na]
at play.sbt.run.PlayReload$$anonfun$compile$2.apply(PlayReload.scala:20) ~[na:na]
at play.sbt.run.PlayReload$$anonfun$compile$2.apply(PlayReload.scala:18) ~[na:na]
Caused by: java.util.NoSuchElementException: key not found: SOURCE
at scala.collection.MapLike$class.default(MapLike.scala:228) ~[scala-library-2.11.7.jar:na]
at scala.collection.AbstractMap.default(Map.scala:58) ~[scala-library-2.11.7.jar:na]
at scala.collection.MapLike$class.apply(MapLike.scala:141) ~[scala-library-2.11.7.jar:na]
at scala.collection.AbstractMap.apply(Map.scala:58) ~[scala-library-2.11.7.jar:na]
at play.twirl.compiler.GeneratedSource.source(TwirlCompiler.scala:129) ~[na:na]
at play.twirl.compiler.GeneratedSource.sync(TwirlCompiler.scala:138) ~[na:na]
at play.twirl.sbt.TemplateCompiler$$anonfun$syncGenerated$2.apply(TemplateCompiler.scala:38) ~[na:na]
at play.twirl.sbt.TemplateCompiler$$anonfun$syncGenerated$2.apply(TemplateCompiler.scala:38) ~[na:na]
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) ~[scala-library-2.11.7.jar:na]
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) ~[scala-library-2.11.7.jar:na]
Related to question in
Rest server (Play Framework) gets "Read Timed out" exception during load test
I have similar situation using play framework 2.2.4:
java version "1.8.0_31"
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)
I am testing play with jmeter in my local PC. I hit with 2000 threads and timeout exception occurs, and Socket is somehow not closing.
How do I catch this exception and close the hanging sockets?
StackTrace:
va.util.concurrent.TimeoutException: Futures timed out after [5 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) [scala-library.jar:na]
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) [scala-library.jar:na]
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107) ~[scala-library.jar:na]
at akka.dispatch.MonitorableThreadFactory$AkkaForkJoinWorkerThread$$anon$3.block(ThreadPoolBuilder.scala:169) ~[akka-actor_2.10.jar:2.2.0]
at scala.concurrent.forkjoin.ForkJoinPool.managedBlock(ForkJoinPool.java:3640) [scala-library.jar:na]
at akka.dispatch.MonitorableThreadFactory$AkkaForkJoinWorkerThread.blockOn(ThreadPoolBuilder.scala:167) ~[akka-actor_2.10.jar:2.2.0]
at scala.concurrent.Await$.result(package.scala:107) ~[scala-library.jar:na]
at play.core.j.FPromiseHelper$.get(FPromiseHelper.scala:53) ~[play_2.10.jar:2.2.4]
at play.core.j.FPromiseHelper.get(FPromiseHelper.scala) ~[play_2.10.jar:2.2.4]
at play.libs.F$Promise.get(F.java:363) ~[play_2.10.jar:2.2.4]
wrap response>>
at com.ganda.common.controllers.filters.WrapResponse.wrap(WrapResponse.java:50) ~[na:na]
db filter below>>>
at com.ganda.common.controllers.filters.CredentialsFilter.call(CredentialsFilter.java:72) ~[na:na]
at play.core.j.JavaAction$$anon$3.apply(JavaAction.scala:91) ~[play_2.10.jar:2.2.4]
at play.core.j.JavaAction$$anon$3.apply(JavaAction.scala:90) ~[play_2.10.jar:2.2.4]
at play.core.j.FPromiseHelper$$anonfun$flatMap$1.apply(FPromiseHelper.scala:82) ~[play_2.10.jar:2.2.4]
at play.core.j.FPromiseHelper$$anonfun$flatMap$1.apply(FPromiseHelper.scala:82) ~[play_2.10.jar:2.2.4]
at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:251) ~[scala-library.jar:na]
at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:249) ~[scala-library.jar:na]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) [scala-library.jar:na]
at play.core.j.HttpExecutionContext$$anon$2.run(HttpExecutionContext.scala:37) ~[play_2.10.jar:2.2.4]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) ~[akka-actor_2.10.jar:2.2.0]
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) ~[akka-actor_2.10.jar:2.2.0]
Wrapped by: play.api.Application$$anon$1: Execution exception[[TimeoutException: Futures timed out after [5 seconds]]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10.jar:2.2.4]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10.jar:2.2.4]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:264) [play_2.10.jar:2.2.4]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:264) [play_2.10.jar:2.2.4]
at scala.Option.map(Option.scala:145) [scala-library.jar:na]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3.applyOrElse(PlayDefaultUpstreamHandler.scala:264) [play_2.10.jar:2.2.4]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3.applyOrElse(PlayDefaultUpstreamHandler.scala:260) [play_2.10.jar:2.2.4]
at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:344) [scala-library.jar:na]
at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:343) [scala-library.jar:na]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) [scala-library.jar:na]
at play.api.libs.iteratee.Execution$$anon$1.execute(Execution.scala:43) [play-iteratees_2.10.jar:2.2.4]
at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40) [scala-library.jar:na]
at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248) [scala-library.jar:na]
at scala.concurrent.Promise$class.complete(Promise.scala:55) [scala-library.jar:na]
at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:153) [scala-library.jar:na]
at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) [scala-library.jar:na]
at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) [scala-library.jar:na]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) [scala-library.jar:na]
at scala.concurrent.forkjoin.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1361) [scala-library.jar:na]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library.jar:na]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library.jar:na]
What is happening here is that the Play promise is timeouting probably due to slow down introduced by load.
So this reveals a performance issue in your application, you could if it's acceptable increase the promise timeout or profile and improve performances of the promise processing so that it proceeds faster.