Play2 run via Sbt in dev mode on Osx Causes Out of Memory - scala

I can't seem to find similar issues.
On OSX, play2 bin seems to run okay but using sbt causes oom errors and scala files never complete compiling
--- (Running the application from SBT, auto-reloading is enabled) ---
[info] play - Listening for HTTP on port 9000...
(Server started, use Ctrl+D to stop and go back to the console...)
[info] Compiling 15 Scala sources and 2 Java sources to
/Users/tripled153/Development/src/Foundation/playApp/target/scala-2.9.1/classes...
Java HotSpot(TM) 64-Bit Server VM warning: Exception
java.lang.OutOfMemoryError occurred dispatching signal Unknown Signal
to handler- the VM may need to be forcibly terminated
I'm running with extended memory settings but still no luck. Running in staged (prod) mode appears okay though.
rundev.sh
export SBT_OPTS="-Xms512M -Xmx3536M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:MaxPermSize=512M"
sudo sbt clean run

Related

sbt server fails to start throwing error 231

When I am trying to start sbt, it fails throwing an IO exception.
Version of SBT - 1.1.2
Version of Scala - 2.12.6
Error:
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
[info] Loading project definition from C:\Users\username\IdeaProjects\projectname\project
[info] Loading settings from build.sbt ...
[info] Set current project to example (in build file:/C:/Users/username/IdeaProjects/projectname/)
[error] server failed to start on local:sbt-server-ad238fc8ee352b69065c. java.io.IOException: Could not create lock for
\\.\pipe\sbt-server-ad238fc8ee352b69065c_lock, error 231
Analysing the source of the throw it seems error code 231 refers to Windows error:
ERROR_PIPE_BUSY
231 (0xE7)
All pipe instances are busy.
The pipe cannot be accessed because another process is using it, so try closing down IntelliJ and any other SBT server running, and try again later. Perhaps even reboot the machine, and then start a single system SBT server (not IntelliJ), and see if the issue persists.

SBT killed when running out of ram

I am trying to run my scala application in a docker container with 800MB of RAM, (docker run -m=800m mindlevel) but it gets killed before it is done compiling.
[info] Compiling 10 Scala sources to /root/mindlevel-backend/target/scala-2.12/classes...
[info] 'compiler-interface' not yet compiled for Scala 2.12.1. Compiling...
[info] Compilation completed in 8.631 s
/usr/local/sbt/bin/sbt-launch-lib.bash: line 58: 57 Killed "$#"
I have tried to restrict the SBT heap with:
CMD ["sbt", "-mem", "700", "run"]
But it still gets killed at the same place.
Without any restrictions on the docker container it runs fine.
When running it in an EC2 micro instance (1GB of RAM) it also crashes, with or without restrictions on SBT and Docker.
Is there some restriction that I am missing?
To get it to run with less than 1GB heap I precompiled the project with sbt assembly which makes a jar packaged with your project and all of your dependencies.
Install it by adding:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.5")
to project/assembly.sbt
Then generate the jar with sbt assembly which will place the jar in ./target/scala-<VERSION>/<PROJECT>-assembly-1.0.jar
Then simply run your project with:
scala target/scala-<VERSION>/<PROJECT>-assembly-1.0.jar
Since no compilation step is needed on the low memory instance now a lot of projects will be able to run even though it is less memory than recommended.

IntelliJ can't connect to its own Scala compile server

I am using latest IntelliJ IDEA (2016.1.1) - but I have seen this behaviour in previous versions too - developing in Scala with latest Scala plugin (3.0.3).
When I try to Run/Debug a test or code, IDEA will start Making the project, but will then stay there several seconds while the "Make" progress bar doesn't progress at all.
After 20-30 seconds of this, the following message appear in the Messages tab:
Warning:scala: Cannot connect to compile server at
localhost/127.0.0.1:59656 Trying to compile without it
The port shown there, 59656, is a port I hard-coded myself in %USER_PROFILE%\.IntelliJIdea2016.1\config\options\scala.xml, after I noticed this problem the first time on the default port, so changing the port did not help...
At this point, the bar will read "Reading compilation settings..." for several more tens of seconds before the code actually runs. It is terribly bothering and I can't seem to be able to fix it definitively. This can easily take up to 3 minutes to run a single test. Very disrupting. I noticed this problem happening intermittently on my work machine (Win7).
Anyone has any idea how to debug/fix this issue?
Thanks
I had the same issue.
I'm using IntelliJ 2017.3. Play framework 2.6. Stb 0.13.15.
Solved with adding -server -Xss1m in the scala compile server JVM parameters:
I had the same problem, and it was caused by conflicting settings files between the settings repository and the local preferences. Your compile server settings are store in a file called scala.xml in your IDEA preferences dir.
If you use "settings sync", i.e. an online repository to sync your IDEA settings, make sure that you have only one scala.xml config file. Go to your Library/Preferences/<IDEA-Dir>/ and remove that file from the options and settingsRepository dirs. This resets to "factory settings".
Restart IntelliJ, setup the compile server again. It should now only write one file, to the settingsRepository dir, with a new ID and port allocation for the compile server.
This fixed it for me.
I don't fully understand this yet, however under Preferences, Languages & Frameworks > Scalac Server there are some settings that might help:
Shutdown server if it is idle for <number> minutes.
Use project home as compiler server working directory (Compile server will be restarted on each compilation of a new project)
JVM SDK is used to instantiate compile server ... (when JVM SDK and module SDK match).
There is a small icon in the bottom right of the IntelliJ window that indicates whether the compile server is running and can be used to stop and start it.
The server will shutdown if it is idle. It should normally start up when needed but this might not happen if the JVM SDK does not match the module SDK. Increasing the idle timeout should help to reduce the frequency of the problem. Matching the JVM SDK to the Module SDK ought to fix the problem, but might not be possible if you have multiple projects using different SDKs.
Using the project home as the compiler server working directory might help if you frequently switch between projects.
Edit
I have this problem again with IntelliJ IDEA Ultimate 2016.3.2 but the usual method of a clean shutdown and reboot hasn't worked. I've pieced together some information on how the compile server works and it seems to me that IntelliJ is looking on the wrong port for the compile server. If I manually force a make then I can see a compile server running, but it isn't the one that IntelliJ is looking for. I still don't have a solution, but this information might get us closer to one.
Finding Files
The first step is working out where IntelliJ stores its configuration and log files. This information is currently published here: IntelliJ Directories
On my Mac this means:
configuration files: ~/Library/Preferences/IntelliJIdea2016.3/options
log files: ~/Library/Logs/IntelliJIdea2016.3
The Compile Server
The compile server is a Java process with a very long command line that ends like this:
org.jetbrains.jps.cmdline.BuildMain <host> <port> <compile-server-id> <compile-server-cache-directory>
The configuration options are stored in ~/Library/Preferences/IntelliJIdea2016.3/options/scala.xml
Some of these are configured in the Preferences -> Languages & Frameworks -> Scala Compiler Server dialog, but other settings can be added by editing the file. Mine currently looks like this:
<application>
<component name="ScalaSettings">
<option name="COMPILE_SERVER_PORT" value="59184" />
<option name="COMPILE_SERVER_ID" value="93f4589e-4707-4e20-a05a-01869045e067" />
<option name="COMPILE_SERVER_SDK" value="1.8" />
<option name="COMPILE_SERVER_MAXIMUM_HEAP_SIZE" value="2048" />
</component>
</application>
I added the COMPILER_SERVER_PORT option by hand, but with this version of IntelliJ the compile server that it uses for single compilations has a different port and id. Unfortunately this process uses a different port if I change COMPILER_SERVER_PORT and restart IntelliJ so I think these are two separate instances.
I used this command to find the compile server process, after manually starting a compile:
ps -ef | grep nailgun
This command finds the ports used by IntelliJ:
lsof -i | grep idea
There is also a log file here:
~/Library/Logs/IntelliJIdea2016.3/idea.log
that indicates that something is not quite right:
2016-12-28 10:19:46,745 [ 50710] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stdout]: Build process started. Classpath: /Users/richard/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/163.10154.41/IntelliJ IDEA.app/Contents/lib/jps-launcher.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/lib/tools.jar:/Users/richard/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/163.10154.41/IntelliJ IDEA.app/Contents/lib/optimizedFileManager.jar
2016-12-28 10:59:43,958 [2447923] INFO - ij.compiler.impl.CompileDriver - COMPILATION STARTED (BUILD PROCESS)
2016-12-28 10:59:43,960 [2447925] INFO - j.compiler.server.BuildManager - Using preloaded build process to compile /Users/richard/workspace-scala
2016-12-28 10:59:44,181 [2448146] ERROR - .impl.MessageBusConnectionImpl - java.lang.NullPointerException
java.lang.RuntimeException: java.lang.NullPointerException
at com.intellij.openapi.application.impl.LaterInvocator.invokeAndWait(LaterInvocator.java:189)
at com.intellij.openapi.application.impl.ApplicationImpl.invokeAndWait(ApplicationImpl.java:662)
at com.intellij.openapi.application.impl.ApplicationImpl.invokeAndWait(ApplicationImpl.java:667)
at org.jetbrains.plugins.scala.extensions.package$$anonfun$invokeAndWait$1.apply$mcV$sp(package.scala:567)
at org.jetbrains.plugins.scala.extensions.package$.preservingControlFlow(package.scala:577)
at org.jetbrains.plugins.scala.extensions.package$.invokeAndWait(package.scala:566)
at org.jetbrains.plugins.scala.compiler.ServerMediator$$anon$1.buildStarted(ServerMediator.scala:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.util.messages.impl.MessageBusConnectionImpl.deliverMessage(MessageBusConnectionImpl.java:117)
at com.intellij.util.messages.impl.MessageBusImpl.doPumpMessages(MessageBusImpl.java:381)
at com.intellij.util.messages.impl.MessageBusImpl.pumpMessages(MessageBusImpl.java:368)
at com.intellij.util.messages.impl.MessageBusImpl.sendMessage(MessageBusImpl.java:347)
at com.intellij.util.messages.impl.MessageBusImpl.access$200(MessageBusImpl.java:42)
at com.intellij.util.messages.impl.MessageBusImpl$2.invoke(MessageBusImpl.java:230)
at com.sun.proxy.$Proxy117.buildStarted(Unknown Source)
at com.intellij.compiler.server.BuildManager$NotifyingMessageHandler.buildStarted(BuildManager.java:1435)
at com.intellij.compiler.server.DelegatingMessageHandler.buildStarted(DelegatingMessageHandler.java:18)
at com.intellij.compiler.server.DelegatingMessageHandler.buildStarted(DelegatingMessageHandler.java:18)
at com.intellij.compiler.server.BuildMessageDispatcher.sendBuildParameters(BuildMessageDispatcher.java:115)
at com.intellij.compiler.server.BuildManager.a(BuildManager.java:724)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.intellij.util.concurrency.BoundedTaskExecutor$2.run(BoundedTaskExecutor.java:210)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.jetbrains.plugins.scala.compiler.CompileServerLauncher$$anonfun$15.apply(CompileServerLauncher.scala:282)
at org.jetbrains.plugins.scala.compiler.CompileServerLauncher$$anonfun$15.apply(CompileServerLauncher.scala:282)
at scala.collection.TraversableLike$$anonfun$filterImpl$1.apply(TraversableLike.scala:259)
at scala.collection.immutable.Set$Set3.foreach(Set.scala:145)
at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:258)
at scala.collection.TraversableLike$class.filter(TraversableLike.scala:270)
at scala.collection.AbstractTraversable.filter(Traversable.scala:104)
at org.jetbrains.plugins.scala.compiler.CompileServerLauncher$.org$jetbrains$plugins$scala$compiler$CompileServerLauncher$$jdkChangeRequired(CompileServerLauncher.scala:282)
at org.jetbrains.plugins.scala.compiler.CompileServerLauncher.start(CompileServerLauncher.scala:56)
at org.jetbrains.plugins.scala.compiler.CompileServerLauncher.tryToStart(CompileServerLauncher.scala:41)
at org.jetbrains.plugins.scala.compiler.ServerMediator$$anon$1$$anonfun$buildStarted$2.apply$mcV$sp(ServerMediator.scala:43)
at org.jetbrains.plugins.scala.extensions.package$$anonfun$invokeAndWait$1$$anon$16.run(package.scala:569)
at com.intellij.openapi.application.TransactionGuardImpl$6.run(TransactionGuardImpl.java:318)
at com.intellij.openapi.application.impl.LaterInvocator$1.run(LaterInvocator.java:166)
at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.a(LaterInvocator.java:417)
at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:401)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
at com.intellij.ide.IdeEventQueue.b(IdeEventQueue.java:843)
at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:679)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:391)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
2016-12-28 10:59:44,183 [2448148] ERROR - .impl.MessageBusConnectionImpl - IntelliJ IDEA 2016.3.2 Build #IU-163.10154.41
2016-12-28 10:59:44,183 [2448148] ERROR - .impl.MessageBusConnectionImpl - JDK: 1.8.0_112-release
2016-12-28 10:59:44,183 [2448148] ERROR - .impl.MessageBusConnectionImpl - VM: OpenJDK 64-Bit Server VM
2016-12-28 10:59:44,183 [2448148] ERROR - .impl.MessageBusConnectionImpl - Vendor: JetBrains s.r.o
2016-12-28 10:59:44,183 [2448148] ERROR - .impl.MessageBusConnectionImpl - OS: Mac OS X
2016-12-28 10:59:44,183 [2448148] ERROR - .impl.MessageBusConnectionImpl - Last Action: CompileDirty
2016-12-28 10:59:48,828 [2452793] INFO - lij.compiler.impl.CompilerUtil - COMPILATION FINISHED (BUILD PROCESS); Errors: 0; warnings: 1 took 4878 ms: 0 min 4sec
CompileServerLauncher.scala indicates that this is something to do with not being able to find a JDK 8 to use, although if it can't find one it looks like it should fail with an error message, not a NullPointerException.
I still got this issue with IntelliJ 2021.2.2 Community Edition and simple restart of IntelliJ IDE resolved it.
I faced the same issue when using IntelliJ 2022.3.1.
I noticed that the compiler server working directory seems to change when switching between projects, so it is good to set the project home as the compiler server's default working directory.
Go to:
File -> Settings -> Build,Execution,Deployment -> Compiler -> ScalaCompiler -> ScalaCompileServer,
then check the box "Start process in project directory"
Also, do ensure that -server -Xss1m is included in the JVM VM options:

Can't build kie-wb-webapp 6.1.x (JBPM)

I need to modify KIE Workbench source and I have two issues that are stopping me from building with Maven 3.2.5, one on Windows and one on Linux.
On Windows 7, I am getting an error that says
[deletia...]SNAPSHOT\uberfire-backend-api-0.4.0-SNAPSHOT-sources.jar,
com.google.gwt.dev.Compiler,-logLevel, INFO, -style, OBF, -war,
c:\kie-wb-webapp\target\kie-wb-webapp-6.1.1-SNAPSHOT, -localWorkers, 1,
-strict, -XfragmentCount, -1, -deploy, c:\kie-wb-webapp\target\gwt-symbols-deploy, -gen,
c:\kie-wb-webapp\target\.generated, org.kie.workbench.FastCompiledKIEWebapp]:
Error while executing process. Cannot run program "c:\development\software\jdk\jdk1.7.0_71\jre\bin\java":
CreateProcess error=206, The filename or extension is too long.
So I looked it up and apparently, this is well known limitation since gwt-maven-plugin creates a command-line that is longer than 8191 chars and this is forbidden. So I gave it a shot on Ubuntu.
On Ubuntu,
I am getting:
[INFO] --- gwt-maven-plugin:2.5.1:compile (gwt-compile) # kie-wb-webapp [ERROR] Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
[ERROR] Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
[INFO] Compiling module org.kie.workbench.FastCompiledKIEWebapp
[INFO] Validating units:
[INFO] [ERROR] Errors in 'jar:file:/home/rich/.m2/repository/com/google/guava/guava-gwt/14.0.1/guava-gwt-14.0.1.jar!/com/google/common/primitives/Booleans.java'
[INFO] [ERROR] Line 29: The import java.util.BitSet cannot be resolved
[INFO] [ERROR] Aborting compile due to errors in some input files
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE*
It's really important that I find a workaround to 1 or both of these issues (my guess is that if I clear the command-line issue on Windows, I'll just run into the guava issue again). Also, unless it's absolutely necessary, I can't upgrade to 6.2.x, because this would cause a lot of work for my project.
Has anyone run into this?
If you don't need Java 8 support, you may be able to solve the problem by using an earlier version of the JDK. I believe that you do this by setting JAVA_HOME for your Maven run. You can download a JDK 7 here.
You could also build Guava yourself from HEAD, as the problem has been fixed there but not included in a release. Or you could wait for a 19.0-rc1, which we hope to have out soon.

Deploying Play Framework 2.3.x (Java style) on Ubuntu 14.04 production server

I have a Play application on my laptop, which I am trying to get running on my production server. Locally the application compiles and works without errors.
On my production server, I do the following:
root#example:~/mysite# ./activator
[info] Loading project definition from home/mysite/project
[info] Set current project to mysite (in build file:/home/mysite/)
[mysite] $ ~run
--- (Running the application, auto-reloading is enabled) ---
[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
(Server started, use Ctrl+D to stop and go back to the console...)
[info] Compiling 4 Scala sources and 2 Java sources to /home/mysite/target/scala-2.11/classes...
[info] 'compiler-interface' not yet compiled for Scala 2.11.1. Compiling...
Killed
root#example:~/mysite#
As you can see, `compiler-interface' is not yet compiled. I am wondering why this is. Is this because I don't have the Scala compiler installed?
Probably the Killed part is because you don't have enough memory to compile your application.
Either way I think it would be better to deploy your app already compiled than compile it on place. You can do that using ./activator dist.
This will generate a zip file in YOUR_PROJECT_DIR/target/universal named YOUR-PROJECT-NAME-VERSION.zip with everything already compiled and with every needed dependency packaged already. You can then upload this file to somewhere in your production server, extract it and then start the server using
$ EXTRACTED_FOLDER/bin/YOUR-APP-NAME
If you still want to build it in your production server, try to reduce the memory used by activator using
$ ./activator -mem 200 # MAX 200mb
// (...)
[PROJECT-NAME] ~start
Note that ~run will start your server in dev mode (ie sensible information like stacktraces etc can be shown on error) and recompile it on changes, so you may want to use start or ~start instead.