I'm trying to use sbt on an amazon ec2 micro instance but I'm getting this error when I execute sbt command.
mkdir prueba
cd prueba
sbt
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (malloc) failed to allocate 715849728 bytes for committing reserved memory
Any ideas?
Thank you in advance!
You have to run sbt with a switch limitting the used memory to be less than the available on machine.
Instead of editing files, I run sbt with sbt -mem 256 for example to get -Xmx and -Xms to 256MB. Works fine.
I used to run sbt on EC2 micro. Previously SBT could be used with 256M heap or possibly even smaller. Try running something like this
export SBT_OPTS="-Xms128M -Xmx256M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256M"
java $SBT_OPTS -jardirname $0/sbt-launch.jar "$#"
Related
I am trying to run my scala application in a docker container with 800MB of RAM, (docker run -m=800m mindlevel) but it gets killed before it is done compiling.
[info] Compiling 10 Scala sources to /root/mindlevel-backend/target/scala-2.12/classes...
[info] 'compiler-interface' not yet compiled for Scala 2.12.1. Compiling...
[info] Compilation completed in 8.631 s
/usr/local/sbt/bin/sbt-launch-lib.bash: line 58: 57 Killed "$#"
I have tried to restrict the SBT heap with:
CMD ["sbt", "-mem", "700", "run"]
But it still gets killed at the same place.
Without any restrictions on the docker container it runs fine.
When running it in an EC2 micro instance (1GB of RAM) it also crashes, with or without restrictions on SBT and Docker.
Is there some restriction that I am missing?
To get it to run with less than 1GB heap I precompiled the project with sbt assembly which makes a jar packaged with your project and all of your dependencies.
Install it by adding:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.5")
to project/assembly.sbt
Then generate the jar with sbt assembly which will place the jar in ./target/scala-<VERSION>/<PROJECT>-assembly-1.0.jar
Then simply run your project with:
scala target/scala-<VERSION>/<PROJECT>-assembly-1.0.jar
Since no compilation step is needed on the low memory instance now a lot of projects will be able to run even though it is less memory than recommended.
I am working on a Scala project that has been in existence for a few years but is new to me. My task is to upgrade it from Scala 2.9.3 to 2.11.7, along with its dependencies. I have gotten past the errors and warnings, but I cannot get the project to compile successfully in SBT. I always get a StackOverflowError in pretty much the same place. The stacktrace looks like this, but details vary with the Xss setting (currently 4M, but have tried as high as 24M):
java.lang.StackOverflowError
at scala.tools.nsc.transform.Erasure$Eraser.typed1(Erasure.scala:698)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5395)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5422)
at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5369)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5373)
at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5471)
at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5479)
at scala.tools.nsc.transform.Erasure$Eraser.adaptMember(Erasure.scala:644)
at scala.tools.nsc.transform.Erasure$Eraser.typed1(Erasure.scala:698)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5395)
at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5422)
SBT_OPTS looks like this:
-Xmx2G -Xss4M -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
I can 'make' the project successfully in Intellij, and others can pull my changes from GitHub and compile the project in sbt, so the issue seems to be local to my machine (a recent quad-core Macbook Pro with 16GB RAM). Other Scala/sbt projects compile successfully for me on this machine.
Here are other relevant details:
Scala version: 2.11.7
Java version: java version "1.8.0_66" (build 1.8.0_66-b17)
sbt version: 0.13.7 (have also tried 0.13.9)
I have completely rebuilt the ivy2 cache and cleared the lib_managed directory. The version of the scala-compiler.jar is the same as is used on at least one machine that can 'sbt compile' the code successfully. I did a clean reinstall of sbt (via brew remove sbt, manual removal of ~/.sbt directory, then brew install sbt).
I have not tried to isolate the line of source code being compiled when the error occurs. I have assumed it would be more productive to look for a configuration issue or dependency conflict somewhere.
Any suggestions for further troubleshooting will be appreciated.
[Added...] It may be helpful to add that, as an experiment, I downloaded the Scala language source code from https://github.com/scala/scala and got the following very similar error trying to sbt compile it:
java.lang.StackOverflowError
at scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerValue(ExplicitOuter.scala:229)
at scala.tools.nsc.transform.ExplicitOuter$ExplicitOuterTransformer.transform(ExplicitOuter.scala:441)
at scala.tools.nsc.transform.ExplicitOuter$ExplicitOuterTransformer.transform(ExplicitOuter.scala:352)
at scala.reflect.internal.Trees$class.itransform(Trees.scala:1345)
at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:44)
at scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.scala$reflect$internal$Trees$UnderConstructionTransformer$$super$transform(ExplicitOuter.scala:219)
at scala.reflect.internal.Trees$UnderConstructionTransformer$class.transform(Trees.scala:1693)
at scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.transform(ExplicitOuter.scala:291)
at scala.tools.nsc.transform.ExplicitOuter$ExplicitOuterTransformer.transform(ExplicitOuter.scala:459)
at scala.tools.nsc.transform.ExplicitOuter$ExplicitOuterTransformer.transform(ExplicitOuter.scala:352)
at scala.reflect.internal.Trees$class.itransform(Trees.scala:1347)
at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
Here is something interesting. From this post I found out about launching sbt with a -d flag for debugging info. Got the following output:
Kevins-MacBook-Pro:scala kdoherty$ sbt -d
[process_args] java_version = '1.8.0_66'
# Executing command line:
java
-Xmx2G
-Xss4M
-XX:+UseConcMarkSweepGC
-XX:+CMSClassUnloadingEnabled
-Xmx384m
-Xss512k
-XX:+UseCompressedOops
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005
-jar
/usr/local/Cellar/sbt/0.13.9/libexec/sbt-launch.jar
So somewhere my SBT_OPTS settings are being overridden (by defaults, I guess). Now I need to find where those defaults are coming from.
I figured it out. Once I knew that the -d flag would tell me what settings SBT was actually using, I saw that the values in my SBT_OPTS environment variable were being clobbered by other, lower settings. Where were those coming from? From my JAVA_OPTS env variable! I should have noticed them sooner, but now I know I can keep those Java options as they are and override them by adding the SBT-specific settings to my /usr/local/etc/sbtopts file, using the somewhat awkward format of
-J-Xmx2G
-J-Xss2M
Using the values shown I was able to run sbt compile successfully on my project.
I hope someone finds this useful.
Add to the bottom of /usr/local/etc/sbtopts
-J-Xmx4G
-J-Xss4M
All set.
I just added -Xss in my Intellij sbt properties and the issue is resolved.
Intellij SBT properties
Relevant parts from the output of sbt -h:
# jvm options and output control
JAVA_OPTS environment variable, if unset uses ""
.jvmopts if this file exists in the current directory, its contents
are appended to JAVA_OPTS
SBT_OPTS environment variable, if unset uses ""
.sbtopts if this file exists in the current directory, its contents
are prepended to the runner args
Thus, in my case I solved the problem by creating a file .sbtopts with content
-J-Xmx4G
-J-Xss4M
in the project directory.
Note: Running sbt -d shows what settings have been used, for instance:
$ sbt -d
[addSbt] arg = '-debug'
[process_args] java_version = '8'
# Executing command line:
java
-Xms1024m
-XX:ReservedCodeCacheSize=128m
-XX:MaxMetaspaceSize=256m
-Xmx2G
-Xss2M
-jar
/path/to/sbt-launch.jar
-debug
I was unable to get this to work via the provided answers. build.sbt settings and the sbtopts file both failed to solve this error for me. What I had to do was run sbt with the -mem flag, for example:
sbt -mem 2048 compile
and now my projects built. If using IntelliJ you can also go to
Preferences > Build, Execution, Deployment > Build Tools > sbt
and set Maximum heap size, MB to whatever target you need.
I discovered the SBT_OPT setting in the bin/sbt file of my sbt install was affecting the memory values set in my projects build.sbt
updating the existing -Xss value in this file from 1M to 8M raised the memory size of the Scalac stack to a point I stopped getting StackOverflow exceptions in the sbt-invoked compiler. This seemed odd because the sbt documented approach to setting stack size in the compiler is to do this is with the -J-Xss setting.
Sbt doesn't seem to actually enable you to set the compiler's stack memory. While a build.sbt accepts the following configuration as a valid setting, it doesn't seem to apply the value in the compiler:
scalacOptions in ThisBuild ++= Seq(-J-Xss8M)
I suspect that is a bug or non-implemented functionality
I too run into this problem recently and I discovered a working solution for it with sbt 1.3.13.
create a .sbtopts file under project root
add -J-Xss100M (or any thread stack size you think suitable) to the .sbtopts file
There are multiple Correct Answers already. But what worked for me is below,
// Created and Added a File: .jvmopts in the Project Root Folder with below Parameters.
-Xms3022m
-Xmx4048m
-Xss124m
-XX:MaxPermSize=4048m
-XX:MaxMetaspaceSize=512m
-XX:+CMSClassUnloadingEnabled
-XX:ReservedCodeCacheSize=128m
Is there a way to increase the memory during assembly (the plugin)? I notice that adding more memory to sbt through SBT_OPTS does't really help. I suspect that assembly is forking a separate JVM that doesn't inherit the JVM configurations from sbt.
Here is the error:
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
Here is what I tried without much luck:
javaOptions in assembly += "-Xmx2g"
I suspect that assembly is forking a separate JVM that doesn't inherit the JVM configurations from sbt.
I'm the author of sbt-assembly. assembly does not fork a separate JVM.
You can see the code here and check - https://github.com/sbt/sbt-assembly/blob/0.13.0/src/main/scala/sbtassembly/Assembly.scala#L239-L246
Make sure to export SBT_OPTS. Mine is "-Xmx2G -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=2G -Xss2M"
I met the same issue before, Please do it as below for windows. The issue could be gone.
set SBT_OPTS="-Xmx2G"
sbt assembly
You can increase heap size by using below mentioned command in linux environment:-
sbt -J-Xms2048m -J-Xmx2048m assembly
I tried to work with easy example of scala language.
I installed sbt successfully. When run sbt -h I see help message from sbt:
Usage: sbt [options]
-h | -help print this message
-v | -verbose this runner is chattier
-d | -debug set sbt log level to debug
-no-colors disable ANSI color codes
-sbt-create start sbt even if current directory contains no sbt project
-sbt-dir <path> path to global settings/plugins directory (default: ~/.sbt)
-sbt-boot <path> path to shared boot directory (default: ~/.sbt/boot in 0.11 series)
-ivy <path> path to local Ivy repository (default: ~/.ivy2)
-mem <integer> set memory options (default: 1536, which is -Xms1536m -Xmx1536m -XX:MaxPermSize=384m -XX:ReservedCodeCacheSize=192m) ....
but when I tried to run in the directory with scala assignments (it's Coursera course) I caught error:
nazar_art#nazar-desctop:~/scala/example$ sbt
Detected sbt version 0.12.1
Starting sbt: invoke with -help for other options
Error occurred during initialization of VM
Could not reserve enough space for object heap
but it should open the sbt command prompt. Smt like this:
shell$ cd /path/to/progfun-project-directory # This is the shell of the operating system
shell$ sbt
> _ # This is the sbt shell
UPDATE:
After I add this following line
java -Xms64M -Xmx256M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=64M -jardirname $0/sbt-launch.jar "$#"
at /usr/bin I see next output:
nazar_art#nazar-desctop:/usr/bin$ sbt
bash: /usr/bin/sbt: Permission denied
nazar_art#nazar-desctop:/usr/bin$ sudo sbt
sudo: sbt: command not found
- Why this happen?
- How to solve this issue?
First of all, The example folder name (and the folder name that you have installed sbt) cannot include "!". this probably applies to other special characters as well. Even, I recommend to install SBT in a folder that has no space on its path (and the same thing for examples folder).
For example, if you have installed SBT (or examples folder) on a path like:
/Users/dashti/My Programs/sbt-0.12.1
You can re-install it a path like:
/Users/dashti/MyPrograms/sbt-0.12.1
By applying this naming change, if the error is gone, please ignore the rest of this post.
Otherwise, to fix the mentioned error, first find sbt executable script, then edit it and correct any problem in it:
If you are using a UNIX-based operating system, please follow these steps:
1- Find SBT: Open a terminal and type this command:
which sbt
and you will receive a path like:
/Users/dashti/Documents/scala-2.10.0/bin/sbt
and in this file, there is a command for executing a java program, like:
java -Xms512M -Xmx1024M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=128M -jar `dirname $0`/sbt-launch.jar "$#"
2- Edit sbt script: by entering this command in terminal:
vim /Users/dashti/Documents/scala-2.10.0/bin/sbt
you will be able to edit this file and maybe you should change the memory usage arguments to a lower specification ( You should think about whether you have enough RAM to be running the sbt). For example, this one will be a better choice:
java -Xms64M -Xmx256M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=64M -jar `dirname $0`/sbt-launch.jar "$#"
SBT needs more memory allocated to the JVM.
This can be done in build.sbt:
javaOptions ++= Seq(
"-Xms64m",
"-Xmx256M",
"-XX:+CMSClassUnloadingEnabled",
"-XX:MaxPermSize=64M"
);
Creating an sbt file(no extentions) and running ./sbt is better, but with the code below(#MohammadDashti's suggestion), sbt-launch.jar needs to be in the project folder.
java -Xms512M -Xmx1024M -Xss1M -XX:+CMSClassUnloadingEnabled \
-XX:MaxPermSize=128M -jar `dirname $0`/sbt-launch.jar "$#"
Later you could add javaagent:/path/to/jrebel -Drebel.remoting_plugin=true and re-deploy changes on save with a free JRebel for Scala license.
I have noticed a strange behavior of my scala compiler. It occasionally throws an OutOfMemoryError when compiling a class. Here's the error message:
[info] Compiling 1 Scala source to /Users/gruetter/Workspaces/scala/helloscala/target/scala-2.9.0/test-classes...
java.lang.OutOfMemoryError: PermGen space
Error during sbt execution: java.lang.OutOfMemoryError: PermGen space
It only happens once in a while and the error is usually not thrown on the subsequent compile run. I use Scala 2.9.0 and compile via SBT.
Does anybody have a clue as to what might be the cause for this error? Thanks in advance for your insights.
I use HomeBrew to install sbt on OS X. It supports a SBT_OPTS argument which can be put in ~/.sbtconfig file with export SBT_OPTS=-XX:MaxPermSize=256M.
The cause for OutOfMemoryError: PermGen space is that it doesn't have enough permanent generation space :) If you are using Oracle JVM, you need to add the -XX:MaxPermSize=256M (or some other amount of space) argument to your sbt script. For other JVMs, look at their documentation.
I assumed you're using sbt 0.13.6 or higher. Create .sbtopts file in your sbt project's root with the following content:
-J-Xmx4G
-J-XX:MaxMetaspaceSize=1G
-J-XX:MaxPermSize=1G
-J-XX:+CMSClassUnloadingEnabled
MaxMetaspaceSize is for Java 8 whereas MaxPermSize is for Java 7. They are critical to prevent out of memory errors related either to permgen or metaspace exhaustion. Of course, consider adapting flag values or adding any other flags required.
More details and alternative approaches can be found in this blog post.
I had this issue, played around with it for 10 minutes looking at sites trying to change the memory size.
Turns out i resolved it by,
user-profile$ sbt
Then,
sbt-project-name 0.1> clean
This cleared it up for me.
It looks like a memory leak in SBT for me as in my case the program compiles and runs successfully for about 3-5 times before hitting the exception which is fixed by SBT restart.
The most adequate solution indeed seems to be -XX:MaxPermSize= JVM parameter as Alexey Romanov suggests or to restart SBT periodically if it helps.
But there is another interesting way: try switching to Java 8. AFAIK it doesn't use PermGen any more and is probably immune to this exception this way.
I still hope SBT authors will address this issue in future versions.
I am building with the Jenkins sbt plugin and had the same problems. They were resolved after copying the SBT_OPTS from the sbt file to the Jenkins job config's JVM flags.
Originally using a command like:
java -jar /path/to/sbt-launch.jar test
I got first OutOfMemoryError: PermGen space which I solved using -XX:MaxPermSize, and then OutOfMemoryError: Java heap space, to which -Xmx was the remedy.
So in my case, a command like this worked:
java -XX:MaxPermSize=256M -Xmx2048M -jar /path/to/sbt-launch.jar test
change following code block in sbt.sh file and save its working fine.
get_mem_opts () {
local mem=${1:-1536}
local perm=$(( $mem / 4 ))
(( $perm > 256 )) || perm=1024 //256 to 1024
(( $perm < 1024 )) || perm=2048 // 1024 to 2048
local codecache=$(( $perm / 2 ))
echo "-Xms${mem}m -Xmx${mem}m -XX:MaxPermSize=${perm}m -XX:ReservedCodeCacheSize=${codecache}m"
}
or
using terminal to export sbt config
export SBT_OPTS="-XX:+CMSClassUnloadingEnabled -XX:PermSize=1024M -XX:MaxPermSize=2048M"
You can also add a .jvmopts file in the root folder of your project, and write inside the file the following:
-Xms1g
-Xmx4g