how can I change the coursier cache directory of the bundled sbt in IntelliJ - scala

I've migrated my local coursier cache to another part of my system and changed the COURSIER_CACHE system property, as described in the coursier docs
My local installation of SBT works fine with this, via shell as well as via IntelliJ's sbt shell. However, when building my app using IntelliJ's build-in SBT using CTRL+F9, I get the following error:
scalac: Scala compiler JARs not found (module 'solipsism'): C:\Users\<user>\AppData\Local\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-library\2.13.5\scala-library-2.13.5.jar, <etc>
with C:\Users\<user>\AppData\Local\Coursier\cache being the default coursier cache location.
So far, I tried to add the following VM Parameters to the Settings > Build Execution Deployment > sbt > VM Parameters without success:
-Dcoursier.cache=<repo dir>
-Dsbt.coursier.home=<repo dir>
as described in the SBT docs and in the coursier docs again
Verifying these settings via IntelliJ's SBT shell using csrCacheDirectory shows that they were picked up correctly, but the build error persists.
How can I change the coursier cache directory for the default build?
Using Windows 10, IntelliJ CE 2021.1.1, SBT 1.5.1
Thanks,
Cheers,
Joost Papendorp

Found the cause: IntelliJ had somehow retained the links to the old repository in the project structure. This is apparently not fixed by re-importing the project.
Solution: Delete entire Intellij project (not just the modules). Close IDE. Start new project. Re-import modules.
Classy.

https://get-coursier.io/docs/cache#sbtd mentions two options:
-Dsbt.coursier.home system property, or
COURSIER_CACHE environment variable

Related

sbt Project name must be valid Scala identifier error while creating new project

I wanted to create a new project in scala in intellij. I have chosen Lightbend Project Starter and then Akka Quickstart Scala. On finish I got this error:
sbt Project name must be valid Scala identifier
The name of my project is "akka-demo" but "akkademo" and default "untitled" don't work either. Empty catalog is created. Same with other templates. How can I overcome this?
Env:
Intellij idea ultimate 18.3.4
sbt 1.2.6
As mentioned, akkademo should indeed work.
It sounds like an IntelliJ problem.
If a project directory was created, you can try to remove the .idea directory and reimport the project.
Furthermore, you can download the Akka Quickstart Scala project from Lightbend's website, and start the sbt shell in this directory.
Try to run and compile to make sure your environment is set up correctly.
If everything works with the sbt shell but not in IntelliJ, then open:
IntelliJ -> Preferences -> Build, Execution, Deployment -> Build Tools -> sbt and check the Use sbt shell for imports and for builds.
Then go to the sbt tab in IntelliJ and refresh.

IntelliJ Cannot Import sbt Project

I have upgraded IntelliJ to version CE 2017.3 from a CE 2017.2.5, and now I cannot build my sbt project. The build gets stuck on the first step:
"Build: sync
MyProjName: syncing...
dump project structure from sbt"
I have deleted the .idea directory and the the MyProjName/project/target directory. I did an sbt build from the command line, which worked.
Then I started IntelliJ again and tried to import MyProjName/build.sbt as a project. I have also logged out/in. Did this several times. The import never progresses past this step.
I work behind a firewall, but I have checked my proxy settings, and they are correct and unchanged from the previous version (if this matters).
How can I troubleshoot / fix this?
The sbt version is 0.13.6. Scala plugin 2017.3.10.
From the log:
IDE: IntelliJ IDEA (build #IC-173.3727.127, 27 Nov 2017 09:32)
OS: Windows 7 (6.1, amd64)
JRE: 1.8.0_152-release-1024-b6 (JetBrains s.r.o)
JVM: 25.152-b6 (OpenJDK 64-Bit Server VM)
I solved the sync issue following info here:
https://youtrack.jetbrains.com/issue/SCL-13024
Key info is "It seems that sbt is trying to re-resolve 'sources' and 'docs' for sbt plugins that are not available at all. If properties to download "Library sources" and "sbt sources" are disabled, then time to dump the structure is reduced from 'forever' to less than 1 minute."
That is , in IntelliJ IDEA, clear the check box "Library sources" under "Project level settings" for Build Tools sbt in IDEA preferences. As shown in image below:
The only way I could work around this was by running sbt clean compile in the SBT console/cmd line first, then refreshing the SBT project in IntelliJ.
Open a terminal inside IDEA by pressing Alt F12 or use sbt shell
Run sbt - if it is first time it will take some time.
Run clean
Run compile - it will download dependencies you have in your project.
Close IDEA project (File -> Close Project) and open it again.
Note: if it asks to import in the right-bottom of the screen activate Enable auto-import, if you don't like to keep it enabled (like me) press Import, cancel the syncronyzation and then Refresh sbt project again. When you try to use import in the middle of a sync it gets stuck with dump project structure from sbt message.
The Refresh sbt project is located at this corner of the Build Sync window - highlighted below:
I have not found a way to make this work in IntelliJ Idea CE 2017.3, so I have reverted to CE 2017.2.5
IntelliJ appears to limit the efficiency of the already slow sbt update. I recommend using sbt-coursier instead. It was written specifically to improve dependency update time, and on one large project I'm involved in it improved dependency update time by about 2 orders of magnitude.
Uncheck "Use auto-import" in Preferences > Build, Execution, Deployment > Build Tools > sbt to prevent IntelliJ from auto updating imports on its own
Add addSbtPlugin("io.get-coursier" % "sbt-coursier" % "1.0.3") to your project's build.sbt file
Or install it globally in ~/.sbt/1.0/plugins/build.sbt
Run sbt update on the command line
If you see [error] java.lang.NoClassDefFoundError: org/vafer/jdeb/Console, then add the following to ~/.sbt/1.0/sbt-coursier.sbt.
import coursier.Keys._
classpathTypes += "maven-plugin"
See coursier issue #450 for more information about that error.
(Optional) Re-enable auto-import from step 1, as subsequent transactional dependency updates may be faster.
I meet the same problem! I delete the directory ~/.sbt/0.13, then it works. You can delete the sbt directory you used.
What worked for me was to increase the memory for SBT inside IntelliJ. On my Mac: preference -> Build, Execution, Depoyment -> sbt
then change "Maximum heap size" from 768MB to 2048MB and then everything was successfully imported after few minutes.
Custom SBT version works for me.
IntelliJ IDEA Preferences -> Build,Execution,Deployment -> Build Tools -> sbt -> Launcher
Choose Custom
For me, simply checking use sbt shell for both build and imports worked.
Had an older checked out project causing this issue. Did the usual rm -rf .idea + re-import but it kept getting stuck. Tried all sorts of things, but when finally fixed it for me was not just deleting the top-level target folder, but instead doing
find . -name target -type d -exec rm -rf {} \;
Next open / import went flawlessly. (This was on 2018.1.5, 2018.1.6 and 2018.2 RC.)
I have suffered from same problems after upgrade intelliJ to 2017.3
But it works fine now.
I think intelliJ couldn't pass the 'No proxy for' option to sbt.
so, I added
-Dhttp.nonProxyHosts="host1|host2"
option in File > Settings > Build, Excution, Deployment > sbt > VM parameters
Simply unchecking 'sbt sources' did the trick for me
I was also facing this issue. Below two configurations worked for me. I tried to confirm this setting in many new sbt projects, for all it worked.
(1) uncheck "Library sources" under "Project level settings" for Build Tools sbt in
IDEA preferences
(2) -Dhttp.nonProxyHosts="host1|host2"
option in File > Settings > Build, Excution, Deployment > sbt > VM parameters
I hit similar issue but found it's related to waiting for locks
~/.sbt/tool/sbt.boot.lock
~/.ivy2/.sbt.ivy.lock
Maybe if killing IntelliJ IDEA and no one will release the lock files, the issue is resolved after deleting locks files manually

intellij/activator/sbt are downloading dependencies previously downloaded by the other

I'm not an expert with sbt so probably my question is a bit noob, but I've notice than when I create a project and download its dependencies with sbt, if I open the project with intellij, all the dependencies are redownloaded again, the same happen in the inverse orden intellij->sbt and also activator..
my (poor) knowledge about sbt is than this use ivy and the dependencies are downloaded in ~/.ivy2/ folder...that is where sbt is downloading my deps, but seems than intellij is using other folder.
personally I don't use so much activator, but I would like configure sbt and intellij for use the same ivy path...
2)recently I publish finagle-postgre to my local ivy using sbt +publishLocal, I can check in my ivy folder
/home/yo/.ivy2/local/com.twitter/finagle-postgres_2.11/0.1.0-SNAPSHOT
but unfortunately intellij is unable to resolve this dependency, I try adding this line to my build
resolvers += Resolver.file("Local", file( Path.userHome.absolutePath + "/.ivy2/local"))(Resolver.ivyStylePatterns)
but seems not works
3) the path where is downloaded the dependencies is related to which sbt-launch.jar file is used? How can I know what sbt-lauch.jar file is using sbt right now...
thanks guys!
If we're talking about IntelliJ appearing to download artifacts after they've already been downloaded by SBT/Activator, then it turns out that it's probably just that IntelliJ is downloading the sources - it's not redownloading the binary artifacts, just the source artifacts that accompany them.
This isn't readily apparent when you're looking at the Refreshing SBT project task in the Background Tasks popup, because the full download path is truncated, so you see something like this:
[info] downloading https://repo1.maven.org/maven2/org/apache/httpcompo...
..it's natural to assume that this is the same binary artifact you already saw SBT download on the console, but you can see the full story if you check the full log (go Help -> Show Log in files and open sbt.last.log in the file browser).
You'll see that the only artifacts getting downloaded end with -sources.jar:
$ grep repo1.maven.org /home/roberto/.IntelliJIdea2016.3/system/log/sbt.last.log
[info] downloading https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.3.6/httpclient-4.3.6-sources.jar ...
[info] downloading https://repo1.maven.org/maven2/com/googlecode/javaewah/JavaEWAH/0.7.9/JavaEWAH-0.7.9-sources.jar ...
[info] downloading https://repo1.maven.org/maven2/org/pegdown/pegdown/1.2.1/pegdown-1.2.1-sources.jar ...
[info] downloading https://repo1.maven.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3-sources.jar ...
```
If you don't have the Sources checkbox checked when you're doing Import project, these source downloads won't happen.
tested using IntelliJ 2016.3.5 and Scala plugin v2016.3.9
First, the activator is just a launcher for SBT itself, so there should be no difference in behaviour.
Second, IntelliJ also uses the files in ~/.ivy2 by default if you have not told it otherwise (by setting SBT_OPTS environment variable for example, but that depends on your IntelliJ version).
A difference might result if you're using different scala versions (e.g. 2.10.x vs. 2.11.x) when you do not have set the scalaVersion in your project explicitly. Then, each tool would download the corresponding libraries for the appropriate scala version it has configured by default.
Another thing is that IntelliJ will download source and javadoc jars for each dependency if you have enabled that in your settings which might look like it downloads the dependencies again.
Note, I'm wildly guessing here because you have not included any output of the programs you're using, so it's hard to say what the real problem is.

Installing SBT on Win 7 64 bit

I want to install Apache Spark for testing purpose. For that I found out that Scala and sbt are necessary. I downloaded scala msi and installed it. For installing sbt I tried various methods but am unable to do so. Can someone tell me what am I doing wrong. What I did is
Install Scala msi
Download sbt msi and install it.
Set sbt_home and path variable to the location where sbt is extracted. Then I opened cmd to check my sbt version by using sbt sbt-version I am getting the following error **unresolved dependency:
org.fusesource.jansi#jansi;1.11: not found
Error during sbt execution: Error retrieving required libraries (see C:\Users\ashish-b\.sbt\boot\update.log for complete log) Error: Could not retrieve jansi 1.11 **
Whats wrong in it?
I saw this issue as well when connecting to the internet via a corporate proxy. In this case, sbt couldn't download its dependencies.
We work with a proxy Maven repository for depedencies. Configure sbt to use a proxy repo.
Our sbt repositories file looks like this:
[repositories]
local
local-maven: file:///C:/data/maven_repo/
aaa-ext-ivy-proxy: http://nexus-ext.company.net:8081/nexus/content/groups/ivy-public/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
aaa-ext-maven-proxy: http://nexus-ext.company.net:8081/nexus/content/groups/public/
aaa-int-maven-repo: http://nexus-int.company.net:8081/nexus/content/groups/public/
Or you can also configure the proxy server directly for SBT, see this question.

Intellij unable to resolve references to a specific jar file

In a given project that is driven by sbt there is some kind of corruption in the project libraries specifically for a MavenLocal repository used for kafka-spark-9.7.2.jar in which:
references to the classes provided by that jar are marked as "symbol not found" by the editor parser
however the editor (strangely) does offer to import the classes
but after accepting the import, the symbols are still marked in red as unresolved.
The following attempts to "clean things up" have already been performed:
Build | Make Project
Build | Rebuild Project
In addition I have verified that the project does build from
sbt package
on the command line
UPDATE After re-running sbt gen-idea the librraries are still not found by the Parser. yet the libraries exist -even IJ knows about them as shown in the following screenshot. Why is it that IJ can find the library
C:\Users\S80035683\.ivy2\cache\org.apache.kafka\kafka\jars\kafka-0.7.2-spark.jar!\kafka\api\FetchRequest.class
However IJ is unable to resolve any classes from that library in the Parser?
You have to build the project for Intellij, try to do this in your project root:
sbt idea with-sources=yes
This should build the project structure from scratch and add the right dependencies, usually I refresh the project after adding a dependency or a jar.
Edit:
To use the command you need this plugin, otherwise you can use gen-idea but I used it only a few times and I'm not sure how it will work out.
Edit2:
There was some confusion, first, for the IDEA SBT console you don't need to prepend the sbt command since you already are inside sbt:
If you have the sbt plugin for idea you can use gen-idea with-source=yes (without prepending sbt)
From the terminal, either you go to your project root and type sbt to enter the sbt console and use gen-idea or idea with-sources=yes (without prepending sbt)
or directly sbt gen-idea or if you have the plugin sbt idea with-sources=yes (prepending sbt)
To reach the sbt console inside idea you need to install the sbt plugin on preferences -> plugin and search for sbt and then View -> Tool Windows -> SBT Console:
To start the console click on the play button, to kill the console on the skull.
I had the same problem. I fixed it by directly writing the CLASSES and SOURCES of the problematic library. This can be found in .idea/libraries/SBT__<problematic library>_jar.xml