In a Scala project I updated the build.properties from 0.13.8 to 0.13.11. That "broke" sbt as it does not start anymore, i.e. it cannot download the 0.13.11 jars?! sbt prints a list of tried repo's, but the repo.typesafe.com was not one of them.
My local installed sbt is 0.13.8.
For some reason the scala-sbt jars are not available anymore in Typesafe's Bintray. Largest version there is 0.13.9.
I know the place to get it is https://repo.typesafe.com/typesafe/ivy-releases/, but how do I tell sbt to use this repo?
I have already tried:
adding a resolver to plugins.sbt
adding a resolver to build.sbt
adding the repo to .sbt/repositories
but I cannot get it working.
How to tell sbt where to get binaries?
Make sure that you're using sbt (launcher) that's at the same version of higher than the version used in your project.
Execute sbt about in an empty directory and find [info] This is sbt X.Y.X.
Make sure that you don't use ~/.sbt/repositories file that sets up the repositories used to resolve artifacts.
try to set your https proxy Information into http.proxyHost and http.proxyPort. This solved it for me.
I'm not an expert with sbt so probably my question is a bit noob, but I've notice than when I create a project and download its dependencies with sbt, if I open the project with intellij, all the dependencies are redownloaded again, the same happen in the inverse orden intellij->sbt and also activator..
my (poor) knowledge about sbt is than this use ivy and the dependencies are downloaded in ~/.ivy2/ folder...that is where sbt is downloading my deps, but seems than intellij is using other folder.
personally I don't use so much activator, but I would like configure sbt and intellij for use the same ivy path...
2)recently I publish finagle-postgre to my local ivy using sbt +publishLocal, I can check in my ivy folder
/home/yo/.ivy2/local/com.twitter/finagle-postgres_2.11/0.1.0-SNAPSHOT
but unfortunately intellij is unable to resolve this dependency, I try adding this line to my build
resolvers += Resolver.file("Local", file( Path.userHome.absolutePath + "/.ivy2/local"))(Resolver.ivyStylePatterns)
but seems not works
3) the path where is downloaded the dependencies is related to which sbt-launch.jar file is used? How can I know what sbt-lauch.jar file is using sbt right now...
thanks guys!
If we're talking about IntelliJ appearing to download artifacts after they've already been downloaded by SBT/Activator, then it turns out that it's probably just that IntelliJ is downloading the sources - it's not redownloading the binary artifacts, just the source artifacts that accompany them.
This isn't readily apparent when you're looking at the Refreshing SBT project task in the Background Tasks popup, because the full download path is truncated, so you see something like this:
[info] downloading https://repo1.maven.org/maven2/org/apache/httpcompo...
..it's natural to assume that this is the same binary artifact you already saw SBT download on the console, but you can see the full story if you check the full log (go Help -> Show Log in files and open sbt.last.log in the file browser).
You'll see that the only artifacts getting downloaded end with -sources.jar:
$ grep repo1.maven.org /home/roberto/.IntelliJIdea2016.3/system/log/sbt.last.log
[info] downloading https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.3.6/httpclient-4.3.6-sources.jar ...
[info] downloading https://repo1.maven.org/maven2/com/googlecode/javaewah/JavaEWAH/0.7.9/JavaEWAH-0.7.9-sources.jar ...
[info] downloading https://repo1.maven.org/maven2/org/pegdown/pegdown/1.2.1/pegdown-1.2.1-sources.jar ...
[info] downloading https://repo1.maven.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3-sources.jar ...
```
If you don't have the Sources checkbox checked when you're doing Import project, these source downloads won't happen.
tested using IntelliJ 2016.3.5 and Scala plugin v2016.3.9
First, the activator is just a launcher for SBT itself, so there should be no difference in behaviour.
Second, IntelliJ also uses the files in ~/.ivy2 by default if you have not told it otherwise (by setting SBT_OPTS environment variable for example, but that depends on your IntelliJ version).
A difference might result if you're using different scala versions (e.g. 2.10.x vs. 2.11.x) when you do not have set the scalaVersion in your project explicitly. Then, each tool would download the corresponding libraries for the appropriate scala version it has configured by default.
Another thing is that IntelliJ will download source and javadoc jars for each dependency if you have enabled that in your settings which might look like it downloads the dependencies again.
Note, I'm wildly guessing here because you have not included any output of the programs you're using, so it's hard to say what the real problem is.
In a freshly checked out sbt ( 0.3.7 ) project and empty ivy cache, is it possible to trigger compilation of the compiler-interface(s) needed without compiling the project itself? I have poked around but haven't found a way.
Currently if a compiler-interface is required it will be created during compilation of the project. I would like to have this compiled directly in a separate command if possible. This would allow CircleCi to cache it saving 1-3 minutes with every build because it could be cached in the dependencies section of the circle.yml.
In sbt 0.13.12 compile:compileIncremental seems to do the trick. I ran inspect compile and inspected its dependencies to find the command.
In a given project that is driven by sbt there is some kind of corruption in the project libraries specifically for a MavenLocal repository used for kafka-spark-9.7.2.jar in which:
references to the classes provided by that jar are marked as "symbol not found" by the editor parser
however the editor (strangely) does offer to import the classes
but after accepting the import, the symbols are still marked in red as unresolved.
The following attempts to "clean things up" have already been performed:
Build | Make Project
Build | Rebuild Project
In addition I have verified that the project does build from
sbt package
on the command line
UPDATE After re-running sbt gen-idea the librraries are still not found by the Parser. yet the libraries exist -even IJ knows about them as shown in the following screenshot. Why is it that IJ can find the library
C:\Users\S80035683\.ivy2\cache\org.apache.kafka\kafka\jars\kafka-0.7.2-spark.jar!\kafka\api\FetchRequest.class
However IJ is unable to resolve any classes from that library in the Parser?
You have to build the project for Intellij, try to do this in your project root:
sbt idea with-sources=yes
This should build the project structure from scratch and add the right dependencies, usually I refresh the project after adding a dependency or a jar.
Edit:
To use the command you need this plugin, otherwise you can use gen-idea but I used it only a few times and I'm not sure how it will work out.
Edit2:
There was some confusion, first, for the IDEA SBT console you don't need to prepend the sbt command since you already are inside sbt:
If you have the sbt plugin for idea you can use gen-idea with-source=yes (without prepending sbt)
From the terminal, either you go to your project root and type sbt to enter the sbt console and use gen-idea or idea with-sources=yes (without prepending sbt)
or directly sbt gen-idea or if you have the plugin sbt idea with-sources=yes (prepending sbt)
To reach the sbt console inside idea you need to install the sbt plugin on preferences -> plugin and search for sbt and then View -> Tool Windows -> SBT Console:
To start the console click on the play button, to kill the console on the skull.
I had the same problem. I fixed it by directly writing the CLASSES and SOURCES of the problematic library. This can be found in .idea/libraries/SBT__<problematic library>_jar.xml
I've got an sbt (Scala) project that currently pulls artifacts from the web. We'd like to move towards a corporate-standardized Nexus repository that would cache artifacts. From the Nexus documentation, I understand how to do that for Maven projects. But sbt obviously uses a different approach. (I understand Ivy is involved somehow, but I've never used it and don't understand how it works.)
How do I tell sbt and/or the underlying Ivy to use the corporate Nexus repository system for all dependencies? I'd like the answer to use some sort of project-level configuration file, so that new clones of our source repository will automatically use the proxy. (I.e., mucking about with per-user config files in a dot-directory is not viable.)
Thanks!
Step 1: Follow the instructions at Detailed Topics: Proxy Repositories, which I have summarised and added to below:
(If you are using Artifactory, you can skip this step.) Create an entirely separate Maven proxy repository (or group) on your corporate Maven repository, to proxy ivy-style repositories such as these two important ones:
http://repo.typesafe.com/typesafe/ivy-releases/
http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/
This is needed because some repository managers cannot handle Ivy-style and Maven-style repositories being mixed together.
Create a file repositories, listing both your main corporate repository and any extra one that you created in step 1, in the format shown below:
[repositories]
my-maven-proxy-releases: http://repo.example.com/maven-releases/
my-ivy-proxy-releases: http://repo.example.com/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
Either save that file in the .sbt directory inside your home directory, or specify it on the sbt command line:
sbt -Dsbt.repository.config=<path-to-your-repo-file>
Good news for those using older versions of sbt: Even though, in the sbt 0.12.0 launcher jar at least, the boot properties files for older sbt versions don't contain the required line (the one that mentions repository.config), it will still work for those versions of sbt if you edit those files to add the required line, and repackage them into the sbt 0.12.0 launcher jar! This is because the feature is implemented in the launcher, not in sbt itself. And the sbt 0.12.0 launcher is claimed to be able to launch all versions of sbt, right back to 0.7!
Step 2: To make sure external repositories are not being used, remove the default repositories from your resolvers. This can be done in one of two ways:
Add the command line option -Dsbt.override.build.repos=true mentioned on the Detailed Topics page above. This will cause the repositories you specified in the file to override any repositories specified in any of your sbt files. This might only work in sbt 0.12 and above, though - I haven't tried it yet.
Use fullResolvers := Seq( resolver(s) for your corporate maven repositories ) in your build files, instead of resolvers ++= or resolvers := or whatever you used to use.
OK, with some help from Mark Harrah on the sbt mailing list, I have an answer that works.
My build class now looks like the following (plus some other repos):
import sbt._
//By extending DefaultWebProject, we get Jetty support
class OurApplication(info: ProjectInfo) extends DefaultWebProject(info) {
// This skips adding the default repositories and only uses the ones you added
// explicitly. --Mark Harrah
override def repositories = Set("OurNexus" at "http://our.nexus.server:9001/nexus/content/groups/public/")
override def ivyRepositories = Seq(Resolver.defaultLocal(None)) ++ repositories
/* Squeryl */
val squeryl = "org.squeryl" % "squeryl_2.8.0.RC3" % "0.9.4beta5"
/* DATE4J */
val date4j = "hirondelle.date4j" % "date4j" % "1.0" from "http://www.date4j.net/date4j.jar"
// etc
}
Now, if I delete the Squeryl tree from my machine's .ivy2/cache directory, sbt tries to grab it from the Nexus tree with the appropriate URL. Problem solved!
All you need is to define a property file sbt.boot.properties which will allow you to:
redefine the ivy cache location (I need that because it would be otherwise part of our roaming Windows profile, which is severely limited in disk space in our shop. See Issue 74)
define any other Maven repo you want
C:\HOMEWARE\apps\sbt-0.74\sbt.boot.properties
[scala]
version: 2.7.7
# classifiers: sources, javadoc
[app]
org: org.scala-tools.sbt
name: sbt
version: read(sbt.version)
class: sbt.xMain
components: xsbti
cross-versioned: true
classifiers: sources, javadoc
[repositories]
local
my-nexus: http://my.nexus/nexus/content/repositories/scala-tools/, [organization]/[module]/[revision]/[type]s/[artifact](-[classifier]).[ext]
maven-local
# sbt-db: http://databinder.net/repo/, [organization]/[module]/[revision]/[type]s/[artifact](-[classifier]).[ext]
# maven-central
# scala-tools-releases
# scala-tools-snapshots
[boot]
directory: project/boot
properties: project/build.properties
prompt-create: Project does not exist, create new project?
prompt-fill: true
quick-option: true
[log]
level: debug
[app-properties]
project.name: quick=set(test), new=prompt(Name)[p], fill=prompt(Name)
project.organization: new=prompt(Organization)[org.vonc]
project.version: quick=set(1.0), new=prompt(Version)[1.0], fill=prompt(Version)[1.0]
build.scala.versions: quick=set(2.8.0.RC2), new=prompt(Scala version)[2.8.0.RC2], fill=prompt(Scala version)[2.8.0.RC2]
sbt.version: quick=set(0.7.4), new=prompt(sbt version)[0.7.4], fill=prompt(sbt version)[0.7.4]
project.scratch: quick=set(true)
project.initialize: quick=set(true), new=set(true)
[ivy]
cache-directory: C:\HOMEWARE\projects\.ivy2\cache
Note: this sbt.boot.properties file is inspired from:
the one mentioned in the "Generalized Launcher" page of the sbt project.
the one found within sbt-0.74 itself!
I have commented any external Maven repository definition, and added a reference to my own Nexus Maven repo.
The launcher may be configured in one of the following ways in increasing order of precedence:
Replace the /sbt/sbt.boot.properties file in the jar.
Put a configuration file named sbt.boot.properties on the classpath. Put it in the classpath root without the /sbt prefix.
Specify the location of an alternate configuration on the command line. This can be done by:
either specifying the location as the system property sbt.boot.properties
or as the first argument to the launcher prefixed by '#'.
The system property has lower precedence.
Resolution of a relative path is:
first attempted against the current working directory,
then against the user's home directory,
and then against the directory containing the launcher jar.
An error is generated if none of these attempts succeed.
Define a sbt.bat wrapper (in order to be sure to specify your sbt.boot.properties) like:
C:\HOMEWARE>more C:\HOMEWARE\bin\sbt.BAT
#echo off
set t=%~dp0
set adp0=%t:C:\="%"
set SBT_DIR=%adp0%..\apps\sbt-0.74
dir C:\%SBT_DIR%\sbt-launch-0.7.4.jar
# if needed, add your proxy settings
set PROXY_OPTIONS=-Dhttp.proxyHost=my.proxy -Dhttp.proxyPort=80xx -Dhttp.proxyUser=auser -Dhttp.proxyPassword=yyyy
set JAVA_OPTIONS=-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256m -Xmx512M -cp C:\HOMEWARE\apps\sbt-0.74\sbt-launch-0.7.4
set SBT_BOOT_PROPERTIES=-Dsbt.boot.properties="sbt.boot.properties"
cmd /C C:\HOMEWARE\apps\jdk4eclipse\bin\java.exe %PROXY_OPTIONS% %JAVA_OPTIONS% %SBT_BOOT_PROPERTIES% -jar C:\HOMEWARE\apps\sbt-0.74\sbt-launch-0.7.4.jar %*
And your sbt will download artifacts only from:
your Nexus
your local Maven repo.
Just tested at home with an old Nexus opensource 1.6 I had running, java 1.6, sbt07.4
C:\Prog\Java\jdk1.6.0_18\jre\bin\java -Xmx512M -Dsbt.boot.properties=sbt.boot.properties - jar "c:\Prog\Scala\sbt\sbt-launch-0.7.4.jar"
That gives:
[success] Build completed successfully.
C:\Prog\Scala\tests\pp>sbt
Getting Scala 2.8.0 ...
downloading http://localhost:8081/nexus/content/repositories/scala/org/scala-lang/scala-compiler/2.8.0/scala-compiler-2.
8.0.jar ...
[SUCCESSFUL ] org.scala-lang#scala-compiler;2.8.0!scala-compiler.jar (311ms)
downloading http://localhost:8081/nexus/content/repositories/scala/org/scala-lang/scala-library/2.8.0/scala-library-2.8.
0.jar ...
[SUCCESSFUL ] org.scala-lang#scala-library;2.8.0!scala-library.jar (185ms)
:: retrieving :: org.scala-tools.sbt#boot-scala
confs: [default]
2 artifacts copied, 0 already retrieved (14484kB/167ms)
[info] Building project test 0.1 against Scala 2.8.0
[info] using sbt.DefaultProject with sbt 0.7.4 and Scala 2.7.7
If I try a funny value in the sbt.boot.properties file:
C:\Prog\Scala\tests\pp>sbt
Getting Scala 2.9.7 ...
:: problems summary ::
:::: WARNINGS
module not found: org.scala-lang#scala-compiler;2.9.7
==== nexus: tried
http://localhost:8081/nexus/content/repositories/scala/org/scala-lang/scala-compiler/2.9.7/scala-compiler-2.9.7.pom
-- artifact org.scala-lang#scala-compiler;2.9.7!scala-compiler.jar:
http://localhost:8081/nexus/content/repositories/scala/org/scala-lang/scala-compiler/2.9.7/scala-compiler-2.9.7.jar
So it does limit itself to the two repo I defined:
[repositories]
nexus: http://localhost:8081/nexus/content/repositories/scala
nexus2: http://localhost:8081/nexus/content/repositories/scala, [organization]/[module]/[revision]/[type]s/[artifact](-[classifier]).[ext]
(I commented everything else: local, maven-local, ...)
If I comment all repositories and put a funny value (2.7.9) for the scala version in the sbt.boot.properties, I do get (like the OP did)
C:\Prog\Scala\tests\pp>sbt
Error during sbt execution: No repositories defined.
If I put 2.7.7 (while still having all repo commented), yes, it won't generate an error:
C:\Prog\Scala\tests\pp>sbt
[info] Building project test 0.1 against Scala 2.8.0
[info] using sbt.DefaultProject with sbt 0.7.4 and Scala 2.7.7
But that's only because it already had downloaded scala2.8.0 during my previous tries.
If I remove that library from my project/boot directory, then it will throw an Exception:
[info] using sbt.DefaultProject with sbt 0.7.4 and Scala 2.7.7
> C:\Prog\Scala\tests\pp>sbt
Error during sbt execution: No repositories defined.
at xsbt.boot.Pre$.error(Pre.scala:18)
at xsbt.boot.Update.addResolvers(Update.scala:197)
...
at xsbt.boot.Boot$.main(Boot.scala:15)
at xsbt.boot.Boot.main(Boot.scala)
Error loading project: Error during sbt execution: No repositories defined.
edit the config file in sbt_home/conf "sbtconfig.txt"
add two line
-Dsbt.override.build.repos=true
-Dsbt.repository.config="C:/Program Files (x86)/sbt/conf/repo.properties"
the repo.properties content is
[repositories]
local
public: http://222.vvfox.com/public <-fix this ,write your local nexus group url
Well this has bugged me for a while so I found a guy that has written an SBT plugin for maven out on github called maven-sbt so all you have to do is include it in your plugins project and make your project mixin with maven.MavenDependencies and all your operations like update and publish-local work with your local maven. The nice thing about that is if you are like me, your org is all maven. So, all you libs are in you local maven repo but if for some reason you build with sbt first, then you start getting a bunch or jars in ivy too. What a waste of space, and time since you will still need to get them for your maven builds.
That said, I wish this were built into sbt so I would not need to add it to every project. Maybe as a processor at least. He mentioned in one thing I read that he would like to add it to 0.9 but I have not been able to find it.
I got this error because I had a blank file in ~/.sbt/repositories. Both adding repositories to the file and removing the file solved the problem.