I have a Maven POM file that the deployment engineers need to deploy the system in the enterprise. I have developers using SBT for a Scala project. They use SBT targets that just aren't supported in Maven. We'd like to use the Maven POM file to define the dependencies, slurp in those dependencies in SBT, and define SBT development targets there.
According to the SBT documentation, the externalPom() command is the way to do that. But even with the simplest POM file (two developers have tried this with two different simple POM files that defined different dependencies), the externalPom() command seems to half work. The SBT targets clearly recognize the dependency defined in the POM, but can't resolve it. This error arises:
Cannot add dependency 'commons-collections#commons-collections;3.2.2'
to configuration 'default' of module
default#maven-sbt$sources_javadoc;0.1-SNAPSHOT because this
configuration doesn't exist!
When the externalPom() command is commented out and the equivalent dependency added directly in the build.sbt file everything goes swimmingly. The dependency comes directly from Maven Central in both cases; one from copying the dependency from the Maven tab and one from copying the dependency from the SBT tab. Once again, two developers are seeing exactly the same thing, from two different dependencies. The thing that's common is that both developers have reduced the build.sbt file down to a single statement. In the "slurp from POM" case, that statement is externalPom(). In the "plain old SBT" case, that statement is the dependency copied from Maven Central. The POM file is a dependency list with a single dependency (as simple as we can make it and still test externalPom()).
We suspect that we need something else in the build.sbt to make the externalPom() command work but we don't know what it is. Any help with that would be greatly appreciated.
I did some experimentation with this, and was able to replicate your error in my experiments.
I'm still a bit of a Scala / SBT newbie, but I created a build.sbt file that looks like so:
val Default = config("default")
lazy val root = (project in file(".")).
configs(Default).
settings(
externalPom()
)
This did compile for me!
One non-obvious catch: I had to make sure to include the scala-library in my POM file as a dependency
For our Scala development we currently use ivy + ant, but we are also trying to use sbt for our development workflow. This would be for the continuous incremental compilation when not using an IDE.
sbt uses ivy, so in theory this should work. But when using an ivy external file the tests won't compile.
To reproduce this you can even use the generated ivy.xml file from any sbt project.
Here are the steps to reproduce the error on a sbt project with tests,
from the sbt console run deliverLocal (deliver-local in previous versions of sbt)
copy the generated ivy file into your project home and rename it to 'ivy.xml'. From my understanding using this file should be equivalent to declaring the dependencies in build.sbt.
edit the build.sbt, add externalIvyFile() on one line and then comment all dependencies declarations
in the console, run reload, then test
compile will run just fine, but test will fail at compile time. None of the dependencies will be honoured, not even the production code of the current project.
What am I missing?
In my case it worked with the following build.sbt:
externalIvyFile()
classpathConfiguration in Compile := Compile
classpathConfiguration in Test := Test
classpathConfiguration in Runtime := Runtime
You just need the extra three lines in the end. Here is a link for more info: http://www.scala-sbt.org/release/docs/Detailed-Topics/Library-Management.html#ivy-file-dependency-configuration
Look for the Full Ivy Example. I hope it helps!
EDIT: Just to be complete - here is what pointed me to the above link: https://github.com/sbt/sbt/issues/849.
I am trying to follow the tutorial on compiling a simple DSL using Delite+LMS. I compiled LMS and Delite succesfully. Now, following this tutorial closely: http://stanford-ppl.github.io/Delite/myfirstdsl.html I run into problems when I try to build my profiling dsl. It seems that the compiler cannot find the delite-collection classes:
felix#felix-UX32VD:~/Documents/phd/delite/Delite$ sbt compile
Loading /home/felix/sbt/bin/sbt-launch-lib.bash
[info] Loading project definition from /home/felix/Documents/phd/delite/Delite/project
[info] Set current project to delite (in build file:/home/felix/Documents/phd/delite/Delite/)
[info] Compiling 5 Scala sources to /home/felix/Documents/phd/delite/Delite/dsls/profiling/target/scala-2.10/classes...
[error] /home/felix/Documents/phd/delite/Delite/dsls/profiling/src/example/profiling/Profile.scala:7: object DeliteCollection is not a member of package ppl.delite.framework.datastruct.scala
[error] import ppl.delite.framework.datastruct.scala.DeliteCollection
[error] ^
[error] /home/felix/Documents/phd/delite/Delite/dsls/profiling/src/example/profiling/Profile.scala:69: not found: type ScalaGenProfileArrayOps
[error] with ScalaGenDeliteOps with ScalaGenProfileOps with ScalaGenProfileArrayOps
[error]
^
Does someone have some insights to what I'm doing wrong?
From SBT manual:
Library dependencies can be added in two ways:
unmanaged dependencies are jars dropped into the lib directory
managed dependencies are configured in the build definition and downloaded
automatically from repositories (through Apache Ivy, exactly like Maven)
In any case, adding code inside a framework project is a bad idea, because you will have to change the build process (for example, adding an extra module). In addition, you might have to recompile all the code of the framework and this would be very slow.
The right way to make your code depending on a framework is:
Reference the library as a managed dependency available in some kind of repository (best solution).
Copy the jar inside the lib folder of your project and add it as an unmanaged dependency.
Since apparently Delite is not available on any Ivy repo, the best approach is to clone the Git repo and publish it locally. See http://www.scala-sbt.org/release/docs/Detailed-Topics/Publishing.html
Publishing Locally
The publishLocal command will publish to the local
Ivy repository. By default, this is in ${user.home}/.ivy2/local. Other
projects on the same machine can then list the project as a
dependency. For example, if the SBT project you are publishing has
configuration parameters like:
name := 'My Project'
organization := 'org.me'
version :=
'0.1-SNAPSHOT'
Then another project can depend on it:
libraryDependencies += "org.me" %% "my-project" % "0.1-SNAPSHOT"
I manage my project using Maven and SBT at same time. The reasons for this are:
Intellij IDEA cannot import SBT
project.(idea-sbt plugin doesn't
work very well)
I don't know how can get sources and
javadocs from SBT.(I'd like to see any answers about this)
The problem is I don't know how to let Maven download SBT dependency. I search through maven repository and couldn't find anything about sbt. I wanna use Maven or SBT to manage all the jars in my project.
If you put a pom.xml to the root of your project, it will be recognized by SBT. When you specify no managed dependencies in the project definition, SBT relies on Maven dependencies.
As it said in SBT doumentation,
sbt performs this dependency handling
when the update action is executed. By
default, sbt does not update your
dependencies before every compilation,
but only does so when you execute
update. sbt supports three ways of
specifying these dependencies:
* Declarations in your project definition
* Maven POM files
* Ivy configuration and settings files
Maven knows nothing about SBT as of now (at least, I've not heard about any plugins so far), so, the best you can do to manage your project both in Maven and SBT, is to generate POMs by SBT. See SBT to Maven Converter for more details.
idea-sbt plugin works great for me with IDEA 10 - all it's really intended to do is open an SBT shell within the IDE and it does that well enough.
A plugin you should look into if you're interested in getting the Maven out of your build is sbt-idea plugin ( https://github.com/mpeltonen/sbt-idea ). This is a great plugin that generates IDEA files from an SBT project. It couldn't be easier to use. At an SBT prompt, run the following commands:
*sbtIdeaRepo at http://mpeltonen.github.com/maven/
*idea is com.github.mpeltonen sbt-idea-processor 0.3.0
update
idea
Note the asterisks - they should be included.
At this point, you can open your project in IDEA. It won't complain about the SBT dependencies. Any time you add new dependencies to your project file, simply run the 'idea' command again to tell IDEA about it. I do that in the SBT window provided by idea-sbt.
As far as getting sources and docs with dependencies, you can do something like this (from the SBT docs):
val sc = "org.scalacheck" % "scalacheck" % "1.5" withSources()
There is a corresponding withJavadoc() method. Hope that helps.
I've got an sbt (Scala) project that currently pulls artifacts from the web. We'd like to move towards a corporate-standardized Nexus repository that would cache artifacts. From the Nexus documentation, I understand how to do that for Maven projects. But sbt obviously uses a different approach. (I understand Ivy is involved somehow, but I've never used it and don't understand how it works.)
How do I tell sbt and/or the underlying Ivy to use the corporate Nexus repository system for all dependencies? I'd like the answer to use some sort of project-level configuration file, so that new clones of our source repository will automatically use the proxy. (I.e., mucking about with per-user config files in a dot-directory is not viable.)
Thanks!
Step 1: Follow the instructions at Detailed Topics: Proxy Repositories, which I have summarised and added to below:
(If you are using Artifactory, you can skip this step.) Create an entirely separate Maven proxy repository (or group) on your corporate Maven repository, to proxy ivy-style repositories such as these two important ones:
http://repo.typesafe.com/typesafe/ivy-releases/
http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/
This is needed because some repository managers cannot handle Ivy-style and Maven-style repositories being mixed together.
Create a file repositories, listing both your main corporate repository and any extra one that you created in step 1, in the format shown below:
[repositories]
my-maven-proxy-releases: http://repo.example.com/maven-releases/
my-ivy-proxy-releases: http://repo.example.com/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
Either save that file in the .sbt directory inside your home directory, or specify it on the sbt command line:
sbt -Dsbt.repository.config=<path-to-your-repo-file>
Good news for those using older versions of sbt: Even though, in the sbt 0.12.0 launcher jar at least, the boot properties files for older sbt versions don't contain the required line (the one that mentions repository.config), it will still work for those versions of sbt if you edit those files to add the required line, and repackage them into the sbt 0.12.0 launcher jar! This is because the feature is implemented in the launcher, not in sbt itself. And the sbt 0.12.0 launcher is claimed to be able to launch all versions of sbt, right back to 0.7!
Step 2: To make sure external repositories are not being used, remove the default repositories from your resolvers. This can be done in one of two ways:
Add the command line option -Dsbt.override.build.repos=true mentioned on the Detailed Topics page above. This will cause the repositories you specified in the file to override any repositories specified in any of your sbt files. This might only work in sbt 0.12 and above, though - I haven't tried it yet.
Use fullResolvers := Seq( resolver(s) for your corporate maven repositories ) in your build files, instead of resolvers ++= or resolvers := or whatever you used to use.
OK, with some help from Mark Harrah on the sbt mailing list, I have an answer that works.
My build class now looks like the following (plus some other repos):
import sbt._
//By extending DefaultWebProject, we get Jetty support
class OurApplication(info: ProjectInfo) extends DefaultWebProject(info) {
// This skips adding the default repositories and only uses the ones you added
// explicitly. --Mark Harrah
override def repositories = Set("OurNexus" at "http://our.nexus.server:9001/nexus/content/groups/public/")
override def ivyRepositories = Seq(Resolver.defaultLocal(None)) ++ repositories
/* Squeryl */
val squeryl = "org.squeryl" % "squeryl_2.8.0.RC3" % "0.9.4beta5"
/* DATE4J */
val date4j = "hirondelle.date4j" % "date4j" % "1.0" from "http://www.date4j.net/date4j.jar"
// etc
}
Now, if I delete the Squeryl tree from my machine's .ivy2/cache directory, sbt tries to grab it from the Nexus tree with the appropriate URL. Problem solved!
All you need is to define a property file sbt.boot.properties which will allow you to:
redefine the ivy cache location (I need that because it would be otherwise part of our roaming Windows profile, which is severely limited in disk space in our shop. See Issue 74)
define any other Maven repo you want
C:\HOMEWARE\apps\sbt-0.74\sbt.boot.properties
[scala]
version: 2.7.7
# classifiers: sources, javadoc
[app]
org: org.scala-tools.sbt
name: sbt
version: read(sbt.version)
class: sbt.xMain
components: xsbti
cross-versioned: true
classifiers: sources, javadoc
[repositories]
local
my-nexus: http://my.nexus/nexus/content/repositories/scala-tools/, [organization]/[module]/[revision]/[type]s/[artifact](-[classifier]).[ext]
maven-local
# sbt-db: http://databinder.net/repo/, [organization]/[module]/[revision]/[type]s/[artifact](-[classifier]).[ext]
# maven-central
# scala-tools-releases
# scala-tools-snapshots
[boot]
directory: project/boot
properties: project/build.properties
prompt-create: Project does not exist, create new project?
prompt-fill: true
quick-option: true
[log]
level: debug
[app-properties]
project.name: quick=set(test), new=prompt(Name)[p], fill=prompt(Name)
project.organization: new=prompt(Organization)[org.vonc]
project.version: quick=set(1.0), new=prompt(Version)[1.0], fill=prompt(Version)[1.0]
build.scala.versions: quick=set(2.8.0.RC2), new=prompt(Scala version)[2.8.0.RC2], fill=prompt(Scala version)[2.8.0.RC2]
sbt.version: quick=set(0.7.4), new=prompt(sbt version)[0.7.4], fill=prompt(sbt version)[0.7.4]
project.scratch: quick=set(true)
project.initialize: quick=set(true), new=set(true)
[ivy]
cache-directory: C:\HOMEWARE\projects\.ivy2\cache
Note: this sbt.boot.properties file is inspired from:
the one mentioned in the "Generalized Launcher" page of the sbt project.
the one found within sbt-0.74 itself!
I have commented any external Maven repository definition, and added a reference to my own Nexus Maven repo.
The launcher may be configured in one of the following ways in increasing order of precedence:
Replace the /sbt/sbt.boot.properties file in the jar.
Put a configuration file named sbt.boot.properties on the classpath. Put it in the classpath root without the /sbt prefix.
Specify the location of an alternate configuration on the command line. This can be done by:
either specifying the location as the system property sbt.boot.properties
or as the first argument to the launcher prefixed by '#'.
The system property has lower precedence.
Resolution of a relative path is:
first attempted against the current working directory,
then against the user's home directory,
and then against the directory containing the launcher jar.
An error is generated if none of these attempts succeed.
Define a sbt.bat wrapper (in order to be sure to specify your sbt.boot.properties) like:
C:\HOMEWARE>more C:\HOMEWARE\bin\sbt.BAT
#echo off
set t=%~dp0
set adp0=%t:C:\="%"
set SBT_DIR=%adp0%..\apps\sbt-0.74
dir C:\%SBT_DIR%\sbt-launch-0.7.4.jar
# if needed, add your proxy settings
set PROXY_OPTIONS=-Dhttp.proxyHost=my.proxy -Dhttp.proxyPort=80xx -Dhttp.proxyUser=auser -Dhttp.proxyPassword=yyyy
set JAVA_OPTIONS=-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256m -Xmx512M -cp C:\HOMEWARE\apps\sbt-0.74\sbt-launch-0.7.4
set SBT_BOOT_PROPERTIES=-Dsbt.boot.properties="sbt.boot.properties"
cmd /C C:\HOMEWARE\apps\jdk4eclipse\bin\java.exe %PROXY_OPTIONS% %JAVA_OPTIONS% %SBT_BOOT_PROPERTIES% -jar C:\HOMEWARE\apps\sbt-0.74\sbt-launch-0.7.4.jar %*
And your sbt will download artifacts only from:
your Nexus
your local Maven repo.
Just tested at home with an old Nexus opensource 1.6 I had running, java 1.6, sbt07.4
C:\Prog\Java\jdk1.6.0_18\jre\bin\java -Xmx512M -Dsbt.boot.properties=sbt.boot.properties - jar "c:\Prog\Scala\sbt\sbt-launch-0.7.4.jar"
That gives:
[success] Build completed successfully.
C:\Prog\Scala\tests\pp>sbt
Getting Scala 2.8.0 ...
downloading http://localhost:8081/nexus/content/repositories/scala/org/scala-lang/scala-compiler/2.8.0/scala-compiler-2.
8.0.jar ...
[SUCCESSFUL ] org.scala-lang#scala-compiler;2.8.0!scala-compiler.jar (311ms)
downloading http://localhost:8081/nexus/content/repositories/scala/org/scala-lang/scala-library/2.8.0/scala-library-2.8.
0.jar ...
[SUCCESSFUL ] org.scala-lang#scala-library;2.8.0!scala-library.jar (185ms)
:: retrieving :: org.scala-tools.sbt#boot-scala
confs: [default]
2 artifacts copied, 0 already retrieved (14484kB/167ms)
[info] Building project test 0.1 against Scala 2.8.0
[info] using sbt.DefaultProject with sbt 0.7.4 and Scala 2.7.7
If I try a funny value in the sbt.boot.properties file:
C:\Prog\Scala\tests\pp>sbt
Getting Scala 2.9.7 ...
:: problems summary ::
:::: WARNINGS
module not found: org.scala-lang#scala-compiler;2.9.7
==== nexus: tried
http://localhost:8081/nexus/content/repositories/scala/org/scala-lang/scala-compiler/2.9.7/scala-compiler-2.9.7.pom
-- artifact org.scala-lang#scala-compiler;2.9.7!scala-compiler.jar:
http://localhost:8081/nexus/content/repositories/scala/org/scala-lang/scala-compiler/2.9.7/scala-compiler-2.9.7.jar
So it does limit itself to the two repo I defined:
[repositories]
nexus: http://localhost:8081/nexus/content/repositories/scala
nexus2: http://localhost:8081/nexus/content/repositories/scala, [organization]/[module]/[revision]/[type]s/[artifact](-[classifier]).[ext]
(I commented everything else: local, maven-local, ...)
If I comment all repositories and put a funny value (2.7.9) for the scala version in the sbt.boot.properties, I do get (like the OP did)
C:\Prog\Scala\tests\pp>sbt
Error during sbt execution: No repositories defined.
If I put 2.7.7 (while still having all repo commented), yes, it won't generate an error:
C:\Prog\Scala\tests\pp>sbt
[info] Building project test 0.1 against Scala 2.8.0
[info] using sbt.DefaultProject with sbt 0.7.4 and Scala 2.7.7
But that's only because it already had downloaded scala2.8.0 during my previous tries.
If I remove that library from my project/boot directory, then it will throw an Exception:
[info] using sbt.DefaultProject with sbt 0.7.4 and Scala 2.7.7
> C:\Prog\Scala\tests\pp>sbt
Error during sbt execution: No repositories defined.
at xsbt.boot.Pre$.error(Pre.scala:18)
at xsbt.boot.Update.addResolvers(Update.scala:197)
...
at xsbt.boot.Boot$.main(Boot.scala:15)
at xsbt.boot.Boot.main(Boot.scala)
Error loading project: Error during sbt execution: No repositories defined.
edit the config file in sbt_home/conf "sbtconfig.txt"
add two line
-Dsbt.override.build.repos=true
-Dsbt.repository.config="C:/Program Files (x86)/sbt/conf/repo.properties"
the repo.properties content is
[repositories]
local
public: http://222.vvfox.com/public <-fix this ,write your local nexus group url
Well this has bugged me for a while so I found a guy that has written an SBT plugin for maven out on github called maven-sbt so all you have to do is include it in your plugins project and make your project mixin with maven.MavenDependencies and all your operations like update and publish-local work with your local maven. The nice thing about that is if you are like me, your org is all maven. So, all you libs are in you local maven repo but if for some reason you build with sbt first, then you start getting a bunch or jars in ivy too. What a waste of space, and time since you will still need to get them for your maven builds.
That said, I wish this were built into sbt so I would not need to add it to every project. Maybe as a processor at least. He mentioned in one thing I read that he would like to add it to 0.9 but I have not been able to find it.
I got this error because I had a blank file in ~/.sbt/repositories. Both adding repositories to the file and removing the file solved the problem.