By default, the Scala Built Tool (SBT) has a set of rules on how to generate URLs when looking up dependencies. For example, if I have the following build file,
// Project settings
name := "MyProject"
version := "0.1"
organization := "com.me"
scalaVersion := "2.8.1"
// Dependencies
libraryDependencies ++= Seq(
"com.google.guava" %% "guava" % "r09"
)
// Repositories
resolvers += "Maven Central Server" at "http://repo1.maven.org/maven2"
Then SBT attempts to find guava at the following URL,
http://repo1.maven.org/maven2/com/google/guava/guava_2.8.1/r09/guava_2.8.1-r09.pom
However, the library I'm looking for in this case isn't even made for Scala, so combining the Scala version just doesn't make sense here. How can I tell SBT what the format is for generating URLs for use with Maven repositories?
EDIT
While it seems that it is possible to edit the layout like so,
Resolver.url("Primary Maven Repository",
new URL("http://repo1.maven.org/maven2/"))( Patterns("[organization]/[module]/[module]-[revision].[ext]") )
the "[module]" keyword is predefined to be the (artifact id)_(scala version) and the "[artifact]" keyword is just "ivy", leaving me back at square one.
As far as I remember "%%" appends the scala version and "%" does not. Try
libraryDependencies ++= Seq(
"com.google.guava" % "guava" % "r09"
)
Check last one paragraph (Custom Layout) of official sbt wiki here.
Basically SBT allows you to use this syntax:
resolvers += Resolver.url("my-test-repo", url)( Patterns("[organisation]/[module]/[revision]/[artifact].[ext]") )
Related
I'm using sbt 1.3.9 and I need to update some libraries that have changed its code but the version stays the same. When I tried to run sbt update command nothing happened the library not downloaded.
I have sbt.build file that looks like the following:
name := """project name"""
organization := "com.example"
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayJava)
lazy val usr = sys.env("MVN_USER")
scalaVersion := "2.13.1"
javacOptions ++= Seq("-source", "11", "-target", "11")
resolvers ++= Seq(
"Jfrog Artifacts".at("https://artifactory.jfrog.com/")
)
credentials += Credentials(
...
)
updateOptions := updateOptions.value.withCachedResolution(false)
updateOptions := updateOptions.value.withLatestSnapshots(false)
libraryDependencies ++= Seq(
guice,
javaWs,
ehcache,
"com.google.api-client" % "google-api-client" % "1.30.7",
"org.apache.commons" % "commons-lang3" % "3.9",
"redis.clients" % "jedis" % "3.2.0"
)
how can I clear sbt cache?
You can just delete the v1 folder.
The default cache location is platform-dependent:
on Linux, ~/.cache/coursier/v1. This also applies to Linux-based CI
environments, and FreeBSD too.
on OS X, ~/Library/Caches/Coursier/v1.
on Windows, %LOCALAPPDATA%\Coursier\Cache\v1, which, for user Alex,
typically corresponds to
C:\Users\Alex\AppData\Local\Coursier\Cache\v1.
I assuming that you are developing something locally and doing publishLocal - normal repository wouldn't let you override published dependency and snapshots are not cached (sbt checks if newer appeared every time you need to build sth).
In such case start using snapshot versions for the future and/or go to ~/.ivy2/your.organisation/library_scalaVersion and remove whole directory with "bad" version. If library is fetched by Maven (with sbt, unlikely these days) it the same idea but with ~/.m2.
I had experienced this while using IntelliJ. After closing IntelliJ, I used to clear .idea from project folder and reimport project into IntelliJ was helping me.
You can use PrettyClean to clean the all of dev tools caches including SBT.
PrettyClean also cleans the SBT project's target folder.
akka-actor_2.13-2.5.23.jar of play-scala-seed build path is cross-compiled with an incompatible version of Scala (2.13.0). In case this report is mistaken, this check can be disabled in the compiler preference page.
I have many errors of this type in my scala project. How do I resolve such errors?
I am using Scala IDE.
build.sbt:
name := """play-scala-seed"""
organization := "com.example"
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.13.0"
libraryDependencies += guice
libraryDependencies += "org.scalatestplus.play" %% "scalatestplus-play" % "4.0.3" % Test
// Adds additional packages into Twirl
//TwirlKeys.templateImports += "com.example.controllers._"
// Adds additional packages into conf/routes
// play.sbt.routes.RoutesKeys.routesImport += "com.example.binders._"
I guess I need to change Scala installation to 2.13, right? ( I am referring to the Scala installation option that shows up in the project properties after right clicking on the project in the IDE )
But, the only options available in the dropdown are 2.10,2.11 and 2.12. How do I add 2.13 into my IDE?
The solutions in similar questions did not work for me.
i'm trying to import a dependency in my build.sbt file from here
https://github.com/dmarcous/spark-betweenness.
When i hover on the error it says:
Expression type ModuleID must confirm to Def.SettingsDefinition in SBT file
Unresolved Dependency
I am new in scala so my question may be silly.Thanks in advance
It is still unclear how your build configuration looks like, but the following build.sbt works (in the sense that it compiles and does not show the error that you mentioned):
name := "test-sbt"
organization := "whatever"
version := "1.0.0"
scalaVersion := "2.10.7"
libraryDependencies += "com.centrality" %% "spark-betweenness" % "1.0.0"
Alternatively, if you have a multi-project build, it could look like this:
lazy val root = project
.settings(
name := "test-sbt",
organization := "whatever",
version := "1.0.0",
scalaVersion := "2.10.7",
libraryDependencies += "com.centrality" %% "spark-betweenness" % "1.0.0"
)
However, you're probably going to find that it still does not work because it cannot resolve this dependency. Indeed, this library does not seem to be available neither in Maven Central nor in jcenter. It is also very old - it appears to only be published for Scala 2.10 and a very old Spark version (1.5), so most likely you won't be able to use it with recent Spark environments (2.x and Scala 2.11).
Question
Is .sbt file is a in scala or in sbt proprietary language? Please help to decipher the sbt build definition.
lazy val root = <--- Instance of the SBT Project object? Why "lazy"? Is "root" the reserved keyword for sbt to identify the project in the build.sbt?
(project in file(".")) <--- Is this defining a Project object regarding the current directory having the SBT expected project structure?
.settings( <--- Is this a scala function call of def settings in the Project object?
name := "NQueen",
version := "1.0",
scalaVersion := "2.11.8",
mainClass in Compile := Some("NQueen")
)
libraryDependencies ++= Seq( <--- libraryDependencies is a reserved keyword of type scala.collection.Seq using which sbt downloads and packages them as part of the output jar?
"org.apache.spark" %% "spark-core" % "2.3.0", <--- Concatenating and creating the full library name including version. I suppose I need to look into respective documentations to find out what to specify.
"org.apache.spark" %% "spark-mllib" % "2.3.0"
)
// <--- Please explain what this block does and where I can find the explanations.
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case x => MergeStrategy.firs
}
Resources
Please suggest good resources to understand the design, mechanism, how .sbt works. I looked into the SBT getting started and documents but as Scala definition itself, it is difficult to understand. If it is make, ant, or maven, how things get pieced together and the design/mechanism are so much clear, but need to find good documentations or tutorials for SBT.
References
I looked into the references below trying to understand.
SBT: How to get started using the Build.scala file (instead of build.sbt)
What is the difference between build.sbt and build.scala?
SBT - Build definition
SBT Project object
scala.collection.Seq
SBT Library dependencies
Spark 2.3 Quick Start
sbt can be really difficult for first time users, and it's ok not to fully understand all of the definitions. It will become clearer over time.
let me first simplify you build.sbt. it contains some unnecessary parts, and will be easier to explain without them
name := "NQueen"
version := "1.0"
scalaVersion := "2.11.8"
mainClass in Compile := Some("NQueen")
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.3.0",
"org.apache.spark" %% "spark-mllib" % "2.3.0"
)
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case x => MergeStrategy.firs
}
and for your questions:
Is .sbt file is a in scala or in sbt proprietary language?
well, it's both. you can do most scala operations in an .sbt file. you can import and use external dependencies, write custom code, etc.. but some things you can't do (define classes for example).
It's also might look as a dedicated different language, but in reality, it's just a DSL written in scala (:=, in, %%, % are all function written in scala)
libraryDependencies is a reserved keyword of type scala.collection.Seq using which sbt downloads and packages them as part of the output jar?
libraryDependencies is not a reserved keyword. you can think of it as a way to configure you project.
writing libraryDependencies := Seq(..) you basically setting the value of libraryDependencies.
But you are right about the meaning. it is a list of dependencies that should be downloaded.
Concatenating and creating the full library name including version. I suppose I need to look into respective documentations to find out what to specify.
keep in mind that %% and % are functions. you use those functions to specify what modules should be downloaded and added to the classpath.
you can find many dependencies (and their versions) in mvnrepository.
for example, for spark: https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.3.0
Please explain what this block does and where I can find the explanations.
assemblyMergeStrategy is a setting coming from the sbt-assembly plugin.
that plugin allows you to pack your application into a single jar with all the dependencies.
you can read about the merge strategy here: https://github.com/sbt/sbt-assembly#merge-strategy
I am new to scala and am currently trying to setup IntelliJ IDEA 13.1 with the Scala plugin. It has support for SBT. I have simply followed the basic tutorial for creating a new project for SBT here: http://confluence.jetbrains.com/display/IntelliJIDEA/Getting+Started+with+SBT
Currently my build.sbt file is:
name := "scalasandpit"
version := "1.0"
scalaVersion := "2.10"
libraryDependencies += "org.scalatest" % "scalatest_2.10" % "2.1.0" % "test"
autoAPIMappings := true
This pulls down various jar binaries, but no sources and no javadoc. I wondered if there is a way to have both sources and javadoc work with IntelliJ and SBT. I think I'm missing something.
There seem to be two issues: getting sbt to pull down sources and docs, and then getting Idea to show them to you. To solve the former problem see the sbt documentation -- about half way down there's a section called "Download Sources" which tells you what to add to your build.sbt:
libraryDependencies +=
"org.scalatest" % "scalatest_2.10" % "2.1.0" % "test" withSources() withJavadoc()