I'm just starting out trying to set up a workflow with scala and sbt, and I'm having trouble with my repository. I am trying to publish a simple test library, which is composed of two projects, and use it from another program.
My source library's build contains the following:
val sharedSettings = Seq(
name := "test-lib",
organization := "com.example",
version := "0.1-SNAPSHOT",
scalaVersion := "2.11.0",
publishTo := Some("Artifactory Realm" at "http://localhost:8081/artifactory/libs-snapshot-local"),
publishMavenStyle := true,
credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")
)
lazy val root = project.in(file(".")).settings(sharedSettings: _*).aggregate(child1, child2)
lazy val sharedCode = project.settings(sharedSettings: _*)
val child1Settings = sharedSettings ++ Seq(unmanagedSourceDirectories in Compile <++= (unmanagedSourceDirectories in sharedCode) in Compile)
val child2Settings = sharedSettings ++ Seq(unmanagedSourceDirectories in Compile <++= (unmanagedSourceDirectories in sharedCode) in Compile)
lazy val child1 = project.settings(child1Settings: _*)
lazy val child2 = project.settings(child2Settings: _*)
I can run sbt publish okay, and it creates the directory com/example/test-lib/XXX in the repo.
In my test program, I have the following:
scalaVersion := "2.11.0",
resolvers += "Artifactory Realm" at "http://localhost:8081/artifactory/libs-snapshot-local",
libraryDependencies += "com.example" %% "test-lib" % "0.1-SNAPSHOT"
When the test program attempts to compile, it cannot resolve com.example, because of the following:
[warn] ==== Artifactory Realm: tried
[warn] http://localhost:8081/artifactory/libs-snapshot-local/com/example/test-lib_2.11/0.1-SNAPSHOT/test-lib_2.11-0.1-SNAPSHOT.pom
Looking at the repository directory itself, I am getting an additional timestamp on my pom files:
test-lib_2.11-0.1-20140510.183027-1.pom 10-May-2014 19:30 793 bytes
test-lib_2.11-0.1-20140510.183027-2.pom 10-May-2014 19:30 793 bytes
...
test-lib_2.11-0.1-20140510.183121-9.pom 10-May-2014 19:31 793 bytes
maven-metadata.xml in the directory is referencing these okay, sbt is looking directly for a pom file without a timestamp and cannot find it. The pom files contain the correct information.
What am I doing wrong?
The issue was not with my sbt configuration after all, but with my repository server.
I'm using Artifactory, and the snapshots repository was configured to use "unique snapshots" by default. The filenames of these snapshots are modified as they are published to include a timestamp, which sbt 13.x doesn't seem to understand.
After changing the repository's "Maven Snapshot Version Behaviour" from "Unique" to "Nonunique", everything started to work.
actually, the inconsistency of timestamp & build number suffix in maven-metadata.xml and jar/pom files genereated by sbt publish lead to such error.
with the following plugin sbt-maven-resolver during deployment procedure, the suffix will be kept same, looks like:
currently, adding this plugin on the deployment side (once the timestamp & build suffix are same, both sbt/maven can find the snapshots):
in plugins.sbt
addSbtPlugin("org.scala-sbt" % "sbt-maven-resolver" % "0.1.0")
hope to solve your case.
Related
I have a set of .proto files (protobuf) which I generate java from using scalapb. I also have in the same sbt 2 sub-projects, one is scalaVersion 2.11 compatible (can't upgrade it to 2.12 due to missing packages) and the other one is scala 2.12.
I created a sub-project to hold my proto, and by default 2.12 is used and my 2.12 sub-project can use it, but my 2.11 can't.
I set the crossScalaVersions to 2.11/2.12, I compiled my project with both, which passed, but then even then I was unable to get the 2.11 sub-project to find that code.
I am "wondering" if that is something supported, or if there is a track I could use a single location to hold my .proto yet have my 2 sub-projects using the same sbt file use those.
lazy val scala212 = "2.12.13"
lazy val scala211 = "2.11.12"
lazy val supportedScalaVersion = List(scala212, scala211)
ThisBuild / scalaVersion := scala212
lazy val root = (project in file("."))
.aggregate(proto, subproject1, subproject2)
.settigns(
crossScalaVersions := Nil,
publish / skip := true
)
lazy val proto = project
.settings(
crossScalaVersions := supportedScalaVersions,
name := "proto",
libraryDependencies += "com.trueaccord.scalapb" %% "scalapb-runtime" % com.trueaccord.scalapb.compiler.Version.scalapbVersion % "protobuf",
PB.targets in Compile := Seq(
scalapb.gen(grpc = false) -> (sourceManaged in Compile).value / "protobuf"
)
)
lazy val subproject1 = project
.dependsOn(proto)
lazy val subproject2 = project
.settings(
scalaVersion := scala211
)
.dependsOn(proto)
So, from the above, if I do sbt "+ proto" I can compile both versions. If I do sbt subproject1/compile it works too. Using sbt subproject2/compile fails indicating that it cannot find the 2.11:proto jar file.
Either, I would like the above somehow to work nicely, or any other trick that I could generate the code from the same proto location but within subproject1/subproject2 would be appreciated.
You could try the sbt-projectmatrix plugin:
https://github.com/sbt/sbt-projectmatrix
The idea is to have separate sbt subprojects for the different Scala versions, so you can simply reference the relevant subproject when calling dependsOn.
I think this plugin is going to end up in sbt some day as it's a much better solution in general than the current built-in stateful cross compilation support, and it's developed by Eugene Yokota, who is also an sbt developer.
I'm trying to create a relatively simple sbt plugin to wrap grpc-swagger artifact.
Therefore, I've created a project with the following structure:
projectDir/
build.sbt
lib/grpc-swagger.jar <- the artifact I've downloaded
src/...
where build.sbt looks like the following:
ThisBuild / version := "0.0.1-SNAPSHOT"
ThisBuild / organization := "org.testPlugin"
ThisBuild / organizationName := "testPlugin"
lazy val root = (project in file("."))
.enable(SbtPlugin)
.settings(name := "grpc-swagger-test-plugin")
According to sbt docs, that's all I have to do in order to include an unmanaged dependecy, that is:
create a lib folder;
store the artifact in there;
However, when I do execute sbt compile publishLocal, the plugin published lacks of that external artifact.
So far I've tried to:
set exportJars := true flag
add Compile / unmanagedJars += file(lib/grpc-swagger.jar") (with also variations of the path)
manual fiddling to libraryDependecies using from file("lib/grpc-swagger.jar") specifier
but none so far seemed to work.
So how am I supposed to add an external artifact to a sbt plugin?
The proper solution to this problem is to publish the grpc-swagger library. If for whatever reason this can't be done from that library's build system, you can do it with sbt. Just add a simple subproject whose only job it is to publish that jar. It should work like so:
...
lazy val `grpc-swagger` = (project in file("."))
.settings(
name := "grpc-swagger",
Compile / packageBin := baseDirectory.value / "lib" / "grpc-swagger.jar",
// maybe other settings, such as grpc-swagger's libraryDependencies
)
lazy val root = (project in file("."))
.enable(SbtPlugin)
.settings(name := "grpc-swagger-test-plugin")
.dependsOn(`grpc-swagger`)
...
The pom file generated for the root project should now specify a dependency on grpc-swagger, and running the publish task in the grpc-swagger project will publish that jar along with a pom file.
That should be enough to make things work, but honestly, it's still a hack. The proper solution is to fix grpc-swagger's build system so you can publish an artifact from there and then just use it via libraryDependencies.
I'm using sbt 1.3.9 and I need to update some libraries that have changed its code but the version stays the same. When I tried to run sbt update command nothing happened the library not downloaded.
I have sbt.build file that looks like the following:
name := """project name"""
organization := "com.example"
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayJava)
lazy val usr = sys.env("MVN_USER")
scalaVersion := "2.13.1"
javacOptions ++= Seq("-source", "11", "-target", "11")
resolvers ++= Seq(
"Jfrog Artifacts".at("https://artifactory.jfrog.com/")
)
credentials += Credentials(
...
)
updateOptions := updateOptions.value.withCachedResolution(false)
updateOptions := updateOptions.value.withLatestSnapshots(false)
libraryDependencies ++= Seq(
guice,
javaWs,
ehcache,
"com.google.api-client" % "google-api-client" % "1.30.7",
"org.apache.commons" % "commons-lang3" % "3.9",
"redis.clients" % "jedis" % "3.2.0"
)
how can I clear sbt cache?
You can just delete the v1 folder.
The default cache location is platform-dependent:
on Linux, ~/.cache/coursier/v1. This also applies to Linux-based CI
environments, and FreeBSD too.
on OS X, ~/Library/Caches/Coursier/v1.
on Windows, %LOCALAPPDATA%\Coursier\Cache\v1, which, for user Alex,
typically corresponds to
C:\Users\Alex\AppData\Local\Coursier\Cache\v1.
I assuming that you are developing something locally and doing publishLocal - normal repository wouldn't let you override published dependency and snapshots are not cached (sbt checks if newer appeared every time you need to build sth).
In such case start using snapshot versions for the future and/or go to ~/.ivy2/your.organisation/library_scalaVersion and remove whole directory with "bad" version. If library is fetched by Maven (with sbt, unlikely these days) it the same idea but with ~/.m2.
I had experienced this while using IntelliJ. After closing IntelliJ, I used to clear .idea from project folder and reimport project into IntelliJ was helping me.
You can use PrettyClean to clean the all of dev tools caches including SBT.
PrettyClean also cleans the SBT project's target folder.
Say I have a project defined as follows:
val commonSettings = Seq(
name := "project1",
version := "1.0",
scalaVersion := "2.12.8",
unmanagedSourceDirectories in Compile +=
baseDirectory.value / ".." / "shared" / "src" / "main" / "scala"
)
val client = project.in(file("client"))
.settings(commonSettings: _*)
val server = project.in(file("server"))
.settings(commonSettings: _*)
And I have a second project that uses a REST api to communicate with this server. Thus the second project uses code defined in the shared source of the first. The second project cannot depend on the entire project1 as there are incompatibilities between the sbt versions for the two projects.
What do I need to add to my build.sbt and/or change in the first project in order for the second project to reference the shared source?
If project1 and project 2 are finally going to run independently it would be better if you write REST APIs to communicate in between them.
You can write an API in project2 and hit it from project1 to run the shared code of project2.
Otherwise, why not just replicate shared code from project2 to project1 ?
You can try jitpack, add below lines in your build.sbt to enable jitpack resolver and add appropriate dependencies
resolvers += "jitpack" at "https://jitpack.io"
// you can also use specific sha or tag instead of master-SNAPSHOT
libraryDependencies += "org.xyz" %% "project" % "master-SNAPSHOT"
i'm trying to import a dependency in my build.sbt file from here
https://github.com/dmarcous/spark-betweenness.
When i hover on the error it says:
Expression type ModuleID must confirm to Def.SettingsDefinition in SBT file
Unresolved Dependency
I am new in scala so my question may be silly.Thanks in advance
It is still unclear how your build configuration looks like, but the following build.sbt works (in the sense that it compiles and does not show the error that you mentioned):
name := "test-sbt"
organization := "whatever"
version := "1.0.0"
scalaVersion := "2.10.7"
libraryDependencies += "com.centrality" %% "spark-betweenness" % "1.0.0"
Alternatively, if you have a multi-project build, it could look like this:
lazy val root = project
.settings(
name := "test-sbt",
organization := "whatever",
version := "1.0.0",
scalaVersion := "2.10.7",
libraryDependencies += "com.centrality" %% "spark-betweenness" % "1.0.0"
)
However, you're probably going to find that it still does not work because it cannot resolve this dependency. Indeed, this library does not seem to be available neither in Maven Central nor in jcenter. It is also very old - it appears to only be published for Scala 2.10 and a very old Spark version (1.5), so most likely you won't be able to use it with recent Spark environments (2.x and Scala 2.11).