sbt ivy2: configuration 'master' not found from 'compile' - scala

I am getting the following error when I build a library bp-kafka-bp2 that depends on library bp-bp2-componentized:
sbt.librarymanagement.ResolveException: unresolved dependency:
com.foo#bp-bp2-componentized_2.11;3.3.+: configuration not found in
com.foo#bp-bp2-componentized_2.11;3.3.0: 'master'. It was required
from com.foo#bp-kafka-bp2_2.11;3.10.1 compile
The unresolved library bp-bp2-componentized does in fact exist in ~/.ivy2/local and does not have a master configuration in its ivy.xml
My questions are:
Should the dependent library (bp-kafka-bp2) be looking for a master configuration of the (not really) missing library?
If it should not be looking for a master config, what can I do to make it stop doing so?
If it should be looking for a master config, how do I induce the build for the (not really) missing library to produce such a config?
I have tried this in sbt versions 1.1.5 and 1.2.1. I have deleted ~/.ivy2/local, ./ivy2/cache and ~/.sbt. I have also removed all the /target and project/target directories in the library I am building and done sbt clean
This library has built just fine for a year or two now. The only recent change I can think of is introducing 2.11 and 2.12 cross-compilation option -- which is not being exercised here, I'm just building 2.11 version on its own.
The direct dependency is declared in a multi-project build.sbt as
lazy val bp2 = (project in file("bp2")).
settings(commonSettings:_*).
settings(Seq(
name := "bp-kafka-bp2"
):_*).
settings(
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % ScalaTestVersion % "test",
"ch.qos.logback" % "logback-classic" % LogbackVersion % "test",
"com.foo" %% "bp-bp2-componentized" % Constellation.dependency("bp-bp2"),
"com.foo" %% "bp-akka-http" % Constellation.dependency("bp-akka")
)
).
dependsOn(reactiveComponentized)
where Constellation.dependency is just a function that looks up a version by name and turns it into a patch range:
object Constellation {
...
def dependency(name: String) : String = versions(name).split("\\.").dropRight(1).mkString(".") + ".+"
}
You can see from the error message that the version is being found and converted to 3.3.+ which is then resolved correctly to a 3.3.0 in the ivy cache. But then it insists on finding a master configuration which is not present.

Related

zio-grpc with Scala 3 (2nd try)

Does anyone have a bare-bones zio-grpc server, with codegen in the project also, working with Scala 3?
I started with the HelloWorld project from their repo and attempted to get it to build with scalaVersion := "3.1.0"
Here is the relevant section in plugins.sbt:
libraryDependencies ++= Seq(
"com.thesamet.scalapb.zio-grpc" % "zio-grpc-codegen_2.13" % zioGrpcVersion,
"com.thesamet.scalapb" % "compilerplugin_2.13" % "0.11.1"
)
excludeDependencies ++= Seq(
ExclusionRule("org.scala-lang.modules", "scala-collection-compat_2.12"),
ExclusionRule("com.thesamet.scalapb", "protoc-bridge_2.12")
)
and in build.sbt:
libraryDependencies ++= Seq(
"io.grpc" % "grpc-netty" % grpcVersion,
"com.thesamet.scalapb" % "scalapb-runtime-grpc_2.13" % scalapb.compiler.Version.scalapbVersion
)
excludeDependencies ++= Seq(
ExclusionRule("org.scala-lang.modules", "scala-collection-compat_2.12"),
ExclusionRule("com.thesamet.scalapb", "protoc-bridge_2.12")
)
Since Scala 3 can use 2.13 libraries, that's what I'm doing. (Of the three zio-grpc-related libs, one, zio-grpc-codegen, does not have a Scala 3 version, so 2.13 must be used at least for that one.)
I get this error from sbt with the versions as above:
java.lang.NoSuchMethodError: scala.package$.Seq()Lscala/collection/immutable/Seq$;
at protocbridge.gens$.java(gens.scala:17)
at protocbridge.gens$.(gens.scala:11)
If I remove either of the scala-collection-compat exclusions, we get
[error] Modules were resolved with conflicting cross-version suffixes in ProjectRef(uri("file:/Users/xxx/dev/zio-grpc/examples/helloworld/project/"), "helloworld-build"):
[error] com.thesamet.scalapb:protoc-bridge _2.12, _2.13
[error] com.thesamet.scalapb:compilerplugin _3, _2.13
In short, I cannot find any permutation of Scala 2.13/3 versions of zio-grpc-codegen, compilerplugin_3, and scalapb-runtime-grpc that will not give some sbt conflicting cross-version suffix error.
TL;DR: this is not possible yet as some of the code you are using rely on macros and is not yet published for Scala3.
SBT plugins runs with Scala 2.12 no matter which Scala version is used in your project, thus you don't have to try to use plugins with _2.13 or _3 suffix, just use the regular syntax that will actually pick _2.12 artifacts.
That is, in plugins.sbt:
libraryDependencies ++= Seq(
"com.thesamet.scalapb.zio-grpc" %% "zio-grpc-codegen" % zioGrpcVersion,
"com.thesamet.scalapb" %% "compilerplugin" % "0.11.8"
)
(No need for any exclusion).
You can confirm this by looking at sbt logs and you should see that it downloads plugins for version 2.12 of Scala:
...
https://somerepository.com/com/thesamet/scalapb/zio-grpc/zio-grpc-codegen_2.12/0.5.1/zio-grpc-codegen_2.12-0.5.1.pom
https://somerepository.com/com/thesamet/scalapb/compilerplugin_2.12/0.11.8/compilerplugin_2.12-0.11.8.pom
...
Once you do that, you'll get an error as following dependencies do not exist:
com.thesamet.scalapb:scalapb-runtime_3:0.11.1
com.thesamet.scalapb:scalapb-runtime-grpc_3:0.11.1
com.thesamet.scalapb.zio-grpc:zio-grpc-core_3:0.5.0
For the 1st and 2nd, you just have to update the compilerplugin version to 0.11.8 as I did already above (the compilerplugin version is used for the main dependency scalapb-runtime-grpc).
For the 3rd, unfortunately it's not published yet for Scala 3. One attempt is to force the _2.13 version for this one with something like that in the build.sbt:
libraryDependencies += ("com.thesamet.scalapb.zio-grpc" %% "zio-grpc-core" % "0.5.1") cross CrossVersion.for3Use2_13
excludeDependencies += "com.thesamet.scalapb.zio-grpc" % "zio-grpc-core_3"
But this doesn't compile with some errors related to macros, and that is something that is not compatible between Scala 2.13 and 3. You cannot workaround that.
Remember you can check available versions of a lib for a Scala version on Maven central:
https://mvnrepository.com/artifact/com.thesamet.scalapb/compilerplugin
https://mvnrepository.com/artifact/com.thesamet.scalapb.zio-grpc/zio-grpc-core
I haven't released zio-grpc for Scala 3 as some tests related to Has were failing and were tricky to fix. At the same ZIO 2 is coming out and deprecated Has. There's a version of zio-grpc with ZIO 2 and Scala 3 support coming soon.

sryza/spark-timeseries: NoSuchMethodError: scala.runtime.IntRef.create(I)Lscala/runtime/IntRef;

I have a Scala project that I build with sbt. It uses the sryza/spark-timeseries library.
I am trying to run the following simple code:
val tsAirPassengers = new DenseVector(Array(
112.0,118.0,132.0,129.0,121.0,135.0,148.0,148.0,136.0,119.0,104.0,118.0,115.0,126.0,
141.0,135.0,125.0,149.0,170.0,170.0,158.0,133.0,114.0,140.0,145.0,150.0,178.0,163.0,
172.0,178.0,199.0,199.0,184.0,162.0,146.0,166.0,171.0,180.0,193.0,181.0,183.0,218.0,
230.0,242.0,209.0,191.0,172.0,194.0,196.0,196.0,236.0,235.0,229.0,243.0,264.0,272.0,
237.0,211.0,180.0,201.0,204.0,188.0,235.0,227.0,234.0,264.0,302.0,293.0,259.0,229.0,
203.0,229.0,242.0,233.0,267.0,269.0,270.0,315.0,364.0,347.0,312.0,274.0,237.0,278.0,
284.0,277.0,317.0,313.0,318.0,374.0,413.0,405.0,355.0,306.0,271.0,306.0,315.0,301.0,
356.0,348.0,355.0,422.0,465.0,467.0,404.0,347.0,305.0,336.0,340.0,318.0,362.0,348.0,
363.0,435.0,491.0,505.0,404.0,359.0,310.0,337.0,360.0,342.0,406.0,396.0,420.0,472.0,
548.0,559.0,463.0,407.0,362.0,405.0,417.0,391.0,419.0,461.0,472.0,535.0,622.0,606.0,
508.0,461.0,390.0,432.0
))
val period = 12
val model = HoltWinters.fitModel(tsAirPassengers, period, "additive", "BOBYQA")
It builds fine, but when I try to run it, I get this error:
Exception in thread "main" java.lang.NoSuchMethodError: scala.runtime.IntRef.create(I)Lscala/runtime/IntRef;
at com.cloudera.sparkts.models.HoltWintersModel.convolve(HoltWinters.scala:252)
at com.cloudera.sparkts.models.HoltWintersModel.initHoltWinters(HoltWinters.scala:277)
at com.cloudera.sparkts.models.HoltWintersModel.getHoltWintersComponents(HoltWinters.scala:190)
.
.
.
The error occurs on this line:
val model = HoltWinters.fitModel(tsAirPassengers, period, "additive", "BOBYQA")
My build.sbt includes:
name := "acme-project"
version := "0.0.1"
scalaVersion := "2.10.5"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-hive" % "1.6.0",
"net.liftweb" %% "lift-json" % "2.5+",
"com.github.seratch" %% "awscala" % "0.3.+",
"org.apache.spark" % "spark-mllib_2.10" % "1.6.2"
)
I have placed sparkts-0.4.0-SNAPSHOT.jar in the lib folder of my project. (I would have preferred to add a libraryDependency, but spark-ts does not appear to be on Maven Central.)
What is causing this run-time error?
The library requires Scala 2.11, not 2.10, and Spark 2.0, not 1.6.2, as you can see from
<scala.minor.version>2.11</scala.minor.version>
<scala.complete.version>${scala.minor.version}.8</scala.complete.version>
<spark.version>2.0.0</spark.version>
in pom.xml. You can try changing these and seeing if it still compiles, find which older version of sparkts is compatible with your versions, or update your project's Scala and Spark versions (don't miss spark-mllib_2.10 in this case).
Also, if you put the jar into lib folder, you also have to put its dependencies there (and their dependencies, etc.) or into libraryDependencies. Instead, publish sparkts into your local repository using mvn install (IIRC) and add it to libraryDependencies, which will allow SBT to resolve its dependencies.

How to use play-plugins-mailer with Play 2.3 and Scala 2.11?

I am trying to use the play plugin for sending emails:
https://github.com/playframework/play-mailer
I have followed the instructions as found on github: added the dependency to build.sbt, created play.plugins with the specified content (do I need to register the file somehow)?
but I get a compilation error:
object mailer is not a member of package play.api.libs
when trying to import
import play.api.libs.mailer._
I get another compilation error on
val mail = use[MailerPlugin].email
MailerPlugin and use are not found.
How to get this working?
Note: the plugin is correctly downloaded (I can find it in my .ivy2 directory), but it is not listed as a dependency in my application.
My build.sbt file:
name := ...
version := "1.0-SNAPSHOT"
scalaVersion := "2.11.2"
resolvers += Resolver.typesafeRepo("releases")
//"mysql" % "mysql-connector-java" % "5.1.31"
libraryDependencies ++= Seq(
"mysql" % "mysql-connector-java" % "5.1.24",
"org.webjars" %% "webjars-play" % "2.3.0-2",
"com.typesafe.play" %% "play-slick" % "0.8.0",
"com.typesafe.play.plugins" %% "play-plugins-mailer" % "2.3.1",
"org.mindrot" % "jbcrypt" % "0.3m"
)
fork in Test := false
lazy val root = (project in file(".")).enablePlugins(PlayScala)
And my play.plugins contains only:
1500:com.typesafe.plugin.CommonsMailerPlugin
UPDATE:
I've downloaded the sample project from https://github.com/playframework/play-mailer
and tried to compile using sbt. It failed with exactly the same problem.
It looks like the problem is somehow with the version of plugin that is available in the typesafe repo:
I've built the plugin from sources, published it to my local repository and then everything compiled fine.
In the build.sbt of the sample app there is:
resolvers += Resolver.file("LocalIvy", file(Path.userHome + File.separator + ".ivy2" + File.separator + "local"))(Resolver.ivyStylePatterns)
So it looks like the authors also had problems in compiling the app using the plugin deployed to the official repository.
UPDATE:
Well, it compiled fine, but then failed at runtime with java.lang.ClassNotFoundException: com.typesafe.plugin.CommonsMailerPlugin
UPDATE 2:
The sample play.plugins is also wrong, the correct one should be:
1500:play.api.libs.mailer.CommonsMailerPlugin
and then eveyrthing finally works
The README is up to date with the latest development (upcoming version).
As of 19/12/2014, the latest version released and available in the Typesafe repository is 2.3.1. If you want to use this version you need to refer to the README in the v2.3.1 tag: https://github.com/playframework/play-mailer/tree/v2.3.1
For the upcoming version we have decided to break compatibility to move the Play mailer into its own package and provide a better implementation. That's the reason why the documentation in the master branch is not working with version 2.3.1.

Akka migration from 2.0 to 2.1

I have started working with Actors, and was following a simple example as mention in the Getting Started Guide.
Specs:
Scala Version: 2.9.2
Akka Version: 2.0
I ran the example and it ran well. Then I changed by sbt build script to:
name := "PracAkka"
scalaVersion := "2.9.2"
resolvers += "Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/"
libraryDependencies += "com.typesafe.akka" % "akka-actor_2.10" % "2.1.2"
i.e. I started using Akka 2.1.2. There were small changes and as per the migration guide, I made the respective changes. But still I am getting the below error:
class file needed by Props is missing. reference type ClassTag of package reflect refers to nonexisting symbol.
What do I need to change?
The documentation is quite clear I'd say: http://doc.akka.io/docs/akka/2.1.2/project/migration-guide-2.0.x-2.1.x.html
(I.e. Akka 2.1.x is for Scala 2.10)
From the sbt documentation
If you use groupID %% artifactID % revision rather than groupID %
artifactID % revision (the difference is the double %% after the
groupID), sbt will add your project's Scala version to the artifact
name. This is just a shortcut. You could write this without the %%:
libraryDependencies += "org.scala-tools" % "scala-stm_2.9.1" % "0.3"
Assuming the scalaVersion for your build is 2.9.1, the following is
identical:
libraryDependencies += "org.scala-tools" %% "scala-stm" % "0.3"
The idea is that many dependencies are compiled for multiple Scala
versions, and you'd like to get the one that matches your project.
Your script seems to be getting Akka for scala 2.10 in "akka-actor_2.10" try renaming to your scala version.

Adding an SBT plugin which does not specify an SBT version in its URL

Specs2 does not define the SBT version in its URL:
https://oss.sonatype.org/content/repositories/releases/org/specs2/specs2_2.9.2/1.12.3/
This is causing problems for SBT when trying to resolve it...
[warn] ==== sonatype-snapshots: tried
[warn] https://oss.sonatype.org/content/repositories/snapshots/org/specs2/specs2_2.9.2_0.12/1.12.3/specs2-1.12.3.pom
[warn] ==== sonatype-releases: tried
[warn] https://oss.sonatype.org/content/repositories/releases/org/specs2/specs2_2.9.2_0.12/1.12.3/specs2-1.12.3.pom
How do I get SBT to resolve the correct URL?
specs2 is not a sbt plugin it's a Scala library for writing executable software specifications.
There are two levels of sbt projects. Your own projects (for now call them "apps") and the build project definition itself (call it "the build").
library dependencies
When apps use other libraries during compilation or test, they are called library dependencies (or "deps" for short). These deps are declared in build.sbt (or *.sbt or project/*.scala) as follows:
libraryDependencies += "org.specs2" %% "specs2" % "2.2" % "test"
By saying %%, artifacts published using sbt automatically appends Scala binary version postfix such as _2.10 on Maven. This is due to the fact that (unlike Java) not all Scala releases are binary compatible with each other. Scala 2.9.1 and 2.9.2 are not compatible, so they both have distinct postfix _2.9.1 and _2.9.2, but Scala 2.10.x are all compatible among the series so they are given _2.10.
Unfortunately, however, different versions of Specs2 are required for Scala versions, you might have to do something more like:
libraryDependencies <+= scalaVersion({
case "2.9.2" => "org.specs2" %% "specs2" % "1.12.3" % "test"
case x if x startsWith "2.10" => "org.specs2" %% "specs2" % "2.2" % "test"
})
For more details check out Getting Started guide.
sbt plugins
There are special type of libraries that the build can depend on to extend its abilities, and they are sbt plugins. These are declared in project/plugins.sbt (or project/*.sbt) as follows:
addSbtPlugin("com.eed3si9n" % "sbt-buildinfo" % "0.2.5")
Since sbt plugins are dependent on sbt version and the Scala version that the build uses, both of those information are encoded somehow into the published artifact path. On Ivy, they are expressed as folder names but on Maven they are expressed as postfix:
http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.eed3si9n/sbt-buildinfo/scala_2.9.2/sbt_0.12/0.2.5/
https://oss.sonatype.org/content/repositories/public/org/scalaxb/sbt-scalaxb_2.10_0.13/1.1.2/