I followed this guide to release a Scala / SBT JAR file in Maven.
I ran the sbt publishSigned and sbt sonatypeRelease for the spark-fast-tests 0.11.0 release and the JAR file was correctly built. See here. These commands come from the sbt-sonatype plugin.
For some reason, when I did the 0.12.0 release, the Maven JAR file only includes HTML files and images. For example, the downloaded JAR file contains this file com/github/mrpowers/spark/fast/tests/DatasetComparer.html, but doesn't include DatasetComparer.class.
The target/scala-2.11/spark-fast-tests_2.11-2.3.0_0.12.0.jar file also only includes the HTML files (either sbt publishSigned or sbt sonatypeRelease must have generated this JAR file).
When I run sbt package, the JAR file that's generated includes the Scala classes like com/github/mrpowers/spark/fast/tests/DatasetComparer.class, as expected.
spark-fast-tests is an open source project and here is the build.sbt file.
How can I include my project classes in the JAR file that's uploaded to Maven? Any tips / tricks on how to debug this better?
Analysing spark-fast-tests build.sbt I would make the following recommendations:
Add sbt-release to plugins.sbt to enable release process customisation:
addSbtPlugin("com.github.gseitz" % "sbt-release" % "1.0.8")
addSbtPlugin("org.xerial.sbt" % "sbt-sonatype" % "2.0")
addSbtPlugin("com.jsuereth" % "sbt-pgp" % "1.1.0")
Create version.sbt at project root and move version setting out of build.sbt:
version in ThisBuild := "0.12.1-SNAPSHOT"
Create sonatype.sbt at project root and move the following settings out of build.sbt:
homepage := Some(url("https://github.com/mrpowers/spark-fast-tests/"))
scmInfo := Some(
ScmInfo(
url("https://github.com/mrpowers/spark-fast-tests/"),
"git#github.com:mrpowers/spark-fast-tests.git"
)
)
developers := List(
Developer(
"mrpowers",
"Matthew Powers",
"matthewkevinpowers#gmail.com",
url("https://github.com/mrpowers/spark-fast-tests/")
)
)
licenses += ("MIT", url("http://opensource.org/licenses/MIT"))
publishMavenStyle := true
Add the following release settings to build.sbt:
import sbtrelease.ReleaseStateTransformations._
publishTo := Some(
if (isSnapshot.value) { Opts.resolver.sonatypeSnapshots }
else { Opts.resolver.sonatypeReleases }
)
releasePublishArtifactsAction := PgpKeys.publishSigned.value
releaseProcess := Seq[ReleaseStep](
checkSnapshotDependencies,
inquireVersions,
runClean,
runTest,
setReleaseVersion,
commitReleaseVersion,
tagRelease,
publishArtifacts,
setNextVersion,
commitNextVersion,
releaseStepCommand("sonatypeReleaseAll"),
pushChanges
)
Release to Maven Central with sbt release
For a working example have a look at build configuration of sbt-sonatype itself.
A little late to the party, but I had the same problem with spark-testing-base.
It stems from the way you modify the artifactName:
artifactName := { (sv: ScalaVersion, module: ModuleID, artifact: Artifact) =>
artifact.name + "-" + module.revision + "." + artifact.extension
}
which is probably copied from the official SBT docs.
Note that the docs have the following hint below the code though:
(Note that in practice you rarely want to drop the classifier.)
My solution for spark-testing-base is the following:
artifactName := { (sv: ScalaVersion, module: ModuleID, artifact: Artifact) =>
Artifact.artifactName(sv, module, artifact).replaceAll(s"-${module.revision}", s"-${sparkVersion.value}${module.revision}")
}
This should make sure to keep the classifier (and whatever additional things might go into the artifactName) and solved the problem of JAR files containing documentation instead of class files.
Related
I stumbled upon this Akka grpc tutorial which suggests that we can create a jar from a project that has .proto file under src/main/proto and add it as a dependency in client and server projects to build their respective stubs.
libraryDependencies += "com.example" %% "my-grpc-service" % "1.0.0" % "protobuf-src"
But this doesn't seem to work!! Are there any example projects that demonstrates how this would work in action? How can we externalize protobuf sources and use the same in a jvm based project?
I was able to figure out how to externalise protobuf files properly as per suggestion from akka-grpc docs.
The problem was that I was not adding sbt-akka-grpc plugin required by sbt to recognise .proto files and include them in the packaged jar. Without this plugin there won't be any .proto file made available in the packaged jar.
addSbtPlugin("com.lightbend.akka.grpc" % "sbt-akka-grpc" % "1.1.0")
Make sure to add organization settings in your build.sbt to prepare jar correctly.
organization := "com.iamsmkr"
Also, if you wish to cross-compile this jar to multiple versions add following entries in your build.sbt:
scalaVersion := "2.13.3"
crossScalaVersions := Seq(scalaVersion.value, "2.12.14")
and then to publish:
$ sbt +publishLocal
With appropriate jars published you can now add them as dependencies in your client and server projects like so:
libraryDependencies +=
"com.iamsmkr" %% "prime-protobuf" % protobufSourceVersion % "protobuf-src"
You can check out this project I am working on to see this in action.
Alternate Way
An alternate way I figured is that you can keep your .proto files in a root directory and then refer them in client and server build.sbt like so:
PB.protoSources.in(Compile) := Seq(sourceDirectory.value / ".." / ".." / "proto")
Checkout this project to see it in action.
I'm trying to create a relatively simple sbt plugin to wrap grpc-swagger artifact.
Therefore, I've created a project with the following structure:
projectDir/
build.sbt
lib/grpc-swagger.jar <- the artifact I've downloaded
src/...
where build.sbt looks like the following:
ThisBuild / version := "0.0.1-SNAPSHOT"
ThisBuild / organization := "org.testPlugin"
ThisBuild / organizationName := "testPlugin"
lazy val root = (project in file("."))
.enable(SbtPlugin)
.settings(name := "grpc-swagger-test-plugin")
According to sbt docs, that's all I have to do in order to include an unmanaged dependecy, that is:
create a lib folder;
store the artifact in there;
However, when I do execute sbt compile publishLocal, the plugin published lacks of that external artifact.
So far I've tried to:
set exportJars := true flag
add Compile / unmanagedJars += file(lib/grpc-swagger.jar") (with also variations of the path)
manual fiddling to libraryDependecies using from file("lib/grpc-swagger.jar") specifier
but none so far seemed to work.
So how am I supposed to add an external artifact to a sbt plugin?
The proper solution to this problem is to publish the grpc-swagger library. If for whatever reason this can't be done from that library's build system, you can do it with sbt. Just add a simple subproject whose only job it is to publish that jar. It should work like so:
...
lazy val `grpc-swagger` = (project in file("."))
.settings(
name := "grpc-swagger",
Compile / packageBin := baseDirectory.value / "lib" / "grpc-swagger.jar",
// maybe other settings, such as grpc-swagger's libraryDependencies
)
lazy val root = (project in file("."))
.enable(SbtPlugin)
.settings(name := "grpc-swagger-test-plugin")
.dependsOn(`grpc-swagger`)
...
The pom file generated for the root project should now specify a dependency on grpc-swagger, and running the publish task in the grpc-swagger project will publish that jar along with a pom file.
That should be enough to make things work, but honestly, it's still a hack. The proper solution is to fix grpc-swagger's build system so you can publish an artifact from there and then just use it via libraryDependencies.
Disclaimer: I am new to sbt and Scala so I might be missing obvious things.
My objective here is to use the Scala compiler as a library from my main project. I was initially doing that by manually placing the scala jars in a libs directory in my project and then including that dir in my classpath. Note that at the time I wasn't using sbt. Now, I want to use sbt and also download the scala sources from github, build the scala jars and then build my project. I start by creating 2 directories: myProject and myProject/project. I then create the following 4 files:
The sbt version file:
// File 1: project/build.properties
sbt.version=0.13.17
The plugins file (not relevant to this question):
// File 2: project/plugins.sbt
addSbtPlugin("com.eed3si9n" % "sbt-buildinfo" % "0.7.0")
The build.sbt file:
// File 3: build.sbt
lazy val root = (project in file(".")).
settings(
inThisBuild(List(
organization := "me",
scalaVersion := "2.11.12",
version := "0.1.0-SNAPSHOT"
)),
name := "a name"
).dependsOn(ScalaDep)
lazy val ScalaDep = RootProject(uri("https://github.com/scala/scala.git"))
My source file:
// File 4: Test.scala
import scala.tools.nsc.MainClass
object Test extends App {
println("Hello World !")
}
If I run sbt inside myProject then sbt will download the scala sources from github and then try to compile them. The problem is that the base-directory is still myProject. This means that if the scala sbt source files refer to something that is in the scala base-directory they won't find it. For example, the scala/project/VersionUtil.scala file tries to open the scala/versions.properties file that lies in the scala base-directory.
Question: How can I set sbt to download a github repo and then build it using that project's base-directory instead of mine's (by that I mean the base-directory of myProject in the above example) ??
Hope that makes sense.
I would really appreciate any feedback on this.
Thanks in advance !
In the Scala ecosystem you usually depend on binary artifacts (libraries) that are published in Maven or Ivy repositories. Virtually all Scala projects publish binaries, including the compiler. So all you have to do is add the line below to your project settings:
libraryDependencies += "org.scala-lang" % "scala-compiler" % scalaVersion.value
dependsOn is used for dependencies between sub-projects in the same build.
For browsing sources you could use an IDE. IntelliJ IDEA can readily import Sbt projects and download/attach sources for library dependencies. Eclipse has an Sbt plugin that does the same. Ensime also, etc. Or just git clone the repository.
I have a CLI tool written in Java which can modify some source with the added params. For example, it can rename an enum value across a whole project.
I want to write an sbt task that can run this tool from my project dir with the given params, like sbt 'enums -rename A B'. My tool can be injected to the project through the sbt dependencies.
I skimmed through the book sbt in Action looking for an answer, but those examples are not this specific.
My build.sbt (far from working):
name := """toolTestWithActivator"""
version := "1.0-SNAPSHOT"
resolvers += "Local Repository" at "file://C:/Users/torcsi/.ivy2/local"
lazy val root = (project in file(".")).enablePlugins(PlayJava)
scalaVersion := "2.11.6"
libraryDependencies ++= Seq(
"tool" % "tool_2.11" % "1.0",
javaJdbc,
javaEbean,
cache,
javaWs
)
val mytool = taskKey[String]("mytool")
mytool := {
com.my.tool.Main
}
Can sbt handle this type of task/dependency structure, or do I need to do this another way?
SBT is recursive: it compiles .sbt files and .scala files under the project folder and use those to execute your build (in fact you can see sbt as a library that helps you producing builds).
So, as you need your library to define a task, that one is a dependency of your build.sbt file (and not a dependency of your project).
To declare that the build.sbt file depends on your library, just create a ".sbt" file in the project folder; example:
project/dependencies.sbt
libraryDependencies += "tool" %% "tool" % "1.0"
and in build.sbt add:
val mytool = taskKey[Unit]("mytool")
mytool := {
com.my.tool.main(Array())
}
Some comments:
be careful with the scala version used: as sbt 0.13 is compiled with scala 2.10; your library should also be compiled for scala 2.10 (the package should be tools_2.10 ). And the new sbt 1.0 is compiled with scala 2.12.
I used the %% notation, so that sbt adds by itself the expected scala version.
I supposed your cli tool defines a classic java main method (or the scala equivalent). So, the argument should be an Array of String (here an empty one) and it returns Unit (void in java).
Some reference to understand the solution:
http://www.scala-sbt.org/0.13/docs/Organizing-Build.html
Hello I have 5 jar files and tried to publish them to a local repository with sbt.
But when I place them in unmanagedBase directory like lib/ they won't get copied with publishLocal. Is there an easy way to include them in the publishing process?
Currently maven has a similar solution here:
http://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
One option is to define a subproject for each jar that you want published. Have your main project depend on each. Give each subproject an appropriate name, version, and organization. For each subproject, put its jar somewhere not on the classpath and make the output of packageBin be that jar.
For example (sbt 0.13 build.sbt),
lazy val main = project.dependsOn(subA)
lazy val subA = project.settings(
name := "third-party",
organization := "org.example",
version := "1.4",
packageBin in Compile := baseDirectory.value / "bin" / "third-party.jar",
// if there aren't doc/src jars use the following to
// avoid publishing empty jars locally
// otherwise, define packageDoc/packageSrc like packageBin
publishArtifact in packageDoc := false,
publishArtifact in packageSrc := false,
// tell sbt to put the jar on main's classpath
// and not the (empty) class directory
exportJars := true,
// set this to not add _<scalaBinaryVersion> to the name
crossPaths := true
)
This approach allows you to change the jar in subA/bin/third-party.jar and have it be used immediately and a subsequent publishLocal will publish it locally.
If you prefer separately publishing it locally, so that it isn't part of the project, define subA as a standalone project instead.