I'm trying to create a standalone jar file from the elastic4s sources on github. I've used sbt compile as detailed on the github page, but I can't find the jar file.
How do I use sbt to create the jar file so I can import it into my projects as a dependency?
The compile task will only compile the project.
> help compile
Compiles sources.
If you want to create a jar file and use it as a dependency in your project, you have two ways of doing that.
Unmanaged dependency (not recommended)
Unmanaged dependency run +package, which will create a jar file for each supported scala version, which you can use in your projects as an unmanaged dependency. Copy the package-generated jar to lib folder in your project.
The jar files will be located in target/scala-2.11 and target/scala-2.10, depending on the Scala version you want to use it with.
Publish to Local Repository (recommended yet imperfect)
If you want to include your custom-built elastic4s, as a managed dependency, you have to run +publishLocal. This will do the same as above, but additionally it will publish the artifact to your local repository. Assuming you've built it with version := "1.2.1.1-SNAPSHOT", you can include it in your project by just adding:
libraryDependencies += "com.sksamuel.elastic4s" %% "elastic4s" % "1.2.1.1-SNAPSHOT"
What makes the approach imperfect is that once you shared the project on GitHub (or any other project sharing platform), people will have to do publishLocal themselves to be able to build your project. The dependency should therefore go to one of the official binary repositories so when a dependency is needed, it's downloaded from Internet. Consult Publishing.
What is the + character in front of the commands
The + in the commands is for cross-building, if you don't use it the command will be executed only using scalaVersion declared in the build.sbt of the project.
Related
I have been using sbt on windows and a custom build.sbt script in conjunction with an import Chisel._ in the top-level file in order to generate Verilog from my Chisel source successfully.
I'm trying to get an IDE working on Windows to expedite Chisel development. I've gone with the Eclipse based SCALA IDE http://scala-ide.org/download/sdk.html/
I want to compile the Chisel library so that the import Chisel._ can be resolved locally, without having to go off and download the source from the repository each timeand recompile the source. When I download the Chisel-master repo from Git and include the src\main folder in my SCALA project in the SCALA IDE, I get lots of syntax errors in the Chisel SCALA files that prevent me from building the project.
Has anyone done anything like this before on Windows or have any knowledge of working with the SCALA IDE as it may just be a case of undefined symbols in the project configuration?
Not sure exactly what you did with build.sbt respect to recompile (I think it download it only the first time, then it caches it for the future). But I'm using ScalaIDE for Chisel on linux, using the default build.sbt files, maybe you can try to get it working out of the box first to help narrow down the issue.
Here are the steps I took in order to get ScalaIDE work with Chisel:
the latest Scala IDE uses 2.11.8, the current Chisel repository defaults to 2.11.7. So I had to change all the build.sbt reference to scalaVersion from 2.11.7 to 2.11.8
I used sbteclipse
https://github.com/typesafehub/sbteclipse
To create importable the workspace to setup the compilation dependencies.
Except for chiselFrontEnd. For some reason, this package is not added to the dependency. I have to Add chiselFrontEnd as a javabuildpath dependency manually (Properties/JavaBuildPath, under Projects) for my own projects.
To resolve undefined symbols, you can also add a JAR onto the project build path using Project Properties > Java Build Path > Libraries > Add External JARs...
If you are getting your JARs through Maven / SBT, they should be in:
C:\Users\<name>\.ivy2\local\edu.berkeley.cs\chisel3_2.11\jars
If you are using publish-local with chisel3, your JARs should be in
C:\Users\<name>\.ivy2\cache\edu.berkeley.cs\chisel3_2.11\jars
Note that chisel3 is compiled into one JAR, including coreMacros and chiselFrontend sub-projects
Of course, this is a more quick-and-dirty solution compared to something that can parse SBT files.
I am trying to use an LSH implementation of Scala(https://github.com/marufaytekin/lsh-spark) in my Spark project.I cloned the repository with some changes to the sbt file (added Organisation)
To use this implementation , I compiled it using sbt compile and moved the jar file to the "lib" folder of my project and updated the sbt configuration file of my project , which looks like this ,
Now when I try to compile my project using sbt compile , It fails to load the external jar file ,showing the error message "unresolved dependency: com.lendap.spark.lsh.LSH#lsh-scala_2.10;0.0.1-SNAPSHOT: not found".
Am i following the right steps for adding an external jar file ?
How do i solve the dependency issue
As an alternative, you can build the lsh-spark project and add the jar in your spark application.
To add the external jars, addJar option can be used while executing spark application. Refer Running spark application on yarn
This issue isn't related to spark but to sbt configuration.
Make sure you followed the correct folder structure imposed by sbt and added your jar in the lib folder, as explained here - lib folder should be at the same level as build.sbt (cf. this post).
You might also want to check out this SO post.
The requirement: an SBT plugin code is in an unmanaged jar, for example in lib/unmanaged-sbt-plugin-0.0.1.jar.
I was expecting the jar file would be picked up by SBT automatically and the plugin would work out-of-the-box, i.e.: the tasks would be accessible from SBT command line, but that is not the case.
The jar file has the sbt/sbt.autoplugins file in it and the plugin works if pulled from a remote repo into the local one and imported via addSbtPlugin(...). Please note I cannot do that - it's a requirement to get it to load from the lib/unmanaged-sbt-plugin-0.0.1.jar and not from the local/remote repo.
Putting the following line in the build.sbt doesn't make the plugin work (there's not error either):
unmanagedJars in Compile += file("lib/unmanaged-sbt-plugin-0.0.1.jar")
The implementation of addSbtPlugin(...) is simply (according to http://www.scala-sbt.org/0.12.2/docs/Getting-Started/Using-Plugins.html):
def addSbtPlugin(dependency: ModuleID): Setting[Seq[ModuleID]] =
libraryDependencies <+= (sbtBinaryVersion in update, scalaBinaryVersion in update)
{ (sbtV, scalaV) => sbtPluginExtra(dependency, sbtV, scalaV) }
I'm wondering if the above info can be used to resolve my issue?
Thank you in advance!
So you can specify an explicit URL for library dependencies (ModuleID):
addSbtPlugin("org.my-org" % "unmanaged-sbt-plugin" % "0.0.1"
from "file:///./lib/unmanaged-sbt-plugin-0.0.1.jar")
Have been struggling to get this to work.
Could not get it to work with proposed solution using from "file://.." (using sbt 1.0.4).
Got it to work by putting the plugin in project/lib folder and adding all the plugin dependencies to plugins.sbt using libraryDependencies ++= Seq(..) like in build.sbt. You can find the plugin dependencies by looking at the plugin pom file, usually in .ivy2/local/<org>/<pluginname>/poms folder.
I have several SBT 0.13 / Play 2.2 projects (websites). They are all multi-module as they share some common functionality. This makes their project configuration files both complex and almost identical, but not quite.
I would like to be able to share as much as possible of these configuration files across the projects (frequent play updates makes keeping 5+ websites up to date a royal pain, not to mention all the almost-identical-but-evolving dependency lists across the projects).
build.properties and plugins.sbt are identical across projects and can be overwritten by a simple script. Great.
Build.scala is trickier - I would like to introduce a shared base class like so:
abstract class MyBuildBase extends Build { ... }
that in Build.scala do:
object ApplicationBuild extends MyBuildBuild { ... }
In order for this to make any sense at all, MyBuildBase.scala needs to be shared across projects. This can be done with svn:external, which operates on directories. Which means I need to somehow make this shared directory accessible when Build.scala is compiled (otherwise sbt complains loudly).
Reading http://www.scala-sbt.org/0.13.0/docs/Detailed-Topics/Classpaths.html and http://www.scala-sbt.org/0.13.0/docs/Getting-Started/Full-Def.html it seems like this should be possible.
However, it is exceptionally unclear to me what to actually put in the project/project/Build.scala file to actually achieve this - I can't find an example of "an sbt build file that's intended to build an sbt build file and include some extra source files in the build".
Any suggestions?
What you probably want to do is create a plugin, or shared library.
You can make an sbt project with a build like follows:
build.sbt
sbtPlugin := true
organization := "you"
name := "common-build"
version := "1.0"
Then create in src/main/scala your abstract class "MyBuildBase". Release this project as an sbt plugin.
Then in your other projects, you can use this as a library/plugin. In project/plugins.sbt add:
addSbtPlugin("you" % "common-build" % "1.0")
And this will resolve your common build library when building your build.
If you need more information, look up more about sbt plugins and ignore the part about making something that extends a Plugin. Plugins are just libraries versioned with sbt's version number and your own. You should be able to put whatever code you want in there to share between builds.
Note: in 2016, Build.scala is deprecated for Build.sbt.
Here is the new (Dec. 2016) multi-module with App Scala sbt template by Michael Lewis.
Usage
sbt new lewismj/sbt-template.g8
You can then run:
sbt compile
sbt publish-local
sbt assembly
It is based on Scala SBT template (Library)
This giter8 template will write SBT build files for a Scala library.
I am trying to use scala to access Amazon's DynamoDB and found this great package on github https://github.com/piotrga/async-dynamo
so I downloaded the code as a zip file , unzipped it and then did "sbt clean test" and getting the following error
error sbt.ResolveException: unresolved dependency: asyncdynamo#async-dynamo;1.6.0: not found
Questions : is this the correct way to generate a jar file that I can include in my Scala program or is there a better way?
thanks in advance.
EDIT:
just for the benefit of others, the SCALA SBT documentation provides lots of information regarding the build process.
Instead of generating a jar file, you can just run 'sbt publish-local' and then include the lines for the managed dependency in the other project.
Sbt/ivy will see you have the artifact that way you don't need to add the jar to the other project which is much cleaner.
Then for example if you need to update the other project you don't need to replace the jar again - just publish-local again and clean and run your other project!
You are not the only one to have problems with this it seems, see github issues page:
https://github.com/piotrga/async-dynamo/issues
The command 'sbt clean test' will run the tests sbt detects. If you want a .jar file you could use 'sbt clean package', which produces a .jar in the target/ folder.
I cloned the repo and was able to run sbt package after changing release.sbt a bit. I had to change the 'publishTo'-variable as it seemed to depend on the repository creators local environment variable, so I just commented it away.
I did not get the dependency problem, so I suppose it is correctly declared. The tests it tries to run do fail though, but sbt package compiles produces the .jar just fine.
EDIT: As Matthias Schlaipfer pointed out in the comments, the more elegant way(and much easier) would just be to add this as an depency in your build.sbt. From the readme, this is what you need to add:
resolvers += "piotrga" at
"https://github.com/piotrga/piotrga.github.com/tree/master/maven-repo"
libraryDependencies += "asyncdynamo" % "async-dynamo" % "1.6"