Disclaimer: I'm scala beginner. All defaults works nice for me, but whenever I want to have custom layout/build I run into a problem.
So, as part of my build I need to pull .war(web app) from project A and .jar(jetty launcher) from project B into some directory of project C(tanuki service wrapper).
Can anybody please provide an an example how to do this in the most effective way.
Not sure if it works with war files, but for making jars locally available you could use sbt's publish-local command. Say you have an sbt project "mylibrary" and another sbt project "mymain". If you locally publish "mylibrary.jar", you can add it as a dependency to "mymain" just like you add any other sbt-managed dependency, i.e., by adding a line such as
libraryDependencies += "foo.bar.com" %% "mylibrary" % "0.1-SNAPSHOT"
to the build.sbt of "mymain".
If that is not possible you might want to write an sbt plugin/command that copies the files into a given directory. I don't have experience with extending sbt, so I can't help with that, but other stackoverflowers surely can :-)
EDIT: (addressing a comment by the OP)
No, I don't have a particular Sbt tutorial. If I need help I turn to the usual suspects, the wiki, the mailing list, Stackoverflow, Sbt's source code. Sbt has an IO package which offers a copyFile method, which, according to this thread, comes in handy. Searching for 'copying files' on the mailing list also yields other results that might help you.
Related
I am working on a sample scalatra webapp. I have created one more service jar which contains dao and service layer. I want to add this dependency in Scalatra-SBT project. How can i achieve this?
I bundled my service jar using command : SBT package which created a jar. How will i include this jar in web application? Can i add this in build.scala file? or can i copy into any Webapp folder ?
Is it possible to push this jar to local repository and pull it from there when my webapp builds?
Whew! Lots of questions here!
The good news is that SBT can do all of the tasks that you are asking for.
DO NOT copy JAR files around to satisfy dependencies! It will end up in tears, virtually guaranteed. Tools like Ivy and Maven (and by extension, SBT) are here to help.
To push your service jar to a local repo:
The SBT task is publish-local, i.e. sbt publish-local from your service jar's root directory.
You'll see lots of descriptive output, finishing up with lines in the following format:
[info] published services_2.10 to /Users/millhouse/.ivy2/local/services/services_2.10/0.1-SNAPSHOT/jars/services_2.10.jar
You don't need to do anything special in your build.sbt to make this work, as long as you have the name and scalaVersion variables set. Note that this will publish to your local Ivy cache, which is $HOME/.ivy2/local for most people.
To get your Scalatra-SBT webapp to pick up your service jar:
Edit your webapp's project/build.scala, adding this dependency under the libraryDependencies key (there should already be a few dependencies, one per line, put yours somewhere in the middle!):
"services" %% "services" % "0.1-SNAPSHOT",
Perform an sbt clean update and your dependency will be pulled in. If it doesn't work, SBT will give you a list of the places where it looked for the artifacts; compare it closely with the location that they were published to (in the previous step) and you'll probably find a typo; fix it and try again.
Please note that there is a lot more to dependency and release management than I've shown above, but this should be enough to get you going.
I have a fairly normal Scala project currently being built using Maven. I would like to support both Scala 2.9.x and the forthcoming 2.10, which is not binary or source compatible. I am willing to entertain converting to SBT if necessary, but I have run into some challenges.
My requirements for this project are:
Single source tree (no branching). I believe that trying to support multiple concurrent "master" branches for each Scala version will be the quickest way to miss bugfixes between the branches.
Version specific source directories. Since the Scala versions are not source compatibile, I need to be able to specify an auxiliary source directory for version specific sources.
Version specific source jars. End users should be able to download the correct source jar, with the correct version specific sources, for their version of Scala for IDE integration.
Integrated deployment. I currently use the Maven release plugin to deploy new versions to the Sonatype OSS repository, and would like to have a similarly simple workflow for releases.
End-user Maven support. My end users are often Maven users, and so a functional POM that accurately reflects dependencies is critical.
Shaded jar support. I need to be able to produce a JAR that includes a subset of my dependenices and removes the shaded dependencies from the published POM.
Things I have tried:
Maven profiles. I created a set of Maven profiles to control what version of Scala is used to build, using the Maven build-helper plugin to select the version specific source tree. This was working well until it came time to publish;
Using classifiers to qualify versions doesn't work well, because the source jars would also need custom classifiers ('source-2.9.2', etc.), and most IDE tools wouldn't know how to locate them.
I tried using a Maven property to add the SBT-style _${scala.version} suffix to the artifact name, but Maven does not like properties in the artifact name.
SBT. This works well once you can grok it (no small task despite extensive documentation). The downside is that there does not seem to be an equivalent to the Maven shade plugin. I've looked at:
Proguard. The plugin is not updated for SBT 0.12.x, and won't build from source because it depends on another SBT plugin that has changed groupIds, and doesn't have a 0.12.x version under the old name. I have not yet been able to work out how to instruct SBT to ignore/replace the plugin dependency.
OneJar. This uses custom class loading to run Main classes out of embedded jars, which is not the desired result; I want the class files of my project to be in the jar along with (possibly renamed) class files from my shaded dependencies.
SBT Assembly plugin. This can work to a degree, but the POM file appears to include the dependencies that I'm trying to shade, which doesn't help my end users.
I accept that there may not be a solution that does what I want for Scala, and/or I may need to write my own Maven or Scala plugins to accomplish the goal. But if I can I'd like to find an existing solution.
Update
I am close to accepting #Jon-Ander's excellent answer, but there is still one outstanding piece for me, which is a unified release process. The current state of my build.sbt is on GitHub. (I'll reproduce it here in an answer later for posterity).
The sbt-release plugin does not support multi-version builds (i.e., + release does not behave as one might like), which makes a sort of sense as the process of release tagging doesn't really need to happen across versions. But I would like two parts of the process to be multi-version: testing and publishing.
What I'd like to have happen is something akin to two-stage maven-release-plugin process. The first stage would do the administrative work of updating Git and running the tests, which in this case would mean running + test so that all versions are tested, tagging, updating to snapshot, and then pushing the result to upstream.
The second stage would checkout the tagged version and + publish, which will rerun the tests and push the tagged versions up to the Sonatype repository.
I suspect that I could write releaseProcess values that do each of these, but I'm not sure if I can support multiple releaseProcess values in my build.sbt. It probably can work with some additional scopes, but that part of SBT is still strange majick to me.
What I currently have done is changed the releaseProcess to not publish. I then have to checkout the tagged version by hand and run + publish after the fact, which is close to what I want but does compromise, especially since the tests are only run on the current scala version in the release process. I could live with a process that isn't two-stage like the maven plugin, but does implement multi-version test and publish.
Any additional feedback that can get me across the last mile would be appreciated.
Most of this is well supported in sbt within a single source tree
Version specific source directories are usually not need. Scala programs tends to be source compatible - so often in fact that
crossbuilding (http://www.scala-sbt.org/release/docs/Detailed-Topics/Cross-Build) has first class support in sbt.
If you really need version specific code, you can add extra source folders.
Putting this in your build.sbt file will add "src/main/scala-[scalaVersion]" as a source directory for each version as you crossbuild in addition to the regular "src/main/scala".
(there is also a plugin available for generating shims between version, but I haven't tried it - https://github.com/sbt/sbt-scalashim)
unmanagedSourceDirectories in Compile <+= (sourceDirectory in Compile, scalaVersion){ (s,v) => s / ("scala-"+v) }
version specific source jars - see crossbuilding, works out of the box
integrated deployment - https://github.com/sbt/sbt-release (has awesome git integration too)
Maven end-users - http://www.scala-sbt.org/release/docs/Detailed-Topics/Publishing.html
Shaded - I have used this one https://github.com/sbt/sbt-assembly which have worked fine for my needs.
Your problem with the assembly plugin can be solved by rewriting the generated pom.
Here is an example ripping out joda-time.
pomPostProcess := {
import xml.transform._
new RuleTransformer(new RewriteRule{
override def transform(node:xml.Node) = {
if((node \ "groupId").text == "joda-time") xml.NodeSeq.Empty else node
}
})
}
Complete build.sbt for for reference
scalaVersion := "2.9.2"
crossScalaVersions := Seq("2.9.2", "2.10.0-RC5")
unmanagedSourceDirectories in Compile <+= (sourceDirectory in Compile, scalaVersion){ (s,v) => s / ("scala-"+v) }
libraryDependencies += "joda-time" % "joda-time" % "1.6.2"
libraryDependencies += "org.mindrot" % "jbcrypt" % "0.3m"
pomPostProcess := {
import xml.transform._
new RuleTransformer(new RewriteRule{
override def transform(node:xml.Node) = {
if((node \ "groupId").text == "joda-time") xml.NodeSeq.Empty else node
}
})
}
I've done something similar to this with SBT as an example:
https://github.com/seanparsons/scalaz/commit/21298eb4af80f107181bfd09eaaa51c9b56bdc28
It's made possible by SBT allowing all the settings to be determined based on other settings, which means that most other things should "just work".
As far as the pom.xml aspect I can only recommend asking the question in the SBT mailing list, I would be surprised if you couldn't do that however.
My blog post http://www.day-to-day-stuff.blogspot.nl/2013/04/fixing-code-and-binary.html contains an example of a slightly more finegrained solution for attaching different source directories; one per major S. Also it explains how to create scala-version-specific code that can be used by not-specific code.
Update 2016-11-08: Sbt now supports this out of the box: http://www.scala-sbt.org/0.13/docs/sbt-0.13-Tech-Previews.html#Cross-version+support+for+Scala+sources
I've come pretty well along with sbt, the Scala Build Tool. If you only have small problems in the code, it's easy.
Now, after a major feature add, much of my code is broken and I sbt seems to be confused as to how the dependencies are. I could help it, compiling the fundamental modules first, but it does not seem to let me.
It's help system is... notorious.
> help compile
Compiles sources.
Yeah, well. I guessed that.
What I wanted to hear was: how do I compile only - say - src/module/A.scala.
This might not even be possible (hello again, make, never abandoned you!). At least I cannot find any reference on the Internet to applying sbt compile just to a single file.
I'm using sbt from the command line prompt, not an IDE.
UPDATE:
It was my fault. :/ Had split a source file into multiple, but forgot to copy a package clause to each of the new ones. Oooops.
Will keep this open for a while, since compiling just a single file (i.e. something like sbt compile filename) would imho not be a bad thing.
You could define a Multi-Project Build where the files you want to compile separately are encapsulated in a project. According to the docs, the following is then possible:
At the sbt interactive prompt, type projects to list your projects and project to select a current project. When you run a task like compile, it runs on the current project. So you don't necessarily have to compile the root project, you could compile only a subproject.
I just wanted to mention here that I came across sbt-compile-quick-plugin (https://github.com/etsy/sbt-compile-quick-plugin). It does what it says on the tin, just add addSbtPlugin("com.etsy" % "sbt-compile-quick-plugin" % "1.3.0") to your project/plugins.sbt, then you can just start up sbt and run compileQuick /path/to/your/file
(See https://stackoverflow.com/a/46849619/1358677)
I would like to give IntelliJ IDEA a try, but I have no idea how to get going.
I am simply trying to create a new project that uses Finagle Echo Server, hosted on github, as starting point.
Assuming I'm starting with a clean install on Mac. I installed IDEA and added the Scala and SBT plugins. What steps should I take to create a project, that uses Finagle and run the code in the http server example?
PLEASE help. I realize my question sounds like a stupid question, but there are so many different approaches to working with Scala projects from SBT command line, Scala-IDE, Idea, etc, that I simply don't know how to get a comfortable development environment going.
A manual solution that doesn't require you to use SBT for your project might be more straightforward, given the SBT versioning issues. You'll still use SBT to build finagle, though.
Install the SBT runner script per step 1 above. (It can handle SBT 0.7 projects too).
Manually git clone git://github.com/twitter/finagle.git.
cd to the finagle directory and type "sbt package". Its dependencies should end up under lib_managed, and it should produce the finagle jars themselves under target/ or some such (just note the locations in the command output).
Create an IDEA project from scratch, and manually add dependencies to the finagle jars and their dependencies (under Project Structure->Dependencies).
This answer assumes that you want to use SBT. Also, I should qualify that this is my usual procedure, but I haven't confirmed that it works with finagle in particular.
0. Install IDEA, with Scala and SBT plugins. (Done by the OP; here for others)
1. Install SBT (automatic method). Copy this handy sbt runner script to a convenient location (or, if you want to keep it up to date, git clone https://github.com/paulp/sbt-extras.git and symlink the script into ~/bin), and make sure it's executable. It will automatically download whatever it needs based on the sbt.version specified in your build.properties.
2. Install sbt-idea. sbt-idea is an SBT plugin (not an IDEA plugin) that generates IDEA module files from an SBT project. It's convenient to install this globally since it's not project-specific. You don't have to download anything manually; just add this to ~/.sbt/plugins/build.sbt:
resolvers += "sbt-idea-repo" at "http://mpeltonen.github.com/maven/"
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "0.11.0")
3. Create SBT project. Create a directory for your project, and a "project" directory within it. Create project/Build.scala as follows:
import sbt._
object MyBuild extends Build {
lazy val root = Project("root", file(".")) dependsOn finagle
lazy val finagle = RootProject(uri("git://github.com/twitter/finagle.git"))
}
See the SBT documentation for lots more options re configuring your project. Note we have to use the Full Configuration here (not just build.sbt) in order to express the github dependency.
It's also a good idea to create project/build.properties:
sbt.version=0.11.2
project.version=0.1
build.scala.versions=2.9.1
4. Generate IDEA project. cd to the directory containing the sbt-based project. type "sbt gen-idea". If all goes well, the directory will now have ".idea" and ".idea_modules" subdirectories.
5. Open the project in IDEA. It may be necessary to fix the target JDK version in the project settings. Aside from that, the project should be ready to go, with all source paths, library dependencies, etc. properly configured.
I'm new to SBT and am unsure how to get a project started. Can someone point me to a beginner's guide to creating a Hello World type project, or give me some clues?
My preferred IDE is IDEA. I have run sbt-idea according to the instruction on the IDEA Plugins page. At the moment I'm a bit confused because
there are no source folders created - where / how am I supposed to create them and how will SBT know where to look?
why is it trying to use Scala 2.8.1, when I have already put scalaVersion := "2.9.0" in the build.sbt file? This means IDEA doesn't recognize object HelloWorld extends App {}.
the instructions on the plugins page above suggest changing the Before Launch options of "a Run Configuration (including the Default Run Configuration)". There are 13 different default configurations for different things listed - which one to change? Should I be creating a new one? Are these default configurations just for this project or will it adversely affect all my other projects that don't use SBT?
Thanks.
This worked for me:
First get sbt and the gen-idea plugin going...
Download the sbt-launch.jar and create the script for launching it as described on the SBT Github wiki.
Create a directory for your new project, such as (on linux) ~/myCode/myNewProject and change to that directory
Run sbt command. This should download the scala libraries and create a 'project' and 'target' directories.
Change to the 'project' directory.
Create a new file 'build.sbt' in this directory with the following lines, as described on the sbt-idea plugin Github wiki:
resolvers += "sbt-idea-repo" at "http://mpeltonen.github.com/maven/"
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.0.0")
Change back to your main project directory such as ~/myCode/myNewProject. Run sbt. It should download the gen-idea plugin.
From the sbt console (which should be running now), run the gen-idea command. It should create the IDEA project directories. For me, it also emits copious warnings.
Now get the IDEA SBT console plugin going...
Open IDEA and install the "SBT" plugin from the plugin manager and restart IDEA. (Note this is the IDEA plugin, not the sbt plugin described above.)
Configure the SBT plugin as described on its wiki (run configurations, location of sbt-launch.jar etc).
Open the freshly generated IDEA project in IDEA.
Put your code and other things in the expected default directories as described on the sbt wiki under 'Directory Layout'. You need to create these directories yourself - sbt doesn't create them automatically. The 'src' and 'test' directories should be at the same level as the 'project' and 'target' directories that sbt created.
Make up a new 'build.sbt' file and put it in ~/myCode/myProject (or whatever you called it). Since I am just figuring out sbt, mine is simple so far - just nominates scalatest as a dependency and uses Scala 2.9:
name := "myProject"
version := "0.1"
organization := "me"
libraryDependencies += "org.scalatest" % "scalatest_2.9.0" % "1.6.1"
scalaVersion := "2.9.0"
Enter the reload command in the SBT console at the bottom of the IDEA screen. It should download scalatest and Scala 2.9 for you. Maybe you need to run 'update' too.
I wrote a very quick guide about it. It is not intended to be an SBT guide -- there's no way to beat the SBT Wiki for that. It would be pointless too, since one can just contribute to the wiki itself.
But, I think my very quick guide will get you up and running as you wish.
As for directory creation, the answer I got was that SBT expects the IDE to handle that -- I don't really like that attitude, but a plugin can do the job. You'll see I install the sbt eclipse plugin just so it will do it for me, even though I use IDEA myself (when I use an IDE).
Also, note that, ideally, you use both the IDEA plugin for SBT that you mentioned, and the SBT plugin for IDEA. See here for the list of plugins.
Unless the IDEA plugin has evolved a lot, you really need to generate an IDEA configuration from SBT iself -- IDEA won't "read" your configuration. That's what the SBT plugin for IDEA does. Install it, and sbt gen-idea. I expect that will solve the problem you mentioned.
Note, however, that the version of Scala you use to compile your project and the version of Scala that SBT uses for itself are different indeed. This is not a problem, it is as expected. I'm not sure from your question if the 2.8.1 version you mentioned is the one used by SBT, or one used by IDEA -- or even one used to compile your project, indicating that something is not working.
Where did you put the example you mentioned anyway? You should follow maven-style directory hierarchy, which means putting it on src/main/scala/, and possibly a subdirectory of that related to the package, if you follow Java convention as well.
And try to compile with sbt, to make sure that is working, before proceeding to IDEA.
I normally put the src into a folder "src/test/scala" and "src/main/scala". sbt-idea will add this folders ase source/test folder in idea.
sbt itself needs Scala 2.8.1 but this version has nothing to do with the version of your code. If you try to compile some source sbt will download Scala 2.9.0.