I have a pretty simple configuration:
//lazy val bananaRdfProject = RootProject( uri("git://github.com:stample/banana-rdf.git#"+bananaGitBranch) )
// lazy val bananaRdfProject = RootProject( uri("https://github.com/stample/banana-rdf.git#"+bananaGitBranch) )
// lazy val bananaRdfProject = ProjectRef( uri("https://github.com/stample/banana-rdf.git#"+bananaGitBranch) ,"banana-rdf")
lazy val bananaRdfProject = RootProject( file("../banana-rdf") )
lazy val main = play.Project(appName, appVersion, appDependencies).settings(...)
.dependsOn( bananaRdfProject )
I tried using the 4 different project declarations above for bananaRdfProject.
As I may edit this banana-rdf locally, I want it to be recompiled each time I build my play project, so that I do not have to publish the banana-rdf...
But when I try to compile my main play project, that uses banana-rdf, it doesn't compile banana-rdf, but tries to compile the main project: the compilation fails because banana-rdf classes are missing in the classpath.
sebastien#clemence-XPS-L412Z:rww-play (master *%)$ ./play.old/play
[info] Loading project definition from /home/sebastien/Bureau/rww-play/project
[warn] there were 1 deprecation warning(s); re-run with -deprecation for details
[warn] one warning found
[info] Loading project definition from /home/sebastien/Bureau/banana-rdf/project
[info] Updating {file:/home/sebastien/Bureau/banana-rdf/project/}banana-rdf-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 2 Scala sources to /home/sebastien/Bureau/banana-rdf/project/target/scala-2.10/sbt-0.13/classes...
[warn] there were 11 deprecation warning(s); re-run with -deprecation for details
[warn] there were 2 feature warning(s); re-run with -feature for details
[warn] two warnings found
[info] Set current project to RWWeb (in build file:/home/sebastien/Bureau/rww-play/)
_
_ __ | | __ _ _ _
| '_ \| |/ _' | || |
| __/|_|\____|\__ /
|_| |__/
play 2.2-TLS built with Scala 2.10.3-RC3 (running Java 1.7.0_45), http://www.playframework.com
> Type "help play" or "license" for more information.
> Type "exit" or use Ctrl+D to leave this console.
[RWWeb] $ compile
[info] Updating {file:/home/sebastien/Bureau/banana-rdf/}banana...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Updating {file:/home/sebastien/Bureau/rww-play/}RWWeb...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 53 Scala sources and 1 Java source to /home/sebastien/Bureau/rww-play/target/scala-2.10/classes...
[error] /home/sebastien/Bureau/rww-play/app/controllers/CORSProxy.scala:4: object banana is not a member of package org.w3
[error] import org.w3.banana.plantain.Plantain
[error] ^
[error] /home/sebastien/Bureau/rww-play/app/controllers/CORSProxy.scala:7: not found: type Plantain
[error] object CORSProxy extends org.www.readwriteweb.play.CORSProxy[Plantain](webClient)
.................
Isn't it supposed to compile banana-rdf before trying to compile my main project? If not, what is the point of depending on an external RootProject?
The RootProject(file("../banana-rdf")) project reference just references the root project, and what you really need is a reference to the banana-rdf subproject (in the rdf subdirectory).
The available subprojects are defined in https://github.com/w3c/banana-rdf/blob/master/project/build.scala. There are a few:
[main]> projects
[info] In file:/Users/jacek/sandbox/stackoverflow/19832655/
[info] * main
[info] In https://github.com/w3c/banana-rdf.git
[info] banana
[info] banana-jena
[info] banana-rdf
[info] banana-rdf-test-suite
[info] banana-sesame
[info] examples
[info] experimental
[info] ldp
[info] patch
To reference banana-rdf you should then use the following ProjectRef pointing at a right module (subproject) in a build configuration. Note ProjectRef as well as the name of the subproject - banana-rdf.
lazy val bananaRdfProject =
ProjectRef(uri("https://github.com/w3c/banana-rdf.git"), "banana-rdf")
With the ProjectRef you should be able to resolve any types defined in the banana-rdf subproject.
There is a difference between aggregate projects and dependsOn. I think you need to aggregate to get all projects to build together, dependsOn only makes sure the classes from that project is on the classpath, but it could of course be an old artifact if you haven't built it recently. Check out the sbt docs for more info: http://www.scala-sbt.org/0.12.3/docs/Getting-Started/Multi-Project.html
Related
Given a project structure:
rootProject
aggregatingSubProject
subA
subB
subC
How do I run the package task for aggregatingSubProject and all its aggregated projects, without specifying them by hand?
So far I can only have subA and subB built when doing package in the rootProject - regardless if I put .aggregate(subA, subB) in aggregatingProject's build.sbt.
I need the rootProject to define common settings for the build and I want to build multiple projects (some aggregating other projects, like aggregatingSubProject does) in a single build.
EDIT: I need to do this without specifying all the sub-sub-projects in the root build.sbt. I'd like to define subA and subB in aggregatingSubProject/build.sbt.
Using sbt 0.13.16.
Given this in build.sbt:
val rootProject = project in file(".")
val aggregatingSubProject = project
val subC = project
and this in aggregatingSubProject/build.sbt:
aggregateProjects(subA, subB)
val subA = project
val subB = project
You can run aggregatingSubProject/package and get:
> aggregatingSubProject/package
[info] Updating {file:/s/t-subaggregate/}subB...
[info] Updating {file:/s/t-subaggregate/}aggregatingSubProject...
[info] Updating {file:/s/t-subaggregate/}subA...
[info] Done updating.
[info] Packaging /s/t-subaggregate/aggregatingSubProject/subA/target/scala-2.12/suba_2.12-0.1-SNAPSHOT.jar ...
[info] Done updating.
[info] Done packaging.
[info] Packaging /s/t-subaggregate/aggregatingSubProject/subB/target/scala-2.12/subb_2.12-0.1-SNAPSHOT.jar ...
[info] Done packaging.
[info] Done updating.
[info] Packaging /s/t-subaggregate/aggregatingSubProject/target/scala-2.12/aggregatingsubproject_2.12-0.1-SNAPSHOT.jar ...
[info] Done packaging.
I'm integrating the sbt-release plugin to our projects, in order to delegate all the build+publish tasks to it.
It basically does all we need, but I'm adding an additional ReleaseStep to it: publishing a Docker image. This is my current releaseProcess (as per this):
releaseProcess := Seq[ReleaseStep](
checkSnapshotDependencies,
inquireVersions,
runTest,
setReleaseVersion,
commitReleaseVersion,
tagRelease,
//publishArtifacts,
releaseStepCommand("docker"),
setNextVersion,
commitNextVersion
//pushChanges,
)
(commented push-related stuff not to break anything upstream)
It's mostly fine and dandy, except the tag that the resulting Docker image gets.
For example, if the project was version 0.17.0-SNAPSHOT, and I wanted to release version 1.0.0, something like this would happen:
[develop] ✓ [17:54:22] lithium : ~/devel/some-company/some-app
➤ sbt clean "release skip-tests"
[info] Loading project definition from /opt/devel/some-company/some-app/project
[info] Updating {file:/opt/devel/some-company/some-app/project/}some-app-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Set current project to someapp (in build file:/opt/devel/some-company/some-app/)
[warn] Multiple resolvers having different access mechanism configured with same name 'local'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`).
[warn] Credentials file /var/jenkins_home/credentials/nexus_creds does not exist
[success] Total time: 0 s, completed May 18, 2017 5:54:56 PM
[info] Starting release process off commit: a7acde4d0e943be1e4befa6cc70dc873e164044a
[warn] Multiple resolvers having different access mechanism configured with same name 'local'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`).
[warn] Credentials file /var/jenkins_home/credentials/nexus_creds does not exist
[info] Updating {file:/opt/devel/some-company/some-app/}root...
[info] Done updating.
[warn] Found intransitive dependency (net.logstash.logback:logstash-logback-encoder:4.7) while publishMavenStyle is true, but Maven repositories
[warn] do not support intransitive dependencies. Use exclusions instead so transitive dependencies
[warn] will be correctly excluded in dependent projects.
[warn]
Release version [0.17.0] : 1.0.0
Next version [1.0.1-SNAPSHOT] :
[info] Setting version to '1.0.0'.
[info] Reapplying settings...
[info] Set current project to someapp (in build file:/opt/devel/some-company/some-app/)
[info] [develop db69b0a] Setting version to 1.0.0
[info] 1 file changed, 1 insertion(+), 1 deletion(-)
[info] Reapplying settings...
[info] Set current project to someapp (in build file:/opt/devel/some-company/some-app/)
[info] Packaging /opt/devel/some-company/some-app/target/scala-2.11/someapp_2.11-0.17.0-SNAPSHOT-sources.jar ...
[warn] Multiple resolvers having different access mechanism configured with same name 'local'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`).
[warn] Credentials file /var/jenkins_home/credentials/nexus_creds does not exist
[info] Done packaging.
[info] Wrote /opt/devel/some-company/some-app/target/scala-2.11/someapp_2.11-0.17.0-SNAPSHOT.pom
Warning: node.js detection failed, sbt will use the Rhino based Trireme JavaScript engine instead to run JavaScript assets compilation, which in some cases may be orders of magnitude slower than using node.js.
[info] Packaging /opt/devel/some-company/some-app/target/scala-2.11/someapp_2.11-0.17.0-SNAPSHOT-web-assets.jar ...
[info] Done packaging.
[info] Compiling 107 Scala sources and 36 Java sources to /opt/devel/some-company/some-app/target/scala-2.11/classes...
[info] Main Scala API documentation to /opt/devel/some-company/some-app/target/scala-2.11/api...
[warn] Class javax.annotation.CheckReturnValue not found - continuing with a stub.
[warn] /opt/devel/some-company/some-app/app/services/DOPXHandlerActor.scala:44: non-variable type argument services.DOPX in type pattern List[services.DOPX] (the underlying of List[services.DOPX]) is unchecked since it is eliminated by erasure
[warn] case reqs:List[DOPX] => withMDC { // PASAR A CASE CLASS
[warn] ^
[warn] /opt/devel/some-company/some-app/app/services/DOPXHandlerActor.scala:44: non-variable type argument services.DOPX in type pattern List[services.DOPX] (the underlying of List[services.DOPX]) is unchecked since it is eliminated by erasure
[warn] case reqs:List[DOPX] => withMDC { // PASAR A CASE CLASS
[warn] ^
[warn] /opt/devel/some-company/some-app/app/controllers/TransactionController.scala:79: match may not be exhaustive.
[warn] It would fail on the following input: (None, Success(_))
[warn] paymentConfirmationsService.confirm(siteId, chargeId, operationResource, request.headers.get("user")) map { _ match {
[warn] ^
[warn] there were 15 feature warnings; re-run with -feature for details
[warn] /opt/devel/some-company/some-app/app/services/DistributedTransactionProcessor.scala:139: match may not be exhaustive.
[warn] It would fail on the following input: Some(_)
[warn] failureResponse match {
[warn] ^
[warn] /opt/devel/some-company/some-app/app/services/DistributedTransactionProcessor.scala:312: match may not be exhaustive.
[warn] It would fail on the following inputs: (_, Failure(_)), (_, Success(_))
[warn] distributedOperationProcessor.processDistributedOPx(opDataFatherFixed.chargeId, Rechazada(), opDataFatherFixed, None, meanPayment, Some(refundSubPaymentOperations)).map( pr => pr match {
[warn] ^
model contains 337 documentable templates
[warn] /opt/devel/some-company/some-app/app/services/refunds/RefundService.scala:346: match may not be exhaustive.
[warn] It would fail on the following input: None
[warn] operation.sub_transactions.find { subTx => subTx.subpayment_id.get == subpaymentId } match {
[warn] ^
[warn] /opt/devel/some-company/some-app/app/services/ApplicationTimer.scala:9: Could not find any member to link for "ApplicationLifecycle".
[warn] /**
[warn] ^
[warn] three warnings found
[info] Main Scala API documentation successful.
[info] Packaging /opt/devel/some-company/some-app/target/scala-2.11/someapp_2.11-0.17.0-SNAPSHOT-javadoc.jar ...
[info] Done packaging.
[warn] there were 15 feature warnings; re-run with -feature for details
[warn] 30 warnings found
[warn] bootstrap class path not set in conjunction with -source 1.6
[info] /opt/devel/some-company/some-app/app/legacy/some-company/sps/domain/DBParametros.java: Some input files use unchecked or unsafe operations.
[info] /opt/devel/some-company/some-app/app/legacy/some-company/sps/domain/DBParametros.java: Recompile with -Xlint:unchecked for details.
[info] Packaging /opt/devel/some-company/some-app/target/scala-2.11/someapp_2.11-0.17.0-SNAPSHOT.jar ...
[info] Done packaging.
[info] Packaging /opt/devel/some-company/some-app/target/scala-2.11/someapp_2.11-0.17.0-SNAPSHOT-sans-externalized.jar ...
[info] Done packaging.
[info] Sending build context to Docker daemon 108.9MB
[info] Step 1/8 : FROM lapp-dvde004:5000/java-alpine:latest
[info] ---> fd94b5262b7b
[info] Step 2/8 : MAINTAINER Redbee
[info] ---> Using cache
[info] ---> 2905e1a8a792
[info] Step 3/8 : WORKDIR /opt/docker
[info] ---> Using cache
[info] ---> 0b2f169737f6
[info] Step 4/8 : ADD opt/ /opt
[info] ---> a973fb66a793
[info] Removing intermediate container f6da68f8a1c3
[info] Step 5/8 : ADD opt/docker/conf/jce_policy-8.tar.gz /usr/lib/jvm/default-jvm/jre/lib/security/
[info] ---> 3fda5972ddd5
[info] Removing intermediate container cb10f1fcf2dd
[info] Step 6/8 : RUN chown -R daemon:daemon .
[info] ---> Running in 867093c046e0
[info] ---> 60a969ff427c
[info] Removing intermediate container 867093c046e0
[info] Step 7/8 : USER daemon
[info] ---> Running in a7fa422e326c
[info] ---> 8857a5a5392b
[info] Removing intermediate container a7fa422e326c
[info] Step 8/8 : ENTRYPOINT /opt/docker/conf/wrapper.sh
[info] ---> Running in 31887383f984
[info] ---> 2418dba2d69e
[info] Removing intermediate container 31887383f984
[info] Successfully built 2418dba2d69e
[info] Successfully tagged lapp-dvde004:5000/someapp:0.17.0-SNAPSHOT
[info] Built image lapp-dvde004:5000/someapp:0.17.0-SNAPSHOT
[info] Update Latest from image lapp-dvde004:5000/someapp:0.17.0-SNAPSHOT
[success] Total time: 58 s, completed May 18, 2017 5:56:07 PM
[info] Setting version to '1.0.1-SNAPSHOT'.
[info] Reapplying settings...
[info] Set current project to someapp (in build file:/opt/devel/some-company/some-app/)
[info] [develop a1dd63d] Setting version to 1.0.1-SNAPSHOT
[info] 1 file changed, 1 insertion(+), 1 deletion(-)
Sadly I cannot highlight code, but it says:
[info] Setting version to '1.0.0'.
[info] Reapplying settings...
And then the built image is:
[info] Built image lapp-dvde004:5000/someapp:0.17.0-SNAPSHOT
In my limited understanding, it seems like sbt-release and sbt-docker (or sbt-native-packager, I tried with both) do not share the same build context? Something along those lines?
sbt-release successfully changes the version.sbt file before sbt-docker kicks in, but still the latter pays no mind to it.
Any ideas?
PS: Funfact: This question apparently had the answer (or at least his requirements matched mine), so, pandaforme, if you read this, why did you delete that gist!? :(
Thanks to the amazing pandaforme who re-uploaded the aforementioned gist, it's now working.
So, I implemented two things: A 'release' process which also builds a Docker image, and also a custom task 'publishSnapshot' which is sort of an extension to 'publish' (publishes the snapshot and it's docker image; also known as "overengineering when I could just do sbt publish dockerBuildAndPush").
Implementation, if anyone needs it:
val publishDocker = ReleaseStep(action = st => {
val extr: Extracted = Project.extract(st)
val ref: ProjectRef = extr.get(thisProjectRef)
extr.runAggregated(
sbtdocker.DockerKeys.dockerBuildAndPush in sbtdocker.DockerPlugin.autoImport.docker in ref,
st
)
st
})
lazy val publishSnapshot = taskKey[Unit]("Publish a Snapshot and it's Docker image")
publishSnapshot in Compile := Def.sequential(
publish in Compile,
sbtdocker.DockerKeys.dockerBuildAndPush in sbtdocker.DockerPlugin.autoImport.docker
).value
releaseProcess := Seq[ReleaseStep](
checkSnapshotDependencies,
inquireVersions,
runTest,
setReleaseVersion,
commitReleaseVersion,
tagRelease,
publishArtifacts,
publishDocker,
setNextVersion,
commitNextVersion,
pushChanges
)
We have a project with several subprojects which can compile under both Scala 2.10 and 2.11, one subproject which only compiles under 2.10 (actually, Scala-Virtualized 2.10.2) and one subproject which only compiles under 2.11. Is there a simple way to create an aggregate project which cross-builds all possible subprojects for both 2.10 and 2.11? Or, alternately, to have different default projects for 2.10 and 2.11?
In particular, here is current Build.scala. If I add lmsBackend to root, I get
> show scalaVersion
[info] common/*:scalaVersion
[info] 2.10.4
[info] lms-backend/*:scalaVersion
[info] 2.10.2
[info] meta/*:scalaVersion
[info] 2.10.4
[info] community-edition/*:scalaVersion
[info] 2.10.4
[info] core/*:scalaVersion
[info] 2.10.4
[info] scalan/*:scalaVersion
[info] 2.10.4
> show crossScalaVersions
[info] common/*:crossScalaVersions
[info] List(2.10.4, 2.11.5)
[info] lms-backend/*:crossScalaVersions
[info] List(2.10.2)
[info] meta/*:crossScalaVersions
[info] List(2.10.4, 2.11.5)
[info] community-edition/*:crossScalaVersions
[info] List(2.10.4, 2.11.5)
[info] core/*:crossScalaVersions
[info] List(2.10.4, 2.11.5)
[info] scalan/*:crossScalaVersions
[info] List(2.10.4, 2.11.5)
SBT is able to run update, compile, etc. fine on this aggregate project. However, once I try any cross-building, things break:
> +update
[info] Setting version to 2.10.4
[info] Reapplying settings...
[info] Set current project to scalan (in build file:/home/aromanov/IdeaProjects/scalan-lite/)
...
[info] Updating {file:/home/aromanov/IdeaProjects/scalan-lite/}lms-backend...
[info] Resolving org.scala-lang.virtualized#scala-library;2.10.4 ...
[warn] module not found: org.scala-lang.virtualized#scala-library;2.10.4
...
[info] Resolving org.scala-lang.virtualized#scala-compiler;2.10.4 ...
[warn] module not found: org.scala-lang.virtualized#scala-compiler;2.10.4
...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.scala-lang.virtualized#scala-library;2.10.4: not found
[warn] :: org.scala-lang.virtualized#scala-compiler;2.10.4: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.scala-lang.virtualized:scala-library:2.10.4 ((sbt.Classpaths) Defaults.scala#L1169)
[warn] +- com.huawei.scalan:lms-backend_2.10:0.2.6-SNAPSHOT
[warn] org.scala-lang.virtualized:scala-compiler:2.10.4
[warn] +- com.huawei.scalan:lms-backend_2.10:0.2.6-SNAPSHOT
[trace] Stack trace suppressed: run last lms-backend/*:update for the full output.
[error] (lms-backend/*:update) sbt.ResolveException: unresolved dependency: org.scala-lang.virtualized#scala-library;2.10.4: not found
[error] unresolved dependency: org.scala-lang.virtualized#scala-compiler;2.10.4: not found
[error] Total time: 2 s, completed Jan 28, 2015 1:33:49 PM
show scalaVersion now shows 2.10.4 for all subprojects. Is there any way to include lms-backend in the aggregate project and still avoid this problem?
I had a similar question. My question had an additional complication of classpath dependencies (dependsOn), rather than just aggregation, but one solution for my problem fixes this one quite well.
sbt-doge, a deceptively cutesy GitHub project, replaces the implementation of + with one of its interchangeable prefixes: much, so, such, very.
Current implementation of + cross building operator does not take in account for the crossScalaVersions of the sub projects. Until that's fixed, here's an alternative implementation of it.
The creator of the project is one of the main contributors to SBT.
Add
addSbtPlugin("com.eed3si9n" % "sbt-doge" % "0.1.3")
to project/plugins.sbt.
Then
> very compile
I believe sbt-doge might not be enough for your requirements as mine were not met by this plugin as well.
I would recommend sbt-cross
This plugin allows you for much more flexibility among your modules, as well as, it provides a way to aggregate modules of different Scala versions, without the need to have any version in common!
Example:
lazy val common = (project in file("common")).cross
lazy val common_2_11 = common("2.11.8")
lazy val common_2_10 = common("2.10.6")
lazy val A = (project in file("A"))
.settings(scalaVersion := "2.10.6")
.dependsOn(common_2_10)
lazy val B = (project in file("B"))
.settings(scalaVersion := "2.11.8")
.dependsOn(common_2_11)
lazy val root = (project in file("."))
.aggregate(common, A, B)
I have already posted a similar answer.
Enjoy!
I am trying to figure out how to create an build.sbt file for my own Scalding-based project.
Scalding source structure has no build.sbt file. Instead it has project/Build.scala build definition.
What would be the right way to integrate my own sbt project with Scalding, so I could also import it later in Eclipse with sbt-eclipse plugin?
Update:
For the following code:
import cascading.tuple.Fields
import com.twitter.scalding._
class Scan(args: Args) extends Job(args) {
val output = TextLine("tmp/out.txt")
val wordsList = List(
("john"),
("liza"),
("nina"),
("x"))
val orderedPipe =
IterableSource[(String)](wordsList, ('word))
.debug
.write(output)
}
With this build.sbt:
name := "Scan"
version := "1.0"
libraryDependencies := Seq("com.twitter" %% "scalding" % "0.11.1")
I get errors:
$ sbt
[info] Loading global plugins from /home/test/.sbt/0.13/plugins
[info] Set current project to Scan (in build file:/home/test/Cascading/Scala/Scan/)
> compile
[info] Updating {file:/home/test/Cascading/Scala/Scan/}scan...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading http://repo1.maven.org/maven2/com/twitter/scalding_2.10/0.11.1/scalding_2.10-0.11.1.jar ...
[info] [SUCCESSFUL ] com.twitter#scalding_2.10;0.11.1!scalding_2.10.jar (641ms)
[info] Done updating.
[info] Compiling 1 Scala source to /home/test/Cascading/Scala/Scan/target/scala-2.10/classes...
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:1: not found: object cascading
[error] import cascading.tuple.Fields
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:2: object twitter is not a member of package com
[error] import com.twitter.scalding._
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:5: not found: type Job
[error] class Scan(args: Args) extends Job(args) {
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:5: not found: type Args
[error] class Scan(args: Args) extends Job(args) {
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:5: too many arguments for constructor Object: ()Object
[error] class Scan(args: Args) extends Job(args) {
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:6: not found: value TextLine
[error] val output = TextLine("tmp/out.txt")
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:15: not found: value IterableSource
[error] IterableSource[(String)](wordsList, ('word))
[error] ^
[error] 7 errors found
[error] (compile:compile) Compilation failed
Update 2
After doing git clone git#github.com:twitter/scalding.git their repository and sbt publishLocal I still have the same compilation errors.
BUT adding two lines that you suggested to build.sbt allowed me to compile my code. So the following build.sbt really works, thanks!
name := "BlockScan"
version := "1.0"
libraryDependencies := Seq("com.twitter" %% "scalding" % "0.11.1")
lazy val scaldingCore = ProjectRef(uri("https://github.com/twitter/scalding.git"), "scalding-core")
lazy val myProject = project in file(".") dependsOn scaldingCore
'sbt eclipse' creates Eclipse project wich does not compile under Eclipse and reports these errors:
Project 'Scan' is missing required Java project: 'scalding-core'
More than one scala library found in the build path (/home/test/usr/eclipse-scala-3.0.3/configuration/org.eclipse.osgi/bundles/290/1/.cp/lib/scala-library.jar, /home/test/wks/Cascading/Scala/scalding/target/scala-2.9.3/scalding-assembly-0.10.0.jar).At least one has an incompatible version. Please update the project build path so it contains only compatible scala libraries.
scalacheck_2.9.3-1.10.0.jar is cross-compiled with an incompatible version of Scala (2.9.3).
specs_2.9.3-1.6.9.jar is cross-compiled with an incompatible version of Scala (2.9.3).
Since they don't seem to publish their libraries to remote repositories where you could pull down the necessary dependencies, you'll have to declare the source dependency on the GitHub repository for the project.
lazy val scaldingCore = ProjectRef(uri("https://github.com/twitter/scalding.git"), "scalding-core")
lazy val myProject = project in file(".") dependsOn scaldingCore
With the above build definition, sbt will git clone the RootProject and load the build.
➜ scalding xsbt
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
Cloning into '/Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding'...
[info] Loading project definition from /Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding/project
[info] Updating {file:/Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding/project/}scalding-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.eed3si9n/sbt-assembly/scala_2.10/sbt_0.13/0.10.2/jars/sbt-assembly.jar ...
[info] [SUCCESSFUL ] com.eed3si9n#sbt-assembly;0.10.2!sbt-assembly.jar (3600ms)
[info] Done updating.
[info] Compiling 3 Scala sources to /Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding/project/target/scala-2.10/sbt-0.13/classes...
[warn] there were 8 deprecation warning(s); re-run with -deprecation for details
[warn] there were 2 feature warning(s); re-run with -feature for details
[warn] two warnings found
[info] Set current project to myProject (in build file:/Users/jacek/sandbox/scalding/)
> projects
[info] In file:/Users/jacek/sandbox/scalding/
[info] * myProject
[info] In https://github.com/twitter/scalding.git
[info] maple
[info] scalding
[info] scalding-args
[info] scalding-avro
[info] scalding-commons
[info] scalding-core
[info] scalding-date
[info] scalding-hadoop-test
[info] scalding-jdbc
[info] scalding-json
[info] scalding-parquet
[info] scalding-repl
The build setup should give you access to scalding classes.
> console
[info] Starting scala interpreter...
[info]
Welcome to Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_60).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import com.twitter.scalding._
import com.twitter.scalding._
And the Scan class compiles fine - it's in src/main/scala directory.
> show sources
[info] ArrayBuffer(/Users/jacek/sandbox/scalding/src/main/scala/Scan.scala)
[success] Total time: 0 s, completed Jul 15, 2014 12:21:14 AM
> compile
[info] Updating {file:/Users/jacek/sandbox/scalding/}myProject...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/jacek/sandbox/scalding/target/scala-2.10/classes...
[success] Total time: 4 s, completed Jul 15, 2014 12:21:20 AM
You could also git clone git#github.com:twitter/scalding.git their repository and sbt publishLocal to be able to declare binary dependency in build.sbt as follows:
libraryDependencies := Seq("com.twitter" %% "scalding" % "0.11.1")
With the dependency in (either way), execute sbt eclipse and be done with it!
My build is simple:
lazy val stampleWebProject = play.Project("stample-web", appVersion, appDependencies,path = file("stample-web"))
.dependsOn(stampleCoreProject,stampleSearchProject)
.aggregate(stampleCoreProject,stampleSearchProject)
lazy val stampleCoreProject = Project(id = "stample-core",base = file("stample-core"))
lazy val stampleSearchProject = Project(id = "stample-search",base = file("stample-search"))
All these projects have a build.sbt file with dependencies, without any scala build (which would be ignored as far as I know)
When I start SBT (12.4), I get the following:
[info] Set current project to stample-core (in build file:/home/sebastien/Bureau/Stample/)
> projects
[info] In file:/home/sebastien/Bureau/Stample/
[info] * stample-core
[info] stample-search
[info] stample-web
> project stample-search
[info] Set current project to stample-search (in build file:/home/sebastien/Bureau/Stample/)
> projects
[info] In file:/home/sebastien/Bureau/Stample/
[info] stample-core
[info] * stample-search
[info] stample-web
> project stample-core
[info] Set current project to stample-core (in build file:/home/sebastien/Bureau/Stample/)
> projects
[info] In file:/home/sebastien/Bureau/Stample/
[info] * stample-core
[info] stample-search
[info] stample-web
> project stample-web
[info] Set current project to stample-search (in build file:/home/sebastien/Bureau/Stample/)
[stample-search] $ projects
[info] In file:/home/sebastien/Bureau/Stample/
[info] stample-core
[info] stample-search
[info] * stample-web
[stample-search] $ compile
[info] Updating {file:/home/sebastien/Bureau/Stample/}stample-core...
[info] Resolving org.slf4j#slf4j-api;1.6.6 ...
[info] Done updating.
[info] Updating {file:/home/sebastien/Bureau/Stample/}stample-web...
[error] a module is not authorized to depend on itself: stample-search#stample-search_2.10;1.0
[error] (stample-web/*:update) java.lang.IllegalArgumentException: a module is not authorized to depend on itself: stample-search#stample-search_2.10;1.0
[error] Total time: 1 s, completed 26 août 2013 21:57:45
I do not understand some stuff here:
How is choosen the project in which we are by default. I've seem documentation was added in SBT 13.0 but did not see it in the 12.4 multibuild documentation.
How comes I type project stample-web and it tells me I'm in stample-search
Why is there a special display in my sbt console for the project I'm in (stample-web or stample-search, I don't really know...) (this appears here: [stample-search] $ compile, is this relative to play projects?
Why it can't compile stample-search, since it doesn't depend on itself in my build (I suspect it tries to compile the web project but there's a naming problem or something?
Is this an SBT bug. If so, is it possible to use the new 13.0 version with Play framework?
Eugene Yokota is right: I have a conflict in the stample-web folder, which was set the stample-web name in the scala build, but it has a wrong build.sbt in the folder which has name := stample-search