Difference between scalaSource and sourceDirectories in sbt - scala

What is the difference between scalaSource and sourceDirectories? I have a non-standard directory structure where source code lives in src/,
This line works in build.sbt:
scalaSource in Compile := file("src/")
but not
sourceDirectories in Compile := Seq(file("src/"))

Sources come in many kinds: Scala and Java, managed and unmanaged. sourceDirectories in Compile combines all of them, but you wouldn't normally want to set it directly; normally you'd set the more specific setting that applies to the particular kind of source you're trying to tell sbt the location of.
Note that baseDirectory.value / "src" would be more correct than file("src") (it works in more scenarios: subprojects, external projects, etc).

I can't tell you the whole picture, someone who worked on sbt may see this post later.
Till then, here is how I reason about this things: inspect
> inspect actual compile:scalaSource
[info] Description:
[info] Default Scala source directory.
[info] Dependencies:
[info] compile:sourceDirectory
[info] Reverse dependencies:
[info] compile:unmanagedSourceDirectories
[info] Delegates:
[info] compile:scalaSource
> inspect actual compile:sourceDirectories
[info] Description:
[info] List of all source directories, both managed and unmanaged.
[info] Dependencies:
[info] compile:unmanagedSourceDirectories
[info] compile:managedSourceDirectories
[info] Delegates:
[info] compile:sourceDirectories
[info] *:sourceDirectories
[info] {.}/compile:sourceDirectories
Now this is how I interpret this:
sourceDirectories are ... well ... completely informal ...
Lets see how this is relevant to something like compile:
> inspect compile
[info] Dependencies:
[info] compile:compile::compileInputs <----
[info] compile:compile::streams
> inspect compile:compile::compileInputs
[info] Dependencies:
...
[info] compile:compile::sources <----
...
> inspect compile:compile::sources
[info] Provided by:
[info] {file:<build-uri>}<project-id>/compile:sources
[info] Delegates:
[info] compile:compile::sources
[info] compile:sources <----
This task is delegated, we can see where we came from in Reverse dependencies with inspect actual, regular inspect woudn't show them.
> inspect actual compile:sources
[info] Description:
[info] All sources, both managed and unmanaged.
[info] Reverse dependencies:
[info] compile:doc
[info] compile:compile::compileInputs
[info] Dependencies:
[info] compile:unmanagedSources <----
[info] compile:managedSources
> inspect compile:unmanagedSources
[info] Dependencies:
[info] compile:unmanagedSourceDirectories
...
> inspect compile:unmanagedSourceDirectories
[info] Dependencies:
[info] compile:javaSource
[info] compile:scalaSource <----
[info] Reverse dependencies:
[info] compile:sourceDirectories
[info] compile:unmanagedSources
Hope this helps.

Related

sbt doesn't find class of managed dependency

I try to include the build of jep (https://mvnrepository.com/artifact/jep/jep/2.24) into this scala project using sbt: https://github.com/shadaj/scalapy
So, instead of building jep manually via an unmanaged dependency, I want to include it as a managed dependency. Therefore I just included:
resolvers += "jep" at "https://mvnrepository.com/artifact/"
libraryDependencies += "jep" % "jep" % "2.24"
in the build.sbt
The .jar is getting downloaded and included in my .ivy2 folder but when compiling, sbt seems not to find it:
> compile
[info] Updating {file:/C:/scalapy_indp/scalapy/}scalapy...
[info] Resolving org.sonatype.oss#oss-parent;9 ...
[info] downloading https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.12.1/scala-library-2.12.1.jar ...
[info] [SUCCESSFUL ] org.scala-lang#scala-library;2.12.1!scala-library.jar (1406ms)
[info] downloading https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.1/scala-reflect-2.12.1.jar ...
[info] [SUCCESSFUL ] org.scala-lang#scala-reflect;2.12.1!scala-reflect.jar (1156ms)
[info] downloading https://repo1.maven.org/maven2/jep/jep/2.24/jep-2.24.jar ...
[info] [SUCCESSFUL ] jep#jep;2.24!jep.jar (188ms)
[info] downloading https://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.12.1/scala-compiler-2.12.1.jar ...
[info] [SUCCESSFUL ] org.scala-lang#scala-compiler;2.12.1!scala-compiler.jar (2469ms)
[info] downloading https://repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.12/1.0.6/scala-xml_2.12-1.0.6.jar ...
[info] [SUCCESSFUL ] org.scala-lang.modules#scala-xml_2.12;1.0.6!scala-xml_2.12.jar(bundle) (360ms)
[info] downloading https://repo1.maven.org/maven2/jline/jline/2.14.1/jline-2.14.1.jar ...
[info] [SUCCESSFUL ] jline#jline;2.14.1!jline.jar (250ms)
[info] Done updating.
[info] Compiling 16 Scala sources to C:\scalapy_indp\scalapy\target\scala-2.12\classes...
[info] 'compiler-interface' not yet compiled for Scala 2.12.1. Compiling...
[info] Compilation completed in 15.285 s
[error] C:\scalapy_indp\scalapy\src\gen\scala\me\shadaj\scalapy\py\ObjectTupleReaders.scala:2: not found: object jep
[error] import jep.Jep
[error] ^
The errors of course keep coming for every Jep appearance in the code.
I'm absolutely not familiar with scala or sbt, I'm just trying to get this running without having to use unmanaged dependencies i.e. having it platform independent. Building jep manually seems to be platform dependent regarding the files which are being create:
If the build succeeds it will create a directory jep/build which will
contain a jep.jar and the compiled C library of Jep, typically named
jep.so or jep.dll depending on your platform.
see https://github.com/mrj0/jep/wiki/Getting-Started
The dependency is correctly setup, but the import is wrong.
Try
import org.nfunk.jep.JEP
instead.
--
Actually, I just realized those may be two different projects. The dependencies you included is this one: http://sens.cse.msu.edu/Software/jep-2.23/doc/website/doc/doc_usage.htm

Where is the default value of the organizationName is coming from in sbt

sbt defines the organizationName setting as SettingKey[String] (see Keys). When starting an sbt build in an empty directory, one can inspect the default value of organizationName:
> inspect organizationName
[info] Setting: java.lang.String = default
[info] Description:
[info] Organization full/formal name.
[info] Provided by:
[info] {file:/Users/bene/workspace/sbt/test/}test/*:organizationName
[info] Defined at:
[info] (sbt.Classpaths) Defaults.scala:1174
[info] Dependencies:
[info] *:organization
[info] Reverse dependencies:
[info] *:projectInfo
[info] Delegates:
[info] *:organizationName
[info] {.}/*:organizationName
[info] */*:organizationName
So the organizationName setting defaults to "default". Now the question is, where is the default value coming from?
Although it may not be completely obvious, all the information is already there. The sbt output gives us a first hint:
[info] Defined at:
[info] (sbt.Classpaths) Defaults.scala:1174
In line 1174 in Default.scala we can see, that organizationName falls back to the value of organization. Inspecting the organization setting gives us the next hint:
> inspect organization
[info] Setting: java.lang.String = default
[info] Description:
[info] Organization/group ID.
[info] Provided by:
[info] {file:/Users/bene/workspace/sbt/test/}test/*:organization
[info] Defined at:
[info] (sbt.Classpaths) Defaults.scala:1173
[info] (sbt.Build) Build.scala:58
[info] Dependencies:
[info] *:normalizedName
[info] *:thisProject
[info] Reverse dependencies:
[info] *:organizationName
[info] *:projectId
[info] Delegates:
[info] *:organization
[info] {.}/*:organization
[info] */*:organization
Finally, in Build.scala:58 we can find the mapping from a missing organization to "default".

IntelliJ No Source to compile with Maven Error

I have created a maven project in intellij and added scala framework support
This project name is sparkexamples and I added the necessary dependencies
Below is the project directory structure .
S:\surender\intellij_1\sparkexamples
S:\surender\intellij_1\sparkexamples\src\main\scala\com\test\spark\examples\DemoMain.scala
S:\surender\intellij_1\sparkexamples\pom.xml
I went inside S:\surender\intellij_1\sparkexamples from command prompt and started running the below mvn commands
mvn clean compile
mvn package
After running mvn package I could see in cons
S:\surender\intellij_1\sparkexamples>mvn clean compile
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building spark-examples 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) # spark-examples ---
[INFO] Deleting S:\surender\intellij_1\sparkexamples\target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # spark-exam
ples ---
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) # spark- examples
---
[INFO] No sources to compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.950 s
[INFO] Finished at: 2017-03-28T10:46:18+01:00
[INFO] Final Memory: 13M/224M
[INFO] ------------------------------------------------------------------------
S:\surender\intellij_1\sparkexamples>mvn package
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building spark-examples 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # spark-exam
ples ---
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) # spark-examples
---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) # sp
ark-examples ---
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory S:\surender\intellij_1\sparkexamples\
src\test\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) # spark-e
xamples ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) # spark- examples ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) # spark-examples ---
[INFO] Building jar: S:\surender\intellij_1\sparkexamples\target\spark-examples-
1.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.651 s
[INFO] Finished at: 2017-03-28T10:47:09+01:00
[INFO] Final Memory: 15M/225M
[INFO] ------------------------------------------------------------------------
S:\surender\intellij_1\sparkexamples>ole that **No Sources to compile**
Could Someone help me to resolve this issue?
The Maven compiler is "incremental" by default. (That means that just the compile goal is "incremental", Maven still makes you go through the whole lifecycle, by default.)
When you first did:
mvn clean compile
Your sources got compiled. (Lifecycles: you ran up to and including the compile lifecycle phase.)
When you then did:
mvn package
The compiler was again invoked, and observed that it had already done its work previously. (Lifecycles: package is a lifecycle phase extending the compile phase.)
This is what seems to happen. Disclaimer: you only included output for mvn package. If your mvn compile also tells you that there's nothing to compile, something else going on, and you'd probably want to rephrase your question (or post a new one).
Also, if this lifecycle thing is messing you up, one more hint: compile is a lifecycle phase, and many people confuse it with doing a particular "task", which it is not. From mvn -h:
usage: mvn [options] [<goal(s)>] [<phase(s)>]
Again, compile is a phase, if you wanted to do a goal ("task"), in case of doing just compilation that'd be: compiler:compile (name_of_the_plugin:name_of_the_goal).

(gcloud.preview.app.deploy) Error Response: [400] "env" setting is not supported for this deployment

I try these tutorial https://cloud.google.com/tools/cloud-repositories/docs/push-to-deploy, and I do mvn gcloud:deploy,but got the error messages bellow:
[dev-jenkins-test-1] $ /bin/sh -xe /tmp/hudson4310631253025446569.sh
+ mvn gcloud:deploy
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building jenkins-test-java 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] >>> gcloud-maven-plugin:2.0.9.88.v20151125:deploy (default-cli) > package # jenkins-test-java >>>
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # jenkins-test-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /var/jenkins/workspace/dev-jenkins-test-1/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) # jenkins-test-java ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) # jenkins-test-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /var/jenkins/workspace/dev-jenkins-test-1/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) # jenkins-test-java ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) # jenkins-test-java ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) # jenkins-test-java ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO]
[INFO] <<< gcloud-maven-plugin:2.0.9.88.v20151125:deploy (default-cli) < package # jenkins-test-java <<<
[INFO]
[INFO] --- gcloud-maven-plugin:2.0.9.88.v20151125:deploy (default-cli) # jenkins-test-java ---
[INFO] Running gcloud app deploy...
[INFO] Running python -S /google/google-cloud-sdk/lib/googlecloudsdk/gcloud/gcloud.py --quiet --project=straw-hat-pirates-dev preview app deploy /var/jenkins/workspace/dev-jenkins-test-1/target/appengine-staging/app.yaml
[INFO] WARNING: Soon, deployments will set the deployed version to receive all traffic by
[INFO] default.
[INFO]
[INFO] To keep the current behavior (where new deployments do not receive any traffic),
[INFO] use the `--no-promote` flag or run the following command:
[INFO]
[INFO] $ gcloud config set app/promote_by_default false
[INFO]
[INFO] To adopt the new behavior early, use the `--promote` flag or run the following
[INFO] command:
[INFO]
[INFO] $ gcloud config set app/promote_by_default true
[INFO]
[INFO] Either passing one of the new flags or setting one of these properties will
[INFO] silence this message.
[INFO]
[INFO] You are about to deploy the following modules:
[INFO] - straw-hat-pirates-dev/default (from [/var/jenkins/workspace/dev-jenkins-test-1/target/appengine-staging/app.yaml])
[INFO] Deployed URL: [https://20151223t054356-dot-straw-hat-pirates-dev.appspot.com]
[INFO] (add --promote if you also want to make this module available from
[INFO] [https://straw-hat-pirates-dev.appspot.com])
[INFO]
[INFO] Beginning deployment...
[INFO] Verifying that Managed VMs are enabled and ready.
[INFO]
[INFO] Provisioning remote build service.
[INFO] Copying certificates for secure access. You may be prompted to create an SSH keypair.
[INFO] Building and pushing image for module [default]
[INFO] ----------------------------- DOCKER BUILD OUTPUT ------------------------------
[INFO] Step 0 : FROM gcr.io/google_appengine/openjdk8
[INFO] ---> 3c058a018ce1
[INFO] Step 1 : ADD jenkins-test-java-1.0-SNAPSHOT.jar /app/
[INFO] ---> 938437aaa1a1
[INFO] Removing intermediate container 4106bc3645ca
[INFO] Step 2 : ENTRYPOINT java -jar /app/jenkins-test-java-1.0-SNAPSHOT.jar
[INFO] ---> Running in f187a76322dd
[INFO] ---> 4600e0f4d83a
[INFO] Removing intermediate container f187a76322dd
[INFO] Successfully built 4600e0f4d83a
[INFO] --------------------------------------------------------------------------------
[INFO]
[INFO] Updating module [default]...
[INFO] Updating module [default].../
[INFO] Updating module [default]...failed.
[INFO] ERROR: (gcloud.preview.app.deploy) Error Response: [400] "env" setting is not supported for this deployment.
[INFO] Deleted [https://www.googleapis.com/compute/v1/projects/straw-hat-pirates-dev/zones/us-central1-f/instances/gae-builder-vm-20151223t054356].
[ERROR] Error: gcloud app command with exit code : 1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:03 min
[INFO] Finished at: 2015-12-23T05:45:57+00:00
[INFO] Final Memory: 11M/56M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.appengine:gcloud-maven-plugin:2.0.9.88.v20151125:deploy (default-cli) on project jenkins-test-java: Error: gcloud app command exit code is: 1 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Build step 'Execute shell' marked build as failure
Finished: FAILURE
I guess :
It seems like something wrong with my environment configuration.
I still cannot fix this problem.
Thank for your help. :)
I figured it out.
I'll bet your maven module is not setup with <packaging>war</packaging>
If your module is set to jar (default), then it tried to deploy as a standard java application where you are going to launch your own server. Since you want a managed vm with appengine running, you need to package as war, and then the plugin will take care of the rest.
Cheers.

Why is crossScalaVersion not scalaVersion?

build.sbt
scalaVersion := "2.11.4"
project/build.properties
sbt.version=0.13.7
Then
> show scalaVersion
[info] 2.11.4
> show crossScalaVersions
[info] List(2.10.4)
> inspect crossScalaVersions
[info] Setting: scala.collection.Seq[java.lang.String] = List(2.10.4)
[info] Description:
[info] The versions of Scala used when cross-building.
[info] Provided by:
[info] */*:crossScalaVersions
[info] Defined at:
[info] (sbt.Defaults) Defaults.scala:237
[info] Delegates:
[info] *:crossScalaVersions
[info] {.}/*:crossScalaVersions
[info] */*:crossScalaVersions
[info] Related:
[info] */*:crossScalaVersions
It seems like crossScalaVersions should be List(2.11.4).
Look at SBT's source code, that's also what I would think.
crossScalaVersions := Seq(scalaVersion.value)
Why doesn't crossScalaVersions correspond to scalaVersion?
scalaVersion.value is context-dependent. So in Defaults.scala it's */*:scalaVersion from appConfiguration.value.provider.scalaProvider. It's the version used to compile your project's definition, including your build.sbt file (sbt 0.13.7 uses 2.10.4 scala-compiler to compile your project definitions). And that the only way as your project's definitions (including scalaVersion) isn't compiled yet when Defaults executed and crossScalaVersions defined. So, */*:crossScalaVersions depends on */*:scalaVersion not proj/*:scalaVersion.
Just compare Provided by with explicit scalaVersion := 2.11.4 inside build.sbt:
> inspect scalaVersion
[info] Setting: java.lang.String = 2.11.4
[info] Description:
[info] The version of Scala used for building.
[info] Provided by:
[info] {file:/Users/user/dev/proj/}proj/*:scalaVersion
[info] Defined at:
[info] /Users/user/dev/proj/build.sbt:1
[info] Reverse dependencies (D=derives):
[info] *:allDependencies
[info] D *:scalaBinaryVersion
[info] *:libraryDependencies
[info] *:scalaInstance
[info] *:crossScalaVersions
[info] *:update
[info] Delegates:
[info] *:scalaVersion
[info] {.}/*:scalaVersion
[info] */*:scalaVersion
[info] Related:
[info] */*:scalaVersion
And without one (just empty project):
> inspect scalaVersion
[info] Setting: java.lang.String = 2.10.4
[info] Description:
[info] The version of Scala used for building.
[info] Provided by:
[info] */*:scalaVersion
[info] Defined at:
[info] (sbt.Defaults) Defaults.scala:232
[info] Reverse dependencies:
[info] *:allDependencies
[info] *:libraryDependencies
[info] *:update
[info] *:scalaInstance
[info] Delegates:
[info] *:scalaVersion
[info] {.}/*:scalaVersion
[info] */*:scalaVersion
[info] Related:
[info] */*:scalaVersion
So, you just need to redefine */*:scalaVersion in your build.sbt:
scalaVersion in GlobalScope := "2.11.2"