SBT failing to resolve dependency [ResolveException] - scala

I have the following in my build.sbt file:
lazy val root = (project in file(".")).
settings(
inThisBuild(List(
scalaVersion := "2.13.4",
)),
name := "ScalaTest",
resolvers += "spigot-repo" at "https://hub.spigotmc.org/nexus/content/repositories/snapshots/",
libraryDependencies += "org.spigotmc" % "spigot-api" % "1.16.5-R0.1-SNAPSHOT" % "provided" intransitive(),
version := "1.0"
)
Which, when using IntelliJ and trying to refresh / reimport the project, a few issues occur:
Dependencies won't resolve despite working with maven / Gradle.
Produces the following error
/Users/harrydrummond/Library/Java/JavaVirtualMachines/openjdk-15.0.1/Contents/Home/bin/java -Djline.terminal=jline.UnsupportedTerminal -Dsbt.log.noformat=true -Dfile.encoding=UTF-8 -Didea.managed=true -Dfile.encoding=UTF-8 -jar /Users/harrydrummond/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/203.7148.57/IntelliJ IDEA.app.plugins/Scala/launcher/sbt-launch.jar
[info] welcome to sbt 1.4.7 (Oracle Corporation Java 15.0.1)
[info] loading global plugins from /Users/harrydrummond/.sbt/1.0/plugins
[info] loading settings for project t-build from plugins.sbt ...
[info] loading project definition from /Users/harrydrummond/Documents/workspace/scala/t/project
[info] loading settings for project root from build.sbt ...
[info] set current project to ScalaTest (in build file:/Users/harrydrummond/Documents/workspace/scala/t/)
[info] sbt server started at local:///Users/harrydrummond/.sbt/1.0/server/26df0aea2f87fd17e7ab/sock
[info] started sbt server
sbt:ScalaTest>
;set _root_.scala.collection.Seq(historyPath := None,shellPrompt := { _ => "" }
,SettingKey[_root_.scala.Option[_root_.sbt.File]]
("sbtStructureOutputFile")
in _root_.sbt.Global := _root_.scala.Some(_root_.sbt.file("/private/var/folders/kf/q70qvrg10ls9qd7vvb8zlvgr0000gn/T/sbt-structure.xml"))
,SettingKey[_root_.java.lang.String]("sbtStructureOptions")
in _root_.sbt.Global := "download, resolveClassifiers")
[info] Defining Global / sbtStructureOptions, Global / sbtStructureOutputFile and 1 others.
[info] The new values will be used by cleanKeepGlobs
[info] Run `last` for details.
[info] Reapplying settings...
[info] set current project to ScalaTest (in build file:/Users/harrydrummond/Documents/workspace/scala/t/)
[info] Applying State transformations org.jetbrains.sbt.CreateTasks from /Users/harrydrummond/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/203.7148.57/IntelliJ IDEA.app.plugins/Scala/repo/org.jetbrains/sbt-structure-extractor/scala_2.12/sbt_1.0/2020.3/jars/sbt-structure-extractor.jar
[info] Reapplying settings...
[info] set current project to ScalaTest (in build file:/Users/harrydrummond/Documents/workspace/scala/t/)
[warn]
[warn] Note: Unresolved dependencies path:
[error] stack trace is suppressed; run 'last update' for the full output
[error] stack trace is suppressed; run 'last ssExtractDependencies' for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.spigotmc:spigot-api:1.16.5-R0.1-SNAPSHOT
[error] Not found
[error] Not found
[error] not found: /Users/harrydrummond/.ivy2/localorg.spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/spigot-api-1.16.5-R0.1-SNAPSHOT.pom
[error] download error: Caught java.io.IOException: Server returned HTTP response code: 403 for URL: https://hub.spigotmc.org/nexus/content/repositories/snapshots/org/spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/spigot-api-1.16.5-R0.1-SNAPSHOT.pom (Server returned HTTP response code: 403 for URL: https://hub.spigotmc.org/nexus/content/repositories/snapshots/org/spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/spigot-api-1.16.5-R0.1-SNAPSHOT.pom) while downloading https://hub.spigotmc.org/nexus/content/repositories/snapshots/org/spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/spigot-api-1.16.5-R0.1-SNAPSHOT.pom
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: Error downloading org.spigotmc:spigot-api:1.16.5-R0.1-SNAPSHOT
[error] Not found
[error] Not found
[error] not found: /Users/harrydrummond/.ivy2/localorg.spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/spigot-api-1.16.5-R0.1-SNAPSHOT.pom
[error] download error: Caught java.io.IOException: Server returned HTTP response code: 403 for URL: https://hub.spigotmc.org/nexus/content/repositories/snapshots/org/spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/spigot-api-1.16.5-R0.1-SNAPSHOT.pom (Server returned HTTP response code: 403 for URL: https://hub.spigotmc.org/nexus/content/repositories/snapshots/org/spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/spigot-api-1.16.5-R0.1-SNAPSHOT.pom) while downloading https://hub.spigotmc.org/nexus/content/repositories/snapshots/org/spigotmc/spigot-api/1.16.5-R0.1-SNAPSHOT/spigot-api-1.16.5-R0.1-SNAPSHOT.pom
[error] Total time: 1 s, completed 21 Feb 2021, 16:54:13
[info] shutting down sbt server

Related

intellij: errors with scala, spark

I have a scala project using spark libraies, and it works fine most of the times(using intellij). But some times it starts giving errors on intellij launch:
[warn] Note: Unresolved dependencies path:
[error] stack trace is suppressed; run 'last update' for the full output
[error] stack trace is suppressed; run 'last ssExtractDependencies' for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.apache.spark:spark-core_2.12:3.0.0-preview2
[error] Not found
[error] Not found
[error] not found: C:\...\.ivy2\localorg.apache.spark\spark-core_2.12\3.0.0-preview2\ivys\ivy.xml
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-core_2.12\3.0.0-preview2\.spark-core_2.12-3.0.0-preview2.pom__sha1
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-core_2.12\3.0.0-preview2\.spark-core_2.12-3.0.0-preview2.pom__sha1
[error] Error downloading org.apache.spark:spark-sql_2.12:3.0.0-preview2
[error] Not found
[error] Not found
[error] not found: C:\...\.ivy2\localorg.apache.spark\spark-sql_2.12\3.0.0-preview2\ivys\ivy.xml
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-sql_2.12\3.0.0-preview2\.spark-sql_2.12-3.0.0-preview2.pom__sha1
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: Error downloading org.apache.spark:spark-core_2.12:3.0.0-preview2
[error] Not found
[error] Not found
[error] not found: C:\...\.ivy2\localorg.apache.spark\spark-core_2.12\3.0.0-preview2\ivys\ivy.xml
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-core_2.12\3.0.0-preview2\.spark-core_2.12-3.0.0-preview2.pom__sha1
[error] Error downloading org.apache.spark:spark-sql_2.12:3.0.0-preview2
[error] Not found
[error] Not found
[error] not found: C:\...\.ivy2\localorg.apache.spark\spark-sql_2.12\3.0.0-preview2\ivys\ivy.xml
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-sql_2.12\3.0.0-preview2\.spark-sql_2.12-3.0.0-preview2.pom__sha1
[error] Total time: 1 s, completed 17 Sep 2022, 15:13:49
[info] shutting down sbt server
build.sbt is:
/*ThisBuild / version := "0.1.0-SNAPSHOT"
ThisBuild / scalaVersion := "2.13.8"
lazy val root = (project in file("."))
.settings(
name := "spark-learning"
)*/
// Name of the package
name := "spark-learning"
// Version of our package
version := "1.0"
// Version of Scala
scalaVersion := "2.12.14"
// Spark library dependencies
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "3.0.0-preview2",
"org.apache.spark" %% "spark-sql" % "3.0.0-preview2"
)
What causes these issues out of sudden?And how can I get rid?
One weird thing in the error messages is the path where it looks for dependencies: ...ivy2\localorg.apache.spark\....
There should be a \ after local.
Any chance your SBT repositories configuration could be messed up? Not sure where it is on Windows, it's /etc/sbt/repositories typically on Linux but maybe Intellij has its own setting.

Facing issue with sbt dependency

I try to use the phoenix-spark jar to laod phoenix table to Spark 2.2.3 DataFrame
adding this dependency:
libraryDependencies += "org.apache.phoenix" % "phoenix-spark2" % "4.7.0.2.6.5.1102-5"
I tested this two resolvers one by one:
resolvers += "Hortonworks Repository" at "http://repo.hortonworks.com/content/repositories/releases/"
resolvers += "Hortonworks Releases" at "http://repo.hortonworks.com/content/groups/public/"
I had the folowing error:
[info] welcome to sbt 1.3.13 (Oracle Corporation Java 1.8.0_261)
[info] loading project definition from /home/ambac61n/IdeaProjects/phoenix_test/project
[info] loading settings for project phoenix_test from build.sbt ...
[info] set current project to phoenix_test (in build file:/home/my_user/IdeaProjects/phoenix_test/)
[info] sbt server started at local:///home/ambac61n/.sbt/1.0/server/0c2856c06fe3f2cf2706/sock
sbt:phoenix_test>
[info] Defining Global / sbtStructureOptions, Global / sbtStructureOutputFile and 1 others.
[info] The new values will be used by cleanKeepGlobs
[info] Run `last` for details.
[info] Reapplying settings...
[info] set current project to phoenix_test (in build file:/home/ambac61n/IdeaProjects/phoenix_test/)
[info] Applying State transformations org.jetbrains.sbt.CreateTasks from /home/my_user/.local/share/JetBrains/IdeaIC2020.2/Scala/repo/org.jetbrains/sbt-structure-extractor/scala_2.12/sbt_1.0/2018.2.1+4-88400d3f/jars/sbt-structure-extractor.jar
[info] Reapplying settings...
[info] set current project to phoenix_test (in build file:/home/my_user/IdeaProjects/phoenix_test/)
[warn]
[warn] Note: Unresolved dependencies path:
[error] stack trace is suppressed; run 'last update' for the full output
[error] stack trace is suppressed; run 'last ssExtractDependencies' for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.apache.phoenix:phoenix-spark2:4.7.0.2.6.5.1102-5
[error] Not found
[error] Not found
[error] not found: /home/ambac61n/.ivy2/local/org.apache.phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/phoenix-spark2-4.7.0.2.6.5.1102-5.pom
[error] not found: http://repo.hortonworks.com/content/repositories/releases/org/apache/phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/phoenix-spark2-4.7.0.2.6.5.1102-5.pom
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: Error downloading org.apache.phoenix:phoenix-spark2:4.7.0.2.6.5.1102-5
[error] Not found
[error] Not found
[error] not found: /home/ambac61n/.ivy2/local/org.apache.phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/phoenix-spark2-4.7.0.2.6.5.1102-5.pom
[error] not found: http://repo.hortonworks.com/content/repositories/releases/org/apache/phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/phoenix-spark2-4.7.0.2.6.5.1102-5.pom
[error] Total time: 3 s, completed 22 août 2020 05:56:14
[info] shutting down sbt server
Do you have any Idea?
After vising those repositories, I noticed that there is no package, indeed.
For the first repository,
https://repo1.maven.org/maven2/org/apache/phoenix/
No package for the phoenix-spark2
and the second repository,
https://repo.hortonworks.com/content/repositories/releases/org/apache/phoenix/phoenix-spark2/
No package for the 4.7.0.2.6.5.1102-5 version
Try with the other versions.

How to run an existing Scala project using VS Code and Metals?

I am brand new to Scala and I find that Scala IDE is very slow on my machine for basic things like searching the codebase and editing code. I am used to Visual Studio Code and was very happy to find this metals extension.
I was able to "import build" and fix issues like bumping up scala version in my projects but I am not sure how to reproduce this step to set up a run configuration and actually launch our app in Scala IDE.
We have a parent folder which has a bunch of projects and a 'consoleapp' project which is the main entry point of our app - it imports the logic/routes of all other projects.
|____parent
| |____consoleapp
| |____project1
| |____project2
I tried sbt run and sbt runMain consoleapp from within the consoleapp folder and also the parent folder but they didn't work.
I am not sure what other information from our setup is relevant - happy to provide more info as needed.
Updated to add more details below:
consoleapp/build.sbt
name := "consoleapp"
version := "1.0"
scalaVersion := "2.12.10"
packMain := Map("consoleapp" -> "consoleapp")
libraryDependencies ++= Seq (...)
Output of commands I ran - sbt run and sbt runMain
Running from ~/scala/parent
> sbt run masterstate [0a8dab85] modified
[info] Loading settings for project global-plugins from metals.sbt,build.sbt ...
[info] Loading global plugins from /Users/pradhyo/.sbt/1.0/plugins
[info] Loading project definition from /Users/pradhyo/scala/parent/project
[info] Loading settings for project consoleapp from build.sbt ...
...
Loading settings for all other projects in parent folder
...
[info] Loading settings for project parent from build.sbt ...
[info] Resolving key references (22435 settings) ...
[info] Set current project to parent (in build file:/Users/pradhyo/scala/parent/)
[error] java.lang.RuntimeException: No main class detected.
[error] at scala.sys.package$.error(package.scala:30)
[error] stack trace is suppressed; run last Compile / bgRun for the full output
[error] (Compile / bgRun) No main class detected.
[error] Total time: 1 s, completed 18-Dec-2019 1:41:25 PM
Running from ~/scala/parent
> sbt "runMain consoleapp.consoleapp" masterstate [0a8dab85] modified
[info] Loading settings for project global-plugins from metals.sbt,build.sbt ...
[info] Loading global plugins from /Users/pradhyo/.sbt/1.0/plugins
[info] Loading project definition from /Users/pradhyo/scala/parent/project
[info] Loading settings for project consoleapp from build.sbt ...
...
Loading settings for all other projects in parent folder
...
[info] Loading settings for project parent from build.sbt ...
[info] Resolving key references (22435 settings) ...
[info] Set current project to parent (in build file:/Users/pradhyo/scala/parent/)
[info] running consoleapp.consoleapp
[error] (run-main-0) java.lang.ClassNotFoundException: consoleapp.consoleapp
[error] java.lang.ClassNotFoundException: consoleapp.consoleapp
[error] at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
[error] stack trace is suppressed; run last Compile / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1
[error] Total time: 0 s, completed 18-Dec-2019 1:46:21 PM
Running from ~/scala/parent/consoleapp
> sbt run masterstate [0a8dab85] modified
[info] Loading settings for project global-plugins from metals.sbt,build.sbt ...
[info] Loading global plugins from /Users/pradhyo/.sbt/1.0/plugins
[info] Loading project definition from /Users/pradhyo/scala/parent/consoleapp/project
[info] Loading settings for project consoleapp from build.sbt ...
[info] Set current project to consoleapp (in build file:/Users/pradhyo/scala/parent/consoleapp/)
[error] java.lang.RuntimeException: No main class detected.
[error] at scala.sys.package$.error(package.scala:30)
[error] stack trace is suppressed; run last Compile / bgRun for the full output
[error] (Compile / bgRun) No main class detected.
[error] Total time: 0 s, completed 18-Dec-2019 1:49:26 PM
Running from ~/scala/parent/consoleapp
> sbt "runMain consoleapp" masterstate [0a8dab85] modified
[info] Loading settings for project global-plugins from metals.sbt,build.sbt ...
[info] Loading global plugins from /Users/pradhyo/.sbt/1.0/plugins
[info] Loading project definition from /Users/pradhyo/scala/parent/consoleapp/project
[info] Loading settings for project consoleapp from build.sbt ...
[info] Set current project to consoleapp (in build file:/Users/pradhyo/scala/parent/consoleapp/)
[info] running consoleapp
[error] (run-main-0) java.lang.ClassNotFoundException: consoleapp
[error] java.lang.ClassNotFoundException: consoleapp
[error] at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
[error] stack trace is suppressed; run last Compile / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1
[error] Total time: 1 s, completed 18-Dec-2019 1:50:06 PM
After following the instructions in the Scala Metals VSCode Readme, use a launch configuration similar to this for the Eclipse screenshots in the question.
.vscode/launch.json
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "scala",
"name": "Debug consoleapp",
"request": "launch",
"mainClass": "consoleapp",
"buildTarget": "consoleapp",
"args": [],
"jvmOptions": ["-J-Dconfig.file=/path/to/config/file"]
}
]
}
I had trouble passing the config file for pureconfig correctly. Here's the Github issue with the correct jvmOptions line.
Metals supports these Scala versions 2.13.0, 2.13.1, 2.12.8, 2.12.9,
2.12.10, 2.12.7 and 2.11.12
This is something I noticed from their documentation(doc). So it could pinpoint to your and mine problem which I addressed in comments.
Your scalaVersion := "2.12.3" is not listed here.

plotply scala sbt unresolved dependency not found

I am trying to get started with plotly using scala. I am following their guide to the letter but sbt returns an error:
Error:Error while importing SBT project:
[info] Resolving org.scala-sbt#run;0.13.8 ...
[info] Resolving org.scala-sbt#task-system;0.13.8 ...
[info] Resolving org.scala-sbt#tasks;0.13.8 ...
[info] Resolving org.scala-sbt#tracking;0.13.8 ...
[info] Resolving org.scala-sbt#cache;0.13.8 ...
[info] Resolving org.scala-sbt#testing;0.13.8 ...
[info] Resolving org.scala-sbt#test-agent;0.13.8 ...
[info] Resolving org.scala-sbt#test-interface;1.0 ...
[info] Resolving org.scala-sbt#main-settings;0.13.8 ...
[info] Resolving org.scala-sbt#apply-macro;0.13.8 ...
[info] Resolving org.scala-sbt#command;0.13.8 ...
[info] Resolving org.scala-sbt#logic;0.13.8 ...
[info] Resolving org.scala-sbt#precompiled-2_8_2;0.13.8 ...
[info] Resolving org.scala-sbt#precompiled-2_9_2;0.13.8 ...
[info] Resolving org.scala-sbt#precompiled-2_9_3;0.13.8 ...
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: co.theasi#plotly_2.11;0.2.0: not found
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: co.theasi#plotly_2.11;0.2.0: not found
[error] Total time: 10 s, completed Jul 16, 2016 3:29:59 PM</pre><br/>See complete log in /home/michael/.IdeaIC2016.1/system/log/sbt.last.log
Can someone please help me resolve this issue?
My build.sbt is as follows:
name := "plotly"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies += "co.theasi" %% "plotly" % "0.2.0"
There is only 0.1 version on maven central - link. And 0.2 is not yet released. To use 0.2 version change your build.sbt file:
resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"
libraryDependencies += "co.theasi" %% "plotly" % "0.2-SNAPSHOT"

How to declare dependency on Scalding in sbt project?

I am trying to figure out how to create an build.sbt file for my own Scalding-based project.
Scalding source structure has no build.sbt file. Instead it has project/Build.scala build definition.
What would be the right way to integrate my own sbt project with Scalding, so I could also import it later in Eclipse with sbt-eclipse plugin?
Update:
For the following code:
import cascading.tuple.Fields
import com.twitter.scalding._
class Scan(args: Args) extends Job(args) {
val output = TextLine("tmp/out.txt")
val wordsList = List(
("john"),
("liza"),
("nina"),
("x"))
val orderedPipe =
IterableSource[(String)](wordsList, ('word))
.debug
.write(output)
}
With this build.sbt:
name := "Scan"
version := "1.0"
libraryDependencies := Seq("com.twitter" %% "scalding" % "0.11.1")
I get errors:
$ sbt
[info] Loading global plugins from /home/test/.sbt/0.13/plugins
[info] Set current project to Scan (in build file:/home/test/Cascading/Scala/Scan/)
> compile
[info] Updating {file:/home/test/Cascading/Scala/Scan/}scan...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading http://repo1.maven.org/maven2/com/twitter/scalding_2.10/0.11.1/scalding_2.10-0.11.1.jar ...
[info] [SUCCESSFUL ] com.twitter#scalding_2.10;0.11.1!scalding_2.10.jar (641ms)
[info] Done updating.
[info] Compiling 1 Scala source to /home/test/Cascading/Scala/Scan/target/scala-2.10/classes...
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:1: not found: object cascading
[error] import cascading.tuple.Fields
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:2: object twitter is not a member of package com
[error] import com.twitter.scalding._
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:5: not found: type Job
[error] class Scan(args: Args) extends Job(args) {
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:5: not found: type Args
[error] class Scan(args: Args) extends Job(args) {
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:5: too many arguments for constructor Object: ()Object
[error] class Scan(args: Args) extends Job(args) {
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:6: not found: value TextLine
[error] val output = TextLine("tmp/out.txt")
[error] ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:15: not found: value IterableSource
[error] IterableSource[(String)](wordsList, ('word))
[error] ^
[error] 7 errors found
[error] (compile:compile) Compilation failed
Update 2
After doing git clone git#github.com:twitter/scalding.git their repository and sbt publishLocal I still have the same compilation errors.
BUT adding two lines that you suggested to build.sbt allowed me to compile my code. So the following build.sbt really works, thanks!
name := "BlockScan"
version := "1.0"
libraryDependencies := Seq("com.twitter" %% "scalding" % "0.11.1")
lazy val scaldingCore = ProjectRef(uri("https://github.com/twitter/scalding.git"), "scalding-core")
lazy val myProject = project in file(".") dependsOn scaldingCore
'sbt eclipse' creates Eclipse project wich does not compile under Eclipse and reports these errors:
Project 'Scan' is missing required Java project: 'scalding-core'
More than one scala library found in the build path (/home/test/usr/eclipse-scala-3.0.3/configuration/org.eclipse.osgi/bundles/290/1/.cp/lib/scala-library.jar, /home/test/wks/Cascading/Scala/scalding/target/scala-2.9.3/scalding-assembly-0.10.0.jar).At least one has an incompatible version. Please update the project build path so it contains only compatible scala libraries.
scalacheck_2.9.3-1.10.0.jar is cross-compiled with an incompatible version of Scala (2.9.3).
specs_2.9.3-1.6.9.jar is cross-compiled with an incompatible version of Scala (2.9.3).
Since they don't seem to publish their libraries to remote repositories where you could pull down the necessary dependencies, you'll have to declare the source dependency on the GitHub repository for the project.
lazy val scaldingCore = ProjectRef(uri("https://github.com/twitter/scalding.git"), "scalding-core")
lazy val myProject = project in file(".") dependsOn scaldingCore
With the above build definition, sbt will git clone the RootProject and load the build.
➜ scalding xsbt
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
Cloning into '/Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding'...
[info] Loading project definition from /Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding/project
[info] Updating {file:/Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding/project/}scalding-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.eed3si9n/sbt-assembly/scala_2.10/sbt_0.13/0.10.2/jars/sbt-assembly.jar ...
[info] [SUCCESSFUL ] com.eed3si9n#sbt-assembly;0.10.2!sbt-assembly.jar (3600ms)
[info] Done updating.
[info] Compiling 3 Scala sources to /Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding/project/target/scala-2.10/sbt-0.13/classes...
[warn] there were 8 deprecation warning(s); re-run with -deprecation for details
[warn] there were 2 feature warning(s); re-run with -feature for details
[warn] two warnings found
[info] Set current project to myProject (in build file:/Users/jacek/sandbox/scalding/)
> projects
[info] In file:/Users/jacek/sandbox/scalding/
[info] * myProject
[info] In https://github.com/twitter/scalding.git
[info] maple
[info] scalding
[info] scalding-args
[info] scalding-avro
[info] scalding-commons
[info] scalding-core
[info] scalding-date
[info] scalding-hadoop-test
[info] scalding-jdbc
[info] scalding-json
[info] scalding-parquet
[info] scalding-repl
The build setup should give you access to scalding classes.
> console
[info] Starting scala interpreter...
[info]
Welcome to Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_60).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import com.twitter.scalding._
import com.twitter.scalding._
And the Scan class compiles fine - it's in src/main/scala directory.
> show sources
[info] ArrayBuffer(/Users/jacek/sandbox/scalding/src/main/scala/Scan.scala)
[success] Total time: 0 s, completed Jul 15, 2014 12:21:14 AM
> compile
[info] Updating {file:/Users/jacek/sandbox/scalding/}myProject...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/jacek/sandbox/scalding/target/scala-2.10/classes...
[success] Total time: 4 s, completed Jul 15, 2014 12:21:20 AM
You could also git clone git#github.com:twitter/scalding.git their repository and sbt publishLocal to be able to declare binary dependency in build.sbt as follows:
libraryDependencies := Seq("com.twitter" %% "scalding" % "0.11.1")
With the dependency in (either way), execute sbt eclipse and be done with it!