I've downloaded Algebird and I want to try out few things in the Scala interpreter using this library. How do I achieve this?
Of course, you can use scala -cp whatever and manually manage your dependencies. But that gets quite tedious, especially if you have multiple dependencies.
A more flexible approach is to use sbt to manage your dependencies. Search for the library you want to use on search.maven.org. Algebird for example is available by simply searching for algebird. Then create a build.sbt referring to that library, enter the directory and enter sbt console. It will download all your dependencies and start a scala console session with all dependencies automatically on the classpath.
Changing things like the scala version or the library version is just a simple change in the build.sbt. To play around you don't need any scala code in your directory. An empty directory with just the build.sbt will do just fine.
Here is a build.sbt for using algebird:
name := "Scala Playground"
version := "1.0"
scalaVersion := "2.10.2"
libraryDependencies += "com.twitter" % "algebird-core" % "0.2.0"
Edit: often when you want to play around with a library, the first thing you have to do is to import the namespace(s) of the library. This can also be automated in the build.sbt by adding the following line:
initialCommands in console += "import com.twitter.algebird._"
Running sbt console will not import libraries declared with a test scope. To use those libraries in the REPL, start the console with
sbt test:consoleQuick
You should be aware, however, that starting the console this way skips compiling your test sources.
Source: http://www.scala-sbt.org/0.13/docs/Howto-Scala.html
You can use the scala's -cp switch to keep jars on the classpath. There are other switches available too, for example, -deprecation and -unchecked for turning on various warnings. Many more to be found with scala -X... and scala -Y.... You can find out more information about these switches with scala -help
This is an answer using Ammonite (as opposed to the Scala REPL) - but it is such a great tool that it is worth mentioning.
You can install it with a one liner such as:
sudo sh -c '(echo "#!/usr/bin/env sh" && curl -L https://github.com/lihaoyi/Ammonite/releases/download/2.1.2/2.13-2.1.2) > /usr/local/bin/amm && chmod +x /usr/local/bin/amm' && amm
or using brew on macOS:
brew install ammonite-repl
For scala 2.10, you need to use an oder version 1.0.3:
sudo sh -c '(echo "#!/usr/bin/env sh" && curl -L https://github.com/lihaoyi/Ammonite/releases/download/1.0.3/2.10-1.0.3) > /usr/local/bin/amm && chmod +x /usr/local/bin/amm' && amm
Run Ammonite in your terminal:
amm
// Displays
Loading...
Welcome to the Ammonite Repl 2.1.0 (Scala 2.12.11 Java 1.8.0_242)
Use in ivy import to import your 3rd part library:
import $ivy.`com.twitter::algebird-core:0.2.0`
Then you can use your library within the Ammonite-REPL:
import com.twitter.algebird._
import com.twitter.algebird.Operators._
Map(1 -> Max(2)) + Map(1 -> Max(3)) + Map(2 -> Max(4))
...
Related
If you follow the steps at the official Scala 3 sites, like Dotty or Scala Lang then it recommends using Coursier to install Scala 3. The problem is that neither or these explain how to run a compiled Scala 3 application after following the steps.
Scala 2:
> cs install scala
> scalac HelloScala2.scala
> scala HelloScala2
Hello, Scala 2!
Scala 3:
> cs install scala3-compiler
> scala3-compiler HelloScala3.scala
Now how do you run the compiled application with Scala 3?
Currently there does not seem to be a way to launch a runner for Scala 3 using coursier, see this issue. As a workaround, you can install the binaries from the github release page. Scroll all the way down passed the contribution list to see the .zip file and download and unpack it to some local folder. Then put the unpacked bin directory on your path. After a restart you will get the scala command (and scalac etc) in terminal.
Another workaround is using the java runner directly with a classpath from coursier by this command:
java -cp $(cs fetch -p org.scala-lang:scala3-library_3:3.0.0):. myMain
Replace myMain with the name of your #main def function. If it is in a package myPack you need to say myPack.myMain (as usual).
Finally, it seems that is possible to run scala application like scala 2 version using scala3 in Coursier:
cs install scala3
Then, you can compile it with scala3-compiler and run with scala3:
scala3-compiler Main.scala
scala3 Main.scala
This work-around seems to work for me:
cs launch scala3-repl:3+ -M dotty.tools.MainGenericRunner -- YourScala3File.scala
This way, you don't even have to compile the source code first.
In case your source depends on third-party libraries, you can specify the dependencies like this:
cs launch scala3-repl:3+ -M dotty.tools.MainGenericRunner -- -classpath \
$(cs fetch --classpath io.circe:circe-generic_3:0.14.1):. \
YourScala3File.scala
This would be an example where you use the circe library that's compiled with Scala 3. You should be able to specify multiple third-party libraries with the fetch sub-command.
I'm trying to get familiar with Scala. I am using macOS.
I've installed scala using brew install scala which is hassle-free and once complete I can launch the scala REPL simply by issuing scala and I'm at the scala> prompt.
I now want to import some packages, so I try:
import org.apache.spark.sql.Column
and unsurprisingly it fails with
error: object apache is not a member of package org
This makes sense, how would it know where to get that package from? Thing is, I don't know what I need to do to make that package available. Is there anything I can do from the command-line that would allow me to import org.apache.spark.sql.Column?
I have googled around a little but not found anything that explains in a jargon-free way. Complete Scala noob here so jargon-free responses would be appreciated.
Here are two ways to start a REPL with dependencies that I'm aware of:
Use SBT to manage dependencies, use console to start a REPL with those dependencies
Use Ammonite REPL
You could create a separate directory with a build.sbt where you set
scalaVersion := "2.11.12"
and then copy the
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.3.0"
snippets from MavenCentral. Then you can run the REPL with sbt console. Note that this will create a project and target subdirectories, so it "leaves traces", you can't use it like the standalone scala-repl. You could also omit the build.sbt, and add the library-dependencies by typing them into the SBT-shell itself.
Alternatively you can just use Ammonite REPL that has been created exactly for that purpose.
You can use classpath to make the lib available i.e. download the jar locally and use the command as follows (here I'm using Apache IO lib to move files from scala prompt )
C0:Desktop pvangala$ scala -cp /Users/pvangala/Downloads/commons-io-2.6/commons-io-2.6.jar
Welcome to Scala 2.12.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_161).
Type in expressions for evaluation. Or try :help.
scala> import java.io.File
import java.io.File
scala> val src = new File("/Users/pvangala/Downloads/commons-io-2.6-bin.tar")
src: java.io.File = /Users/pvangala/Downloads/commons-io-2.6-bin.tar
scala> val dst = new File("/Users/pvangala/Desktop")
dst: java.io.File = /Users/pvangala/Desktop
scala> org.apache.commons.io.FileUtils.moveFileToDirectory(src, dst, true)
If you want to use spark stuff I'd recommend you use the spark-shell that comes with the spark-installation. I don't know macOS so I can't help you much with the install of Spark there.
For normal Scala I recommend ammonite http://ammonite.io/#Ammonite-REPL that has included syntax to allow to pull packages/dependencies.
If you want to use spark, you should use the spark-shell instead the scala REPL. It has almost the same behaviour but includes all the spark dependencies by default.
You should download spark binaries from here
Then if you are using Linux, you should create the variable SPARK_HOME pointing to the downloaded folder and include its bin folder in PATH.
then you can start it in any console with the command spark-shell
In Windows i'm not sure, but i think that you should have a spark-shell.cmd file or something similar which you can use to start the spark-shell,
I did the following in Windows:
for /f "tokens=*" %%a in ('java -jar coursier fetch -p "com.lihaoyi::requests:0.2.0" "com.lihaoyi::upickle:0.7.5"') do set SCP=%%a
scala -nc -classpath %SCP% %1 %2 %3
Instead of the two libraries listed here you can use an unlimited number of other libraries you need. They must be available in maven central, though. The %1 could be a scala script (".sc" extension). But you could leave it empty and the REPL will start with the libraries on the classpath.
I wish to create a docker image with several SBT components built in (docker part not relevant). If I just install sbt from deb or rpm and not do anything, the first time it is invoked it still has to download the internet before it's able to start.
Thus, I need to be able to specify the following:
1 SBT Versions - e.g. 0.12, 0.13.1, 0.13.2, ...
2 Scala Versions - 2.9.1, 2.10.4, 2.11.5, ...
3 Commonly used libraries - play framework, etc.
I'm wondering what's the best way to ensure those are already pre-cached.
Here is what I do in my Dockerfile to prefetch multiple scala versions :
RUN echo 'crossScalaVersions := Seq("2.10.6", "2.11.7")' > build.sbt \
&& echo 'object Hi { def main(args: Array[String]) = println("Done") }' > src.scala \
&& sbt "+run" \
&& rm build.sbt src.scala
You can easily customize this further by specifying the sbt version on cli ( sbt -Dsbt.version=0.x.y "+run" ), or add some libraryDependencies to the build.sbt.
HTH
Well, if you create one project which has everything you want listed as dependency -- including SBT and Scala versions, and then run update to retrieve all these dependencies, they'll all be cached on .ivy2\cache. This should serve for pretty much everything, but SBT itself seems to have tons of separate dependencies as seen here, or on your local disk at the ivy2 cache.
In Python, if I install a package with pip install package_name, I can open a Python repl by typing python and simply import the package by its name, regardless of what directory I'm currently in in the filesystem.
Like so
$ python
Python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>
and the requests library is imported and I can play with it in the repl.
In Scala, I know how to do this in a project that uses sbt, but for learning purposes, I would like to install a package in such a way so that I can simply type scala at the command line and then import the installed package, without being tied to a specific project.
$ scala
Welcome to Scala version 2.10.2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_40).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import scalaz._
<console>:7: error: not found: value scalaz
import scalaz._
How can I do this?
Scala is different from Python. Code compiled for Scala 2.9.x is not compatible to 2.10.x. So global definitions can cause a lot of problems if you work with different versions.
You can use SBT and add to $HOME/.sbt/plugins/build.sbt
libraryDependencies += "org.scalaz" %% "scalaz-core" % "7.0.4"
or
libraryDependencies += "org.scalaz" % "scalaz-core_2.10" % "7.0.4"
and then go to /tmp and start a Scala REPL with SBT:
sbt console
But on long term it is not a good idea.
The best thing would be to install SBT, create a file build.sbt
and put this in it:
libraryDependencies += "org.scalaz" % "scalaz-core_2.10" % "7.0.4"
scalaVersion := "2.10.2"
initialCommands in console := "import scalaz._, Scalaz._"
Now change with the console into the folder of build.sbt and run
sbt console
With this you can experiment with the REPL and have already scalaz imported and in the class path. In addition it is easy to add additional dependencies.
SBT is cool, you don't need to install new Scala versions manually, just declare it in build.sbt.
In addition to S.R.I i'm using the following solution, shell script:
/usr/bin/scalaz
#!/bin/sh
scala -cp ~/.ivy2/cache/org.scalaz/scalaz-core_2.10/bundles/scalaz-core_2.10-7.1.0-M3.jar ... other libs
then just call it in the terminal:
$ scalaz
Welcome to Scala version 2.10.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_40).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import scalaz._; import Scalaz._
import scalaz._
import Scalaz._
scala> Monad[Option].point(1)
res0: Option[Int] = Some(1)
scala>
This is generally not recommended since most jar libraries are meant to be used along with programmer projects. Also, unlike other ecosystems, jar libraries are usually installed via some user mode library management tool like ivy or maven or sbt, as you may have observed.
If you really want to do this, you can install jars to scala's TOOL_CLASSPATH location which you can find out from scala.bat or scala shell script file bundled along with your scala distribution. Alternatively, you can build your own custom scala repl, which can load globally installable libraries from some configured location. Either way, it requires futzing with the TOOL_CLASSPATH.
P.S: I currently do not have access to the actual scala.bat file to help you with this, but you can look it up here and here to understand what I mean. Note that these files may not show how the .bat files are structured as those in the distribution(and may be quite dated). Please look it up in the official distribution for information.
EDIT
I can explain a bit more now that I'm back and looked at the actual scala batch and shell scripts included in official distribution :-)
Like I said above, the scala script loads all jar files present in its TOOL_CLASSPATH folder which is usually ${SCALA_HOME}/lib. It also offers the ability to add to TOOL_CLASSPATH with the promising -toolcp option - Let's see what it shows: (the batch script is similar - I'll just show things from scala shell script)
while [[ $# -gt 0 ]]; do
case "$1" in
-D*)
# pass to scala as well: otherwise we lose it sometimes when we
# need it, e.g. communicating with a server compiler.
java_args=("${java_args[#]}" "$1")
scala_args=("${scala_args[#]}" "$1")
shift
;;
-J*)
# as with -D, pass to scala even though it will almost
# never be used.
java_args=("${java_args[#]}" "${1:2}")
scala_args=("${scala_args[#]}" "$1")
shift
;;
-toolcp)
TOOL_CLASSPATH="${TOOL_CLASSPATH}${SEP}${2}"
shift 2
;;
-nobootcp)
unset usebootcp
shift
;;
-usebootcp)
usebootcp="true"
shift
;;
-debug)
SCALA_RUNNER_DEBUG=1
shift
;;
*)
scala_args=("${scala_args[#]}" "$1")
shift
;;
esac
done
As you can see, this is sorely limiting - you'd have to specify each jar to be added. You could just use -cp! Can we make it better? Sure, we'd have to muck around in this toolcp business.
addtoToolCP() {
for i in $(find $1 -name "*.jar")
do
if [[ -z "$TOOLCP" ]]
then
TOOLCP="$i"
else
TOOLCP="${TOOLCP}:$i"
fi
done
}
So, you could just check for the emptiness of our TOOLCP parameter and accordingly call scala as scala -toolcp $TOOLCP if it's non-empty. Now, you can just invoke your shell script as: myscalascript <list-of-paths-to-be-added-to-toolcp>. Or you could just keep one folder and go on adding new libraries to that folder. Hope this helps - as others have said, do watch out for binary compatibility issues. Binary incompatibility issues would only affect major scala versions, minor versions should be perfectly compatible. Lastly, at risk of repeating myself to death, use this only if you're sure you want this. :-)
How can I run an sbt app from the shell, so that I can run my app as a normal command-line program (as if run directly via scala but without having to set up an enormous classpath)?
I know I can do:
echo hello | sbt 'run-main com.foo.MyMain3 arg1 arg2' > out.txt
But this (1) takes forever to start because it starts sbt, (2) causes all stdout and stderr to go to stdout, and (3) causes all output to be decorated with a logger [info] or [error].
I looked at https://github.com/harrah/xsbt/wiki/Launcher but it seems too heavyweight, since it downloads dependencies and sets up a new environment and whatnot. I just want to run this app within my existing development environment.
Thus far I've cobbled together my own script to build up a classpath, and you can also do some other things like modify your project file to get sbt to print the raw classpath, but I feel like there must be a better way.
Here's what I have in my SBT (version 0.10) project definition,
val Mklauncher = config("mklauncher") extend(Compile)
val mklauncher = TaskKey[Unit]("mklauncher")
val mklauncherTask = mklauncher <<= (target, fullClasspath in Runtime) map { (target, cp) =>
def writeFile(file: File, str: String) {
val writer = new PrintWriter(file)
writer.println(str)
writer.close()
}
val cpString = cp.map(_.data).mkString(":")
val launchString = """
CLASSPATH="%s"
scala -usejavacp -Djava.class.path="${CLASSPATH}" "$#"
""".format(cpString)
val targetFile = (target / "scala-sbt").asFile
writeFile(targetFile, launchString)
targetFile.setExecutable(true)
}
... // remember to add mklauncherTask to Project Settings
The mklauncher task creates a script target/scala-sbt that executes scala with the project classpath already set. It would be nice to have mklauncher executed automatically whenever the classpath changes, but I haven't looked into doing this yet.
(I use the Java classpath, rather than Scala's, for ease of creating embedded interpreters.)
The start-script SBT plugin is now at:
https://github.com/sbt/sbt-start-script
It requires a few steps to set up and generates scripts that do not work on OS X, but that can be easily fixed if you're on that platform (see below).
Setup
Install greadlink (OS X only):
a) brew install coreutils
b) map readlink to the new function (greadlink) by adding these lines to ~/.bashrc:
function readlink() { greadlink "$#"; }
export -f readlink`
Add start-script plugin to ~/.sbt/plugins/build.sbt:
addSbtPlugin("com.typesafe.sbt" % "sbt-start-script" % "0.8.0")
Add start-script task to current project:
$ sbt add-start-script-tasks # execute from directory where build.sbt resides
Add start-script support to current build.sbt:
import com.typesafe.sbt.SbtStartScript
seq(SbtStartScript.startScriptForClassesSettings: _*)
Note the blank line in between statements (de rigueur for SBT build files).
Generate Start Script
Then, whenever you want to create a script to start your app like sbt run-main, but without sbt, execute:
$ sbt start-script
Run
target/start mypackage.MyMainClass
Time flies by and a lot have changed since the other answers. It's currently SBT 0.13.6 time.
I think what you may need is the sbt-onejar plugin or the SBT Native Packager plugin.
sbt-onejar "is a simple-build-tool plugin for building a single executable JAR containing all your code and dependencies as nested JARs."
SBT Native Packager's "goal is to be able to bundle up Scala software built with SBT for native packaging systems, like deb, rpm, homebrew, msi."
Just discovered the sbt start script plugin: https://github.com/typesafehub/xsbt-start-script-plugin:
This plugin allows you to generate a script target/start for a
project. The script will run the project "in-place" (without having to
build a package first).
The target/start script is similar to sbt run but it doesn't rely on
SBT. sbt run is not recommended for production use because it keeps
SBT itself in-memory. target/start is intended to run an app in
production.
The plugin adds a task start-script which generates target/start. It
also adds a stage task, aliased to the start-script task.