I have recently installed SBT on Docker Ubuntu machine to get started with Scala. When I started docker initially, it started grabbing all the Java, sbt JAR's from the remote locations (https://repo.scala-sbt.org/scalasbt/debian/sbt-0.13.17.deb).
But, whenever I run sbt command, it again starts downloading the sbt JAR. Is there a way of maintaining a global cache whereby artifacts are only downloaded once and not every time I remote to docker container?
My solution to this was a multi stage build.
Have a “base” docker image.
Copy in only build.sbt, projects.sbt and the file which sets out the sbt version from your project.
That defines the required dependencies. The last line in that base image is “sbt update” - I.e fetch them. That “base image” has the dependencies in it… and is reuseable. Just remember to run it when you change library versions etc to rebuild it.
In the ”build” image… copy over the project and proceed as normal… make sure sbt is resolving from maven-local, and it should use the “cache”… which is already in place from the paragraph above.
I’d be interested to hear other answers, but that’s my solution… YMMV :-).
That works for me on a cloud / Kube pipeline.
Related
This question is a bit embarrassing to ask, but several searches have yielded no clear answer. I have created a Scala Play project (Play 2.8.1; sbt 1.3.4; Scala 2.13.1) using the command sbt new playframework/play-scala-seed.g8 All works as expected. I added a few classes and objects of Scala code and experienced no problems in building and running on the Ubuntu 18.04 cmd line using sbt commands (e.g. sbt run, sbt clean).
Then I checked my code into a git repo and pushed to a remote repository. The project is a simple one, consisting only of a about 5 .scala files and some standard Play Framework front-end views. I made sure to push build.sbt, and the build.properties and plugins.sbt files under the project directory, in addition to my .scala files. I also checked in the sbt and sbt.bat scripts.
I cloned this git project into a new location.
I run the code, as always using sbt run. Some dependencies are pulled down. But before long I get an error: java.lang.RuntimeException: No main class detected.
I tried several times, and then again after running sbt clean just for good measure. I note that a brief grep revealed no clear main() candidates, but this was true also of my working code in the original--not the freshly cloned--local git repository. What do I need to do in order to insure that I or someone else can simply check out this or any other Play project from github, and run it successfully without this error? This ought to be automatic. Thanks in advance.
Ooh, yuk. Despite checkng, it seems I pulled down an empty build.sbt file. I made sure the correct, non-empty build.sbt file was checked in, I pulled to get it, and the project built with no problem. Sorry for this.
I've tried to run application inside firejail without network. When I run sbt in host system, all goes ok, sbt finds all libraries. Then I try to run sbt inside restricted environment with read-write access to ~/.sbt, ~/.ivy2, ~/.m2 and project folders and read-only access to .sbt binary. Sbt loses all dependent libraries and try go get them from network, which is disabled.
Where do sbt store library cache and why do it refuse to take old compiled and ready to use state?
update: sbt fails on the time it tries to build global plugins. It does not get to the stage where it performs project build. I've tried to add offline := true both to the project and global sbt configuration, but sbt tries to update plugins nevertheless.
update2: I've managed to run sbt successfully without rebuilding all global plugins. That requires access to /etc. But I would prefer not to give access to /etc. What information do SBT need from /etc? Hostname or something else? Why on earth it make cache depend on such things?
update3:
I've discovered the exact cause for SBT failing to recognize already downloaded libraries. /etc/passwd is strictly needed for SBT to work correctly. Could anyone point in what way SBT uses login info exactly?
I'm trying to run a scala/akka based program on a linux cluster machine. I was following the tutorial link (this was the only example I could find):
http://doc.akka.io/docs/akka/2.0/intro/getting-started-first-scala.html
It says to use the command to obtain the akka library:
git clone git://github.com/akka/akka.git
But it doesn't have any jars files inside it as per the tutorial.
I'm not able to get a basic akka-scala-sbt combo working on linux. So any help on this is much appreciated. I'm not aware (not able to find any clear source) of the commands needed to compile/execute with and without using SBT.
Java version: "1.8.0_31"
Scala version: "2.11.5"
Akka: I'm not sure, I did git clone, believe its the latest
SBT: 0.13.9
Java,Scala are already installed on the cluster, I had to just use module load.
You can start with this simple example: https://github.com/izmailoff/SmallAkkaProjectExample. It will help you to understand how to compile and run a project.
Usually the steps are:
Start SBT
Compile code
Run it from SBT
or:
Start SBT
Compile executable JAR
Run the JAR from command line
If you want a more advanced example take a look at: https://github.com/izmailoff/remote-akka-server-template
In either of these projects you don't need to download any libraries/JARs. SBT will download everything you need automatically.
In short, you need to understand how to build projects with SBT and how to run them - not related to Akka. Separately from that you need to know how Akka runs, i.e. ActorSystem, kernel, etc.
I am trying to monitor kafka using KafkaOffsetMonitor tool. It is working fine as I used already built jar available at its github page. Now I want to make some changes in this tool but I dont know how to manually build it. I have downloaded the zip file from github page. Now how should I build it?
PS: Steps would be helpful
Below are the steps which may help you:
1. Checkout and get the source code into your local machine or you may have to unzip it if you have downloaded zip file.
2. Go to the folder which you have extracted or checked-out.
3. Run below command:
mvn clean package -U -DskipTests
Note: make sure you have installed maven in your machine.
Make the changes in the source if you want to modify the tool and build it using above steps.
I know that I'm coming at this pretty late, but I ran into the same problem. Basically, you need a Java jdk, scala, and sbt installed first. You didn't post what os you're dealing with, so it would be hard for me to give you steps for that. I use Gentoo Linux and you can install it by running emerge -av sbt
Once sbt is installed, just clone KafkaOffsetMonitor Github repository change to the top level directory and run the following sbt command: sbt assembly
The jar you will be looking for will be in: ./target/scala-<scala_version>/KafkaOffsetMonitor-assembly-<kafkaoffsetmonitor_version>.jar
As you must known, the first time you run SBT, it starts to download his dependencies:
>sbt sbt-version
Getting org.scala-sbt sbt 0.12.4
downloading repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/main/0.12.4/jars/main.jar ...
The problem is that I'm living in Africa and the ISP is not very good: for some unknown reason, they are some domains from which I can not download any file. For example, if I enter to the site of WAMP and try to download the installer, the download begins but it never ends. Normally I fix this issue by using a proxy (such as HideMyAss or ZendProxy), which servers as an intermediate between the site and the ISP, allowing me of download the file.
typesafe.com seems to be one of the not-work-for-download-in-africa domains, as consequence, SBT can not download his dependencies.
The question is, there is a way of download manually the dependencies? (I have downloaded them using a proxy, but I do not known in which folder put them in order to avoid SBT to try to download them)?
Create a directory in the root of your sbt project called lib and put the jars there.