I have a SBT projec on Ubuntu 14.04. When I run sbt I get the following message:
Detected sbt version sbt.version = 0.13.8
Starting sbt: invoke with -help for other options
Cannot find sbt launcher sbt.version = 0.13.8
Please download:
From http://typesafe.artifactoryonline.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/sbt.version = 0.13.8/sbt-launch.jar
To /home/vagrant/.sbt/.lib/sbt.version/sbt-launch.jar
When I try and download from the above url I am met with a 404 response. In fact even the url http://typesafe.artifactoryonline.com is giving 404.
Any ideas how I can get the correct sbt version on my system?
One solution is not to use the package manager at all. Uninstall sbt, and simply use the sbt shell script by Paul Phillips: https://github.com/paulp/sbt-extras :
curl -s https://raw.githubusercontent.com/paulp/sbt-extras/master/sbt > ~/bin/sbt \
&& chmod 0755 ~/bin/sbt
This is quite up-to-date and should have the correct repositories for the launchers.
How did you install sbt?
If you're using sbt-extras you might be hitting #107. Try upgrading it to a newer version.
An alternative is to switch to the official version of sbt. See Installing sbt on Linux:
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823
sudo apt-get update
sudo apt-get install sbt
Your installed sbt version is probably a bit older and still refers to the old repository.
Typesafe moved to a bintray repository so the new url would be:
https://dl.bintray.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.8/sbt-launch.jar
or with alias
https://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.8/sbt-launch.jar
repo.typesafe.com may be replaced by repo.lightbend.com in the future ;)
Reinstalling the sbt run script would probably help, too.
Related
I am trying to use Github actions for my scala project and created a Docker workflow for it. Basically, I am trying to install sbt into my container and run the project.
Dockerfile looks like this:
FROM centos:centos8
ENV SCALA_VERSION 2.13.1
ENV SBT_VERSION 1.5.2
RUN yum install -y epel-release
RUN yum update -y && yum install -y wget
# INSTALL JAVA
RUN yum install -y java-11-openjdk
# INSTALL SBT
RUN wget http://dl.bintray.com/sbt/rpm/sbt-${SBT_VERSION}.rpm
RUN yum install -y sbt-${SBT_VERSION}.rpm
RUN wget -O /usr/local/bin/sbt-launch.jar http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/$SBT_VERSION/sbt-launch.jar
WORKDIR /root
EXPOSE 8080
RUN sbt compile
CMD sbt run
But when I push anything, I get the following error:
The command '/bin/sh -c wget http://dl.bintray.com/sbt/rpm/sbt-${SBT_VERSION}.rpm' returned a non-zero code: 8
When I check the link manually (by setting the sbt version), I see indeed bintray responds with 403 forbidden error but status.bintray.com tells all systems are operational.
Am I doing something wrong or is something wrong with bintray?
Forbidden doesnt mean non operational.
I think that url is incorrect as its not hosted on bintray rather jfrog, please see section on Centos which states
remove old Bintray repo file
https://www.scala-sbt.org/1.x/docs/Installing-sbt-on-Linux.html
Fedora 25. sbt is installed and I have been using it, probably last time was a week ago. But today it is not found.
$ sbt
bash: sbt: command not found...
Install package 'sbt' to provide command 'sbt'? [N/y]
Tried a reinstall but dnf knows what I do
$ sudo dnf install sbt
[sudo] password for xxx:
Last metadata expiration check: 0:31:12 ago on Thu Apr 27 19:39:34 2017.
Package sbt-0.13.15.2-2.noarch is already installed, skipping.
Dependencies resolved.
Nothing to do.
Complete!
Checked for install location with which but no luck
$ which sbt
/usr/bin/which: no sbt in (/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/xxx/.local/bin:/home/xxx/bin)
I would guess the problem is in my $PATH however I have not altered it, although I have installed some packages
$ echo $PATH
/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/xxx/.local/bin:/home/xxx/bin
Finally i did find a jar named sbt-launch.jar in /usr/share/sbt/bin/ but I don't think /usr/share has ever been part of my $PATH.
Recommendations? thx
fixed it by running
$dnf reinstall sbt
Out of curiosity I checked $PATH again afterwards and it was the same. So it was something else evidently.
I have resolved it reinstalling:
First, uninstall sbt:
yum remove sbt
Then download the version you want from https://dl.bintray.com/sbt/debian/
wget http://dl.bintray.com/sbt/rpm/sbt-0.13.16.rpm
And install by yum
sudo yum install sbt-0.13.16.rpm
Check it with sbt about
sbt about
It must show you similar to
sbt about
[info] This is sbt 0.13.16
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.6
I have been trying to install spark using the tutorial and everytime I run the command sbt/sbt assembly, I get the error "Error: Invalid or corrupt jarfile sbt/sbt-launch-0.13.5.jar"
I have tried everything: seperately adding the sbt file to the sbt folder in the spark folder, installing sbt individually, checking the download and reinstalling it over again, but in vain. Any advice about what I am doing wrong? Thanks.
Ok, After playing around for a while I finally got it and hopefully this will work for you aswell. That tutorial builds spark, where they do provide prebuilt binaries. I'm using Spark 1.2.0 just as a note (1.4.1 wouldn't work for me)
This is on Ubuntu 15.04 but should work on 14.04 the same
1) Remove the following lines from your bashrc
export SCALA_HOME=/usr/local/src/scala/scala-2.10.4
export PATH=$SCALA_HOME/bin:$PATH
2) Remove and reinstall scala
sudo rm -rf /usr/local/src/scala
# The following line is only needed if you installed scala another way, if so remove the #
# sudo apt-get remove scala-library scala
wget http://www.scala-lang.org/files/archive/scala-2.11.7.deb
sudo dpkg -i scala-2.11.7.deb
sudo apt-get update
sudo apt-get install scala
3) Download PreBuilt Spark and extract
wget http://d3kbcqa49mib13.cloudfront.net/spark-1.2.0-bin-hadoop2.4.tgz
tar -xzvf spark-1.2.0-bin-hadoop2.4.tgz
4) Run spark-shell
cd spark-1.2.0-bin-hadoop2.4/
./bin/spark-shell
Sources (basically where I've read from, this solution has been trial and error)
https://chongyaorobin.wordpress.com/2015/07/01/step-by-step-of-installing-apache-spark-on-apache-hadoop/
https://gist.github.com/visenger/5496675
If you have downloaded spark package from http://d3kbcqa49mib13.cloudfront.net/spark-1.1.0.tgz then cross check file - "sbt/sbt-launch-0.13.5.jar". If it just contains small (5-6lines) html content then you need to download jar file manually. This html file just indicate that required jar file was not found. You may use follow following steps for centos:
Download jar manually:
wget http://dl.bintray.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.1/sbt-launch.jar ./sbt/sbt-launch-0.13.5.jar
Prevent automatic downloading of jar file:
sed -i '47,68s/^/#/' sbt/sbt-launch-lib.bash
Install spark again:
sbt/sbt assembly
It worked for me without altering scala installation. Hope it helps.
The sbt script does not download properly sbt-launch-0.13.5.jar because there must be something wrong with the URLs it is using. As a result the file that it downloads contains just an HTML header (wither 400 or 302 codes). Until a better solution becomes available, as a workaround I would download manually sbt-launch-0.13.5.jar beforehand.
In the SPARK_HOME/sbt/sbt-launch-lib.bash script replace line 53 to line 57 with following
if hash curl 2>/dev/null; then
(curl --fail --location --silent ${URL1} > ${JAR_DL} ||\
(rm -f "${JAR_DL}" && curl --fail --location --silent ${URL2} > ${JAR_DL})) && \
mv "${JAR_DL}" "${JAR}"
elif hash wget 2>/dev/null; then
(wget --quiet ${URL1} -O ${JAR_DL} ||\
(rm -f "${JAR_DL}" && wget --quiet ${URL2} -O ${JAR_DL})) &&\
mv "${JAR_DL}" "${JAR}"
else
Then try again, run the sbt assembly command
sbt/sbt assembly
Simplest method is install sbt manually as follows
download sbt deb file
wget http://dl.bintray.com/sbt/debian/sbt-0.13.5.deb
Then run
sudo dpkg -i sbt-0.13.5.deb
sudo apt-get update
sudo apt-get install sbt
then build using
sbt assembly instead of sbt/sbt assembly from spark home folder
#Frozenfire, I'am not sure if it's possible but the Spark documentation Overview says :
For the Scala API, Spark 1.4.1 uses Scala 2.10. You will need to use a
compatible Scala version (2.10.x).
And I wonder if it would be the reason why you have this problem:
I'm using Spark 1.2.0 just as a note (1.4.1 wouldn't work for me)
Because you do :
sudo dpkg -i scala-2.11.7.deb
which downloads and installs scala-2.11.7.
I don't know but this might be a clue !
PS1: this is more a comment to Frozenfire's answer, but I can't comment because of a lack of reputation and I wanted to share this.
PS2: Building for Scala 2.11
I try to include sbt into docker images. However, it never works and always stuck at Getting org.scala-sbt sbt 0.13.7 ... Also, it is also not working for changing the sbt version.
Here is the snippet of docker file
FROM ubuntu:14.04
RUN echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
RUN sudo apt-get update
RUN sudo apt-get install sbt //also I used force--yes
also, I try to install it on the container manually by using
wget http://dl.bintray.com/sbt/debian/sbt-0.13.5.deb
sudo apt-get update
sudo dpkg -i sbt-0.13.5.deb
When I run sbt compile, it also stuck trying to get Getting org.scala-sbt ...
but it is working for sbt --version
Basically, I dont know why sbt stucks trying to get Getting org.scala-sbt ...
You will need a Java Virtual Machine for SBT and so I think it's a good think you start from the java official docker image. Here is a basic docker file that uses the official ubuntu installation method:
FROM java
RUN echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list
RUN apt-key update
RUN apt-get update
RUN apt-get -y --force-yes install sbt
NOTE: --force-yes has to be there because it is not an authenticated package. I tried adding RUN apt-key update, but that didn't make a difference so you can omit this line.
Then, build a test image: docker build -t test/sbt ., create an interactive container docker run -i -t test/sbt sbt and play with it.
This works for me, but I noticed the download times were slow for launching SBT, so be patient at this step.
This is because the SBT executable in itself is really light and will fetch a bunch of libraries on the first run to accomplish its task. It's also a way for SBT to support multiple projects using multiple SBT versions. If you are stuck at libraries resolution, check your networking configuration. For SBT errors, they are mostly printed on the command line, but you can configure logging if you want.
What's left for you to figure out is to add your project files and issue a compile command to test it.
sbt will try to download a higher version of itself if the project require a higher version than the currently installed sbt version. Usually there is a {projectFolder}/project/build.properties which specifies the desired sbt version for a sbt project. for example: sbt.version=0.13.7 requires version 0.13.7
You seem to get stuck at Getting org.scala-sbt sbt 0.13.7 .... But I believe sbt is actually trying to download sbt 0.13.7 to your local. As the package is not small, depending on your network speed, it may take a while.
It is also likely that there is a network connectivity issue that prevents sbt from downloading its package. So you can try to verify first your network connectivity is not a problem.
If your network is fine, another approach you can try is to go to sbt site to download 0.13.7 package manually to your docker and install it there by following instructions you can find from sbt site.
Hope this helps.
Sometimes sbt stuck when downloading files. You can periodically check size ~/.ivy2 folder and if size isn't grow kick sbt process and rerun sbt.
For me only after 5 kicks sbt download all files!!!
When I am trying to install sbt in Ubuntu as it is suggested here:
wget http://apt.typesafe.com/repo-deb-build-0002.deb
sudo dpkg -i repo-deb-build-0002.deb
sudo apt-get update
sudo apt-get install sbt
The last command sudo apt-get install sbt asks if I want to install GJI -- GNU java interpreter.
The following extra packages will be installed:
gcj-4.5-jre gcj-4.5-jre-headless
Suggested packages:
fastjar gcj-4.5-jdk
The following NEW packages will be installed:
gcj-4.5-jre gcj-4.5-jre-headless sbt
I do not want to install it. How can I install an sbt package in Ubuntu without GCJ ?
You can go on the sbt website, download it,extract it somewhere in your home directory and add it in the path.
I have it on ~/apps/sbt/ and i added the following in the end of ~/.bashrc
export PATH=$HOME/apps/sbt/bin:$PATH
I found it the simplest way to achive this because the apt typesafe respository is outdated.