I started to use a project/Settings.scala to help clean up my main build.sbt. This is a scalajs project with Play backend where I use Play's WS & Cache dependencies. In built.sbt the 'string/keyword/' <-not sure correct term here but ws and cache properly resolve. However in my Seq[String] in Settings.scala where I store my server library dependencies they won't. Currently I am using
libraryDependencies ++= Seq(ws, cache) ++ Settings.jvmDependencies.value,
which works but it makes me wonder how I might be able to move everything to Settings or if that is possible. When I dig one layer deeper in IDE I see for example ws is defined as :
val ws : sbt.ModuleID = { /* compiled code */ }
in the object PlayImport but I can't see the proper values to populate a full/typical dependency definition for sbt
Ultimately I am curious can I successfully export ws & cache such that I can have this line in my build.sbt
libraryDependencies ++= Settings.jvmDependencies.value,
You can have a direct look at the Play SBT plugin sources, it's usually the easiest way. Here's how the ws defined:
val ws = component("play-ahc-ws")
where component is defined in the same file like this:
def component(id: String) = "com.typesafe.play" %% id % play.core.PlayVersion.current
With this information we know the ws dependency amounts to "com.typesafe.play" %% "play-ahc-ws" % "2.5.10" for the current Play version.
If you want to have all the Play symbols in your plugin - which is what your project/Settings.scala file is - just import the fields from the Play plugin's autoImport member:
import play.sbt.Play.autoImport._
This will let you use ws, cache, and any other symbols the plugin exposes.
Related
I want to build a 'fat' jar of my code. I understand how to do this mostly but all the examples I have use the idea that the jar is not local and I am not sure how to include into my assembled jar another JAR that I built that the scala code uses. Like what folder does this JAR I have to include reside in?
Normally when I run my current code as a test using spark-shell it looks like this:
spark-shell --jars magellan_2.11-1.0.6-SNAPSHOT.jar -i st_magellan_abby2.scala
(the jar file is right in the same path as the .scala file)
So now I want to build a build.sbt file that does the same and includes that SNAPSHOT.jar file?
name := "PSGApp"
version := "1.0"
scalaVersion := "2.11.8"
resolvers += "Spark Packages Repo" at "http://dl.bintray.com/spark-packages/maven"
//provided means don't included it is there. already on cluster?
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.2.0" % "provided",
"org.apache.spark" %% "spark-sql" % "2.2.0" % "provided",
"org.apache.spark" %% "spark-streaming" % "2.2.0" % "provided",
//add magellan here somehow?
)
So where would I put the jar in the SBT project folder structure so it gets picked up when I run sbt assembly? Is that in the main/resources folder? Which the reference manual says is where 'files to include in the main jar' go?
What would I put in the libraryDependencies here so it knows to add that specific jar and not go out into the internet to get it?
One last thing, I was also doing some imports in my test code that doesn't seem to fly now that I put this code in an object with a def main attached to it.
I had things like:
import sqlContext.implicits._ which was right in the code above where it was about to be used like so:
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext.implicits._
import org.apache.spark.sql.functions.udf
val distance =udf {(a: Point, b: Point) =>
a.withinCircle(b, .001f); //current radius set to .0001
}
I am not sure can I just keep these imports inside the def main? or do I have to move them elsewhere somehow? (Still learning scala and wrangling the scoping I guess).
One way is to build your fat jar using the assembly plugin (https://github.com/sbt/sbt-assembly) locally and publishLocal to store the resulting jar into your local ivy2 cache
This will make it available for inclusion in your other project based on build.sbt settings in this project, eg:
name := "My Project"
organization := "org.me"
version := "0.1-SNAPSHOT"
Will be locally available as "org.me" %% "my-project" % "0.1-SNAPSHOT"
SBT will search local cache before trying to download from external repo.
However, this is considered bad practise, because only final project should ever be a fat-jar. You should never include one as dependency (many headaches).
There is no reason to make project magellan a fat-jar if library is included in PGapp. Just publishLocal without assembly
Another way is to make projects dependant on each other as code, not library.
lazy val projMagellan = RootProject("../magellan")
lazy val projPSGApp = project.in(file(".")).dependsOn(projMagellan)
This makes compilation in projPSGApp tigger compilation in projMagellan.
It depends on your use case though.
Just don't get in a situation where you have to manage your .jar manually
The other question:
import sqlContext.implicits._ should always be included in the scope where dataframe actions are required, so you shouldn't put that import near the other ones in the header
Update
Based on discussion in comments, my advise would be:
Get the magellan repo
git clone git#github.com:harsha2010/magellan.git
Create a branch to work on, eg.
git checkout -b new-stuff
Change the code you want
Then update the versioning number, eg.
version := "1.0.7-SNAPSHOT"
Publish locally
sbt publishLocal
You'll see something like (after a while):
[info] published ivy to /Users/tomlous/.ivy2/local/harsha2010/magellan_2.11/1.0.7-SNAPSHOT/ivys/ivy.xml
Go to your other project
Change build.sbt to include
"harsha2010" %% "magellan" % "1.0.7-SNAPSHOT" in your libraryDependencies
Now you have a good (temp) reference to your library.
Your PSGApp should be build as an fat jar assembly to pass to Spark
sbt clean assembly
This will pull in the custom build jar
If the change in the magellan project is usefull for the rest of the world, you should push your changes and create a pull request, so that in the future you can just include the latest build of this library
I am reading this article
https://playframework.github.io/play-soap/SbtWsdl.html
and based on this. I wrote the following build.sbt file
name := "PlaySOAPClient"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
"com.typesafe.play" % "play-soap-client_2.11" % "1.1.3"
)
WsdlKeys.packageName := Some("com.foo")
WsdlKeys.wsdlTasks in Compile := Seq(
WsdlKeys.WsdlTask((sourceDirectory.value / "main" / "wsdl" / "foo.wsdl").toURI.toURL)
)
and plugins.sbt
resolvers += Resolver.url("play-sbt-plugins", url("https://dl.bintray.com/playframework/sbt-plugin-releases/"))(Resolver.ivyStylePatterns)
addSbtPlugin("com.typesafe.sbt" % "sbt-play-soap" % "1.1.3")
When I do sbt compile the plugin does generate some code. but that code does not compile
Error:scalac: missing or invalid dependency detected while loading class file 'PlaySoapClient.class'.
Could not access type Configuration in value play.api,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'PlaySoapClient.class' was compiled against an incompatible version of play.api.
Warning:scalac: Class javax.inject.Singleton not found - continuing with a stub.
/Users/User/IdeaProjects/PlaySOAPClient/target/scala-2.11/wsdl/main/sources/com/foo/webservices/FooWebService.scala
Error:(13, 8) object inject is not a member of package javax
#javax.inject.Singleton
Error:(14, 107) object api is not a member of package play
class FooWebService #javax.inject.Inject() (apacheCxfBus: play.soap.ApacheCxfBus, configuration: play.api.Configuration) extends play.soap.PlaySoapClient(apacheCxfBus, configuration) {
Error:(14, 32) object inject is not a member of package javax
class FooWebService #javax.inject.Inject() (apacheCxfBus: play.soap.ApacheCxfBus, configuration: play.api.Configuration) extends play.soap.PlaySoapClient(apacheCxfBus, configuration) {
Does anyone have an idea of what dependencies are missing. Note that this is a client side application only using this library to make a soap call. I don't want any server side dependencies of play framework.
My hope is that I will be able to use the play-soap as a standalone library in my console application to make soap calls.
Add dependency to build.sbt
libraryDependencies += "com.typesafe.play" %% "play" % "2.6.7" intransitive()
Then sbt compile should work (after sbt update).
Honestly, including the entire Play Framework into the application purely for the sake of WSDL client seems to be too much. All you need is to generate annotated Java beans and make a dependency just on them. And you can actually do that with common tools i.e. using Java's wsimport and sbt tasks to wrap it around.
Consider the following bootstrap template for this: https://github.com/sainnr/sbt-scala-wsdl-template. It generates all the boilerplate in-flight, compiles before the main sbt project and eliminates the need to commit this boilerplate Java code to your pristine Scala repo. If you notice, it doesn't even require a complete application server, just throw in some JavaEE-ish libs like rt.jar or its alternatives. Hope that helps someone.
I'm trying to make my first tests with Scala and with Play framework.
I have installed play 2.2.0, which seems to be the last version, with the standalone package. After that, I've been able to create a new application, compile and run it.
I've tried to start to use Anorm package to access to the database, but I've found a blocking error which I can't find on the docs. I don't know if that means that is so obvious, but after adding:
package controllers
import play.api._
import play.api.mvc._
import play.db.anorm._ //(this is the new line)
object Application extends Controller {
def index = Action {
Ok(views.html.index("Your new application is ready."))
}
}
It fails with:
object db is not a member of package play
I've seen this:
https://groups.google.com/forum/#!msg/play-framework/RUbmEVsw2rY/UOE5mNs1WjoJ
Where they talk about adding the dependency to jdbc, which seems to be already in my build.sbt.
libraryDependencies ++= Seq(
jdbc,
anorm,
cache
)
I've also found this thread here:
play.db package not found in play 2.1.3
But I can't find a build.scala file on my project. Not using any IDE by now, just play console (run & compile commands).
Thanks a lot!
In fact (as the error explains), there is no package play.db.anorm._ in version 2.2.0. Try use import anorm._ instead.
You need the following libraries
slick
play-jdbc
anorm
This is how my dependencies look like in build.sbt :
libraryDependencies ++= Seq(
"com.typesafe.slick" % "slick_2.10" % "2.1.0",
"org.postgresql" % "postgresql" % "9.4-1201-jdbc41",
"com.typesafe.play" % "play-jdbc_2.10" % "2.4.0-RC1",
cache,
anorm
)
Search for the latest version of library at Maven Central Repository
I want to use http://dispatch.databinder.net/Dispatch.html .
The site indicates I must add this to project/plugins.sbt:
libraryDependencies += "net.databinder.dispatch" %% "core" % "0.9.1"
which I did. I then restarted the play console and compiled.
Importing doesnt work:
import dispatch._
Guess I have been silly, but then I never used a build system when using Java.
How must I trigger the process that downloads/builds the package? Where are the jars (or equivalent) stored; can I reuse them? When is the package available for use by the Play application?
It doesn't say you should add it to project/plugins.sbt. That is the wrong place. It says to add to the build.sbt file, on the root of your project. Being a Play project, project/Build.scala might be more appropriate -- I don't know if it will pick up settings from build.sbt or not.
To add the dependency in your Build.scala:
val appDependencies = Seq(
"net.databinder.dispatch" %% "core" % "0.9.1"
)
You probably need to run sbt update.
From the sbt Command Line Reference:
update Resolves and retrieves external dependencies as described in library dependencies.
I want to develop a rather simple web application in Scala, with Lift or Play framework, using Eclipse as an environment and some simple webserver like Jetty. However, to my inexpressibly great surprise, I cannot setup the whole thing to work together. I also could not find any sensible simple and clear guide on how to do this. After a half day of searching, I came to an opinion that everyone around seem to use a mix of sbt/maven and feel it ok migrating from one to another and writing project config files manually just to get a simple blank project to begin with.
There is no plain simple way to create even an empty project. With Java I remember that it was a couple of clicks - to integrate a webserver into Eclipse, create a simple web app project and run it right from there. Where had gone the power and simplicity of Scala in this case? And that's only if I want to try Lift. What if I would like to try Play also, should I travel the same path again?
Is there anywhere a simple and complete guide that describes how to setup the environment so that it is possible to start developing the apps right away?
UPDATE: I have reached a successful Play project integration with Eclipse, with all the capabilities that Play has out-of-the-box, thanks to the advice of Peter Gwiazda. I am using this setup for developing right now. However, my question of interest remains: what are other ways to acheive similar functionality with other frameworks such as Lift, Scalatra and others?
With Playframework 2.0 it's pretty simple. Just follow the tutorial
http://www.playframework.org/documentation/2.0/ScalaTodoList
With Play you don't need anything else - Play already contains a server.
IMHO Play is way easier to work with than Lift.
EDIT
OK, you asked for it ;-)
Here's a bleeding edge setup for Scalatra with SBT Coffeescript & LESS (see HERE for SBT-Eclipse dependency management)
1) eclipsify a test project
2) in project root create "build.sbt" file:
import AssemblyKeys._
import Keys._
name := "your project name"
version := "1.0"
scalaVersion := "2.9.1"
fork in run := true
resolvers ++= Seq(
"Sonatype OSS Snapshots" at "http://oss.sonatype.org/content/repositories/snapshots/",
"Typesafe repository" at "http://typesafe.artifactoryonline.com/typesafe/ivy-releases/"
)
seq(webSettings :_*)
seq(assemblySettings: _*)
seq(coffeeSettings: _*)
seq(lessSettings:_*)
(LessKeys.mini in (Compile, LessKeys.less)) := false
libraryDependencies ++= Seq(
"org.scalatra" %% "scalatra" % "2.1.0-SNAPSHOT",
"org.scalatra" %% "scalatra-scalate" % "2.1.0-SNAPSHOT",
"org.scalatra" %% "scalatra-lift-json" % "2.1.0-SNAPSHOT",
"org.scalatra" %% "scalatra-anti-xml" % "2.1.0-SNAPSHOT",
"org.scalatra" %% "scalatra-fileupload" % "2.1.0-SNAPSHOT",
"org.eclipse.jetty" % "jetty-webapp" % "8.1.0.RC2" % "test;container;provided",
"javax.servlet" % "javax.servlet-api" % "3.0.1" % "provided"
)
unmanagedBase <<= baseDirectory { base => base / "/src/main/webapp/WEB-INF/lib/" }
3) create folder "project" in root with plugins.sbt file:
libraryDependencies <+= sbtVersion(v => "com.github.siasia" %% "xsbt-web-plugin" % (v+"-0.2.10"))
resolvers += Resolver.url("sbt-plugin-releases", new URL("http://scalasbt.artifactoryonline.com/scalasbt/sbt-plugin-releases/"))(Resolver.ivyStylePatterns)
addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "2.0.0-M3")
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.7.2")
addSbtPlugin("me.lessis" % "coffeescripted-sbt" % "0.2.2")
addSbtPlugin("me.lessis" % "less-sbt" % "0.1.9")
4) from terminal/command prompt start sbt & enable code reloading:
# sbt
> ~;container:start;container:reload /
Open up the Scalatra Book and start hacking ;-)
ORIGINAL
Have to mention it, but a micro framework a la Scalatra, Spray, or Unfiltered might be of interest as well.
That is, if you're not looking for the kitchen sink that Lift and Play provide; if you are looking for the kitchen sink and want to get rolling quickly, Play 2.0 does indeed look quite nice.
Disclaimer: I'm a member of the Vaadin team.
You could also try out Vaadin which works perfectly with Scala, HOWTO here. You can also use Maven or SBT if you want. You should also check out Scaladin, the semi-official Scala wrapper for Vaadin.
Vaadin is a component based library (just one JAR with no dependencies) and it allows you to create your Ajax and HTML5 enabled UI in pure Scala without any HTML templates, RPC, XML or JavaScript.
You could use my Maven prototype for Scalatra, then simply import the maven project into Eclipse. Quite nice and you're not forced to use SBT.
https://github.com/fancellu/scalatra-maven-prototype
You can have a look on my Github repo where I have a project that uses Lift and Jetty (as an embedded server). It's not well documented yet but is small enough to grasp how it's working
P4G Server Repo
The server starts from com.p4g.Server object (which is called within com.p4g.Main Application object )
My Lift boostrap object is located in boostrap.liftweb package
BTW, I'm also using ScalaQuery and ScalaZ