I tried to create a simple RestHelper "Hello world" example but I'm having trouble. When I start my container with container:start command, the
serve {
case Nil Get _ => Extraction.decompose("Hello Restful world!")
}
is not invoked in my RestHelper extension. I get the following message:
"The Requested URL / was not found on this server"
So, it seems that for some reason lift ignores
LiftRules.statelessDispatch.append(Service)
line in bootstrap.Boot.boot definition. And I have absolutely no clue why it happens.
Here's my Boot class:
package bootstrap
import net.liftweb.http.LiftRules
import com.yac.restfultest.Service
class Boot {
def boot {
LiftRules.statelessDispatch.append(Service)
}
}
And here's Service:
package com.yac.restfultest
import net.liftweb.http.rest.RestHelper
import net.liftweb.json.Extraction
object Service extends RestHelper {
serve {
case Nil Get _ => Extraction.decompose("Hello Restful world!")
}
}
and in case it helps here's my web.xml:
<!DOCTYPE web-app SYSTEM "http://java.sun.com/dtd/web-app_2_3.dtd">
<web-app>
<filter>
<filter-name>LiftFilter</filter-name>
<display-name>Lift Filter</display-name>
<description>The Filter that intercepts Lift calls</description>
<filter-class>net.liftweb.http.LiftFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>LiftFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
</web-app>
and in case all of the above is not enough here is my build.sbt:
name := "TestRest"
version := "1.0"
scalaVersion := "2.11.6"
resolvers ++= Seq("snapshots" at "https://oss.sonatype.org/content/repositories/snapshots",
"releases" at "https://oss.sonatype.org/content/repositories/releases"
)
jetty()
libraryDependencies ++= {
val liftVersion = "2.6.2"
Seq(
"net.liftweb" %% "lift-webkit" % liftVersion % "compile"
)
}
As you can see it's almost the most minimalistic lift project setup possible. Still I can't get it working. Any help would be appreciated.
And here's sbt log on container:start:
[info] Compiling 1 Scala source to /home/yac/IdeaProjects/TestRest/target/scala-2.11/classes...
[info] Packaging /home/yac/IdeaProjects/TestRest/target/scala-2.11/testrest_2.11-1.0.jar ...
[info] Done packaging.
[info] starting server ...
[success] Total time: 2 s, completed Apr 22, 2015 7:51:25 PM
> 2015-04-22 19:51:25.640:INFO::main: Logging initialized #44ms
2015-04-22 19:51:25.646:INFO:oejr.Runner:main: Runner
2015-04-22 19:51:25.726:INFO:oejs.Server:main: jetty-9.2.1.v20140609
2015-04-22 19:51:29.818:WARN:oeja.AnnotationConfiguration:main: ServletContainerInitializers: detected. Class hierarchy: empty
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2015-04-22 19:51:30.405:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext#32e377c7{/,file:/home/yac/IdeaProjects/TestRest/target/webapp/,AVAILABLE}{file:/home/yac/IdeaProjects/TestRest/target/webapp/}
2015-04-22 19:51:30.406:WARN:oejsh.RequestLogHandler:main: !RequestLog
2015-04-22 19:51:30.417:INFO:oejs.ServerConnector:main: Started ServerConnector#7a601e4{HTTP/1.1}{0.0.0.0:8080}
2015-04-22 19:51:30.418:INFO:oejs.Server:main: Started #4848ms
So, turns out it's a usual noobish non-attentiveness. The package, containing Boot.scala should be called bootstrap.liftweb and not just bootstrap as in my case.
And yes, as suggested by jcern it should be "index" :: Nil in routing pattern.
Related
I've written unit tests referring to DataframeGenerator example, which allows you to generate mock dataframes on the fly
After having executed the following commands successfully
sbt clean
sbt update
sbt compile
I get the errors shown in output upon running either of the following commands
sbt assembly
sbt test -- -oF
Output
...
[info] SearchClicksProcessorTest:
17/11/24 14:19:04 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/11/24 14:19:07 WARN SparkContext: Using an existing SparkContext; some configuration may not take effect.
17/11/24 14:19:18 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
17/11/24 14:19:18 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException
17/11/24 14:19:19 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
[info] - testExplodeMap *** FAILED ***
[info] ExceptionInInitializerError was thrown during property evaluation.
[info] Message: "None"
[info] Occurred when passed generated values (
[info]
[info] )
[info] - testFilterByClicks *** FAILED ***
[info] NoClassDefFoundError was thrown during property evaluation.
[info] Message: Could not initialize class org.apache.spark.rdd.RDDOperationScope$
[info] Occurred when passed generated values (
[info]
[info] )
[info] - testGetClicksData *** FAILED ***
[info] NoClassDefFoundError was thrown during property evaluation.
[info] Message: Could not initialize class org.apache.spark.rdd.RDDOperationScope$
[info] Occurred when passed generated values (
[info]
[info] )
...
[info] *** 3 TESTS FAILED ***
[error] Failed: Total 6, Failed 3, Errors 0, Passed 3
[error] Failed tests:
[error] com.company.spark.ml.pipelines.search.SearchClicksProcessorTest
[error] (root/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 73 s, completed 24 Nov, 2017 2:19:28 PM
Things that I've tried unsuccessfully
Running sbt test with F flag to show full stacktrace (no stacktrace output appears as shown above)
Re-build the project in IntelliJ Idea
My questions are
What could be the possible cause of this error?
How can I enable the stack-trace output in SBT to be able to debug it?
EDIT-1
My unit-test class contains several methods like below
class SearchClicksProcessorTest extends FunSuite with Checkers {
import spark.implicits._
test("testGetClicksData") {
val schemaIn = StructType(List(
StructField("rank", IntegerType),
StructField("city_id", IntegerType),
StructField("target", IntegerType)
))
val schemaOut = StructType(List(
StructField("clicked_res_rank", IntegerType),
StructField("city_id", IntegerType),
))
val dataFrameGen = DataframeGenerator.arbitraryDataFrame(spark.sqlContext, schemaIn)
val property = Prop.forAll(dataFrameGen.arbitrary) { dfIn: DataFrame =>
dfIn.cache()
val dfOut: DataFrame = dfIn.transform(SearchClicksProcessor.getClicksData)
dfIn.schema === schemaIn &&
dfOut.schema === schemaOut &&
dfIn.filter($"target" === 1).count() === dfOut.count()
}
check(property)
}
}
while build.sbt looks like this
// core settings
organization := "com.company"
scalaVersion := "2.11.11"
name := "repo-name"
version := "0.0.1"
// cache options
offline := false
updateOptions := updateOptions.value.withCachedResolution(true)
// aggregate options
aggregate in assembly := false
aggregate in update := false
// fork options
fork in Test := true
//common libraryDependencies
libraryDependencies ++= Seq(
scalaTest,
typesafeConfig,
...
scalajHttp
)
libraryDependencies ++= allAwsDependencies
libraryDependencies ++= SparkDependencies.allSparkDependencies
assemblyMergeStrategy in assembly := {
case m if m.toLowerCase.endsWith("manifest.mf") => MergeStrategy.discard
...
case _ => MergeStrategy.first
}
lazy val module-1 = project in file("directory-1")
lazy val module-2 = (project in file("directory-2")).
dependsOn(module-1).
aggregate(module-1)
lazy val root = (project in file(".")).
dependsOn(module-2).
aggregate(module-2)
P.S. Please read comments on original question before reading this answer
Even the popular solution of overriding SBT's transitive dependency over faster-xml.jackson didn't work for me; in that some more changes were required (ExceptionInInitializerError was gone but some other error cropped up)
Finally (in addition to above mentioned fix) I ended up creating DataFrames in a different way (as opposed to StructType used here). I created them as
spark.sparkContext.parallelize(Seq(MyType)).toDF()
where MyType is a case class as per the schema of DataFrame
While implementing this solution, I encountered a small problem that while datatypes of schema generated from case class were correct, the nullability of fields was often mismatching; fix for this issue was found here
Here I'm blatantly admitting that I'm not sure what was the correct fix: faster-xml.jackson dependency or the alternate way of creating DataFrame, so please feel free to fill the lapses in understanding / investigating the issue
i have had a similar problem case, and after investigating I found out, that adding a lazy before a val solved my issue. My estimate is, running a Scala program with Scalatest invokes a little different initializing sequence. Where a normal scala execution initializes vals in an sourecode line numbers top-down order - having nested object {...} blocks initialized in the same way - using the same coding with Scalatest, the execution initializes the valss in nested object { ... } blocks before the vals line-number wise above the object { ... }.
This is absolutely vague I know but deferring initialization with prefixing vals with lazy could solve the test issue here.
The crucial thing here is that it doesn't occur in normal execution, only test execution, and in my case it was only occuring when using lambdas with taps in this form:
...
.tap(x =>
hook_feld_erweiterungen_hook(
abc = theProblematicVal
)
)
...
I'm trying to write a test case for a simple REST API in Play2/Scala that send/receives JSON. My test looks like the following:
import org.junit.runner.RunWith
import org.specs2.matcher.JsonMatchers
import org.specs2.mutable._
import org.specs2.runner.JUnitRunner
import play.api.libs.json.{Json, JsArray, JsValue}
import play.api.test.Helpers._
import play.api.test._
import play.test.WithApplication
/**
* Add your spec here.
* You can mock out a whole application including requests, plugins etc.
* For more information, consult the wiki.
*/
#RunWith(classOf[JUnitRunner])
class APIv1Spec extends Specification with JsonMatchers {
val registrationJson = Json.parse("""{"device":"576b9cdc-d3c3-4a3d-9689-8cd2a3e84442", |
"firstName":"", "lastName":"Johnny", "email":"justjohnny#test.com", |
"pass":"myPassword", "acceptTermsOfService":true}
""")
def dropJsonElement(json : JsValue, element : String) = (json \ element).get match {
case JsArray(items) => util.dropAt(items, 1)
}
def invalidRegistrationData(remove : String) = {
dropJsonElement(registrationJson,remove)
}
"API" should {
"Return Error on missing first name" in new WithApplication {
val result= route(
FakeRequest(
POST,
"/api/v1/security/register",
FakeHeaders(Seq( ("Content-Type", "application/json") )),
invalidRegistrationData("firstName").toString()
)
).get
status(result) must equalTo(BAD_REQUEST)
contentType(result) must beSome("application/json")
}
...
However when I attempt to run sbt test, I get the following error:
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0
[info] Loading project definition from /home/cassius/brentspace/esalestracker/project
[info] Set current project to eSalesTracker (in build file:/home/cassius/brentspace/esalestracker/)
[info] Compiling 3 Scala sources to /home/cassius/brentspace/esalestracker/target/scala-2.11/test-classes...
[error] /home/cassius/brentspace/esalestracker/test/APIv1Spec.scala:34: could not find implicit value for evidence parameter of type org.specs2.main.CommandLineAsResult[play.test.WithApplication{val result: scala.concurrent.Future[play.api.mvc.Result]}]
[error] "Return Error on missing first name" in new WithApplication {
[error] ^
[error] one error found
[error] (test:compileIncremental) Compilation failed
[error] Total time: 3 s, completed 18/01/2016 9:30:42 PM
I have similar tests in other applications, but it looks like the new version of specs adds a lot of support for Futures and other things that invalidate previous tutorials. I'm on Scala 2.11.6, Activator 1.3.6 and my build.sbt looks like the following:
name := """eSalesTracker"""
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.11.6"
libraryDependencies ++= Seq(
jdbc,
cache,
ws,
"com.typesafe.slick" %% "slick" % "3.1.0",
"org.postgresql" % "postgresql" % "9.4-1206-jdbc42",
"org.slf4j" % "slf4j-api" % "1.7.13",
"ch.qos.logback" % "logback-classic" % "1.1.3",
"ch.qos.logback" % "logback-core" % "1.1.3",
evolutions,
specs2 % Test,
"org.specs2" %% "specs2-matcher-extra" % "3.7" % Test
)
resolvers += "scalaz-bintray" at "http://dl.bintray.com/scalaz/releases"
resolvers += Resolver.url("Typesafe Ivy releases", url("https://repo.typesafe.com/typesafe/ivy-releases"))(Resolver.ivyStylePatterns)
// Play provides two styles of routers, one expects its actions to be injected, the
// other, legacy style, accesses its actions statically.
routesGenerator := InjectedRoutesGenerator
I think you are using the wrong WithApplication import.
Use this one:
import play.api.test.WithApplication
The last line of the testcase should be the assertion/evaluation statement.
e.g. before the last } of the failing testcase method put the statement false must beEqualTo(true) and run it again.
I have used specs2 many times successfully in vanilla SBT projects. now I am starting to learn typesafe activator platform.
I did the following steps
activator new Shop just-play-scala
this is my build.sbt file
name := """Shop"""
version := "1.0-SNAPSHOT"
// Read here for optional jars and dependencies
libraryDependencies ++= Seq("org.specs2" %% "specs2-core" % "3.6.1" % "test")
resolvers += "scalaz-bintray" at "http://dl.bintray.com/scalaz/releases"
scalacOptions in Test ++= Seq("-Yrangepos")
lazy val root = project.in(file(".")).enablePlugins(PlayScala)
I created a file Shop/app/test/models/ShopSpec.scala
import org.specs2.mutable.Specification
class ShopSpec extends Specification {
def foo = s2"""
| This is a specification to check the 'Hello world' string
| The 'Hello world' string should
| contain 11 characters $e1
| start with 'Hello' $e2
| end with 'world' $e3
| """.stripMargin
def e1 = "Hello world" must haveSize(11)
def e2 = "Hello world" must startWith("Hello")
def e3 = "Hello world" must endWith("world")
}
When I run activator test I get an error
[success] Total time: 0 s, completed Jun 24, 2015 12:21:32 AM
Mohitas-MBP:Shop abhi$ activator test
[info] Loading project definition from /Users/abhi/ScalaProjects/Shop/project
[info] Set current project to Shop (in build file:/Users/abhi/ScalaProjects/Shop/)
**cannot create a JUnit XML printer. Please check that specs2-junit.jar is on the classpath**
org.specs2.reporter.JUnitXmlPrinter$
java.net.URLClassLoader.findClass(URLClassLoader.java:381)
java.lang.ClassLoader.loadClass(ClassLoader.java:424)
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.jav
I have previously written spec2 test cases successfully when I was using SBT projects. but only when I use the typesafe activator that I get this issue with test cases.
I even changed the code of my test to something as simple as
import org.specs2.mutable.Specification
class ShopSpec extends Specification {
"A shop " should {
"create item" in {
failure
}
}
}
But still the same problem.
Wait .. I think I resolved it.
The activator play platform already has specs2 included so there is no need for me to tweak the built.sbt file for specs 2.
So I removed everything I had added to build.sbt file and left the file as
name := """Shop"""
version := "1.0-SNAPSHOT"
lazy val root = project.in(file(".")).enablePlugins(PlayScala)
Now it works fine. So basically, I don't need to add anything in a activator project for specs2.
I could have deleted the question... but leaving it here so that it can be of help to someone.
What worked for me was adding the following to build.sbt:
libraryDependencies ++= Seq("org.specs2" %% "specs2-core" % "3.6.2" % "test",
"org.specs2" %% "specs2-junit" % "3.6.2" % "test")
I have a play application and I need to ignore my functional tests when I compile and build, then later run only the integration tests.
This is my test code:
ApplicationSpec.scala
#RunWith(classOf[JUnitRunner])
class ApplicationSpec extends Specification {
"Application" should {
"send 404 on a bad request" in new WithApplication {
route(FakeRequest(GET, "/boum")) must beNone
}
}
IntegrationSpec.scala
#RunWith(classOf[JUnitRunner])
class IntegrationSpec extends Specification {
"Application" should {
"work from within a browser" in {
running(TestServer(9000), FIREFOX) { browser =>
browser.goTo("http://localhost:9000/app")
browser.pageSource must contain("Your new application is ready.")
}
}
} section "integration"
}
The docs tells me I can use something like this from the command line:
play "test-only -- exclude integration"
The only problem is that this doesn't actually exclude any tests and my integration tests invoke firefox and start running. What am I doing wrong? How can I exclude the integration tests and then later run them by themselves?
sbt's Testing documentation describes several ways to define separate test configurations.
For instance, the Additional test configurations with shared sources example could be used in this situation:
In this approach, the sources are compiled together using the same classpath and are packaged together. However, different tests are run depending on the configuration.
In a default Play Framework 2.2.2 application created with play new, add a file named project/Build.scala with this content:
import sbt._
import Keys._
object B extends Build {
lazy val root =
Project("root", file("."))
.configs(FunTest)
.settings(inConfig(FunTest)(Defaults.testTasks) : _*)
.settings(
libraryDependencies += specs,
testOptions in Test := Seq(Tests.Filter(unitFilter)),
testOptions in FunTest := Seq(Tests.Filter(itFilter))
)
def itFilter(name: String): Boolean = (name endsWith "IntegrationSpec")
def unitFilter(name: String): Boolean = (name endsWith "Spec") && !itFilter(name)
lazy val FunTest = config("fun") extend(Test)
lazy val specs = "org.specs2" %% "specs2" % "2.0" % "test"
}
To run standard unit tests:
$ sbt test
Which produces:
[info] Loading project definition from /home/fernando/work/scratch/so23160453/project
[info] Set current project to so23160453 (in build file:/home/fernando/work/scratch/so23160453/)
[info] ApplicationSpec
[info] Application should
[info] + send 404 on a bad request
[info] + render the index page
[info] Total for specification ApplicationSpec
[info] Finished in 974 ms
[info] 2 examples, 0 failure, 0 error
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 3 s, completed Apr 22, 2014 12:42:37 PM
To run tests for the added configuration (here, "fun"), prefix it with the configuration name:
$ sbt fun:test
Which produces:
[info] Loading project definition from /home/fernando/work/scratch/so23160453/project
[info] Set current project to so23160453 (in build file:/home/fernando/work/scratch/so23160453/)
[info] IntegrationSpec
[info] Application should
[info] + work from within a browser
[info] Total for specification IntegrationSpec
[info] Finished in 0 ms
[info] 1 example, 0 failure, 0 error
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 6 s, completed Apr 22, 2014 12:43:17 PM
You can also test only a particular class, like this:
$ sbt "fun:testOnly IntegrationSpec"
See this example in GitHub.
I have following SBT/Play2 multi-project setup:
import sbt._
import Keys._
import PlayProject._
object ApplicationBuild extends Build {
val appName = "traveltime-api"
val appVersion = "1.0"
val appDependencies = Seq(
// Google geocoding library
"com.google.code.geocoder-java" % "geocoder-java" % "0.9",
// Emailer
"org.apache.commons" % "commons-email" % "1.2",
// CSV generator
"net.sf.opencsv" % "opencsv" % "2.0",
"org.scalatest" %% "scalatest" % "1.7.2" % "test",
"org.scalacheck" %% "scalacheck" % "1.10.0" % "test",
"org.mockito" % "mockito-core" % "1.9.0" % "test"
)
val lib = RootProject(file("../lib"))
val chiShape = RootProject(file("../chi-shape"))
lazy val main = PlayProject(
appName, appVersion, appDependencies, mainLang = SCALA
).settings(
// Add your own project settings here
resolvers ++= Seq(
"Sonatype Snapshots" at
"http://oss.sonatype.org/content/repositories/snapshots",
"Sonatype Releases" at
"http://oss.sonatype.org/content/repositories/releases"
),
// Scalatest compatibility
testOptions in Test := Nil
).aggregate(lib, chiShape).dependsOn(lib, chiShape)
}
As you can see this project depends on two independant subprojects: lib and chiShape.
Now compile works fine - all sources are correctly compiled. However if I try run or test, neither task in runtime has classes from subprojects on classpath loaded and things go haywire with NoClassFound exceptions.
For example - my application has to load serialized data from file and it goes like this: test starts FakeApplication, it tries to load data and boom:
[info] CsvGeneratorsTest:
[info] #markerFilterCsv
[info] - should fail on bad json *** FAILED ***
[info] java.lang.ClassNotFoundException: com.library.Node
[info] at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
[info] at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
[info] at java.security.AccessController.doPrivileged(Native Method)
[info] at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
[info] at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
[info] at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
[info] at java.lang.Class.forName0(Native Method)
[info] at java.lang.Class.forName(Class.java:264)
[info] at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:622)
[info] at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1593)
[info] ...
Strangely enough stage creates a directory structure with chi-shapes_2.9.1-1.0.jar and lib_2.9.1-1.0.jar in staged/.
How can I get my runtime/test configurations get subprojects into classpath?
Update:
I've added following code to Global#onStart:
override def onStart(app: Application) {
println(app)
ClassLoader.getSystemClassLoader.asInstanceOf[URLClassLoader].getURLs.
foreach(println)
throw new RuntimeException("foo!")
}
When I launch tests, the classpath is very very ill populated, to say at least :)
FakeApplication(.,sbt.classpath.ClasspathUtilities$$anon$1#182253a,List(),List(),Map(application.load-data -> test, mailer.smtp.test-mode -> true))
file:/home/arturas/Software/sdks/play-2.0.3/framework/sbt/sbt-launch.jar
[info] CsvGeneratorsTest:
When launching staged app, there's a lot of stuff, how it's supposed to be :)
$ target/start
Play server process ID is 29045
play.api.Application#1c2862b
file:/home/arturas/work/traveltime-api/api/target/staged/jul-to-slf4j.jar
That's strange, because there should be at least testing jars in the classpath I suppose?
It seems I've solved it.
The culprit was that ObjectInputStream ignores thread local class loaders by default and only uses system class loader.
So I changed from:
def unserialize[T](file: File): T = {
val in = new ObjectInputStream(new FileInputStream(file))
try {
in.readObject().asInstanceOf[T]
}
finally {
in.close
}
}
To:
/**
* Object input stream which respects thread local class loader.
*
* TL class loader is used by SBT to avoid polluting system class loader when
* running different tasks.
*/
class TLObjectInputStream(in: InputStream) extends ObjectInputStream(in) {
override protected def resolveClass(desc: ObjectStreamClass): Class[_] = {
Option(Thread.currentThread().getContextClassLoader).map { cl =>
try { return cl.loadClass(desc.getName)}
catch { case (e: java.lang.ClassNotFoundException) => () }
}
super.resolveClass(desc)
}
}
def unserialize[T](file: File): T = {
val in = new TLObjectInputStream(new FileInputStream(file))
try {
in.readObject().asInstanceOf[T]
}
finally {
in.close
}
}
And my class not found problems went away!
Thanks to How to put custom ClassLoader to use? and http://tech-tauk.blogspot.com/2010/05/thread-context-classlaoder-in.html on useful insight about deserializing and thread local class loaders.
This sounds similar to this bug https://play.lighthouseapp.com/projects/82401/tickets/659-play-dist-broken-with-sub-projects, though that bug is about dist and not test. I think that the fix has not made it to the latest stable release, so try building Play from source (and don't forget to use aggregate and dependsOn as demonstrated in that link.
Alternatively, as a workaround, inside sbt, you can navigate to the sub-project with project lib and then type test. It's a bit manual, but you can script that if you'd like.