Exception when using elastic4s with elastic-search and spray-routing - scala

I'm trying to write a little REST api using Scala, Spray.io, Elastic4s and ElasticSearch.
My ES instance is running with default parameters, I just changed the parameter network.host to 127.0.0.1.
Here is my spray routing definition
package com.example
import akka.actor.Actor
import spray.routing._
import com.example.core.control.CrudController
class ServiceActor extends Actor with Service {
def actorRefFactory = context
def receive = runRoute(routes)
}
trait Service extends HttpService {
val crudController = new CrudController()
val routes = {
path("ads" / IntNumber) {
id =>
get {
ctx =>
ctx.complete(
crudController.getFromElasticSearch
)
}
}
}
}
My crudController :
package com.example.core.control
import com.example._
import org.elasticsearch.action.search.SearchResponse
import scala.concurrent._
import scala.util.{Success, Failure}
import ExecutionContext.Implicits.global
class CrudController extends elastic4s
{
def getFromElasticSearch : String = {
val something: Future[SearchResponse] = get
something onComplete {
case Success(p) => println(p)
case Failure(t) => println("An error has occured: " + t)
}
"GET received \n"
}
}
And a trait elastic4s who is encapsulating the call to elastic4s
package com.example
import com.sksamuel.elastic4s.ElasticClient
import com.sksamuel.elastic4s.ElasticDsl._
import scala.concurrent._
import org.elasticsearch.action.search.SearchResponse
trait elastic4s {
def get: Future[SearchResponse] = {
val client = ElasticClient.remote("127.0.0.1", 9300)
client execute { search in "ads"->"categories" }
}
}
This code runs well, and gives me this output :
[INFO] [03/26/2014 11:41:50.957] [on-spray-can-akka.actor.default-dispatcher-4] [akka://on-spray-can/user/IO-HTTP/listener-0] Bound to localhost/127.0.0.1:8080
But when a try to access to the route "localhost/ads/8" with my browser, the case Failure is always triggered and I got this error output on my intellij console :
An error has occured: org.elasticsearch.transport.RemoteTransportException: [Skinhead][inet[/127.0.0.1:9300]][search]
(No console output with elasticSearch running on my terminal)
Is this exception related to ElasticSearch, or am I doing wrong with my Future declaration ?

I suppose you should use ElasticClient.local in this case, as specified in elastic4s docs:
https://github.com/sksamuel/elastic4s
To specify settings for the local node you can pass in a settings object like this:
val settings = ImmutableSettings.settingsBuilder()
.put("http.enabled", false)
.put("path.home", "/var/elastic/")
val client = ElasticClient.local(settings.build)

Related

Creating functional tests Scala Playframework 2.6 Macwire

I wrote some traits to use it as a base for my functional tests
This file is for creating a DB in memory (H2 + Evolutions)
BlogApiDBTest.scala
package functional.common
import play.api.db.Databases
import play.api.db.evolutions.Evolutions
trait BlogApiDBTest {
implicit val testDatabase = Databases.inMemory(
name = "blog_db",
urlOptions = Map(
"MODE" -> "MYSQL"
),
config = Map(
"logStatements" -> true
)
)
org.h2.engine.Mode.getInstance("MYSQL").convertInsertNullToZero = false
Evolutions.applyEvolutions(testDatabase)
}
Here I am overriding some injected components for testing purposes
BlogApiComponentsTest.scala
package functional.common
import common.BlogApiComponents
import org.scalatestplus.play.components.WithApplicationComponents
import play.api.{BuiltInComponents, Configuration}
trait BlogApiComponentsTest extends WithApplicationComponents with BlogApiDBTest {
override def components: BuiltInComponents = new BlogApiComponents(context) {
override lazy val configuration: Configuration = context.initialConfiguration
override lazy val blogDatabase = testDatabase
}
}
This is the base class for my functional tests
BlogApiOneServerPerTestWithComponents.scala
package functional.common
import org.scalatestplus.play.PlaySpec
import org.scalatestplus.play.components.{OneServerPerTestWithComponents}
trait BlogApiOneServerPerTestWithComponents extends PlaySpec with OneServerPerTestWithComponents with BlogApiComponentsTest {
}
Finally the test I am trying to execute
PostControllerSpec.scala
package functional.controllers
import functional.common.BlogApiOneServerPerTestWithComponents
import org.scalatest.concurrent.{IntegrationPatience, ScalaFutures}
import play.api.mvc.{Results}
import play.api.test.{FakeRequest, Helpers}
import play.api.test.Helpers.{GET, route}
class PostControllerSpec extends BlogApiOneServerPerTestWithComponents
with Results
with ScalaFutures
with IntegrationPatience {
"Server query should" should {
"provide an Application" in {
val Some(result) = route(app, FakeRequest(GET, "/posts"))
Helpers.contentAsString(result) must be("success!")
}
}
}
Then I get
blog-api/test/functional/controllers/PostControllerSpec.scala:18:31: Cannot write an instance of play.api.mvc.AnyContentAsEmpty.type to HTTP response. Try to define a Writeable[play.api.mvc.AnyContentAsEmpty.type]
Here is the code
Adding the following import should make it work:
import play.api.test.Helpers._
Looking at the signature of route
def route[T](app: Application, req: Request[T])(implicit w: Writeable[T]): Option[Future[Result]]
we see it expects an implicit w: Writeable[T]. The above import will provide it via Writables

could not find implicit value for parameter env: com.mohiva.play.silhouette.api.Environment[utils.auth.DefaultEnv]

I'm using a Silhouette v4.0 library with play framework 2.5.
And have been trying to write test code using play specs2.
But, I get the following error with my test class as below.
Error Message
[error] could not find implicit value for parameter env: com.mohiva.play.silhouette.api.Environment[utils.auth.DefaultEnv]
.withAuthenticator[DefaultEnv](identity.loginInfo)
^
Here's the test class
package controllers
import com.google.inject.AbstractModule
import org.joda.time.DateTime
import org.specs2.specification.Scope
import org.specs2.matcher._
import org.specs2.mock._
import play.api.test._
import play.api.libs.json._
import play.api.libs.json.Json
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
import play.api.libs.concurrent.Execution.Implicits._
import play.api.libs.mailer.{ MailerClient, Email }
import play.api.inject.guice.GuiceApplicationBuilder
import play.api.inject.bind
import com.mohiva.play.silhouette.test._
import com.mohiva.play.silhouette.api._
import com.mohiva.play.silhouette.api.repositories.AuthInfoRepository
import com.mohiva.play.silhouette.api.util._
import com.mohiva.play.silhouette.impl.providers._
import net.codingwell.scalaguice.ScalaModule
import utils.auth.DefaultEnv
class TestControllerSpec extends PlaySpecification with Mockito {
"case" in new Context {
new WithApplication(application) {
val request = FakeRequest(POST, "/api/test")
.withAuthenticator[DefaultEnv](identity.loginInfo) // <-
val result = route(app, request).get
status(result) must be equalTo OK
}
}
trait Context extends Scope {
val identity = User(
loginInfo = LoginInfo(..)
..
)
implicit val env = FakeEnvironment[DefaultEnv](Seq(identity.loginInfo -> identity))
class FakeModule extends AbstractModule with ScalaModule {
def configure() = {
bind[Environment[DefaultEnv]].toInstance(env)
}
}
lazy val application = new GuiceApplicationBuilder()
.overrides(new FakeModule)
.build
}
}
There are some other test classes similar to this class are properly able to compile and execute.
It's kind of implicit problem with scope..
Therefore, I tried to import all the same as another test class which's able to compile properly. But, still unable to compile.
Missing some import?
As the compiler states, you're missing an implicit value. Use the following, which is modeled after one of Silhouette's specs:
class TestControllerSpec extends PlaySpecification with Mockito {
"the POST request" should {
"return an OK response" in new Context {
new WithApplication(application) {
val identity = User(LoginInfo(...))
implicit val env = FakeEnvironment[DefaultEnv](Seq(identity.loginInfo -> identity))
val request = FakeRequest(POST, "/api/test")
.withAuthenticator(identity.loginInfo)
val result = route(app, request).get
status(result) must be equalTo OK
}
}
}
trait Context extends Scope {
...
}
}

how to mock external WS API calls in Scala Play framework

I have an existing Scala play application which has a REST API that calls another external REST API. I want to mock the external Web service returning fake JSON data for internal tests. Based on example from: https://www.playframework.com/documentation/2.6.x/ScalaTestingWebServiceClients
I followed example exactly as in Documentation and I'm getting compiler errors due to deprecated class Action.
import play.core.server.Server
import play.api.routing.sird._
import play.api.mvc._
import play.api.libs.json._
import play.api.test._
import scala.concurrent.Await
import scala.concurrent.duration._
import org.specs2.mutable.Specification
import product.services.market.common.GitHubClient
class GitHubClientSpec extends Specification {
import scala.concurrent.ExecutionContext.Implicits.global
"GitHubClient" should {
"get all repositories" in {
Server.withRouter() {
case GET(p"/repositories") => Action {
Results.Ok(Json.arr(Json.obj("full_name" -> "octocat/Hello-World")))
}
} { implicit port =>
WsTestClient.withClient { client =>
val result = Await.result(
new GitHubClient(client, "").repositories(), 10.seconds)
result must_== Seq("octocat/Hello-World")
}
}
}
}
}
object Action in package mvc is deprecated: Inject an ActionBuilder
(e.g. DefaultActionBuilder) or extend
BaseController/AbstractController/InjectedController
And this is the primary example from latest official docs which in fact contains a compile time error, given this example doesn't work how should be the proper way to easily mock an external API using Scala Play?
You may change your example to:
Server.withRouterFromComponents() { cs => {
case GET(p"/repositories") => cs.defaultActionBuilder {
Results.Ok(Json.arr(Json.obj("full_name" -> "octocat/Hello-World")))
}
}
} { implicit port =>
WsTestClient.withClient { client =>
val result = Await.result(
new GitHubClient(client, "").repositories(), 10.seconds)
result should be(Seq("octocat/Hello-World"))
}
}
To be honest, I'm not 100% sure if this is the nicest way. However I have submitted a PR to the play framework so you might watch that space for comments from the makers.
If you're using standalone version of play-ws you can use this library https://github.com/f100ded/play-fake-ws-standalone like this
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import org.f100ded.play.fakews._
import org.scalatest._
import play.api.libs.ws.JsonBodyWritables._
import scala.concurrent.duration.Duration
import scala.concurrent._
import scala.language.reflectiveCalls
/**
* Tests MyApi HTTP client implementation
*/
class MyApiClientSpec extends AsyncFlatSpec with BeforeAndAfterAll with Matchers {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
behavior of "MyApiClient"
it should "put access token to Authorization header" in {
val accessToken = "fake_access_token"
val ws = StandaloneFakeWSClient {
case request # GET(url"http://host/v1/foo/$id") =>
// this is here just to demonstrate how you can use URL extractor
id shouldBe "1"
// verify access token
request.headers should contain ("Authorization" -> Seq(s"Bearer $accessToken"))
Ok(FakeAnswers.foo)
}
val api = new MyApiClient(ws, baseUrl = "http://host/", accessToken = accessToken)
api.getFoo(1).map(_ => succeed)
}
// ... more tests
override def afterAll(): Unit = {
Await.result(system.terminate(), Duration.Inf)
}
}

No such method error akka.util.Helpers$.toRootLowerCase(Ljava/lang/String;)Ljava/lang/String;

I am trying to run the following program
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport
import akka.http.scaladsl.marshalling.ToResponseMarshallable
import akka.http.scaladsl.model.{ContentTypes, HttpEntity}
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.server.directives.BasicDirectives
import akka.io.IO
import akka.stream.ActorMaterializer
import com.typesafe.config.ConfigFactory
import spray.json.DefaultJsonProtocol
import scala.concurrent.duration._
import scala.concurrent.Await
trait JsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
implicit val itemFormat = jsonFormat2(Item)
}
final case class Item(name: String, id: Long)
object Main extends App with RestInterface {
val config = ConfigFactory.load()
val host = config.getString("http.host")
val port = config.getInt("http.port")
implicit val system = ActorSystem("My-ActorSystem")
implicit val executionContext = system.dispatcher
implicit val materializer = ActorMaterializer()
//implicit val timeout = Timeout(10 seconds)
val api = routes
Http().bindAndHandle(api, host, port) map {
binding => println(s"The binding local address is ${binding.localAddress}")
}
// recover
// { case ex => println(s"some shit occurred and it is"+ex.getMessage) }
// sys.addShutdownHook {
// system.log.info("Shutting down Spray HTTP in 10 seconds...")
// // NOTE(soakley): We are waiting to unbind to allow the VIP time to query healthcheck status.
// Thread.sleep(10000)
// system.log.info("Shutting down Spray HTTP and allowing 10 seconds for connections to drain...")
// val unbindFuture = Http.unbind(10.seconds)
//
// Await.ready(unbindFuture, 10.seconds)
// }
}
case class Stringer(str1 : String, str2 : String, str3 : String)
trait RestInterface extends Resource {
val routes = questionroutes ~ mydirectiveroute01 ~ mydirectiveroute02
}
trait Resource extends QuestionResource with mydirective01 with mydirective02
trait QuestionResource {
val questionroutes = {
path("hi") {
get {
val stringer_instance = Stringer("questionairre created","whatever","whatever-whatever")
complete(ToResponseMarshallable(stringer_instance.toString()))
}
}
}
}
trait mydirective01 extends BasicDirectives {
val getandputdirective = get | put
val mydirectiveroute01 = {
path("customhi" / IntNumber) {
passednumber1 => {
path(IntNumber) {
passednumber2 => {
getandputdirective {
ctx =>
ctx.complete("Testing get and put directive" + s"The passed string is " + passednumber2 + passednumber1)
}
}
}
}
}
}
}
trait mydirective02 extends BasicDirectives with JsonSupport {
val mydirectiveroute02 =
path("hello") {
get {
complete(HttpEntity(
ContentTypes.`text/html(UTF-8)`,
"<h1>Say hello to akka-http</h1>"))
}
} ~
path("randomitem") {
get {
// will marshal Item to JSON
complete(Item("thing", 42))
}
} ~
path("saveitem") {
post {
// will unmarshal JSON to Item
entity(as[Item]) { item =>
println(s"Server saw Item : $item")
complete(item)
}
}
}
}
But I get the following error
Exception in thread "main" java.lang.NoSuchMethodError: akka.util.Helpers$.toRootLowerCase(Ljava/lang/String;)Ljava/lang/String;
The error seems to be because of this line
implicit val materializer = ActorMaterializer()
May I know where I am going wrong. I have identical pom.xml file and the program in another module and it seems to run fine. I dont understand where I am going wrong. Thanks
I get errors like that when something that should have been re-compiled has not been re-compiled. clean" followed by "compile" in sbt resolves it.
The issue is due to the conflict in dependencies make sure you are using same version for all the related packages you are importing.
example:
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-http-core_2.11</artifactId>
<version>10.0.0</version>
</dependency>
2.11 make sure you have other dependencies with same version
then make a clean build.
Your project dependencies have conflicting versions. The version that has that function gets discarded, and the other version included in the jar (or classpath - depending on how you execute).
Please provide your dependencies file (i.e. sbt.build or gradle.build etc.)

Scala compile error: value is not member of

Sorry for my bad english!
I'm newbie in scala language, I'm compiling a project with some modifications and getting the error
"scala value is not member of"... some body can help me? follow my code:
Command.scala
package com.bla.cms.domain
import scala.collection.Map
import redis.clients.jedis.Jedis
sealed trait CommandResult
object Success extends CommandResult
object Failure extends CommandResult
object Unauthorized extends CommandResult
trait Command {
def captchaValidator:Validator
def logger:Logging
val properties = PropertiesHandler.readProperties(System.getProperty(PropertiesHandler.configFileNameKey))
}
RedisIcrementCommand.scala
package com.bla.cms
package domain
import scala.collection.Map
import redis.clients.jedis.JedisPool
import dispatch._
import org.apache.log4j.Logger
import redis.clients.jedis.Jedis
class RedisIncrementCommand(database: Jedis, val captchaValidator:Validator, val logger:Logging) extends Command {
def execute(parameters: Map[String,String]):CommandResult = {
captchaValidator.validate(parameters) match {
case Right(params) =>
executeInDatabase(params)
case Left(status) =>
logger.info("Falha ao tentar executar comando. Captcha inválido. Status: " + status)
Failure
}
}
def executeInDatabase(parameters: Map[String,String]):CommandResult = {
parameters.get("selectedAnswer") match {
case Some(selectedAnswer) =>
database.incr(selectedAnswer)
database.sadd(Answers.key, selectedAnswer)
logger.info("Incremento recebido com sucesso. Opção: " + selectedAnswer)
Success
case None =>
logger.info("Falha ao tentar computar incremento. Alternativa não informada.")
Failure
}
}
}
PollServle.scala
package com.bla.cms
import javax.servlet.http._
import redis.clients.jedis.JedisPool
import scala.collection.JavaConverters._
import scala.collection.JavaConversions._
import com.bla.cms.domain.Logging
import org.apache.log4j.Logger
import scala.util.matching.Regex
class PollServlet extends HttpServlet with RequiresConnection {
import com.bla.cms.domain._
import URLHandler._
override def doPost(request:HttpServletRequest, response:HttpServletResponse):Unit = {
val parametersMap = new ServletApiAdapter(request,response).parameters
val outcome = newIncrementCommand(request).execute(parametersMap)
redirect(request, response, outcome)
}
}
And when I compile, I've got:
[error]PollServlet.scala:19: value execute is nota member of com.bla.cms.domain.Command
val outcome = newIncrementCommand(request).execute(parametersMap)
Somebody can help me please?
You can:
Add def execute(parameters: Map[String,String]):CommandResult to Command trait.
Change newIncrementCommand(request) to return RedisIncrementCommand instead of Command.
Cast result of newIncrementCommand(request) to RedisIncrementCommand using asInstanceOf.
I prefer 1 which would be:
trait Command {
val captchaValidator: Validator
val logger: Logging
val properties = PropertiesHandler.readProperties(System.getProperty(PropertiesHandler.configFileNameKey))
def execute(parameters: Map[String,String]): CommandResult
def executeInDatabase(parameters: Map[String,String]): CommandResult
}