Stubbing SOAP requests in Scala - scala

I use scalaxb to generate models and client part of the SOAP interface. For testing I use Betamax, which can also be used in Scala. However, scalaxb uses Netty as a transport, which ignores proxy settings set up by Betamax. How would you cope with this situation?
scalaxb uses cake pattern, so the service is built from 3 parts like in the following example:
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent._
import scala.concurrent.duration._
val service = (new stockquote.StockQuoteSoap12Bindings with
scalaxb.SoapClientsAsync with
scalaxb.DispatchHttpClientsAsync {}).service
val fresponse = service.getQuote(Some("GOOG"))
val response = Await.result(fresponse, 5 seconds)
println(response)
And tests:
import co.freeside.betamax.{TapeMode, Recorder}
import co.freeside.betamax.proxy.jetty.ProxyServer
import dispatch._
import org.scalatest.{Tag, FunSuite}
import scala.concurrent.duration._
import scala.concurrent.{Await, Future}
class StockquoteSpec extends FunSuite with Betamax {
testWithBetamax("stockquote", Some(TapeMode.READ_WRITE))("stockquote") {
val fresponse = service.getQuote(Some("GOOG"))
val response = Await.result(fresponse, 5 seconds)
println(response)
}
}
trait Betamax {
protected def test(testName: String, testTags: Tag*)(testFun: => Unit)
def testWithBetamax(tape: String, mode: Option[TapeMode] = None)(testName: String, testTags: Tag*)(testFun: => Unit) = {
test(testName, testTags: _*) {
val recorder = new Recorder
val proxyServer = new ProxyServer(recorder)
recorder.insertTape(tape)
recorder.getTape.setMode(mode.getOrElse(recorder.getDefaultMode()))
proxyServer.start()
try {
testFun
} finally {
recorder.ejectTape()
proxyServer.stop()
}
}
}
}
Versions:
net.databinder.dispatch 0.11.2
co.freeside.betamax 1.1.2
com.ning.async-http-client 1.8.10
io.netty.netty 3.9.2.Final

It is indeed possible to use proxy with Netty. Although Netty does not read system properties for proxy settings, the settings can be injected using ProxyServerSelector. It is created in build method of AsyncHttpClientConfig:
if (proxyServerSelector == null && useProxySelector) {
proxyServerSelector = ProxyUtils.getJdkDefaultProxyServerSelector();
}
if (proxyServerSelector == null && useProxyProperties) {
proxyServerSelector = ProxyUtils.createProxyServerSelector(System.getProperties());
}
if (proxyServerSelector == null) {
proxyServerSelector = ProxyServerSelector.NO_PROXY_SELECTOR;
}
The only obstacle is that scalaxb uses default config with useProxyProperties=false. You can override it with custom MyDispatchHttpClientsAsync that you can use when creating the service:
val service = (new stockquote.StockQuoteSoap12Bindings with
scalaxb.SoapClientsAsync with
MyDispatchHttpClientsAsync {}).service
And the source code of MyDispatchHttpClientsAsync (the key point is calling setUseProxyProperties(true)):
import com.ning.http.client.providers.netty.NettyAsyncHttpProvider
import com.ning.http.client.{AsyncHttpClientConfig, AsyncHttpClient}
import scalaxb.HttpClientsAsync
/**
* #author miso
*/
trait MyDispatchHttpClientsAsync extends HttpClientsAsync {
lazy val httpClient = new DispatchHttpClient {}
trait DispatchHttpClient extends HttpClient {
import dispatch._, Defaults._
// Keep it lazy. See https://github.com/eed3si9n/scalaxb/pull/279
lazy val http = new Http(new AsyncHttpClient(new NettyAsyncHttpProvider(new AsyncHttpClientConfig.Builder().setUseProxyProperties(true).build())))
// lazy val http = Http.configure(_.setUseProxyProperties(true)) // Maybe later. See https://github.com/eed3si9n/scalaxb/issues/312
def request(in: String, address: java.net.URI, headers: Map[String, String]): concurrent.Future[String] = {
val req = url(address.toString).setBodyEncoding("UTF-8") <:< headers << in
http(req > as.String)
}
}
}

Related

Akka HTTP - max-open-requests and substreams?

I'm writing an app using Scala 2.13 with Akka HTTP 10.2.4 and Akka Stream 2.6.15. I'm trying to query a web service in a parallel manner, like so:
package com.example
import akka.actor.typed.scaladsl.ActorContext
import akka.http.scaladsl.Http
import akka.http.scaladsl.client.RequestBuilding.Get
import akka.http.scaladsl.model.HttpResponse
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.stream.scaladsl.{Flow, JsonFraming, Sink, Source}
import spray.json.DefaultJsonProtocol
import spray.json.DefaultJsonProtocol.jsonFormat2
import scala.util.Try
case class ClientStockPortfolio(id: Long, symbol: String)
case class StockTicker(symbol: String, price: Double)
trait SprayFormat extends DefaultJsonProtocol {
implicit val stockTickerFormat = jsonFormat2(StockTicker)
}
class StockTrader(context: ActorContext[_]) extends SprayFormat {
implicit val system = context.system.classicSystem
val httpPool = Http().superPool()[Seq[ClientStockPortfolio]]
def collectPrices() = {
val src = Source(Seq(
ClientStockPortfolio(1, "GOOG"),
ClientStockPortfolio(2, "AMZN"),
ClientStockPortfolio(3, "MSFT")
)
)
val graph = src
.groupBy(8, _.id % 8)
.via(createPost)
.via(httpPool)
.via(decodeTicker)
.mergeSubstreamsWithParallelism(8)
.to(
Sink.fold(0.0) { (totalPrice, ticker) =>
insertIntoDatabase(ticker)
totalPrice + ticker.price
}
)
graph.run()
}
def createPost = Flow[ClientStockPortfolio]
.grouped(10)
.map { port =>
(
Get(uri = s"http://wherever/?symbols=${port.map(_.symbol).mkString(",")}"),
port
)
}
def decodeTicker = Flow[(Try[HttpResponse], Seq[ClientStockPortfolio])]
.flatMapConcat { x =>
x._1.get.entity.dataBytes
.via(JsonFraming.objectScanner(Int.MaxValue))
.mapAsync(4)(bytes => Unmarshal(bytes).to[StockTicker])
.mapConcat { ticker =>
lookupPreviousPrices(ticker)
}
}
def lookupPreviousPrices(ticker: StockTicker): List[StockTicker] = ???
def insertIntoDatabase(ticker: StockTicker) = ???
}
I have two questions. First, will the groupBy call that splits the stream into substreams run them in parallel like I want? And second, when I call this code, I run into the max-open-requests error, since I haven't increased the setting from the default. But even if I am running in parallel, I'm only running 8 threads - how is the Http().superPool() getting backed up with 32 requests?

how to mock external WS API calls in Scala Play framework

I have an existing Scala play application which has a REST API that calls another external REST API. I want to mock the external Web service returning fake JSON data for internal tests. Based on example from: https://www.playframework.com/documentation/2.6.x/ScalaTestingWebServiceClients
I followed example exactly as in Documentation and I'm getting compiler errors due to deprecated class Action.
import play.core.server.Server
import play.api.routing.sird._
import play.api.mvc._
import play.api.libs.json._
import play.api.test._
import scala.concurrent.Await
import scala.concurrent.duration._
import org.specs2.mutable.Specification
import product.services.market.common.GitHubClient
class GitHubClientSpec extends Specification {
import scala.concurrent.ExecutionContext.Implicits.global
"GitHubClient" should {
"get all repositories" in {
Server.withRouter() {
case GET(p"/repositories") => Action {
Results.Ok(Json.arr(Json.obj("full_name" -> "octocat/Hello-World")))
}
} { implicit port =>
WsTestClient.withClient { client =>
val result = Await.result(
new GitHubClient(client, "").repositories(), 10.seconds)
result must_== Seq("octocat/Hello-World")
}
}
}
}
}
object Action in package mvc is deprecated: Inject an ActionBuilder
(e.g. DefaultActionBuilder) or extend
BaseController/AbstractController/InjectedController
And this is the primary example from latest official docs which in fact contains a compile time error, given this example doesn't work how should be the proper way to easily mock an external API using Scala Play?
You may change your example to:
Server.withRouterFromComponents() { cs => {
case GET(p"/repositories") => cs.defaultActionBuilder {
Results.Ok(Json.arr(Json.obj("full_name" -> "octocat/Hello-World")))
}
}
} { implicit port =>
WsTestClient.withClient { client =>
val result = Await.result(
new GitHubClient(client, "").repositories(), 10.seconds)
result should be(Seq("octocat/Hello-World"))
}
}
To be honest, I'm not 100% sure if this is the nicest way. However I have submitted a PR to the play framework so you might watch that space for comments from the makers.
If you're using standalone version of play-ws you can use this library https://github.com/f100ded/play-fake-ws-standalone like this
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import org.f100ded.play.fakews._
import org.scalatest._
import play.api.libs.ws.JsonBodyWritables._
import scala.concurrent.duration.Duration
import scala.concurrent._
import scala.language.reflectiveCalls
/**
* Tests MyApi HTTP client implementation
*/
class MyApiClientSpec extends AsyncFlatSpec with BeforeAndAfterAll with Matchers {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
behavior of "MyApiClient"
it should "put access token to Authorization header" in {
val accessToken = "fake_access_token"
val ws = StandaloneFakeWSClient {
case request # GET(url"http://host/v1/foo/$id") =>
// this is here just to demonstrate how you can use URL extractor
id shouldBe "1"
// verify access token
request.headers should contain ("Authorization" -> Seq(s"Bearer $accessToken"))
Ok(FakeAnswers.foo)
}
val api = new MyApiClient(ws, baseUrl = "http://host/", accessToken = accessToken)
api.getFoo(1).map(_ => succeed)
}
// ... more tests
override def afterAll(): Unit = {
Await.result(system.terminate(), Duration.Inf)
}
}

No such method error akka.util.Helpers$.toRootLowerCase(Ljava/lang/String;)Ljava/lang/String;

I am trying to run the following program
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport
import akka.http.scaladsl.marshalling.ToResponseMarshallable
import akka.http.scaladsl.model.{ContentTypes, HttpEntity}
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.server.directives.BasicDirectives
import akka.io.IO
import akka.stream.ActorMaterializer
import com.typesafe.config.ConfigFactory
import spray.json.DefaultJsonProtocol
import scala.concurrent.duration._
import scala.concurrent.Await
trait JsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
implicit val itemFormat = jsonFormat2(Item)
}
final case class Item(name: String, id: Long)
object Main extends App with RestInterface {
val config = ConfigFactory.load()
val host = config.getString("http.host")
val port = config.getInt("http.port")
implicit val system = ActorSystem("My-ActorSystem")
implicit val executionContext = system.dispatcher
implicit val materializer = ActorMaterializer()
//implicit val timeout = Timeout(10 seconds)
val api = routes
Http().bindAndHandle(api, host, port) map {
binding => println(s"The binding local address is ${binding.localAddress}")
}
// recover
// { case ex => println(s"some shit occurred and it is"+ex.getMessage) }
// sys.addShutdownHook {
// system.log.info("Shutting down Spray HTTP in 10 seconds...")
// // NOTE(soakley): We are waiting to unbind to allow the VIP time to query healthcheck status.
// Thread.sleep(10000)
// system.log.info("Shutting down Spray HTTP and allowing 10 seconds for connections to drain...")
// val unbindFuture = Http.unbind(10.seconds)
//
// Await.ready(unbindFuture, 10.seconds)
// }
}
case class Stringer(str1 : String, str2 : String, str3 : String)
trait RestInterface extends Resource {
val routes = questionroutes ~ mydirectiveroute01 ~ mydirectiveroute02
}
trait Resource extends QuestionResource with mydirective01 with mydirective02
trait QuestionResource {
val questionroutes = {
path("hi") {
get {
val stringer_instance = Stringer("questionairre created","whatever","whatever-whatever")
complete(ToResponseMarshallable(stringer_instance.toString()))
}
}
}
}
trait mydirective01 extends BasicDirectives {
val getandputdirective = get | put
val mydirectiveroute01 = {
path("customhi" / IntNumber) {
passednumber1 => {
path(IntNumber) {
passednumber2 => {
getandputdirective {
ctx =>
ctx.complete("Testing get and put directive" + s"The passed string is " + passednumber2 + passednumber1)
}
}
}
}
}
}
}
trait mydirective02 extends BasicDirectives with JsonSupport {
val mydirectiveroute02 =
path("hello") {
get {
complete(HttpEntity(
ContentTypes.`text/html(UTF-8)`,
"<h1>Say hello to akka-http</h1>"))
}
} ~
path("randomitem") {
get {
// will marshal Item to JSON
complete(Item("thing", 42))
}
} ~
path("saveitem") {
post {
// will unmarshal JSON to Item
entity(as[Item]) { item =>
println(s"Server saw Item : $item")
complete(item)
}
}
}
}
But I get the following error
Exception in thread "main" java.lang.NoSuchMethodError: akka.util.Helpers$.toRootLowerCase(Ljava/lang/String;)Ljava/lang/String;
The error seems to be because of this line
implicit val materializer = ActorMaterializer()
May I know where I am going wrong. I have identical pom.xml file and the program in another module and it seems to run fine. I dont understand where I am going wrong. Thanks
I get errors like that when something that should have been re-compiled has not been re-compiled. clean" followed by "compile" in sbt resolves it.
The issue is due to the conflict in dependencies make sure you are using same version for all the related packages you are importing.
example:
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-http-core_2.11</artifactId>
<version>10.0.0</version>
</dependency>
2.11 make sure you have other dependencies with same version
then make a clean build.
Your project dependencies have conflicting versions. The version that has that function gets discarded, and the other version included in the jar (or classpath - depending on how you execute).
Please provide your dependencies file (i.e. sbt.build or gradle.build etc.)

Play 2.4.1 injecting Logging filter as in given example

I try to implement the LoggingFilter as in the example given on https://www.playframework.com/documentation/2.4.1/ScalaHttpFilters with Play 2.4.1 .
FilterConfiguration
import javax.inject.Inject
import play.api.http.HttpFilters
class FiltersConfiguaration #Inject()(loggingFilter:com.holidaycheck.filters.RequestLoggingFilter) extends HttpFilters {
val filters = Seq(loggingFilter)
}
Filter
import play.api.Logger
import play.api.mvc._
import play.api.libs.concurrent.Execution.Implicits.defaultContext
object RequestLoggingFilter extends EssentialFilter {
def apply(nextFilter: EssentialAction) = new EssentialAction {
def apply(requestHeader: RequestHeader) = {
val startTime = System.currentTimeMillis
nextFilter(requestHeader).map { result =>
val endTime = System.currentTimeMillis
val requestTime = endTime - startTime
Logger.info(s"${requestHeader.method} ${requestHeader.uri}" +
s" took ${requestTime}ms and returned ${result.header.status}")
result.withHeaders("Request-Time" -> requestTime.toString)
}
}
}
}
configuration
I do as given in the explanation, also giving the filters in the application.conf.
play.http.filters = com.holidaycheck.filters.FiltersConfiguaration
But I get the following Error:
What am I missing here ?
You are trying to inject an object in your FiltersConfiguaration. Make RequestLoggingFilter a class should solve your problem.

How do you set the viewport, width, with phantomjs with play 2 framework

I have been struggling with this for a while, I can't find a way to instruct phantomjs about the viewport. I am using play 2.2 (code is based on: Set Accept-Language on PhantomJSDriver in a Play Framework Specification )
package selenium
import org.specs2.mutable.Around
import org.specs2.specification.Scope
import play.api.test.{TestServer, TestBrowser, FakeApplication}
import org.openqa.selenium.remote.DesiredCapabilities
import org.specs2.execute.{Result, AsResult}
import play.api.test.Helpers._
import scala.Some
import org.openqa.selenium.phantomjs.PhantomJSDriver
import org.openqa.selenium.NoSuchElementException
import java.io.File
import java.io.PrintWriter
import java.util.concurrent.TimeUnit
import collection.JavaConversions._
abstract class WithPhantomJS(val additionalOptions: Map[String, String] = Map()) extends Around with Scope {
implicit def app = FakeApplication()
implicit def port = play.api.test.Helpers.testServerPort
// for phantomjs
lazy val browser: TestBrowser = {
val defaultCapabilities = DesiredCapabilities.phantomjs
print(defaultCapabilities.toString)
val additionalCapabilities = new DesiredCapabilities(mapAsJavaMap(additionalOptions))
val capabilities = new DesiredCapabilities(defaultCapabilities, additionalCapabilities)
val driver = new PhantomJSDriver(capabilities)
val br = TestBrowser(driver, Some("http://localhost:" + port))
br.webDriver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS)
br
}
override def around[T: AsResult](body: => T):Result = {
try {
running(TestServer(port, FakeApplication()))(AsResult.effectively(body))
} catch {
case e: Exception => {
val fn: String = "/tmp/test_" + e.getClass.getCanonicalName;
browser.takeScreenShot(fn + ".png")
val out = new PrintWriter(new File(fn + ".html"), "UTF-8");
try {
out.print(browser.pageSource())
} finally {
out.close
}
throw e;
}
} finally {
browser.quit()
}
}
}
and the actual test looks like this:
class ChangeName extends Specification {
"User" should {
"be able to change his name in account settings" in new WithPhantomJS() {
// ... this throws
I'd like the exception handler to render the site, which works, however, i can see the with matching a mobile device (narrow).
How can I change the width?
This seems to work:
import org.openqa.selenium.Dimension
...
browser.webDriver.manage().window().setSize(new Dimension(1280, 800))