Given the following code snippet
class Config {
private val conf = ConfigFactory.load()
object Http {
val host = conf.getString("http.host")
val port = conf.getInt("http.port")
}
}
how can host and port fields of inner singleton object Http be mocked?
mockito-scala provides ReturnsDeepStubs via IdiomaticMockito
import org.mockito.stubbing.ReturnsDeepStubs
import org.mockito.{ArgumentMatchersSugar, IdiomaticMockito}
val config = mock[Config](ReturnsDeepStubs)
config.Http.host returns "www.example.com"
config.Http.port returns 80
Without deep stubbing we could do
import org.scalatest.mockito.MockitoSugar
import org.mockito.Mockito.when
val config = mock[Config]
val httpConfig = mock[config.Http.type]
when(httpConfig.host).thenReturn("www.example.com")
when(httpConfig.port).thenReturn(80)
when(config.Http).thenReturn(httpConfig)
Related
I've found several resources that provide details on configuring ssl-config options within the application.conf file and I've identified how to access these configurations using AkkaSSLConfig.get(). I've seen that an https context can be created using a AkkaSSLConfig object as a parameter to ConnectionContext.https().
Is it possible to use this for non-http servers? Is the context returned somehow specific to http? I'm trying to take advantage of ssl-config but it isn't clear to me that it provides any advantages for non-http servers and I don't see any convenient way of building a context from the ssl-config definition, in which case it seems I may as well define the context manually.
Lastly, any examples of building the context for non-http servers are difficult to find. It seems the process may be the same as for http servers, but I'm finding that examples often include the use of classes/methods that have 'http' in the name. If anyone knows of a good example I'd be very appreciative.
import java.io.{File, FileInputStream}
import java.security.{KeyStore, SecureRandom}
import akka.actor.ActorSystem
import akka.http.scaladsl.Http.ServerBinding
import akka.http.scaladsl.model.{HttpResponse, StatusCodes}
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.server.Route
import akka.http.scaladsl.server.Directives.pathSingleSlash
import akka.http.scaladsl.{ConnectionContext, Http}
import akka.stream.{ActorMaterializer, TLSClientAuth}
import com.typesafe.sslconfig.akka.AkkaSSLConfig
import com.typesafe.sslconfig.ssl.{KeyManagerConfig, KeyManagerFactoryWrapper, KeyStoreConfig, SSLConfigFactory, SSLConfigSettings}
import javax.net.ssl.{SSLContext, TrustManagerFactory}
import scala.concurrent.{ExecutionContext, Future}
object Test extends App{
implicit val actorSystem: ActorSystem = ActorSystem("test")
implicit val materializer: ActorMaterializer = ActorMaterializer()
implicit val executionContext: ExecutionContext = actorSystem.dispatcher
val ksConfig: KeyStoreConfig = KeyStoreConfig.apply(data = None,
filePath = Some("/Users/mshaik/testApp/src/main/resources/keystore/localhost.p12")
).withPassword(Some("test"))
val kmConfig: KeyManagerConfig = KeyManagerConfig().withKeyStoreConfigs(List(ksConfig))
val sslConfigSettings: SSLConfigSettings = SSLConfigFactory.defaultConfig.withKeyManagerConfig(kmConfig)
val akkaSSLConfig: AkkaSSLConfig = AkkaSSLConfig.get(actorSystem).withSettings(sslConfigSettings)
val ks: KeyStore = KeyStore.getInstance("PKCS12")
ks.load(new FileInputStream(new File(ksConfig.filePath.get)), ksConfig.password.get.toCharArray)
val kmf: KeyManagerFactoryWrapper = akkaSSLConfig.buildKeyManagerFactory(sslConfigSettings)
kmf.init(ks, ksConfig.password.get.toCharArray)
val tmf: TrustManagerFactory = TrustManagerFactory.getInstance("SunX509")
tmf.init(ks)
val sslContext: SSLContext = SSLContext.getInstance("TLS")
sslContext.init(kmf.getKeyManagers, tmf.getTrustManagers, new SecureRandom)
val ctx: ConnectionContext = ConnectionContext.https(sslContext,
sslConfig = Some(akkaSSLConfig),
clientAuth = Some(TLSClientAuth.Want)
)
var bindingFuture: Future[ServerBinding] = _
Http().setDefaultServerHttpContext(ctx)
val route: Route = pathSingleSlash {
get {
complete(HttpResponse(StatusCodes.OK, entity = "Welcome to base path!"))
}
}
try{
bindingFuture = Http().bindAndHandle(route, "localhost", 8085, connectionContext = ctx)
println( s"Server online at https://localhost:8085/")
} catch {
case ex: Exception =>
println(this.getClass, ex.getMessage, ex)
materializer.shutdown()
actorSystem.terminate()
}
}
I believe the answer to my question is that there isn't much use in thoroughly configuring TLS options within ssl-config when creating a non-HTTP TLS connection.
Not a single example I found shows how to define keystore and truststore parameters within the config and then use those configurations to create the SSLContext object (all examples configure the keystore/truststore parameters manually, within the code). Ultimately I found it wasn't useful to use ssl-config for storing configurations. The only place I found it useful is to obtain the list of default ciphers and default protocols (and hence I still use it in my code).
For reference, below is what I ended up doing to configure the context and initial session structure and create the TCP server. This is very similar to other examples found within documentation as well as some responses here on SO. Some differences in this response: 1) This requires client certificates, 2) This is for a server (as opposed to a client), 3) This code shows how to use factory methods to create the TLS BidiFlow (note the Tcp().bindTls call) 4) This allows you to pass in the Flow that will handle the incoming communications.
object TcpServerBindTls extends StrictLogging {
def apply(hostInterface: String, tcpPort: Int, handler: Flow[ByteString, ByteString, NotUsed])(implicit system: ActorSystem, materializer: ActorMaterializer) = {
val sslContext = buildSSLContext
val firstSession = prepareFirstSession(sslContext)
val connections: Source[Tcp.IncomingConnection, Future[Tcp.ServerBinding]] = Tcp().bindTls(hostInterface, tcpPort, sslContext, firstSession)
connections runForeach { connection =>
logger.info(s"New connection: ${connection}")
connection.handleWith(handler)
}
}
def prepareFirstSession(sslContext: SSLContext)(implicit system: ActorSystem) = {
val sslConfig = AkkaSSLConfig.get(system);
val config = sslConfig.config;
val defaultParams = sslContext.getDefaultSSLParameters();
val defaultProtocols = defaultParams.getProtocols();
val defaultCiphers = defaultParams.getCipherSuites();
val clientAuth = TLSClientAuth.need
defaultParams.setProtocols(defaultProtocols)
defaultParams.setCipherSuites(defaultCiphers)
val firstSession = new TLSProtocol.NegotiateNewSession(None, None, None, None)
.withCipherSuites(defaultCiphers: _*)
.withProtocols(defaultProtocols: _*)
.withParameters(defaultParams)
firstSession
}
def buildSSLContext: SSLContext = {
val bufferedSource = io.Source.fromFile("/path/to/password/file")
val keyStorePassword = bufferedSource.getLines.mkString
bufferedSource.close
val keyStore = KeyStore.getInstance("PKCS12");
val keyStoreLocation = "/path/to/keystore/file/server.p12"
val keyStoreFIS = new FileInputStream(keyStoreLocation)
keyStore.load(keyStoreFIS, keyStorePassword.toCharArray())
val trustStore = KeyStore.getInstance("PKCS12");
val trustStoreLocation = settings.tls.keyStoreLocation;
val trustStoreFIS = new FileInputStream(keyStoreLocation)
trustStore.load(trustStoreFIS, keyStorePassword.toCharArray())
val kmf = KeyManagerFactory.getInstance("SunX509")
kmf.init(keyStore, keyStorePassword.toCharArray())
val tmf = TrustManagerFactory.getInstance("SunX509")
tmf.init(trustStore)
val sslContext = SSLContext.getInstance("TLS")
sslContext.init(kmf.getKeyManagers, tmf.getTrustManagers, new SecureRandom())
sslContext
}
}
I am rewriting some application layer code in scala from using scalaj to akka-http
in order to reduce the number of third party dependencies in the project (we already use akka for other things in the same project.) The code simply wraps common types of request to an underlying general request provided by the library
Mostly it has been fine, but I am stuck on the problem of optionally adding a proxy to a request.
Requests should either be direct to the destination or via a proxy, determined by a parameter at runtime.
In my scalaj implementation, I have the following helper class and methods
object HttpUtils {
private def request(
host: Host,
method: HttpMethod,
params: Map[String, String],
postData: Option[String],
timeout: Duration,
headers: Seq[(String, String)],
proxy: Option[ProxyConfig]
): HttpResponse[String] = {
// most general request builder. Other methods in the object fill in parameters and wrap this in a Future
val baseRequest = Http(host.url)
val proxiedRequest = addProxy(proxy, baseRequest)
val fullRequest = addPostData(postData)(proxiedRequest)
.method(method.toString)
.params(params)
.headers(headers)
.option(HttpOptions.connTimeout(timeout.toMillis.toInt))
.option(HttpOptions.readTimeout(timeout.toMillis.toInt))
fullRequest.asString // scalaj for send off request and block until response
}
// Other methods ...
private def addProxy(proxy: Option[ProxyConfig], request: HttpRequest): HttpRequest =
proxy.fold(request)((p: ProxyConfig) => request.proxy(p.host, p.port))
}
case class ProxyConfig(host: String, port: Int)
Is there a way to build a similar construct with akka-http?
Akka HTTP does have proxy support that, as of version 10.0.9, is still unstable. Keeping in mind that the API could change, you could do something like the following to handle optional proxy settings:
import java.net.InetSocketAddress
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.http.scaladsl.{ClientTransport, Http}
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
case class ProxyConfig(host: String, port: Int)
val proxyConfig = Option(ProxyConfig("localhost", 8888))
val clientTransport =
proxyConfig.map(p => ClientTransport.httpsProxy(InetSocketAddress.createUnresolved(p.host, p.port)))
.getOrElse(ClientTransport.TCP)
val settings = ConnectionPoolSettings(system).withTransport(clientTransport)
Http().singleRequest(HttpRequest(uri = "https://google.com"), settings = settings)
In Akka Http 10.2.0, use bindflow for a Flow[HttpRequest, HttpResponse, NotUsed] defined by a RunnableGraph with Flowshape. Insided the RunnableGraph, an Http() outgoingConnection is used to connect to the remote proxy. Some example code:
import akka.actor.typed.ActorSystem
import akka.actor.typed.scaladsl.Behaviors
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.{HttpRequest, HttpResponse}
import akka.stream._
import akka.stream.scaladsl.{Broadcast, Flow, GraphDSL, Merge}
import scala.concurrent.ExecutionContextExecutor
import scala.concurrent.duration.DurationInt
import scala.io.StdIn
import scala.util.{Failure, Success}
object Main {
def main(args: Array[String]) {
implicit val system: ActorSystem[Nothing] = ActorSystem(Behaviors.empty, "testproxy")
implicit val executionContext: ExecutionContextExecutor = system.executionContext
system.log.info("TestAkkaHttpProxy Main started...")
val remoteHost = "xxx.xxx.xxx.x"
val remotePort = 8000
val proxyHost = "0.0.0.0"
val proxyPort = 8080
val gateway = Flow.fromGraph(GraphDSL.create() { implicit b =>
import GraphDSL.Implicits._
// Broadcast for flow input
val broadcaster = b.add(Broadcast[HttpRequest](1))
// Merge for flow output
val responseMerge = b.add(Merge[HttpResponse](1))
// outgoing client for remote proxy
val remote = Http().outgoingConnection(remoteHost, remotePort)
// filter out header that creates Akka Http warning
val requestConvert = Flow[HttpRequest]
.map(req => { req.mapHeaders(headers => headers.filter(h => h.isNot("timeout-access")))
})
// connect graph
broadcaster.out(0) ~> requestConvert ~> remote ~> responseMerge
// expose ports
FlowShape(broadcaster.in, responseMerge.out)
})
// Akka Http server that binds to Flow (for remote proxy)
Http().newServerAt(proxyHost, proxyPort).bindFlow(gateway)
.onComplete({
case Success(binding) ⇒
println(s"Server is listening on 0.0.0.0:8080")
binding.addToCoordinatedShutdown(hardTerminationDeadline = 10.seconds)
case Failure(e) ⇒
println(s"Binding failed with ${e.getMessage}")
system.terminate()
})
system.log.info("Press RETURN to stop...")
StdIn.readLine()
system.terminate()
}
}
Trying to create a schema into cassandra using phantom-dsl for unit testing following this tutorial:
http://outworkers.com/blog/post/phantom-tips-3-understanding-phantom-connectors
I ran into this issue when trying to auto-generate schema
[ERROR] /home/.../test/BaseCassandraSpec.scala:54: error: not enough arguments for method autocreate: (keySpace: com.websudos.phantom.connectors.KeySpace)
com.websudos.phantom.builder.query.CreateQuery.Default[com.neruti.db.models.ConcreteUserModel,com.neruti.User].
[ERROR] Unspecified value parameter keySpace.
[ERROR] Await.result(database.userModel.autocreate().future(),10.seconds)
Any advice?
Currently using version 1.29.6
BaseCassandraSpec
import com.neruti.User
import com.neruti.db.models._
import com.neruti.db.databases._
import com.neruti.db.services._
import com.neruti.db.Connector._
import org.scalatest._
import org.scalatest.{BeforeAndAfterAll,FlatSpec,Matchers,ShouldMatchers}
import org.scalatest.concurrent.ScalaFutures
import org.scalamock.scalatest.MockFactory
import scala.concurrent.duration._
import scala.concurrent.{Await, Future}
import scala.concurrent.ExecutionContext.Implicits.global
override protected def beforeAll(): Unit = {
Await.result(database.userModel.autocreate().future(),10.seconds)
}
Database
class UserDatabase (val connector: KeySpaceDef){
object userModel extends ConcreteUserModel with connector.Connector
}
object ProductionDb extends UserDatabase(connector)
trait ProductionDatabaseProvider {
def database: UserDatabase
}
trait ProductionDatabase extends ProductionDatabaseProvider {
override val database = ProductionDb
}
object testDB extends UserDatabase(testConnector)
trait testDatabaseProvider {
def database: UserDatabase
}
trait testDatabase extends testDatabaseProvider{
override val database = testDB
}
Connector
package com.neruti.db
import com.neruti.db.models._
import com.websudos.phantom.database.Database
import com.websudos.phantom.connectors.ContactPoints
import com.websudos.phantom.dsl.KeySpaceDef
object Connector {
// TODO: these key value pairs shld get from HOCON config file
val host= Seq("127.0.0.1")
val port = 9042
val keySpace: String = "nrt_entities"
// val inet = InetAddress.getByName
lazy val connector = ContactPoints(host,port).withClusterBuilder(
_.withCredentials("dev", "nrtDev1989")
).keySpace(keySpace)
// embedded cassandra is not supported anymore. Check phantom-sbt.
// lazy val testConnector: KeySpaceDef = ContactPoint.embedded.keySpace(keySpace)
lazy val testConnector: KeySpaceDef = ContactPoints(host,port).noHeartbeat().keySpace(keySpace)
}
I would upgrade to phantom 2.0.0 as a side note. Next, there are many things to improve in your code, starting with capitalisation of the traits.
You should use database.create or database.createAsync, which no longer require you to re-pass the implicit keyspace or session. Bear in mind this API is version 2.1.1 of phantom-dsl, available on Maven Central.
package com.neruti.db
import com.neruti.db.models._
import com.outworkers.phantom.connectors.ContactPoints
import com.outworkers.phantom.dsl._
object Connector {
// TODO: these key value pairs shld get from HOCON config file
val host= Seq("127.0.0.1")
val port = 9042
val keySpace: String = "nrt_entities"
// val inet = InetAddress.getByName
lazy val connector = ContactPoints(host,port).withClusterBuilder(
_.withCredentials("dev", "nrtDev1989")
).keySpace(keySpace)
lazy val testConnector: KeySpaceDef = ContactPoints(host, port).noHeartbeat().keySpace(keySpace)
}
class MyDb(override val connector: CassandraConnection) extends Database(connector) {
... tables
}
object TestMyDb extends MyDb(Connector.testConnector)
import com.outworkers.phantom.dsl.context
// Now this will only require an execution context, nothing more
TestMyDb.create()
I use scalaxb to generate models and client part of the SOAP interface. For testing I use Betamax, which can also be used in Scala. However, scalaxb uses Netty as a transport, which ignores proxy settings set up by Betamax. How would you cope with this situation?
scalaxb uses cake pattern, so the service is built from 3 parts like in the following example:
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent._
import scala.concurrent.duration._
val service = (new stockquote.StockQuoteSoap12Bindings with
scalaxb.SoapClientsAsync with
scalaxb.DispatchHttpClientsAsync {}).service
val fresponse = service.getQuote(Some("GOOG"))
val response = Await.result(fresponse, 5 seconds)
println(response)
And tests:
import co.freeside.betamax.{TapeMode, Recorder}
import co.freeside.betamax.proxy.jetty.ProxyServer
import dispatch._
import org.scalatest.{Tag, FunSuite}
import scala.concurrent.duration._
import scala.concurrent.{Await, Future}
class StockquoteSpec extends FunSuite with Betamax {
testWithBetamax("stockquote", Some(TapeMode.READ_WRITE))("stockquote") {
val fresponse = service.getQuote(Some("GOOG"))
val response = Await.result(fresponse, 5 seconds)
println(response)
}
}
trait Betamax {
protected def test(testName: String, testTags: Tag*)(testFun: => Unit)
def testWithBetamax(tape: String, mode: Option[TapeMode] = None)(testName: String, testTags: Tag*)(testFun: => Unit) = {
test(testName, testTags: _*) {
val recorder = new Recorder
val proxyServer = new ProxyServer(recorder)
recorder.insertTape(tape)
recorder.getTape.setMode(mode.getOrElse(recorder.getDefaultMode()))
proxyServer.start()
try {
testFun
} finally {
recorder.ejectTape()
proxyServer.stop()
}
}
}
}
Versions:
net.databinder.dispatch 0.11.2
co.freeside.betamax 1.1.2
com.ning.async-http-client 1.8.10
io.netty.netty 3.9.2.Final
It is indeed possible to use proxy with Netty. Although Netty does not read system properties for proxy settings, the settings can be injected using ProxyServerSelector. It is created in build method of AsyncHttpClientConfig:
if (proxyServerSelector == null && useProxySelector) {
proxyServerSelector = ProxyUtils.getJdkDefaultProxyServerSelector();
}
if (proxyServerSelector == null && useProxyProperties) {
proxyServerSelector = ProxyUtils.createProxyServerSelector(System.getProperties());
}
if (proxyServerSelector == null) {
proxyServerSelector = ProxyServerSelector.NO_PROXY_SELECTOR;
}
The only obstacle is that scalaxb uses default config with useProxyProperties=false. You can override it with custom MyDispatchHttpClientsAsync that you can use when creating the service:
val service = (new stockquote.StockQuoteSoap12Bindings with
scalaxb.SoapClientsAsync with
MyDispatchHttpClientsAsync {}).service
And the source code of MyDispatchHttpClientsAsync (the key point is calling setUseProxyProperties(true)):
import com.ning.http.client.providers.netty.NettyAsyncHttpProvider
import com.ning.http.client.{AsyncHttpClientConfig, AsyncHttpClient}
import scalaxb.HttpClientsAsync
/**
* #author miso
*/
trait MyDispatchHttpClientsAsync extends HttpClientsAsync {
lazy val httpClient = new DispatchHttpClient {}
trait DispatchHttpClient extends HttpClient {
import dispatch._, Defaults._
// Keep it lazy. See https://github.com/eed3si9n/scalaxb/pull/279
lazy val http = new Http(new AsyncHttpClient(new NettyAsyncHttpProvider(new AsyncHttpClientConfig.Builder().setUseProxyProperties(true).build())))
// lazy val http = Http.configure(_.setUseProxyProperties(true)) // Maybe later. See https://github.com/eed3si9n/scalaxb/issues/312
def request(in: String, address: java.net.URI, headers: Map[String, String]): concurrent.Future[String] = {
val req = url(address.toString).setBodyEncoding("UTF-8") <:< headers << in
http(req > as.String)
}
}
}
While playing with, akka-http experimental 1.0-M2 I am trying to create a simple Hello world example.
import akka.actor.ActorSystem
import akka.http.Http
import akka.http.model.HttpResponse
import akka.http.server.Route
import akka.stream.FlowMaterializer
import akka.http.server.Directives._
object Server extends App {
val host = "127.0.0.1"
val port = "8080"
implicit val system = ActorSystem("my-testing-system")
implicit val fm = FlowMaterializer()
val serverBinding = Http(system).bind(interface = host, port = port)
serverBinding.connections.foreach { connection ⇒
println("Accepted new connection from: " + connection.remoteAddress)
connection handleWith Route.handlerFlow {
path("") {
get {
complete(HttpResponse(entity = "Hello world?"))
}
}
}
}
}
Compilation fails with could not find implicit value for parameter setup: akka.http.server.RoutingSetup
Also, if I change
complete(HttpResponse(entity = "Hello world?"))
with
complete("Hello world?")
I get another error: type mismatch; found : String("Hello world?") required: akka.http.marshalling.ToResponseMarshallable
With research I was able to understand the issue to be lack of Execution Context. To solve both the issue I needed to include this:
implicit val executionContext = system.dispatcher
Looking into akka/http/marshalling/ToResponseMarshallable.scala I see ToResponseMarshallable.apply requires it which returns a Future[HttpResponse].
Also, in akka/http/server/RoutingSetup.scala, RoutingSetup.apply needs it.
May be akka team just needs to add some more #implicitNotFounds. I was able to find not exact but related answer at: direct use of Futures in Akka and spray Marshaller for futures not in implicit scope after upgrading to spray 1.2
Well found - this problem still exists with Akka HTTP 1.0-RC2, so the code for that now must look like this (given their API changes):
import akka.actor.ActorSystem
import akka.http.scaladsl.server._
import akka.http.scaladsl._
import akka.stream.ActorFlowMaterializer
import akka.stream.scaladsl.{Sink, Source}
import akka.http.scaladsl.model.HttpResponse
import Directives._
import scala.concurrent.Future
object BootWithRouting extends App {
val host = "127.0.0.1"
val port = 8080
implicit val system = ActorSystem("my-testing-system")
implicit val fm = ActorFlowMaterializer()
implicit val executionContext = system.dispatcher
val serverSource: Source[Http.IncomingConnection, Future[Http.ServerBinding]] =
Http(system).bind(interface = host, port = port)
serverSource.to(Sink.foreach {
connection =>
println("Accepted new connection from: " + connection.remoteAddress)
connection handleWith Route.handlerFlow {
path("") {
get {
complete(HttpResponse(entity = "Hello world?"))
}
}
}
}).run()
}