Any ideas on how to best test an Akka Stream containing an Akka Http Flow? I'm struggling with the following method in particular:
def akkaHttpFlow(server: String)(implicit actorSystem: ActorSystem, actorMaterializer: ActorMaterializer) = {
val uri = new java.net.URI(server)
val port: Int = if( uri.getPort != -1) { uri.getPort } else { 80 }
Http().cachedHostConnectionPool[Seq[String]](uri.getHost, port)
.withAttributes(ActorAttributes.supervisionStrategy(decider))
}
This is the test code
val emails = Set("tonymurphy#example.com")
val source: Source[String, NotUsed] = Source(emails)
val f = source
.grouped(10)
.via(requestBuilderFlow)
.via(akkaHttpFlow)
.map(responseHandler)
.runForeach(println)
f.futureValue.shouldBe(Done)
It fails with the following error (not unexpected tbh) >>>
The future returned an exception of type: akka.stream.StreamTcpException, with message: Tcp command [Connect(localhost:9001,None,List(),Some(10 seconds),true)] failed because of Connection refused.
Would it be possible to embed akka http server in the test? Or how best could I structure the code to be able to mock it?
The supporting code
object MyOperations extends StrictLogging {
val requestBuilderFunc : Seq[String] => (HttpRequest, Seq[String]) = { emails : Seq[String] =>
HttpRequest(method = HttpMethods.POST, uri = "/subscribers").withEntity(ContentTypes.`application/json`, ByteString(Json.toJson(emails).toString())) -> emails.toVector
}
val requestBuilderFlow : Flow[Seq[String],(HttpRequest, Seq[String]),NotUsed] = Flow[Seq[String]] map requestBuilderFunc
val responseHandler: ((Try[HttpResponse], Seq[String])) => (HttpResponse, Seq[String]) = {
case (responseTry, context) =>
logger.debug(s"Response: $responseTry")
(responseTry.get, context.asInstanceOf[Seq[String]])
}
}
I have to admit I'm struggling with how to organise my scala applications into objects, traits, classes, higher order functions etc and test them
What you'll want to do is use something like dependency injection to inject a Flow[(HttpRequest, Seq[String]), (Try[HttpResponse], Seq[String]), Any].
In production that flow will be from akka http, but in test you can mock it yourself to return whatever you need.
Related
We have a Caliban GraphQL application, using it with Play framework. It is well covered with integration tests for queries and mutations, now we're about to add some integration tests for subscriptions and wondering how to do it correctly.
For queries/mutations testing we're using usual FakeRequest, sending it to our router that extends Caliban's PlayRouter, it works very good. Is there any similar way to test websockets/subscriptions?
There is very short amount of information in the Internet about websocket testing in Play and no information at all about GraphQL subscription testing.
Will be grateful for any ideas!
Ok, I managed it. There are couple of rules to follow:
Use websocket header "WebSocket-Protocol" -> "graphql-ws"
After connection is established, send GraphQLWSRequest of type "connection_init"
After receiving response "connection_ack", send GraphQLWSRequest of type "start" with subscription query as payload
After those steps server is listening and you can send your mutation queries.
Some draft example:
import caliban.client.GraphQLRequest
import caliban.client.ws.GraphQLWSRequest
import io.circe.syntax.EncoderOps
import play.api.libs.json.{JsValue, Json}
import play.api.test.Helpers.{POST, contentAsJson, contentAsString, contentType, route, status, _}
import org.awaitility.Awaitility
def getWS(subscriptionQuery: String, postQuery: String): JsValue = {
lazy val port = Helpers.testServerPort
val initRequest = prepareWSRequest("connection_init")
val startRequest = prepareWSRequest("start", Some(GraphQLRequest(subscriptionQuery, Map())))
Helpers.running(TestServer(port, app)) {
val headers = new java.util.HashMap[String, String]()
headers.put("WebSocket-Protocol", "graphql-ws")
val queue = new ArrayBlockingQueue[String](1)
lazy val ws = new WebSocketClient(new URI(s"ws://localhost:$port/ws/graphql"), headers) {
override def onOpen(handshakedata: ServerHandshake): Unit =
logger.info("Websocket connection established")
override def onClose(code: Port, reason: String, remote: Boolean): Unit =
logger.info(s"Websocket connection closed, reason: $reason")
override def onError(ex: Exception): Unit =
logger.error("Error handling websocket connection", ex)
override def onMessage(message: String): Unit = {
val ttp = (Json.parse(message) \ "type").as[JsString].value
if (ttp != "connection_ack" && ttp != "ka") queue.put(message)
}
}
ws.connectBlocking()
Future(ws.send(initRequest))
.flatMap(_ => Future(ws.send(startRequest)))
.flatMap(_ => post(query = postQuery)) // post is my local method, it sends usual FakeRequest
Awaitility.await().until(() => queue.peek() != null)
Json.parse(queue.take())
}
def prepareWSRequest(ttp: String, payload: Option[GraphQLRequest] = None) =
GraphQLWSRequest(ttp, None, payload).asJson.noSpaces
}
I try to develop gRPC server with Akka-gRPC and Slick. I also use Airframe for DI.
Source code is here
The issue is that it cause failure if it receive request when execute as gRPC server.
If it doesn't start as a gRPC server, but just reads resources from the database, the process succeeds.
What is the difference?
At Follows, It read object from database with slick.
...Component is airframe object. It will use by main module.
trait UserRepository {
def getUser: Future[Seq[Tables.UsersRow]]
}
class UserRepositoryImpl(val profile: JdbcProfile, val db: JdbcProfile#Backend#Database) extends UserRepository {
import profile.api._
def getUser: Future[Seq[Tables.UsersRow]] = db.run(Tables.Users.result)
}
trait UserResolveService {
private val repository = bind[UserRepository]
def getAll: Future[Seq[Tables.UsersRow]] =
repository.getUser
}
object userServiceComponent {
val design = newDesign
.bind[UserResolveService]
.toSingleton
}
Follows is gRPC Server source code.
trait UserServiceImpl extends UserService {
private val userResolveService = bind[UserResolveService]
private val system: ActorSystem = bind[ActorSystem]
implicit val ec: ExecutionContextExecutor = system.dispatcher
override def getAll(in: GetUserListRequest): Future[GetUserListResponse] = {
userResolveService.getAll.map(us =>
GetUserListResponse(
us.map(u =>
myapp.proto.user.User(
1,
"t_horikoshi#example.com",
"t_horikoshi",
myapp.proto.user.User.UserRole.Admin
)
)
)
)
}
}
trait GRPCServer {
private val userServiceImpl = bind[UserServiceImpl]
implicit val system: ActorSystem = bind[ActorSystem]
def run(): Future[Http.ServerBinding] = {
implicit def ec: ExecutionContext = system.dispatcher
val service: PartialFunction[HttpRequest, Future[HttpResponse]] =
UserServiceHandler.partial(userServiceImpl)
val reflection: PartialFunction[HttpRequest, Future[HttpResponse]] =
ServerReflection.partial(List(UserService))
// Akka HTTP 10.1 requires adapters to accept the new actors APIs
val bound = Http().bindAndHandleAsync(
ServiceHandler.concatOrNotFound(service, reflection),
interface = "127.0.0.1",
port = 8080,
settings = ServerSettings(system)
)
bound.onComplete {
case Success(binding) =>
system.log.info(
s"gRPC Server online at http://${binding.localAddress.getHostName}:${binding.localAddress.getPort}/"
)
case Failure(ex) =>
system.log.error(ex, "occurred error")
}
bound
}
}
object grpcComponent {
val design = newDesign
.bind[UserServiceImpl]
.toSingleton
.bind[GRPCServer]
.toSingleton
}
Follows is main module.
object Main extends App {
val conf = ConfigFactory
.parseString("akka.http.server.preview.enable-http2 = on")
.withFallback(ConfigFactory.defaultApplication())
val system = ActorSystem("GRPCServer", conf)
val dbConfig: DatabaseConfig[JdbcProfile] =
DatabaseConfig.forConfig[JdbcProfile](path = "mydb")
val design = newDesign
.bind[JdbcProfile]
.toInstance(dbConfig.profile)
.bind[JdbcProfile#Backend#Database]
.toInstance(dbConfig.db)
.bind[UserRepository]
.to[UserRepositoryImpl]
.bind[ActorSystem]
.toInstance(system)
.add(userServiceComponent.design)
.add(grpcComponent.design)
design.withSession(s =>
// Await.result(s.build[UserResolveService].getUser, Duration.Inf)) // success
// Await.result(s.build[UserServiceImpl].getAll(GetUserListRequest()), Duration.Inf)) // success
s.build[GRPCServer].run() // cause IllegalStateException when reciece request.
)
}
When UserResolveService and UserServiceImpl are called directly, the process of loading an object from the database is successful.
However, when running the application as a gRPC Server, an error occurs when a request is received.
Though I was thinking all day, I couldn't resolve...
Will you please help me to resolve.
It resolved. if execute async process, It has to start gRPC server with newSession.
I fix like that.
This is my code for the server written using Akka framework:
case class Sentence(data: String)
case class RawTriples(triples: List[String])
trait Protocols extends DefaultJsonProtocol {
implicit val sentenceRequestFormat = jsonFormat1(Sentence)
implicit val rawTriplesFormat = jsonFormat1(RawTriples)
}
trait Service extends Protocols {
implicit val system: ActorSystem
implicit def executor: ExecutionContextExecutor
implicit val materializer: Materializer
val openie = new OpenIE
def config: Config
val logger: LoggingAdapter
lazy val ipApiConnectionFlow: Flow[HttpRequest, HttpResponse, Any] =
Http().outgoingConnection(config.getString("services.ip-api.host"), config.getInt("services.ip-api.port"))
def ipApiRequest(request: HttpRequest): Future[HttpResponse] = Source.single(request).via(ipApiConnectionFlow).runWith(Sink.head)
val routes = {
logRequestResult("akka-http-microservice") {
pathPrefix("openie") {
post {
decodeRequest{
entity(as[Sentence]){ sentence =>
complete {
var rawTriples = openie.extract(sentence.data)
val resp: MutableList[String] = MutableList()
for(rtrip <- rawTriples){
resp += (rtrip.toString())
}
val response: List[String] = resp.toList
println(response)
response
}
}
}
}
}
}
}
}
object AkkaHttpMicroservice extends App with Service {
override implicit val system = ActorSystem()
override implicit val executor = system.dispatcher
override implicit val materializer = ActorMaterializer()
override val config = ConfigFactory.load()
override val logger = Logging(system, getClass)
Http().bindAndHandle(routes, config.getString("http.interface"), config.getInt("http.port"))
}
The server accepts a POST request containing a sentence and returns a json array in return. It works fine but if I am making requests to it too frequently using parallelized code, then it gives 500 Internal server error. I wanted to know is there any parameter which I can set in the server to avoid that (number of ready threads for accepting requests etc).
In log files, the error is logged as:
[ERROR] [05/31/2017 11:48:38.110]
[default-akka.actor.default-dispatcher-6]
[akka.actor.ActorSystemImpl(default)] Error during processing of
request: 'null'. Completing with 500 Internal Server Error response.
The doc on the bindAndHandle method shows what you want:
/**
* Convenience method which starts a new HTTP server at the given endpoint and uses the given `handler`
* [[akka.stream.scaladsl.Flow]] for processing all incoming connections.
*
* The number of concurrently accepted connections can be configured by overriding
* the `akka.http.server.max-connections` setting. Please see the documentation in the reference.conf for more
* information about what kind of guarantees to expect.
*
* To configure additional settings for a server started using this method,
* use the `akka.http.server` config section or pass in a [[akka.http.scaladsl.settings.ServerSettings]] explicitly.
*/
akka.http.server.max-connections is probably what you want. As the doc suggests, you can also dig deeper into the akka.http.server config section.
Add following in application.conf file
akka.http {
server {
server-header = akka-http/${akka.http.version}
idle-timeout = infinite
request-timeout = infinite
}
}
I am developing a simple HTTP server using Akka-Http in Scala.
My code is as given below:
object HttpServer extends App {
override def main(args: Array[String]): Unit = {
implicit val system = ActorSystem("my-system")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
val route : Route = post {
path("echo") {
val json = ???
complete((StatusCodes.OK, json))
}
}
val bindingFuture = Http().bindAndHandle(route, "localhost", 8080)
println(s"Server online at http://localhost:8080/\nPress RETURN to stop...")
StdIn.readLine()
bindingFuture.flatMap(_.unbind())
port.onComplete(_ => system.terminate())
}
}
I do not know Scala enough yet. For that, I need some help.
I do not know how I can get JSON from Http POST body to give back that json to client.
You only need to add an extractor to your route definition:
val route : Route = post {
path("echo") {
entity(as[String]) { json =>
complete(json)
}
}
Note that you don't need to set the status code explicitly, as akka-http will automatically set status 200 OK for you when you pass a value to complete
I am working on Play framework 2.5.3 and Akka 2.4.7 using Scala. I have an application made in play based webSockets using Akka. I am using the project as a jar in another project.
My code in new project looks like this:
For accepting connections:
val postActor = actorSystem.actorOf(Props[PostActorClass])
def wsSocket = WebSocket.accept[JsValue,JsValue]{ request =>
ActorFlow.actorRef(out => wsConnObj.HandlerClass.props(out,postActor))
}
For getting references of all connections stored in my jar project:
def getConnReferences:Set[ActorRef] = {
implicit val timeout = Timeout(5 seconds)
val future = postActor ? GetReferences
val result = Await.result(future, timeout.duration).asInstanceOf[Set[ActorRef]]
(result)
}
For sending messages to all the references:
def sendMsg = {
val msg = Json.parse("""{"username":"Server","message":"Reached finally.","pub_key":"empty","target":"user"}""")
val refs = getConnReferences
refs.foreach( _ ! msg)
}
I am trying to use these methods in my new project's HomeController. But unable to work them together, like wsSocket should be executed , then sendMsg (which itself calls getConnReferences also).
How will I achieve that? If am calling all these blocks in single action method, I can't establish connection even.
Code for WebSocketConnect class:
class WebSocketConnect #Inject() (cache:CacheApi) (implicit actorSystem: ActorSystem,materializer: Materializer){
object HandlerClass {
homelogger.info(logMessages.passingActorRef)
def props(out: ActorRef, postActor: ActorRef) = {
Props(new SocketHandlerClass(out, postActor, cache, postActorToUsernameMap))
}
}
}