I was trying to insert websocket API's in a project that uses REST API's using scala.When I compile the program this error occurs - scala

I'm trying to establish a websocket connection. I'm trying this in a already created project. And this error occurs.
Type mismatch, expected: Flow[HttpRequest, HttpResponse, Any], actual:
server.Route
The error occurs at route
val binding = Http().bindAndHandle(route, interface, port)
println(s"Server is now online at http://$interface:$port\nPress RETURN
to stop...")
StdIn.readLine()
binding.flatMap(_.unbind()).onComplete(_ => actorSystem.terminate())
println("Server is down...")
val route=path("ws-echo") {
(get) {
val requestHandler: HttpRequest ⇒ HttpResponse = {
case req#HttpRequest( GET, Uri.Path( "/greeter" ), _, _, _ ) ⇒
req.header[UpgradeToWebSocket] match {
case Some( upgrade ) ⇒ upgrade.handleMessages(
greeterWebSocketService
)
case None ⇒ HttpResponse( 400, entity = "Not a valid websocket
request!" )
}
case r: HttpRequest =>
r.discardEntityBytes()
HttpResponse( 404, entity = "Unknown resource!" )
}
}
}
override implicit val system = ActorSystem()
override implicit val executor = system.dispatcher
override implicit val materializer = ActorMaterializer()
override val logger = Logging(system, getClass)
implicit val mat: Materializer
implicit val actorSystem = ActorSystem("akka-system")
val router: server.Route = routes
val config = ConfigFactory.load()
val interface = config.getString("app.interface")
val port = config.getInt("app.port")
Http().bindAndHandle(router, Configuration.httpInterface,
Configuration.httpPort)

I suspect you're getting this error on the line val binding = Http().bindAndHandle(route, interface, port), is that correct?
Indeed bindAndHandle expects a Flow:
def bindAndHandle(
handler: Flow[HttpRequest, HttpResponse, Any],
interface: String,
port: Int = DefaultPortForProtocol,
connectionContext: ConnectionContext = defaultServerHttpContext,
settings: ServerSettings = ServerSettings(system),
log: LoggingAdapter = system.log)(implicit fm: Materializer
): Future[ServerBinding]
So why do so many examples show passing in a Route? The 'magic' here that there's an implicit conversion available to turn a Route into a Flow.
This is basically an implementation detail, you normally shouldn't have to care about this, but the implicit conversion is route2HandlerFlow:
implicit def route2HandlerFlow(
route: Route
)(
implicit routingSettings: RoutingSettings,
parserSettings: ParserSettings,
materializer: Materializer,
routingLog: RoutingLog,
executionContext: ExecutionContext = null,
rejectionHandler: RejectionHandler = RejectionHandler.default,
exceptionHandler: ExceptionHandler = null
): Flow[HttpRequest, HttpResponse, NotUsed]
As you can see, this implicit conversion requires some other implicit values to be available. I suspect you are missing one of those. I think it might be enough to introduce an implicit Materializer, could you give that a try?

Related

Akka Http: Calling TypedActor.context outside of a TypedActor implementation method

I'm trying to build akka http client, by using this example Test akka http client
This is my code:
trait HttpServer {
def sendAndReceive: HttpRequest => Future[HttpResponse]
}
trait ApiHandler extends HttpServer {
implicit def as: ActorSystem
override def sendAndReceive: HttpRequest => Future[HttpResponse] =
Http().singleRequest(_)
}
trait Api {
this: ApiHandler =>
implicit def as: ActorSystem
implicit def mat: Materializer
implicit def ec: ExecutionContext
def request(httpRequest: HttpRequest): Future[HttpResponse] = {
sendAndReceive(httpRequest).flatMap { response =>
response.status match {
case StatusCodes.OK => Future.successful(response)
case _ => throw new RuntimeException ("error")
}
}
}
}
class ApiImpl() (implicit val as: ActorSystem, val mat: Materializer, val ec: ExecutionContext) extends Api with ApiHandler {
def request: Future[HttpResponse] = {
val httpRequest = ???
request(httpRequest)
}
}
And this is how I call request method:
implicit val actorSystem = ActorSystem("system")
implicit val executionContext = actorSystem.dispatcher
implicit val materializer = ActorMaterializer()
val api = new ApiImpl()
api.request
I receive the following error in place, where trait Api request calls sendAndReceive-method (in this line: sendAndReceive(httpRequest).flatMap { response => ):
Calling TypedActor.context outside of a TypedActor implementation method!
What am I doing wrong?
This code:
trait HttpServer {
def sendAndReceive: HttpRequest => Future[HttpResponse]
}
trait ApiHandler extends HttpServer {
implicit def as: ActorSystem
override def sendAndReceive: HttpRequest => Future[HttpResponse] =
Http().singleRequest(_)
}
trait Api {
this: ApiHandler =>
implicit def as: ActorSystem
implicit def mat: Materializer
implicit def ec: ExecutionContext
def request(httpRequest: HttpRequest) = {
sendAndReceive(httpRequest).flatMap {
response =>
response.status match {
case StatusCodes.OK => Future.successful(response)
case _ => throw new RuntimeException("error")
}
}
}
}
class ApiImpl()(implicit val as: ActorSystem, val mat: Materializer, val ec: ExecutionContext) extends Api with ApiHandler {
def request2 = {
val httpRequest = Get("https://akka.io")
request(httpRequest)
}
}
def main(args: Array[String]): Unit = {
implicit val actorSystem = ActorSystem("system")
implicit val executionContext = actorSystem.dispatcher
implicit val materializer = ActorMaterializer()
val api = new ApiImpl()
api.request2.map(println(_))
}
with versions:
val akkaVersion = "2.6.7"
val akkaHttpVersion = "10.1.12"
Works properly:
HttpResponse(200 OK,List(Date: Mon, 06 Feb 2023 12:19:39 GMT, Connection: keep-alive, Last-Modified: Tue, 17 Jan 2023 13:53:07 GMT, Vary: Accept-Encoding, CF-Cache-Status: DYNAMIC, Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=GZ7HZLwXVyB%2BXRzpcYY4A%2BjpGd0h0S8EsbSRWwMm0b8UbQrnqWpmB8e%2FFEYkNsZUVUdCWboRsWP%2B0aC6YJETfHGZU%2FU%2BFyhetR6%2FA8527KxVKfuGTjpr0Chp"}],"group":"cf-nel","max_age":604800}, NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}, Server: cloudflare, CF-RAY: 7953d5d82a3c384d-MAD, alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400),HttpEntity.Chunked(text/html),HttpProtocol(HTTP/1.1))

Function never reached in Akka Streams Source mapping

Given the following code:
class MigrationHandler #Inject()(database: Database, doUpdate: UpdateHandler)
(implicit #PropagateContext executor: ExecutionContext, actorSystem: ActorSystem)
extends Handler with Logging {
implicit val materializer: ActorMaterializer = ActorMaterializer()
val buttonTypeId = "someId1"
val promotionButtonTypeId = "someId2"
val typeId3 = "someId3"
val typeId4 = "someId4"
val typeIds = Seq[String](buttonTypeId, promotionButtonTypeId, typeId3, typeId4)
def apply(requestHeaders: Headers): Seq[Future[Done]] = {
for {
typeId <- typeIds
result = database.allByType(typeId)
.map(contents => contents.map(content => migrate(content, typeId))
.map(migratedContent => doUpdate(ContentId(migratedContent.id),
NewContent(migratedContent.raw, isDefaultContent = false), requestHeaders))
.runForeach(_ => ())).flatten
} yield result
}
def migrate(content: SomeContent, typeId: String): SomeContent = {
logger.info(s"$content with type $typeId")
content
}}
Future[Source[SomeContent, _]] is returned by the database.allByType(typeId)
In unit test where I am mocking database.allByType(typeId) to return Source.single(SomeContent()), I saw that I wasn't able reach the migrate function. Any idea what could be the problem here?

akka-http: send element to akka sink from http route

How can I send elements/messages to an Akka Sink from an Akka HTTP route? My HTTP route still needs to return a normal HTTP response.
I imagine this requires a stream branch/junction. The normal HTTP routes are flows from HttpRequest -> HttpResponse. I would like to add a branch/junction so that HttpRequests can trigger events to my separate sink as well as generate the normal HttpResponse.
Below is a very simple single route akka-http app. For simplicity, I'm using a simple println sink. My production use case, will obviously involve a less trivial sink.
def main(args: Array[String]): Unit = {
implicit val actorSystem = ActorSystem("my-akka-http-test")
val executor = actorSystem.dispatcher
implicit val materializer = ActorMaterializer()(actorSystem)
// I would like to send elements to this sink in response to HTTP GET operations.
val sink: Sink[Any, Future[Done]] = Sink.foreach(println)
val route: akka.http.scaladsl.server.Route =
path("hello" / Segment) { p =>
get {
// I'd like to send a message to an Akka Sink as well as return an HTTP response.
complete {
s"<h1>Say hello to akka-http. p=$p</h1>"
}
}
}
val httpExt: akka.http.scaladsl.HttpExt = Http(actorSystem)
val bindingFuture = httpExt.bindAndHandle(RouteResult.route2HandlerFlow(route), "localhost", 8080)
println("Server online at http://localhost:8080/")
println("Press RETURN to stop...")
scala.io.StdIn.readLine()
bindingFuture
.flatMap(_.unbind())(executor) // trigger unbinding from the port
.onComplete(_ => Await.result(actorSystem.terminate(), Duration.Inf))(executor) // and shutdown when done
}
EDIT: Or in using the low-level akka-http API, how could I send specific messages to a sink from a specific route handler?
def main(args: Array[String]): Unit = {
implicit val actorSystem = ActorSystem("my-akka-http-test")
val executor = actorSystem.dispatcher
implicit val materializer = ActorMaterializer()(actorSystem)
// I would like to send elements to this sink in response to HTTP GET operations.
val sink: Sink[Any, Future[Done]] = Sink.foreach(println)
val requestHandler: HttpRequest => HttpResponse = {
case HttpRequest(GET, Uri.Path("/"), _, _, _) =>
HttpResponse(entity = HttpEntity(
ContentTypes.`text/html(UTF-8)`,
"<html><body>Hello world!</body></html>"))
case HttpRequest(GET, Uri.Path("/ping"), _, _, _) =>
HttpResponse(entity = "PONG!")
case HttpRequest(GET, Uri.Path("/crash"), _, _, _) =>
sys.error("BOOM!")
case r: HttpRequest =>
r.discardEntityBytes() // important to drain incoming HTTP Entity stream
HttpResponse(404, entity = "Unknown resource!")
}
val serverSource = Http().bind(interface = "localhost", port = 8080)
val bindingFuture: Future[Http.ServerBinding] =
serverSource.to(Sink.foreach { connection =>
println("Accepted new connection from " + connection.remoteAddress)
connection handleWithSyncHandler requestHandler
// this is equivalent to
// connection handleWith { Flow[HttpRequest] map requestHandler }
}).run()
println("Server online at http://localhost:8080/")
println("Press RETURN to stop...")
scala.io.StdIn.readLine()
bindingFuture
.flatMap(_.unbind())(executor) // trigger unbinding from the port
.onComplete(_ => Await.result(actorSystem.terminate(), Duration.Inf))(executor) // and shutdown when done
}
IF you want to send the whole HttpRequest to a sink of yours, I'd say the simplest way is to use the alsoTo combinator. The result would be something along the lines of
val mySink: Sink[HttpRequest, NotUsed] = ???
val handlerFlow = Flow[HttpRequest].alsoTo(mySink).via(RouteResult.route2HandlerFlow(route))
val bindingFuture = Http().bindAndHandle(handlerFlow, "localhost", 8080)
FYI: alsoTo in fact hides a Broadcast stage.
IF instead you need to selectively send a message to a Sink from a specific subroute, you have no other choice but to materialize a new flow for each incoming request. See example below
val sink: Sink[Any, Future[Done]] = Sink.foreach(println)
val route: akka.http.scaladsl.server.Route =
path("hello" / Segment) { p =>
get {
(extract(_.request) & extractMaterializer) { (req, mat) ⇒
Source.single(req).runWith(sink)(mat)
complete {
s"<h1>Say hello to akka-http. p=$p</h1>"
}
}
}
}
Also, keep in mind you can always ditch the high-level DSL completely, and model you whole route using the lower-level streams DSL. This will result in more verbose code - but will give you full control of your stream materialization.
EDIT: example below
val sink: Sink[Any, Future[Done]] = Sink.foreach(println)
val handlerFlow =
Flow.fromGraph(GraphDSL.create() { implicit b =>
import GraphDSL.Implicits._
val partition = b.add(Partition[HttpRequest](2, {
case HttpRequest(GET, Uri.Path("/"), _, _, _) ⇒ 0
case _ ⇒ 1
}))
val merge = b.add(Merge[HttpResponse](2))
val happyPath = Flow[HttpRequest].map{ req ⇒
HttpResponse(entity = HttpEntity(
ContentTypes.`text/html(UTF-8)`,
"<html><body>Hello world!</body></html>"))
}
val unhappyPath = Flow[HttpRequest].map{
case HttpRequest(GET, Uri.Path("/ping"), _, _, _) =>
HttpResponse(entity = "PONG!")
case HttpRequest(GET, Uri.Path("/crash"), _, _, _) =>
sys.error("BOOM!")
case r: HttpRequest =>
r.discardEntityBytes() // important to drain incoming HTTP Entity stream
HttpResponse(404, entity = "Unknown resource!")
}
partition.out(0).alsoTo(sink) ~> happyPath ~> merge
partition.out(1) ~> unhappyPath ~> merge
FlowShape(partition.in, merge.out)
})
val bindingFuture = Http().bindAndHandle(handlerFlow, "localhost", 8080)
This is the solution I used that seems ideal. Akka Http seems like it's designed so that your routes are simple HttpRequest->HttpResponse flows and don't involve any extra branches.
Rather than build everything into a single Akka stream graph, I have a separate QueueSource->Sink graph, and the normal Akka Http HttpRequest->HttpResponse flow just adds elements to the source queue as needed.
object HttpWithSinkTest {
def buildQueueSourceGraph(): RunnableGraph[(SourceQueueWithComplete[String], Future[Done])] = {
val annotateMessage: Flow[String, String, NotUsed] = Flow.fromFunction[String, String](s => s"got message from queue: $s")
val sourceQueue = Source.queue[String](100, OverflowStrategy.dropNew)
val sink: Sink[String, Future[Done]] = Sink.foreach(println)
val annotatedSink = annotateMessage.toMat(sink)(Keep.right)
val queueGraph = sourceQueue.toMat(annotatedSink)(Keep.both)
queueGraph
}
def buildHttpFlow(queue: SourceQueueWithComplete[String],
actorSystem: ActorSystem, materializer: ActorMaterializer): Flow[HttpRequest, HttpResponse, NotUsed] = {
implicit val actorSystemI = actorSystem
implicit val materializerI = materializer
val route: akka.http.scaladsl.server.Route =
path("hello" / Segment) { p =>
get {
complete {
queue.offer(s"got http event p=$p")
s"<h1>Say hello to akka-http. p=$p</h1>"
}
}
}
val routeFlow = RouteResult.route2HandlerFlow(route)
routeFlow
}
def main(args: Array[String]): Unit = {
val actorSystem = ActorSystem("my-akka-http-test")
val executor = actorSystem.dispatcher
implicit val materializer = ActorMaterializer()(actorSystem)
val (queue, _) = buildQueueSourceGraph().run()(materializer)
val httpFlow = buildHttpFlow(queue, actorSystem, materializer)
val httpExt: akka.http.scaladsl.HttpExt = Http(actorSystem)
val bindingFuture = httpExt.bindAndHandle(httpFlow, "localhost", 8080)
println("Server online at http://localhost:8080/")
println("Press RETURN to stop...")
scala.io.StdIn.readLine()
println("Shutting down...")
val serverBinding = Await.result(bindingFuture, Duration.Inf)
Await.result(serverBinding.unbind(), Duration.Inf)
Await.result(actorSystem.terminate(), Duration.Inf)
println("Done. Exiting")
}
}

Do i need to create ActorMaterializer multiple times?

I have an app that generate reports, with akka-http + akka-actors + akka-camel + akka-streams. When a post request arrives , the ActiveMqProducerActor enqueue the request into ActiveMq Broker. Then the ActiveMqConsumerActor consumes the message and start the task using akka-streams(in this actor i need the materializer) .
The main class create the ActorSystem and the ActorMaterializer, but i dont know how is the correct way to "inject" the materializer into the akka-actor
object ReportGeneratorApplication extends App {
implicit val system: ActorSystem = ActorSystem()
implicit val executor = system.dispatcher
implicit val materializer = ActorMaterializer()
val camelExtension: Camel = CamelExtension(system);
val amqc: ActiveMQComponent = ActiveMQComponent.activeMQComponent(env.getString("jms.url"))
amqc.setUsePooledConnection(true)
amqc.setAsyncConsumer(true)
amqc.setTrustAllPackages(true)
amqc.setConcurrentConsumers(1)
camelExtension.context.addComponent("jms", amqc);
val jmsProducer: ActorRef = system.actorOf(Props[ActiveMQProducerActor])
//Is this the correct way to pass the materializer?
val jmsConsumer: ActorRef = system.actorOf(Props(new ActiveMQConsumerActor()(materializer)), name = "jmsConsumer")
val endpoint: ReportEndpoint = new ReportEndpoint(jmsProducer);
Http().bindAndHandle(endpoint.routes, "localhost", 8881)
}
The ReportEndPoint class, that have the jmsProducerActor . Mongo is a trait with CRUD methods. JsonSupport(==SprayJsonSupport)
class ReportEndpoint(jmsProducer: ActorRef)
(implicit val system:ActorSystem,
implicit val executor: ExecutionContext,
implicit val materializer : ActorMaterializer)
extends JsonSupport with Mongo {
val routes =
pathPrefix("reports"){
post {
path("generate"){
entity(as[DataRequest]) { request =>
val id = java.util.UUID.randomUUID.toString
// **Enqueue the request into ActiveMq**
jmsProducer ! request
val future: Future[Seq[Completed]] = insertReport(request)
complete {
future.map[ToResponseMarshallable](r => r.head match {
case r : Completed => println(r); s"Reporte Generado con id $id"
case _ => HttpResponse(StatusCodes.InternalServerError, entity = "Error al generar reporte")
})
}
}
}
} ....
The idea of ActiveMqConsumerActor, is send the messages, with streams and backpressure, one by one,because ReportBuilderActor makes many mongo operations (and the datacenter it`s not very good).
//Is this the correct way to pass the materializer??
class ActiveMQConsumerActor (implicit materializer : ActorMaterializer) extends Consumer with Base {
override def endpointUri: String = env.getString("jms.queue")
val log = Logging(context.system, this)
val reportActor: ActorRef = context.actorOf(Props(new ReportBuilderActor()(materializer)), name = "reportActor")
override def receive: Receive = {
case msg: CamelMessage => msg.body match {
case data: DataRequest => {
//I need only one task running
Source.single(data).buffer(1, OverflowStrategy.backpressure).to(Sink.foreach(d => reportActor ! d)).run()
}
case _ => log.info("Invalid")
}
case _ => UnhandledMessage
}
}
Is a good idea have implicit values in companion objects?
Thanks!!

How do you throttle Flow in the latest Akka (2.4.6)?

How do you throttle Flow in the latest Akka (2.4.6) ? I'd like to throttle Http client flow to limit number of requests to 3 requests per second. I found following example online but it's for old Akka and akka-streams API changed so much that I can't figure out how to rewrite it.
def throttled[T](rate: FiniteDuration): Flow[T, T] = {
val tickSource: Source[Unit] = TickSource(rate, rate, () => ())
val zip = Zip[T, Unit]
val in = UndefinedSource[T]
val out = UndefinedSink[T]
PartialFlowGraph { implicit builder =>
import FlowGraphImplicits._
in ~> zip.left ~> Flow[(T, Unit)].map { case (t, _) => t } ~> out
tickSource ~> zip.right
}.toFlow(in, out)
}
Here is my best attempt so far
def throttleFlow[T](rate: FiniteDuration) = Flow.fromGraph(GraphDSL.create() { implicit builder =>
import GraphDSL.Implicits._
val ticker = Source.tick(rate, rate, Unit)
val zip = builder.add(Zip[T, Unit.type])
val map = Flow[(T, Unit.type)].map { case (value, _) => value }
val messageExtractor = builder.add(map)
val in = Inlet[T]("Req.in")
val out = Outlet[T]("Req.out")
out ~> zip.in0
ticker ~> zip.in1
zip.out ~> messageExtractor.in
FlowShape.of(in, messageExtractor.out)
})
it throws exception in my main flow though :)
private val queueHttp = Source.queue[(HttpRequest, (Any, Promise[(Try[HttpResponse], Any)]))](1000, OverflowStrategy.backpressure)
.via(throttleFlow(rate))
.via(poolClientFlow)
.mapAsync(4) {
case (util.Success(resp), any) =>
val strictFut = resp.entity.toStrict(5 seconds)
strictFut.map(ent => (util.Success(resp.copy(entity = ent)), any))
case other =>
Future.successful(other)
}
.toMat(Sink.foreach({
case (triedResp, (value: Any, p: Promise[(Try[HttpResponse], Any)])) =>
p.success(triedResp -> value)
case _ =>
throw new RuntimeException()
}))(Keep.left)
.run
where poolClientFlow is Http()(system).cachedHostConnectionPool[Any](baseDomain)
Exception is:
Caused by: java.lang.IllegalArgumentException: requirement failed: The output port [Req.out] is not part of the underlying graph.
at scala.Predef$.require(Predef.scala:219)
at akka.stream.impl.StreamLayout$Module$class.wire(StreamLayout.scala:204)
Here is an attempt that uses the throttle method as mentioned by #Qingwei. The key is to not use bindAndHandle(), but to use bind() and throttle the flow of incoming connections before handling them. The code is taken from the implementation of bindAndHandle(), but leaves out some error handling for simplicity. Please don't do that in production.
implicit val system = ActorSystem("test")
implicit val mat = ActorMaterializer()
import system.dispatcher
val maxConcurrentConnections = 4
val handler: Flow[HttpRequest, HttpResponse, NotUsed] = complete(LocalDateTime.now().toString)
def handleOneConnection(incomingConnection: IncomingConnection): Future[Done] =
incomingConnection.flow
.watchTermination()(Keep.right)
.joinMat(handler)(Keep.left)
.run()
Http().bind("127.0.0.1", 8080)
.throttle(3, 1.second, 1, ThrottleMode.Shaping)
.mapAsyncUnordered(maxConcurrentConnections)(handleOneConnection)
.to(Sink.ignore)
.run()