I have a method which send picture from client to CDN throught FormData. Code:
def uploadToCDN(formData: Multipart.FormData): Future[HttpResponse] = {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
Http().singleRequest(
HttpRequest(
method = HttpMethods.POST,
uri = "http://cdn.example.com",
entity = formData.toEntity(),
protocol = HttpProtocols.`HTTP/1.1`))
}
How I can add "secret_key": "12345678" to FormData which I receive from the client?
Multipart.FormData is basically made up of its parts. To join two FormDatas you need to concatenate the formdata parts and create a new instance of FormData:
val newFormData =
Multipart.FormData(
Source.single(Multipart.FormData.BodyPart("secret_key", "12345678"))
.concat(originalFormData.parts)
)
See also the Scaladocs of Multipart.FormData.
Related
I am trying to make a singleRequest from Akka using this code:
val request = HttpRequest(
method = HttpMethods.GET,
uri = "url"
)
val responseFuture: Future[HttpResponse] = Http().singleRequest(request)
val entityFuture: Future[HttpEntity.Strict] = responseFuture.flatMap(response => response.entity.toStrict(2.seconds))
entityFuture.map(entity => entity.data.utf8String)
However when I request a big json string I get the following exception.
akka.http.scaladsl.model.EntityStreamException: HTTP chunk size exceeds the configured limit of 1048576 bytes
How do I configure this, I am not using Akka typed(I think), just this:
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
You can use the following configuration to increase the request size:
akka.http.parsing.max-chunk-size = 16m
I have a controller that sends a chunked response:
def streamDatase2t(query:String): Action[AnyContent] = Action.async {
req =>
serivce.getIterator(query).map(res => {
Ok.chunked(Source.apply(res))
})
}
When I try to inspect the returned content in the controller spec I get an exception:
"return 200 response with the content of the iterator" in {
when(serivce.getIterator
(Matchers.any[Request.DatasetLoad],
Matchers.any[ResponseFormat], Matchers.any[Int]))
.thenReturn(Future.successful(new FakeIterable(List("One", "Two", "Three").iterator)))
val fakeRequest = FakeRequest.apply("GET", s"/data")
val result = Helpers.route(fakeApp, fakeRequest).get
checkStatus(result, OK)
contentAsString(result) // <-- exception here !
}
Exception:
NoMaterializer cannot materialize
java.lang.UnsupportedOperationException: NoMaterializer cannot materialize
at play.api.test.NoMaterializer$.materialize(Helpers.scala:732)
at akka.stream.scaladsl.RunnableGraph.run(Flow.scala:629)
at akka.stream.scaladsl.Source.runWith(Source.scala:106)
at akka.stream.scaladsl.Source.runFold(Source.scala:117)
at play.api.http.HttpEntity.consumeData(HttpEntity.scala:49)
at play.api.http.HttpEntity.consumeData$(HttpEntity.scala:48)
at play.api.http.HttpEntity$Chunked.consumeData(HttpEntity.scala:117)
at play.api.test.ResultExtractors.contentAsBytes(Helpers.scala:381)
at play.api.test.ResultExtractors.contentAsBytes$(Helpers.scala:379)
at play.api.test.Helpers$.contentAsBytes(Helpers.scala:676)
As the Exception states NoMaterializer cannot materialize you may need to add a Materializer:
implicit lazy val mat = ActorMaterializer()
implicit lazy val ec = instanceOf[ExecutionContext]
contentAsString has NoMaterializer as the default argument
def contentAsString(of: Future[Result])(implicit timeout: Timeout, mat: Materializer = NoMaterializer): String
NoMaterializer just throws UnsupportedOperationException for everything so try providing your own
implicit val actorSystem = ActorSystem("test")
implicit val materializer = ActorMaterializer()
play-scala-streaming-example demonstrates how we might write a test for streaming controller.
Addressing the comment, consider the following two routes which illustrate the difference between a strict and non-strict (chunked, streamed) body
def nonStrictBody = Action {
val source = Source.apply(List("woo", "h", "oo"))
Ok.chunked(source)
}
def strictBody = Action {
Ok("woohoo")
}
When calling contentAsString on a strict body, then materializer will not be used, hence NoMaterializer is sufficient
In 99% of cases, when running tests against the result body, you don't
actually need a materializer since it's a strict body. So, rather than
always requiring an implicit materializer, we use one if provided,
otherwise we have a default one that simply throws an exception if
used.
However when calling contentAsString on a chunked or streamed body, as it is the case in the nonStrictBody route, then we need to provide a proper Materializer.
I am extracting tweets using Twitter4J and Akka Streams. I have chosen a few fields like userId, tweetId, tweet text and so on. This Tweet entity gets written to the database:
class Counter extends StatusAdapter with Databases{
implicit val system = ActorSystem("TweetsExtractor")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
implicit val LoggingAdapter =
Logging(system, classOf[Counter])
val overflowStrategy = OverflowStrategy.backpressure
val bufferSize = 1000
val statusSource = Source.queue[Status](
bufferSize,
overflowStrategy
)
val insertFlow: Flow[Status, Tweet, NotUsed] =
Flow[Status].map(status => Tweet(status.getId, status.getUser.getId, status.getText, status.getLang,
status.getFavoriteCount, status.getRetweetCount))
val insertSink: Sink[Tweet, Future[Done]] = Sink.foreach(tweetRepository.create)
val insertGraph = statusSource via insertFlow to insertSink
val queueInsert = insertGraph.run()
override def onStatus(status: Status) =
Await.result(queueInsert.offer(status), Duration.Inf)
}
My intention is to add location field. There is a specific GeoLocation type for that in Twitter4J which contains latitude and longitude of double type. However, when I try to extract latitude and longitude directly through the flow nothing is written to the database:
Flow[Status].map(status => Tweet(status.getId, status.getUser.getId, status.getText, status.getLang, status.getFavoriteCount, status.getRetweetCount, status.getGeoLocation.getLatitude, status.getGeoLocation.getLongitude))
What may be the reason of such behavior and how can I fix it?
What is happening here, as confirmed in the comments to the question, is that most tweets do not come with geolocation data attached, making those fields empty and resulting in the misbehavior.
A couple of simple checks for empty values should solve the issue.
I have this post request and have two issues.
1.is with the headers value. The documentation says it takes in a Seq[HttpHeader] and I pass in a Seq[RawHeader] which extends HttpHeader but it says it's a type mismatch. Why?
2.I pass in the data I want to post, but the HttpEntity.Default() takes in a Source[Bytestring]. How do I convert my data to Source[Bytestring]
def post(data: String): Unit = {
val headers = Seq(RawHeader("X-Access-Token", "access token"))
val responseFuture: Future[HttpResponse] =
Http(system).singleRequest(
HttpRequest(
HttpMethods.POST,
"https://beta-legacy-api.ojointernal.com/centaur/user",
headers,
entity = HttpEntity.Default(data)
)
)
}
I pass in a Seq[RawHeader] which extends HttpHeader but it says it's
a type mismatch. Why?
Because Seq[A] is invariant in A.
How do I convert my data to Source[Bytestring]
You don't have to. You can use use the HttpEntity apply method which takes an Array[Byte], and use withHeaders:
import akka.http.scaladsl.model._
def post(data: String): Unit = {
val responseFuture: Future[HttpResponse] =
Http(system).singleRequest(
HttpRequest(
HttpMethods.POST,
"https://beta-legacy-api.ojointernal.com/centaur/user",
entity = HttpEntity(ContentTypes.`application/json`, data.getBytes())
).withHeaders(RawHeader("X-Access-Token", "access token"))
)
}
I'm trying to build event sourced service with REST interface using scala. I somewhat new to scala, although I'm familiar with functional programming (haskell at beginner level).
So I've build persistent actor and view without major problems. The idea of actors is quite simple I think.
object Main extends App {
val system = ActorSystem("HelloSystem")
val systemActor = system.actorOf(Props[SystemActor], name = "systemactor")
val trajectoryView = system.actorOf(Props[TrajectoryView], name = "trajectoryView")
var datas = List()
val processData = ProcessData(0, List(1,2,3), Coordinates(50, 50))
implicit val timeout = Timeout(5 seconds)
def intialDatas(): List[ProcessData] =
(for (i <- 1 to 3) yield ProcessData(i, List(1,2,3), Coordinates(50 + i, 50 + i)))(collection.breakOut)
val command = RegisterProcessCommand(3, this.intialDatas())
val id = Await.result(systemActor ? command, timeout.duration).asInstanceOf[String]
println(id)
systemActor ! MoveProcessCommand(4, ProcessData(4, List(3,4,5), Coordinates(54, 54)), id)
val processes = Await.result(systemActor ? "get", timeout.duration).asInstanceOf[Set[Process]]
println(processes)
implicit val json4sFormats = DefaultFormats
println(write(processes))
println("*****************")
systemActor ! "print"
val getTrajectoryCommand = GetTrajectoryCommand(id)
Thread.sleep(10000)
trajectoryView ! "print"
// val trajectory = Await.result(trajectoryView ? getTrajectoryCommand, timeout.duration).asInstanceOf[ListBuffer[Coordinates]]
println("******* TRAJECTORY *********")
trajectoryView ! "print"
// println(trajectory)
system.shutdown()
}
I've been able to create a script for playing with actor that I've created.
I've read the tutorials for spray routing, but I've been unable to grasp what exactly should I do to provide REST interface for actors that I've created.
object Boot extends App{
implicit val system = ActorSystem("example")
val systemActor = system.actorOf(Props[SystemActor], name = "systemactor")
val trajectoryView = system.actorOf(Props[TrajectoryView], name = "trajectoryView")
val service = system.actorOf(Props[ProcessesService], "processes-rest-service")
implicit val timeout = Timeout(5 seconds)
IO(Http) ? Http.Bind(service, interface = "localhost", port = 8080)
}
And a service
class ProcessesService(systemActor: ActorRef) extends Actor with HttpService {
def actorRefFactory = context
def receive = runRoute(route)
val json4sFormats = DefaultFormats
implicit val timeout = Timeout(5 seconds)
val route = path("processes") {
get {
respondWithMediaType(`application/json`) {
complete {
write(Await.result(systemActor ? "get", timeout.duration).asInstanceOf[Set[Process]])
}
}
}
}
}
I think I need to somehow pass actorRef for SystemActor to this ProcessesService, but I'm not sure how. Also I'm not sure how should I return a response to the request. I understand that I need to somehow pass the "get" message to SystemActor through ActorRef and then serialize the answer to json, but I don't know how to do that.
I would appreciate help!
In spray you can complete routes with a Future.
You should be able to do something like
complete { systemActor ? "get" }
Json serialization is a separate issue.
Oh, your question is vague. Yes you need to be able to reference an actor within your routes. You could just import the val from boot where you define it. They're just Scala variables so where you put them is up to you.