I am new to AKKA world,
How to send a file as a response using akka http?
I got a solution for how to send the file in RESTAPI response with Akka-HTTP, from the above post but I want to provide a name and extension to that file.
so what is a way to send the file as a response with filename and extension means I want downloaded file should have a name with .zip extension in Akka-HTTP with scala
Adding onto the other answer you can use the Akka HTTP DSL as follows:
val fileContentsSource: (String, String) => Source[ChunkStreamPart, _] =
(fileName, enc) =>
Source
.fromIterator(() => scala.io.Source.fromFile(fileName, enc).getLines)
.map(ChunkStreamPart.apply)
val fileEntityResponse: (String, String) => HttpResponse =
(fileName, enc) =>
HttpResponse(entity = Chunked(ContentTypes.`text/plain(UTF-8)`,
fileContentsSource(fileName, enc)))
val route: Route = {
path("file" / Segment) { fileName =>
complete(fileEntityResponse(fileName, "UTF-8"))
}
}
The Segment is called a PathMatcher and will extract the file name from the route so call it as follows
curl --location --request GET 'localhost:PORT/file/filename.txt'
Related
I have a service that takes an HttpRequest from a client to get a file from another server via REST and then forward the file to the client as an HttpResponse.
Don't ask me why the client doesn't ask for the file him/herself because that is a long story.
I compiled a strategy to download the file to the file system and then send the file to the client. This is using extracts from other stackoveflow responses from #RamonJRomeroyVigil.
def downloadFile(request: HttpRequest, fileName: String): Future[IOResult] = {
Http().singleRequest(request).flatMap { response =>
val source = response.entity.dataBytes
source.runWith(FileIO.toPath(filePath))
}
}
def buildResponse(fileName: String)
val bufferedSrc = scala.io.Source.fromFile(fileName)
val source = Source
.fromIterator(() => bufferedSrc.getLines())
.map(ChunkStreamPart.apply)
HttpResponse(entity = HttpEntity.Chunked(ContentTypes.`application/octet-stream`, source))
}
However, I would like to do this in one step without saving the file system and taking advantage of the streaming abilities.
I also would like to limit the amount of request the client can serve at the same time to 5.
Thanks
As you are already getting the file as a stream from the second server, you can forward it directly to the client. You only need to build your HttpResponse on the fly :
def downloadFile(request: HttpRequest) : Future[HttpResponse] = {
Http().singleRequest(request).map {
case okResponse # HttpResponse(StatusCodes.OK, _, _, _) =>
HttpResponse(
entity = HttpEntity.Chunked(ContentTypes.`application/octet-stream`,
okResponse
.entity
.dataBytes
.map(ChunkStreamPart.apply)
))
case nokResponse # HttpResponse(_, _, _, _) =>
nokResponse
}
}
To change the maximum number of concurrent requests allowed for the client, you would need to set akka.http.client.host-connection-pool.max-connections and
akka.http.client.host-connection-pool.max-open-requests. More details can be found here.
So the Play documentation you can find here gives a very simple example for a GET call without anything like auth header or parameters.
Could somebody help me figure out how to use a similar strategy for something more complex like a POST request with JSON data as body and auth header required? I can't get it to work right now.
I want to know how to test a client using ws to do it's external http requests.
Thanks
Here is a snippet of code from one of my projects which sends sms via twilio api:
class SmsServiceImpl #Inject()(config: Configuration, ws: WSClient) extends SmsService {
val twilloAccountSid = config.getString("twillo.accountSid").get
val twilloAuthToken = config.getString("twillo.authToken").get
val smsApiUrl = config.getString("twillo.apiBaseUrl").get + "/" +twilloAccountSid+"/Messages.json"
override def send(phone: String, message: String): Future[Unit] = {
val data = Map(
"To" -> Seq(phone),
"From" -> Seq(config.getString("twillo.fromPhone").get),
"Body" -> Seq(message)
)
val response: Future[WSResponse] = ws.url(smsApiUrl).withAuth(twilloAccountSid, twilloAuthToken, WSAuthScheme.BASIC).post(data)
response.map { response =>
response.status match {
case 201 => Unit
case _ => throw new SmsSendingException(s"Failed to send sms. Got https status: ${response.status}")
}
}
}
It is POST request with authentication.
I have been having issues with downloading media from the media uri provided on the mms messages.
val url = https://api.twilio.com/2010-04-01/Accounts/xx/Messages/xx/Media/xx
the media url provided is in the above structure,
new URL(url) #> new File("file.png") !! //this fails, due to multiple redirects
When I open the URI in browser the redirect ends up in
http://media.twiliocdn.com.s3-external-1.amazonaws.com/xx/xx
1st url -> 2nd url -> above url ;so,all in all 2 redirects
And if I try the snippet posted above with the new url, it works. I am sure its because of the multiple redirects, the snippet didnt work in the first place.
Been using play framework with scala, can I get any source example to download the file. Any help or pointers is appreciated. Tried various examples but still could not solve the issue.
Some findings =>
Accessing Twilio MMS images
anything similar for scala?
Update: #millhouse
def fileDownloader(urls: String, location: String) = {
import play.api.Play.current
import scala.concurrent.ExecutionContext.Implicits.global
// Make the request
val futureResponse: Future[(WSResponseHeaders, Enumerator[Array[Byte]])] =
WS.url(urls).withFollowRedirects(true).getStream()
futureResponse.flatMap {
case (headers, body) =>
val file = new File(location)
val outputStream = new FileOutputStream(file)
// The iteratee that writes to the output stream
val iteratee = Iteratee.foreach[Array[Byte]] { bytes =>
outputStream.write(bytes)
}
// Feed the body into the iteratee
(body |>>> iteratee).andThen {
case result =>
// Close the output stream whether there was an error or not
outputStream.close()
// Get the result or rethrow the error
result.get
}.map(_ => file)
}
}
This is the approach I had been using till now(works), as explained in the play docs. But I needed a sync approach, meaning I would need to carry out another step on successful file download. Sorry, for not clarifying out ahead.
Update 2 : Solved in this manner,
def fileDownloader(urls: String, location: String) = {
import play.api.Play.current
import scala.concurrent.ExecutionContext.Implicits.global
// Make the request
val futureResponse: Future[(WSResponseHeaders, Enumerator[Array[Byte]])] =
WS.url(urls).withFollowRedirects(true).getStream()
val downloadedFile: Future[File] = futureResponse.flatMap {
case (headers, body) =>
val file = new File(location)
val outputStream = new FileOutputStream(file)
// The iteratee that writes to the output stream
val iteratee = Iteratee.foreach[Array[Byte]] { bytes =>
outputStream.write(bytes)
}
// Feed the body into the iteratee
(body |>>> iteratee).andThen {
case result =>
// Close the output stream whether there was an error or not
outputStream.close()
// Get the result or rethrow the error
result.get
}.map(_ => file)
}
downloadedFile.map{ fileIn =>
//things needed to do
}
}
Thanks,
I haven't used the Twilio MMS API but it should be very straightforward to get the Play Framework HTTP client to follow redirects, using the documented option to the client:
val url = "https://api.twilio.com/2010-04-01/Accounts/xx/Messages/xx/Media/xx"
ws.url(url).withFollowRedirects(true).get().map { response =>
val theBytes:Array[Byte] = response.bodyAsBytes // Play 2.4 and lower
// ... save it
}
Note that the above code works for Play 2.4.x and lower; the bodyAsBytes method of WSResponse returns an Array[Byte]. If you're on the current cutting-edge and using Play 2.5.x, bodyAsBytes gives you an Akka ByteString with lots of nice functional methods, but you probably just want to call toArray on it if all you want is to store the data:
ws.url(url).withFollowRedirects(true).get().map { response =>
val theBytes:Array[Byte] = response.bodyAsBytes.toArray // Play 2.5
// ... save it
}
How do I do a full proxy with Play Framework?
I want to keep the headers and body intact for both the request and the response. Basically, a transparent proxying layer to both the client and server.
Note: I've got something working. Will post it when SO allows me to.
This is what I end up with.
With my (not comprehensive) testing this works for all methods with various body types.
Note my use of _.head. I haven't dug into why headers have the type Map[String, Seq[String]]. I might be dropping duplicate header contents (e.g. having more than on Content-Type in the header). Perhaps joining the Seq[String] with ; is the better way.
import play.api.libs.ws._
import play.api.libs.iteratee.Enumerator
import play.api.mvc._
def proxy(proxyUrl: String) = Action.async(BodyParsers.parse.raw) { request =>
// filter out the Host and potentially any other header we don't want
val headers: Seq[(String, String)] = request.headers.toSimpleMap.toSeq.filter {
case (headerStr, _) if headerStr != "Host" => true
case _ => false
}
val wsRequestBase: WSRequestHolder = WS.url(s"http://localhost:9000/$proxyUrl") // set the proxy path
.withMethod(request.method) // set our HTTP method
.withHeaders(headers : _*) // Set our headers, function takes var args so we need to "explode" the Seq to var args
.withQueryString(request.queryString.mapValues(_.head).toSeq: _*) // similarly for query strings
// depending on whether we have a body, append it in our request
val wsRequest: WSRequestHolder = request.body.asBytes() match {
case Some(bytes) => wsRequestBase.withBody(bytes)
case None => wsRequestBase
}
wsRequest
.stream() // send the request. We want to process the response body as a stream
.map { case (responseHeader: WSResponseHeaders, bodyStream: Enumerator[Array[Byte]]) => // we want to read the raw bytes for the body
// Content stream is an enumerator. It's a 'stream' that generates Array[Byte]
new Result(
new ResponseHeader(responseHeader.status),
bodyStream
)
.withHeaders(responseHeader.headers.mapValues(_.head).toSeq: _*)
}
}
routes file entry will look something like this:
GET /proxy/*proxyUrl #controllers.Application.proxy(proxyUrl: String)
You will need to other lines to support other methods (e.g. POST)
Feel free to suggest edits.
When I try to GET amazon identity data like that
val pipeline: HttpRequest => Future[IdentityData] = sendReceive ~> unmarshal[IdentityData]
pipeline(Get("http://169.254.169.254/latest/dynamic/instance-identity/document"))
with appropriate case class and formatter, I receive the following exception
UnsupportedContentType(Expected 'application/json')
because amazon mark their response as text/plain content type. They also don't care about the Accept header param. Is there an easy way to tell spray-json to ignore this on unmarshalling?
After digging in spray mail list I wrote a function that works
def mapTextPlainToApplicationJson: HttpResponse => HttpResponse = {
case r# HttpResponse(_, entity, _, _) =>
r.withEntity(entity.flatMap(amazonEntity => HttpEntity(ContentType(MediaTypes.`application/json`), amazonEntity.data)))
case x => x
}
and the used it in the pipeline
val pipeline: HttpRequest => Future[IdentityData] = sendReceive ~> mapTextPlainToApplicationJson ~> unmarshal[IdentityData]
pipeline(Get("http://169.254.169.254/latest/dynamic/instance-identity/document"))
The cool thing is here you can intercept & alter any HttpResponse as long as your intercepting function has the appropriate signature.
If you want to extract some IdentityData (which is a case class with a defined jsonFormat) from amazon response, which is a valid json, but with text/plain context type you can simply extract text data, parse it a json and convert into your data, e.g:
entity.asString.parseJson.convertTo(identityDataJsonFormat)
I've come up with a simpler/cleaner version of #yevgeniy-mordovkin's solution.
def setContentType(mediaType: MediaType)(r: HttpResponse): HttpResponse = {
r.withEntity(HttpEntity(ContentType(mediaType), r.entity.data))
}
Usage:
val pipeline: HttpRequest => Future[IdentityData] = (
sendReceive
~> setContentType(MediaTypes.`application/json`)
~> unmarshal[IdentityData]
)
pipeline(Get("http://169.254.169.254/latest/dynamic/instance-identity/document"))