playframework 2.4 - Unspecified value parameter headers error - scala

I am upgrading playframework 2.4 from 2.3, I changed versions then if I compile same code, I see following error. Since I am novice at Scala, I am trying to learn Scala to solve this issue but still don't know what is the problem. What I want to do is adding a request header value from original request headers. Any help will be appreciated.
[error] /mnt/garner/project/app-service/app/com/company/playframework/filters/LoggingFilter.scala:26: not enough arguments for constructor Headers: (headers: Seq[(String, String)])play.api.mvc.Headers.
[error] Unspecified value parameter headers.
[error] val newHeaders = new Headers { val data = (requestHeader.headers.toMap
The LoggingFilter class
class LoggingFilter extends Filter {
val logger = AccessLogger.getInstance();
def apply(next: (RequestHeader) => Future[Result])(requestHeader: RequestHeader): Future[Result] = {
val startTime = System.currentTimeMillis
val requestId = logger.createLog();
val newHeaders = new Headers { val data = (requestHeader.headers.toMap
+ (AccessLogger.X_HEADER__REQUEST_ID -> Seq(requestId))).toList }
val newRequestHeader = requestHeader.copy(headers = newHeaders)
next(newRequestHeader).map { result =>
val endTime = System.currentTimeMillis
val requestTime = endTime - startTime
val bytesToString: Enumeratee[ Array[Byte], String ] = Enumeratee.map[Array[Byte]]{ bytes => new String(bytes) }
val consume: Iteratee[String,String] = Iteratee.consume[String]()
val resultBody : Future[String] = result.body |>>> bytesToString &>> consume
resultBody.map {
body =>
logger.finish(requestId, result.header.status, requestTime, body)
}
result;
}
}
}
Edit
I updated codes as following and it compiled well
following codes changed
val newHeaders = new Headers { val data = (requestHeader.headers.toMap
+ (AccessLogger.X_HEADER__REQUEST_ID -> Seq(requestId))).toList }
to
val newHeaders = new Headers((requestHeader.headers.toSimpleMap
+ (AccessLogger.X_HEADER__REQUEST_ID -> requestId)).toList)

It simply states that if you want to construct Headers you need to supply a field named headers which is of type Seq[(String, String)]. If you omit the inital new you will be using the apply function of the corresponding object for Headers which will just take a parameter of a vararg of (String, String) and your code should work. If you look at documentation https://www.playframework.com/documentation/2.4.x/api/scala/index.html#play.api.mvc.Headers and flip between the docs for object and class it should become clear.

Related

Reading multiple files with akka streams in scala

I'am trying to read multiple files with akka streams and put result in a list.
I can read one file with no problem. the return type is Future[Seq[String]]. problem is processing the sequence inside the Future must go inside an onComplete{}.
i'am trying the following code but abviously it will not work. the list acc outside of the onComplete is empty. but holds values inside the inComplete. I understand the problem but i don't know how to approach this.
// works fine
def readStream(path: String, date: String): Future[Seq[String]] = {
implicit val system = ActorSystem("Sys")
val settings = ActorMaterializerSettings(system)
implicit val materializer = ActorMaterializer(settings)
val result: Future[Seq[String]] =
FileIO.fromPath(Paths.get(path + "transactions_" + date +
".data"))
.via(Framing.delimiter(ByteString("\n"), 256, true))
.map(_.utf8String)
.toMat(Sink.seq)(Keep.right)
.run()
var aa: List[scala.Array[String]] = Nil
result.onComplete(x => {
aa = x.get.map(line => line.split('|')).toList
})
result
}
//this won't work
def concatFiles(path : String, date : String, numberOfDays : Int) :
List[scala.Array[String]] = {
val formatter = DateTimeFormatter.ofPattern("yyyyMMdd");
val formattedDate = LocalDate.parse(date, formatter);
var acc = List[scala.Array[String]]()
for( a <- 0 to numberOfDays){
val date = formattedDate.minusDays(a).toString().replace("-", "")
val transactions = readStream(path , date)
var result: List[scala.Array[String]] = Nil
transactions.onComplete(x => {
result = x.get.map(line => line.split('|')).toList
acc= acc ++ result })
}
acc}
General Solution
Given an Iterator of Paths values a Source of the file lines can be created by combining FileIO & flatMapConcat:
val lineSourceFromPaths : (() => Iterator[Path]) => Source[String, _] = pathsIterator =>
Source
.fromIterator(pathsIterator)
.flatMapConcat { path =>
FileIO
.fromPath(path)
.via(Framing.delimiter(ByteString("\n"), 256, true))
.map(_.utf8String)
}
Application to Question
The reason your List is empty is because the Future values have not completed and therefore your mutable list is not be updated before the function returns the list.
Critique of Code in Question
The organization and style of the code within the question suggest several misunderstandings related to akka & Future. I think you are attempting a rather complex workflow without understanding the fundamentals of the tools you are trying to use.
1.You should not create an ActorSystem each time a function is being called. There is usually 1 ActorSystem per application and it's created only once.
implicit val system = ActorSystem("Sys")
val settings = ActorMaterializerSettings(system)
implicit val materializer = ActorMaterializer(settings)
def readStream(...
2.You should try to avoid mutable collections and instead use Iterator with corresponding functionality:
def concatFiles(path : String, date : String, numberOfDays : Int) : List[scala.Array[String]] = {
val formattedDate = LocalDate.parse(date, DateTimeFormatter.ofPattern("yyyyMMdd"))
val pathsIterator : () => Iterator[Path] = () =>
Iterator
.range(0, numberOfDays+1)
.map(formattedDate.minusDays)
.map(_.String().replace("-", "")
.map(path => Paths.get(path + "transactions_" + date + ".data")
lineSourceFromPaths(pathsIterator)
3.Since you are dealing with Futures you should not wait for Futures to complete and should instead change the return type of concateFiles to Future[List[Array[String]]].

How can i write the AWS lamda unit test cases in scala?

How we can implement the unit test cases for aws lamda serverless.
My code is
object Test1 extends RequestHandler[APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent] with ResponseObjProcess {
override def handleRequest(input: APIGatewayProxyRequestEvent, context: Context): APIGatewayProxyResponseEvent = {
var response = new APIGatewayProxyResponseEvent()
val gson = new Gson
val requestHttpMethod = input.getHttpMethod
val requestBody = input.getBody
val requestHeaders = input.getHeaders
val requestPath = input.getPath
val requestPathParameters = input.getPathParameters
val requestQueryStringParameters = input.getQueryStringParameters
val parsedBody = JSON.parseFull(requestBody).getOrElse(0).asInstanceOf[Map[String, String]]
println(" parsedBody is:: " + parsedBody)
val active = parsedBody.get("active").getOrElse("false")
val created = parsedBody.get("created").getOrElse("0").toLong
val updated = parsedBody.get("updated").getOrElse("0").toLong
requestHttpMethod match {
case "PUT" =>
println(" PUT Request method ")
// insertRecords("alert_summary_report", requestBody)
response.setStatusCode(200)
response.setBody(gson.toJson("PUT"))
case _ =>
println("")
response.setStatusCode(400)
response.setBody(gson.toJson("None"))
}
response
}
}
And I tried to implement unit test cases for the above code.
Below code is:
test("testing record success case") {
var request = new APIGatewayProxyRequestEvent();
request.setHttpMethod(Constants.PUTREQUESTMETHOD)
DELETEREQUESTBODY.put("id", "")
request.setBody(EMPTYREQUESTBODY)
request.setPathParameters(DELETEREQUESTBODY)
println(s"body = ${request.getBody}")
println(s"headers = ${request.getHeaders}")
val response = ProxyRequestMain.handleRequest(subject, testContext)
val assertEqual = response.getStatusCode.equals(200)
assertEqual
}
Actually, I'm getting response.getStatusCode=400 bad requests but test case passed how can I write handle this.
I am looking at your test code and it's not clear to me what you are trying to achieve with your assertions. I think you might have mixed quite a few things. In the code as it currently stands, you have a val, not assertion. I'd encourage you to have a look at the relevant docs and research the options available to you:
http://www.scalatest.org/user_guide/using_assertions
http://www.scalatest.org/user_guide/using_matchers

File Upload and processing using akka-http websockets

I'm using some sample Scala code to make a server that receives a file over websocket, stores the file temporarily, runs a bash script on it, and then returns stdout by TextMessage.
Sample code was taken from this github project.
I edited the code slightly within echoService so that it runs another function that processes the temporary file.
object WebServer {
def main(args: Array[String]) {
implicit val actorSystem = ActorSystem("akka-system")
implicit val flowMaterializer = ActorMaterializer()
val interface = "localhost"
val port = 3000
import Directives._
val route = get {
pathEndOrSingleSlash {
complete("Welcome to websocket server")
}
} ~
path("upload") {
handleWebSocketMessages(echoService)
}
val binding = Http().bindAndHandle(route, interface, port)
println(s"Server is now online at http://$interface:$port\nPress RETURN to stop...")
StdIn.readLine()
binding.flatMap(_.unbind()).onComplete(_ => actorSystem.shutdown())
println("Server is down...")
}
implicit val actorSystem = ActorSystem("akka-system")
implicit val flowMaterializer = ActorMaterializer()
val echoService: Flow[Message, Message, _] = Flow[Message].mapConcat {
case BinaryMessage.Strict(msg) => {
val decoded: Array[Byte] = msg.toArray
val imgOutFile = new File("/tmp/" + "filename")
val fileOuputStream = new FileOutputStream(imgOutFile)
fileOuputStream.write(decoded)
fileOuputStream.close()
TextMessage(analyze(imgOutFile))
}
case BinaryMessage.Streamed(stream) => {
stream
.limit(Int.MaxValue) // Max frames we are willing to wait for
.completionTimeout(50 seconds) // Max time until last frame
.runFold(ByteString(""))(_ ++ _) // Merges the frames
.flatMap { (msg: ByteString) =>
val decoded: Array[Byte] = msg.toArray
val imgOutFile = new File("/tmp/" + "filename")
val fileOuputStream = new FileOutputStream(imgOutFile)
fileOuputStream.write(decoded)
fileOuputStream.close()
Future(Source.single(""))
}
TextMessage(analyze(imgOutFile))
}
private def analyze(imgfile: File): String = {
val p = Runtime.getRuntime.exec(Array("./run-vision.sh", imgfile.toString))
val br = new BufferedReader(new InputStreamReader(p.getInputStream, StandardCharsets.UTF_8))
try {
val result = Stream
.continually(br.readLine())
.takeWhile(_ ne null)
.mkString
result
} finally {
br.close()
}
}
}
}
During testing using Dark WebSocket Terminal, case BinaryMessage.Strict works fine.
Problem: However, case BinaryMessage.Streaming doesn't finish writing the file before running the analyze function, resulting in a blank response from the server.
I'm trying to wrap my head around how Futures are being used here with the Flows in Akka-HTTP, but I'm not having much luck outside trying to get through all the official documentation.
Currently, .mapAsync seems promising, or basically finding a way to chain futures.
I'd really appreciate some insight.
Yes, mapAsync will help you in this occasion. It is a combinator to execute Futures (potentially in parallel) in your stream, and present their results on the output side.
In your case to make things homogenous and make the type checker happy, you'll need to wrap the result of the Strict case into a Future.successful.
A quick fix for your code could be:
val echoService: Flow[Message, Message, _] = Flow[Message].mapAsync(parallelism = 5) {
case BinaryMessage.Strict(msg) => {
val decoded: Array[Byte] = msg.toArray
val imgOutFile = new File("/tmp/" + "filename")
val fileOuputStream = new FileOutputStream(imgOutFile)
fileOuputStream.write(decoded)
fileOuputStream.close()
Future.successful(TextMessage(analyze(imgOutFile)))
}
case BinaryMessage.Streamed(stream) =>
stream
.limit(Int.MaxValue) // Max frames we are willing to wait for
.completionTimeout(50 seconds) // Max time until last frame
.runFold(ByteString(""))(_ ++ _) // Merges the frames
.flatMap { (msg: ByteString) =>
val decoded: Array[Byte] = msg.toArray
val imgOutFile = new File("/tmp/" + "filename")
val fileOuputStream = new FileOutputStream(imgOutFile)
fileOuputStream.write(decoded)
fileOuputStream.close()
Future.successful(TextMessage(analyze(imgOutFile)))
}
}

Compilation error not found: value x but I declared it already, what's wrong

result.map { res =>
val totaldocs: Int = res.value
// do something with this number
}
//val totaldocs = 60
val totalpages:Int = (totaldocs/ipp)+1
Compilation error not found: value x but I declared it already, what is wrong with my implementation, sorry I am new to play framework and scala programming language.
I would say this line is the problem:
val totalpages:Int = (totaldocs/ipp)+1
because totaldocs is only defined inside the map scope
maybe you want something like:
private def getTotalPages(query:BSONDocument, ipp:Int) (implicit ec: ExecutionContext) = {
val key = collectionName + ":" + BSONDocument.pretty(query)
Logger.debug("Query key = "+key)
val command = Count(query)
val result: Future[CountResult] = collection.runCommand(command)
result.map { res =>
val totaldocs: Int = res.value
// do something with this number
val totalpages:Int = (totaldocs/ipp)+1
Logger.debug(s"Total docs $totaldocs, Total pages $totalpages, Items per page, $ipp")
totalpages
}
}
but now it will return a Future[Int] and you will have to deal with the future on the caller.
Note: this is just one solution, depending on your code it may not be the most adequate one

Scala Play 2.5.0 2.5.1 Filtering with access to result body

I didn't find a way to get the body within a Filter in Play 2.5.x.
I want to create a "BadRequestLogFilter", which should log the request AND the result, if my appliation returns a status code 400-500
In Play 2.4.x, I used Iteratees and it worked.
I was not able to migrate this piece of code to Play 2.5.x. Can somebody give me a hint here? Maybe the hole approach to get the body in an Filter is an bad idea?
Here is my old (in 2.4.x working properly) Filter for Play 2.4.x
class BadRequestLogFilter #Inject() (implicit val mat: Materializer, ec: ExecutionContext) extends Filter {
val logger = Logger("bad_status").underlyingLogger
override def apply(next: (RequestHeader) => Future[Result])(request: RequestHeader): Future[Result] = {
val resultFuture = next(request)
resultFuture.foreach(result => {
val status = result.header.status
if (status < 200 || status >= 400) {
val c = Try(request.tags(Router.Tags.RouteController))
val a = Try(request.tags(Router.Tags.RouteActionMethod))
val body = result.body.run(Iteratee.fold(Array.empty[Byte]) { (memo, nextChunk) => memo ++ nextChunk })
val futResponse = body.map(bytes => new String(bytes))
futResponse.map { response =>
val m = Map("method" -> request.method,
"uri" -> request.uri,
"status" -> status,
"response" -> response,
"request" -> request,
"controller" -> c.getOrElse("empty"),
"actionMethod" -> a.getOrElse("empty"))
val msg = m.map { case (k, v) => s"$k=$v" }.mkString(", ")
logger.info(appendEntries(m), msg)
}
}
})
resultFuture
}
}
I guess I just need an valid replacement for this line here:
val body = result.body.run(Iteratee.fold(Array.empty[Byte]) { (memo, nextChunk) => memo ++ nextChunk })
The body of a result in Play 2.5.x is of type HttpEntity
So once you have the result you can get the body and then materialize it:
val body = result.body.consumeData(mat)
Here mat is the implicit Materializer you have. This is going to return you a Future[ByteString] which you can then decode to get a String representation (I have omitted future handling here for simplicity):
val bodyAsString = body.decodeString("UTF-8")
logger.info(bodyAsString)