How to use dispatch.json in lift project - scala

i am confused on how to combine the json library in dispatch and lift to parse my json response.
I am apparently a scala newbie.
I have written this code :
val status = {
val httpPackage = http(Status(screenName).timeline)
val json1 = httpPackage
json1
}
Now i am stuck on how to parse the twitter json response
I've tried to use the JsonParser:
val status1 = JsonParser.parse(status)
but got this error:
<console>:38: error: overloaded method value parse with alternatives:
(s: java.io.Reader)net.liftweb.json.JsonAST.JValue<and>
(s: String)net.liftweb.json.JsonAST.JValue
cannot be applied to (http.HttpPackage[List[dispatch.json.JsObject]])
val status1 = JsonParser.parse(status1)
I unsure and can't figure out what to do next in order to iterate through the data, extract it and render it to my web page.

Here's another way to use Dispatch HTTP with Lift-JSON. This example fetches JSON document from google, parses all "titles" from it and prints them.
import dispatch._
import net.liftweb.json.JsonParser
import net.liftweb.json.JsonAST._
object App extends Application {
val http = new Http
val req = :/("www.google.com") / "base" / "feeds" / "snippets" <<? Map("bq" -> "scala", "alt" -> "json")
val json = http(req >- JsonParser.parse)
val titles = for {
JField("title", title) <- json
JField("$t", JString(name)) <- title
} yield name
titles.foreach(println)
}

The error that you are getting back is letting your know that the type of status is neither a String or java.io.Reader. Instead, what you have is a List of already parsed JSON responses as Dispatch has already done all of the hard work in parsing the response into a JSON response. Dispatch has a very compact syntax which is nice when you are used to it but it can be very obtuse initially, especially when you are first approaching Scala. Often times, you'll find that you have to dive into the source code of the library when you are first learning to see what is going on. For instance, if you look into the dispatch-twitter source code, you can see that the timeline method actually performs a JSON extraction on the response:
def timeline = this ># (list ! obj)
What this method is defining is a Dispatch Handler which converts the Response object into a JsonResponse object, and then parses the response into a list of JSON Objects. That's quite a bit going on in one line. You can see the definition for the operand ># in the JsHttp.scala file in the http+json Dispatch module. Dispatch defines lots of Handlers that do a conversion behind the scenes into different types of data which you can then pass to block to work with. Check out the StdOut Walkthrough and the Common Tasks pages for some of the handlers but you'll need to dive into the various modules source code or Scaladoc to see what else is there.
All of this is a long way to get to what you want, which I believe is essentially this:
val statuses = http(Status(screenName).timeline)
statuses.map(Status.text).foreach(println _)
Only instead of doing a println, you can push it out to your web page in whatever way you want. Check out the Status object for some of the various pre-built extractors to pull information out of the status response.

Related

Scala Customizing Refined Type Exceptions in Http4s

I am using refined for my requests in order to ensure that the request body confronts to certain specs, such as
case class NewGenreRequest(
name: GenreNameParam,
parent: Option[GenreParentParam]
)
where
type GenreNameString = String Refined MinSize[3]
When the request body sends something with 2 characters, this results in the error Predicate taking size(me) = 2 failed: Predicate (2 < 3) did not fail.: DownField(name) in the response body. However I think this isn't the best error to return to the client.
I would like to find a way to customize this response. The logical thing to try was to write a middleware
def handleErr(resp: Response[F]): Response[F] =
resp match {
case Status.BadRequest(resp) =>
logger.info("HERE IS MY ERR")
resp.????
case _ =>
resp
}
However I need to extract the info regarding which field has failed the predicates. The response body is an fs2 Stream and I am not quite sure about how to go about effectively modifying it using the existing info in the body. I tried maping it but to no result.
Is there an easy way of modifying these error messages?
P.S.: Cross posting from http4s discord as I wasn't able to get an answer

validating response outside of context manager?

[EDITED: I realized after reading response that I oversimplified my question.]
I am new to Locust and not sure how to solve this problem.
I have function (call it "get_doc") that is passed a locust.HttpSession() and uses it to issue an HTTP request. It gets the response and parses it, returning it up several layers of call. One of these higher-level calls looks at the returned, parsed document to decide if the response was what was expected or not. If not, I want Locust to mark the request/response as failed. A code sketch would be:
class MyUser (HttpUser):
#task
def mytask(self):
behavior1 (self.client)
def bahavior1(session):
doc = get_doc(session, url1)
if not doc_ok (doc):
??? how to register a failure with Locust here...
doc2 = get_doc(session, url2)
...
def get_doc(http_session, url):
page = http_session.get(url)
doc = parse (page)
return doc
There may be several behavior[n] functions and several Locust users calling them.
A constraint is that I would like to keep Locust-specific stuff out of bahavior1() so that I can call it with an ordinary Requests session. I have tried to do something like this in get_doc() (the catch_response parameter and success/fail stuff is actually conditionalized on 'session' being an HttpSession object):
def get_doc (session, meth, url):
resp = session.request (meth, url, catch_response=True)
doc = parse (resp.content)
doc.logfns = resp.success, resp.failure
return doc
and then in behavior1() or some higher up-chain caller I can
doc.logfns[1]("Document not as expected")
or
doc.logfns[0] # Looks good!
Unfortunately this is not working; the calls to them produce no errors but Locust doesn't seem to record any successes or failures either. I am not sure if it should work or I bungled something in my code. Is this feasible? Is there a better way?
You can make get_doc a context manager, call .get with catch_response=True and yield instead of return inside it. Similar to how it is done here: https://github.com/SvenskaSpel/locust-plugins/blob/2cbbdda9ae37b6cbb0a11cf69aca80b164198aec/locust_plugins/users/rest.py#L22
And then use it like this
def mytask(self):
with get_doc(self.client, url) as doc:
if not doc_ok(doc):
doc.failure(”doc was not ok :(”)
If you want, you can add the parsed doc as a field on the response before yielding in your doc function, or call doc.failure() inside doc_ok.

Gatling - Reading JSON file and sending content using ElFileBody to a method

I am new to Scala and Gatling.
I am trying to write framework for Load and performance testing using Gatling API in Scala for REST API endpoints.
I have a query regarding one of the code snippet which is supposed to generate signature(calling another method) and save the value in the session.
.exec(session => {
session.set("sign", SignatureGeneration.getSignature(key, ElFileBody("abc.json").toString()))
})
abc.json -
{"device": "${device}"}
In above code getSignature takes arguments (String, String). I want to read the json file and replace ${} value in it with the feeders and send it as String to the method.
While debugging the code I found out, ElFileBody send object as <function1> and not the json content of it.
Solution -
val bodyExpr = ElFileBody("abc.json")
val bodyStr = bodyExpr(session).get

Scala Spray+Swagger , how to auto-generate UI for API calls

Working on a project and using Spray
Want to add Swagger to get a nice UI for all the calls
Found :
http://github.com/gettyimages/spray-swagger
Problem is I cant seem to get it to work with my project, with no docs or
run examples its like walking in the dark .... (and time consuming)
did any 1 get to work with this and has a
test example
wiki page
any thing helpful
So i can get get it working?
Thanks!
So I took a stab at this and got it working. You can check out my repo here.
Granted, it is my scratchpad for exploring Spray, so some things may change. Also, this is based on Spray 1.3.0.
The general gist of getting this to work, as I understand it, is:
You have your traits/classes that hold your routes; lets call this a "Resource". Each one of these traits/classes roughly corresponds to a "path" (e.g. /posts)
In your Resources, instead of directly using the Spray DSL to construct one big route, you need to have one method for each endpoint(roughly corresponding to an "action" in Rails controller, or a route in Sinatra/Scalatra). These methods need to be annotated. So for example you would have separate def postCreate, def postDeletes, etc. These would then need to be composed separately for your Resource.
You instantiate a SwaggerHttpService, implementing the necessary methods (including a Sequence of all the traits/classes in 1, and a Sequence of all your ApiModels that are used in your annotations).
The combination of all your routes (across your actual API and the SwaggerHttpService) needs to get passed to your central routing actor's receive method wrapped in a runRoute (this is the actor that gets in Boot.scala)
I believe 1. and 2. are necessary because spray-swagger works with Java annotations, which cannot be retrieved when wrapped inside an opaque function (which is what the Spray routing DSL normally composes for you).
the project has been updated with a wiki and usage information!
This looks promising :
https://github.com/gettyimages/spray-swagger
didn't try it out yet but looking good!
One of my concerns spray-swagger is that the annotations make the code harder to read. Any ideas for how to make the code readable & still get the benefits of Swagger?
#Api(value = "/pet", description = "Operations about pets")
trait PetHttpService extends HttpService {
#ApiOperation(httpMethod = "GET", response = classOf[Pet], value = "Returns a pet based on ID")
#ApiImplicitParams(Array(
new ApiImplicitParam(name = "petId", required = false, dataType = "integer", paramType = "path", value = "ID of pet that needs to be fetched")
))
#ApiResponses(Array(
new ApiResponse(code = 400, message = "Invalid ID Supplied"),
new ApiResponse(code = 404, message = "Pet not found")))
def petGetRoute = get { path("pet" / IntNumber) { petId =>
complete(s"Hello, I'm pet ${petId}!")
} }
}
I'm going to create a trait with annotations & extend to see if the annotations work. Will let you guys know what happens!

Play Framework - Store Information About Current Request

In my play framework 2 application I'd like to have a log message with the request, response, and some details about the response - such as the number of search results returned from an external web call.
What I have now is a filter like this:
object AccessLog extends Filter {
import play.api.mvc._
import play.api.libs.concurrent.Execution.Implicits._
def apply(next: RequestHeader => Future[SimpleResult])(request: RequestHeader): Future[SimpleResult] = {
val result = next(request)
result map { r =>
play.Logger.info(s"Request: ${request.uri} - Response: ${r.header.status}")
}
result
}
}
At the point of logging, I've alread converted my classes into json, so it seems wasteful to parse the json back into objects so I can log information about it.
Is it possible to compute the number of search results earlier in the request pipeline, maybe into a dictionary, and pull them out when I log the message here?
I was looking at flash, but don't want the values to be sent out in a cookie at any cost. Maybe I can clear the flash instead. Buf if there's a more suitable way I'd like to see that.
This is part of a read-only API that does not involve user accounts or sessions.
You could try using the play.api.cache.Cache object if you can come up with a reproducible unique request identifier. Once you have logged your request, you can remove it from the Cache.