Grails 2.5. I need to log the request and the response to log file.
This is to log REST/JSON api calls. I need to log both the request JSON (which is easy with a filter), and the response JSON (Which is looking impossible)
an api call looks like this:
class myController {
def myMethod() {
def json = request.JSON
:
render(status:200, contentType:'application/json' {
['something': "something",'sometingElse' : 42]
}
Tried creating a filter, e.g:
class LoggingFilters {
def filters = {
after = { Map model ->
log.debug "${response.???"
}
The question is, how do you get a handle on the output? Tried respose.outputStream.toString() etc, but it says something has already got the output stream.
I saw one suggestion which said don't use render, return a model with a map with a single item: data, and render this outside against some dummy jsp. Then you can get the model in the filter. The problem with this is that we have many hundreds of render statements, and we dont really want to retest everything (we have no automated testing).
Another other work around is to use tomcat request dumper filter, but this would not meet our requirements (e.g. we need to scrub certain output, such as passwords, want to be able to log to db, only want to log very specific things etc).
We tried this plugin: https://grails.org/plugin/httplogger , but it doesnt work with grails 2.5.
Related
I'm developing a scala application with play frame work, i have created a filter that filters every request coming from outside server,but now i'm stuck on how can i run a filter on demand since two days,i have 80 APIs 30 of them needs to run a specific filter, how can i read the request route template while the requests like this
GET /api/v1/:locale/:uuid core.controllers.MyClass.myAction1(locale: String)
GET /api/v1/:locale/:uuid/MyRoute core.controllers.MyClass.myAction2(locale: String)
GET /api/v1/:locale/:uuid/Foo core.controllers.MyClass.myAction3(locale: String)
GET /api/v1/:locale/orders/:orderId core.controllers.MyClass.myAction4(locale: String)
well, those routes are placed in routes file,
in filter i need to check weather if the route has :uuid variable or :orderId in order to run its specific filter, because both of their ids, i getting them as uuid so i couldn't expect the request, could i read the route template ?
You can access to some routing information from the RequestHeader#attrs:
// in your filter
val handlerDef: Option[HandlerDef] = request.attrs.get(Router.Attrs.HandlerDef)
See HandlerDef api
If you want to choose 30 out of 80 actions to run some common logic, you could also consider using "action builders" to provide that logic.
When you use Action { ... } you get a vanilla action. You can also make your own MyAction { ... } that wraps a normal Action and runs custom logic. This is an ActionBuilder. If you use this approach you just need to update your 30 actions to use that custom action builder.
See: https://www.playframework.com/documentation/2.6.x/ScalaActionsComposition#Custom-action-builders
I am building a REST API and facing this issue: How can REST API pass very large JSON?
Basically, I want to connect to Database and return the training data. The problem is in Database I have 400,000 data. If I wrap them into a JSON file and pass through GET method, the server would throw Heap overflow exception.
What methods we can use to solve this problem?
DBTraining trainingdata = new DBTraining();
#GET
#Produces("application/json")
#Path("/{cat_id}")
public Response getAllDataById(#PathParam("cat_id") String cat_id) {
List<TrainingData> list = new ArrayList<TrainingData>();
try {
list = trainingdata.getAllDataById(cat_id);
Gson gson = new Gson();
Type dataListType = new TypeToken<List<TrainingData>>() {
}.getType();
String jsonString = gson.toJson(list, dataListType);
return Response.ok().entity(jsonString).header("Access-Control-Allow-Origin", "*").header("Access-Control-Allow-Methods", "GET").build();
} catch (SQLException e) {
logger.warn(e.getMessage());
}
return null;
}
The RESTful way of doing this is to create a paginated API. First, add query parameters to set page size, page number, and maximum number of items per page. Use sensible defaults if any of these are not provided or unrealistic values are provided. Second, modify the database query to retrieve only a subset of the data. Convert that to JSON and use that as the payload of your response. Finally, in following HATEOAS principles, provide links to the next page (provided you're not on the last page) and previous page (provided you're not on the first page). For bonus points, provide links to the first page and last page as well.
By designing your endpoint this way, you get very consistent performance characteristics and can handle data sets that continue to grow.
The GitHub API provides a good example of this.
My suggestion is no to pass the data as a JSON but as a file using multipart/form-data. In your file, each line could be a JSON representing a data record. Then, it would be easy to use a FileOutputStream to receive te file. Then, you can process the file line by line to avoid memory problems.
A Grails example:
if(params.myFile){
if(params.myFile instanceof org.springframework.web.multipart.commons.CommonsMultipartFile){
def fileName = "/tmp/myReceivedFile.txt"
new FileOutputStream(fileName).leftShift(params.myFile.getInputStream())
}
else
//print or signal error
}
You can use curl to pass your file:
curl -F "myFile=#/mySendigFile.txt" http://acme.com/my-service
More details on a similar solution on https://stackoverflow.com/a/13076550/2476435
HTTP has the notion of chunked encoding that allows you send a HTTP response body in smaller pieces to prevent the server from having to hold the entire response in memory. You need to find out how your server framework supports chunked encoding.
I'm using Grails 2.3.2 working on a REST API using the built in Grails REST support. I'm having trouble getting rid of the "class" element in the JSON response. Based on a tutorial by Bobby Warner, I have found adding the following to the resources.groovy file:
meterRenderer(JsonRenderer, Meter) {
excludes = ['class']
}
This works fine for show, but for the index controller function, I'm responding with a list of Meters. In this, the "class" doesn't go away. What does it take to get rid of this in the list response?
Edit: To clarify, I am looking for a way to leverage the Content Negotiation feature of Grails new respond functionality without locking myself down to render as JSON implementions.
I guess if you switch to using GSON (github) instead of JSON then you need not worry about that particular exclusion.
That is driven by a config setting provided by the plugin as grails.converters.gson.domain.include.class (default is false).
nickdos' SO link had the answer. I added the following to my BootStrap.groovy:
grails.converters.JSON.registerObjectMarshaller(Meter) {
return it.properties.findAll {k,v -> k != 'class'}
}
And the respond call results in no "class" item. Oddly enough, I lost the "id" item in the process, but I'll save that for another SO question. :)
In my play framework 2 application I'd like to have a log message with the request, response, and some details about the response - such as the number of search results returned from an external web call.
What I have now is a filter like this:
object AccessLog extends Filter {
import play.api.mvc._
import play.api.libs.concurrent.Execution.Implicits._
def apply(next: RequestHeader => Future[SimpleResult])(request: RequestHeader): Future[SimpleResult] = {
val result = next(request)
result map { r =>
play.Logger.info(s"Request: ${request.uri} - Response: ${r.header.status}")
}
result
}
}
At the point of logging, I've alread converted my classes into json, so it seems wasteful to parse the json back into objects so I can log information about it.
Is it possible to compute the number of search results earlier in the request pipeline, maybe into a dictionary, and pull them out when I log the message here?
I was looking at flash, but don't want the values to be sent out in a cookie at any cost. Maybe I can clear the flash instead. Buf if there's a more suitable way I'd like to see that.
This is part of a read-only API that does not involve user accounts or sessions.
You could try using the play.api.cache.Cache object if you can come up with a reproducible unique request identifier. Once you have logged your request, you can remove it from the Cache.
I am trying to use HTTP to POST a file to an outside API from within a grails service. I've installed the rest plugin and I'm using code like the following:
def theFile = new File("/tmp/blah.txt")
def postBody = [myFile: theFile, foo:'bar']
withHttp(uri: "http://picard:8080/breeze/project/acceptFile") {
def html = post(body: postBody, requestContentType: URLENC)
}
The post works, however, the 'myFile' param appears to be a string rather than an actual file. I have not had any success trying to google for things like "how to post a file in grails" since most of the results end up dealing with handling an uploaded file from a form.
I think I'm using the right requestContentType, but I might have missed something in the documentation.
POSTing a file is not as simple as what you have included in your question (sadly). Also, it depends on what the API you are calling is expecting, e.g. some API expect files as base64 encoded text, while others accept them as mime-multipart.
Since you are using the rest plugin, as far as I can recall it uses the Apache HttpClient, I think this link should provide enough info to get you started (assuming you are dealing with mime-multipart). It shouldn't be too hard to change it around to work with your API and perhaps make it a bit 'groovy-ier'