When writing a function for AWS Lambda (in Java/Scala), the handler function can have one of different signatures:
// Raw input / output
def handleRequest(is: InputStream): OutputStream = ???
// Using the AWS dependency
def handleRequest(input: APIGatewayProxyRequestEvent, context: Context): APIGatewayProxyResponseEvent = ???
This is the two possible signatures I know of, at least. Maybe there is more.
The difference is that the first one is a raw response, that needs to be packed into a REST response by the API Gateway with manually configured properties like media type and response code.
The second response seems to utilize the Lambda Proxy Integration and will extract all configuration from the APIGatewayProxyResponse.
Now for my question:
The field body of APIGatewayProxyResponse is of type String.
To me it looks like this POJO is serialized to JSON before being sent to the API Gateway.
This would make it impossible to serve binary data like images or PDF files.
The raw OutputStream cannot carry the information about headers etc. (or can it?), which means I cannot serve multiple different media types.
Of course I could convert images, for example, to Base64. But that is far from optimal.
Is there a way I can serve different (binary and non-binary) media types (with correct headers etc.) in a single AWS Lambda handler? How do I have to configure the API Gateway for that?
Note: This answer is based on reading the docs. I haven't tried it in practice so it may not work.
If you open the "Output Format of a Lambda Function for Proxy Integration" section of the documentation you may see isBase64Encoded field and following text:
The output body is marshalled to the frontend as the method response payload. If body is a binary blob, you can encode it as a Base64-encoded string and set isBase64Encoded to true. Otherwise, you can set it to false or leave it unspecified.
And if you open APIGatewayProxyResponse in the .Net library you can see the IsBase64Encoded property there. It looks like it is just Java API that does not expose this field. But all the rest of the infrastructure should support it. You may also see that a similar field was added to APIGatewayProxyRequestEvent.java at some point but not to APIGatewayProxyResponse. So I think a following workaround should work: create your own APIGatewayProxyResponseEvent class with isBase64Encoded field and use it. Or just extend from the standard com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent and add this field in your subclass. I expect that if you match the naming convention, it should work. It should be marshalled to a JSON after all.
Related
I am working with AWS FireLense using Fluent Bit to forward logs from ECS. There is a parsers file with a JSON parser, and the inputs are dynamically generated by the AWS FireLense image. The generated inputs are both of type forward.
Outside of the AWS specific stuff, I know that some input plugins like Tail have the ability to assign a parser. How do forward, dummy, tcp, etc., know which, if any, parser to use?
For reference, here is a link to a sample partial config section https://github.com/aws-samples/amazon-ecs-firelens-under-the-hood/blob/mainline/generated-configs/fluent-bit/generated_by_firelens.conf
I would like to modify an url parameter /resource/{VaRiAbLe} in an API gateway to S3 mapping so that it actually points to /my-bucket/{variable}. That is, it accepts mixed-case input, and maps it to a lower-case name. Mapping path variables is relatively simple enough to S3 integrations, but I can't seem to get a lower-case mapping working.
Reading through the documentation for mapping parameters, it looks like the path parameters are simple string values (and not templated values), and so defining a mapping as method.request.path.variable.toLowerCase() won't work.
Does anyone have any ideas how to implement this mapping?
Map path variables to a JSON body, and then call another API method that actually does the S3 call?
Bite the bullet, and implement a Lambda function to do the S3 get for me?
Find another api method for S3 that accepts a JSON body that I can use to get the data?
Update using Orchestrated calls
Following the info from Jack, I figured I should try doing the orchestrated call, since the traffic volume is low enough that I am sure that I won't be able to keep the lambda hot.
As a proof of concept, I added two methods to my resource (sitting at /resource/{variable} - GET and POST. The GET method chains to the POST, which does the actual retrieving of the data.
POST method configuration
This is a vanilla S3 proxying method, where you set the URL Path parameter for {variable} to be method.request.body.variable.
GET method configuration
This is a HTTPS proxying method. You'll need an URL for the POST method, so you'll need to deploy the API to get the URL. The only other configuration needed here is a body mapping template with content like:
{
"variable" : "$input.params('variable').toLowerCase()",
"something" : "$input.params('something')"
}
This should be enough to get this working.
The downside to this looks to be that I'm adding an extra method (POST) to my API for that resource that could confuse consumers of the API. I think it should be possible to make the POST on the /resource resource, which would at least make a bit more sense from an API design standpoint.
Depending on how frequently this API will be called, I'd either go with the Lambda proxy or chaining two API Gateway methods together. If the API is called frequently enough to keep a Lambda function warm (say once a minute), then go with Lambda. If not, go with the orchestrated API call.
The orchestrated API call would be interesting, I'd be happy to help with that if you have questions.
As far as I know the only S3 API for getting object data is the GET that is documented in their API reference.
I have a Java client that calls a RESTEasy (JAX-RS) Java server. It is possible that some of my users may have a newer version of the client than the server.
That client may call a resource on the server that contains query parameters that the server does not know about. Is it possible to detect this on the server side and return an error?
I understand that if the client calls a URL that has not been implemented yet on the server, the client will get a 404 error, but what happens if the client passes in a query parameter that is not implemented (e.g.: ?sort_by=last_name)?
Is it possible to detect this on the server side and return an error?
Yes, you can do it. I think the easiest way is to use #Context UriInfo. You can obtain all query parameters by calling getQueryParameters() method. So you know if there are any unknown parameters and you can return error.
but what happens if the client passes in a query parameter that is not implemented
If you implement no special support of handling "unknown" parameters, the resource will be called and the parameter will be silently ignored.
Personally I think that it's better to ignore the unknown parameters. If you just ignore them, it may help to make the API backward compatible.
You should definitely check out the JAX-RS filters (org.apache.cxf.jaxrs.ext.RequestHandler) to intercept, validate, manipulate request, e.g. for security or validatng query parameters.
If you declared all your parameters using annotations you can parse the web.xml file for the resource class names (see possible regex below) and use the full qualified class names to access the declared annotations for methods (like javax.ws.rs.GET) and method parameters (like javax.ws.rs.QueryParam) to scan all available web service resources - this way you don't have to manually add all resource classes to your filter.
Store this information in static variables so you just have to parse this stuff the first time you hit your filter.
In your filter you can access the org.apache.cxf.message.Message for the incoming request. The query string is easy to access - if you also want to validate form parameters and multipart names, you have to reas the message content and write it back to the message (this gets a bit nasty since you have to deal with multipart boundaries etc).
To 'index' the resources I just take the HTTP method and append the path (which is then used as key to access the declared parameters.
You can use the ServletContext to read the web.xml file. For extracting the resource classes this regex might be helpful
String webxml = readInputStreamAsString(context.getResourceAsStream("WEB-INF/web.xml"));
Pattern serviceClassesPattern = Pattern.compile("<param-name>jaxrs.serviceClasses</param-name>.*?<param-value>(.*?)</param-value>", Pattern.DOTALL | Pattern.MULTILINE);
IMHO, there are two techiques to handle a query for a resource:
For http GET you can override represent(Variant variant) or handleGet().
For http POST the same applies with acceptRepresentation(Representation entity) and handlePost().
The doc for handleGet says:
Handles a GET call by automatically returning the best representation available. The content negotiation is automatically supported based on the client's preferences available in the request. This feature can be turned off using the "negotiateContent" property.
and for represent:
Returns a full representation for a given variant previously returned via the getVariants() method. The default implementation directly returns the variant in case the variants are already full representations. In all other cases, you will need to override this method in order to provide your own implementation.
What are the main differences between these two types of implementations? In which case should I prefer one over the other? Is it right that I can achieve with e.g. handleGet() everything that would work with represent()?
I first started using handleGet setting the entity for the response. When I implemented another project I used represent. Looking back i can't really say one way is better or clearer than the other. What are your expirences for that?
I recommend using represent(Variant) because then you’ll be leveraging the content negotiation functionality provided by the default implementation of handleGet(Request, Response).
BTW, lately I've started using the annotation-based syntax instead of overriding superclass methods, and I like it. I find it clearer, simpler, and more flexible.
For example:
#Post('html')
Representation doSearch(Form form) throws ResourceException {
// get a field from the form
String query = form.getFirstValue("query");
// validate the form - primitive example of course
if (query == null || query.trim().length() == 0)
throw new ResourceException(Status.CLIENT_ERROR_BAD_REQUEST, "Query is required.");
// do something
SearchResults searchResults = SearchEngine.doSearch(query);
// return a HTML representation
return new StringRepresentation(searchResults.asHtmlString(), MediaType.TEXT_HTML);
}
The advantages of using this approach include the incoming representation being automatically converted to a useful form, the method can be named whatever makes sense for your application, and just by scanning the class you can see which class methods handle which HTTP methods, for what kind of representations.
Im developeing a RESTful Service in which Processes can be executed and proivde a resulting calculation. For this i have modeled the process itself as a Resource (Example: /processes/translate). I want to execute the process by sending a GET request with appended Input Parameter as Query Parameter (Example: /processes/translate?input1=xxxx&input2=xxxxx).
Each process has different Input Parameter which are defined during the process creation in the backend. My Question is how should i document or describe which inputs are needed to execute a process in machine readable form. For Example in XML.
Until now ive integrated atom:link elements in the Representation. i thought that maybe including XFORM could be a soluttion?
Best Regards
Andre
I would not model this with a GET. While it's the easier solution, it's also (IMO) the least RESTful. I would have clients POST a document describing what they want you to translate and your service sends them back a URI where their answer can be found (some translations might take a while).
Example (ommiting a lot of HTTP headers/context)
POST /processes/translate
Content-Type: application/xml
...
<translation-request>
<input1 type="type1">....</input1>
<input2 type="type5">....</input2>
</translation-request>
Response:
200 OK
Content-Location: /processes/translate/jobs/1234
....
That's always an interesting question. We have a project called RESTx (http://restx.org), with which you can create RESTful web services very easily. You can write custom component code in either Java or Python and then create RESTful resources by sending parameter sets to the server, which are then stored. Each parameter set gets its own URI, though, so you can always just run the code with those parameters by accessing the new parameter set's URI.
Importantly, the entire RESTful API, is automatically created. RESTx examines the component code and then assembles the API description. We decided to describe parameters in a way that is human as well as machine readable. You can see examples of what that looks like in a browser or in plain JSON.
I'm the lead developer on that, so please feel free to contact me about any questions you might have.