I would like to modify an url parameter /resource/{VaRiAbLe} in an API gateway to S3 mapping so that it actually points to /my-bucket/{variable}. That is, it accepts mixed-case input, and maps it to a lower-case name. Mapping path variables is relatively simple enough to S3 integrations, but I can't seem to get a lower-case mapping working.
Reading through the documentation for mapping parameters, it looks like the path parameters are simple string values (and not templated values), and so defining a mapping as method.request.path.variable.toLowerCase() won't work.
Does anyone have any ideas how to implement this mapping?
Map path variables to a JSON body, and then call another API method that actually does the S3 call?
Bite the bullet, and implement a Lambda function to do the S3 get for me?
Find another api method for S3 that accepts a JSON body that I can use to get the data?
Update using Orchestrated calls
Following the info from Jack, I figured I should try doing the orchestrated call, since the traffic volume is low enough that I am sure that I won't be able to keep the lambda hot.
As a proof of concept, I added two methods to my resource (sitting at /resource/{variable} - GET and POST. The GET method chains to the POST, which does the actual retrieving of the data.
POST method configuration
This is a vanilla S3 proxying method, where you set the URL Path parameter for {variable} to be method.request.body.variable.
GET method configuration
This is a HTTPS proxying method. You'll need an URL for the POST method, so you'll need to deploy the API to get the URL. The only other configuration needed here is a body mapping template with content like:
{
"variable" : "$input.params('variable').toLowerCase()",
"something" : "$input.params('something')"
}
This should be enough to get this working.
The downside to this looks to be that I'm adding an extra method (POST) to my API for that resource that could confuse consumers of the API. I think it should be possible to make the POST on the /resource resource, which would at least make a bit more sense from an API design standpoint.
Depending on how frequently this API will be called, I'd either go with the Lambda proxy or chaining two API Gateway methods together. If the API is called frequently enough to keep a Lambda function warm (say once a minute), then go with Lambda. If not, go with the orchestrated API call.
The orchestrated API call would be interesting, I'd be happy to help with that if you have questions.
As far as I know the only S3 API for getting object data is the GET that is documented in their API reference.
Related
Using Argument setter by supplying the parameter value I want to make the Datafusion pipeline as resuable. As said by many other answer's have tried implementing using the cloud reusable pipeline example given in Google guide.I was not able to pass the parameter Json file.So how to create the API to that parameter Json file stored in Google storage.Please explain the values to be passed to Argument setter like URL,Request response etc., If any one of you had implemented in your projects.
Thank you.
ArgumentSetter plugin reads from a HTTP endpoint and it must be publicly accessible as is depicted within the GCP documentation. Currently, there is not a way to read from a non-public file stored in GCS. This behavior has been reported in order to be improved to CDAP through this ticket.
Can you please provide what you've tried so far and where you're stuck?
The URL field in argument setter would contain the API endpoint you're making a call to. Make sure you include any headers your call would need like Authorization, Accept etc.
If you're having issues with argument setter a good check is to use Curl or any other tool to make sure you're able to talk to the endpoint you're trying to use.
Here's some documentation about Argument setter: https://github.com/data-integrations/argument-setter
Define a JSON file with appropriate name/value pairs. Upload it in a GCS bucket - make it public by changing permissions (add "allUsers" in permissions list). When you save it, the file will say "Public to Internet"
Copy the https path to the file and use it in Arguments Setter. If you're able to access this path from curl/ your browser, Argument Setter will be able to do too..
There are other problems I've encountered while using Argument Setter though - the pipe doesn't supersede runtime arguments over default values provided in the URL many a times, specially when the pipe is duplicated.
To make file public
You have to make your bucket public, currently there is no other way.
gsutil iam ch allUsers:objectViewer gs://BUCKET_NAME
I have code that contains several different DELETE methods, one of which takes a parameter, or at least ideally it would take a parameter. However when I make this request, either through CURL or through a web client, it doesn't work. The other DELETE requests function fine and I've hard-coded in the parameter that I want to pass just to see if the call works and it does. I've tried this with both PathParam and QueryParam and neither works. Here's how I'm using the PathParams, which I'm pretty sure is correct, QueryParams looks very similar so I don't want to post that too.
#DELETE
#Path("/byId/{id}")
public void deleteById(#PathParam("id") String id)
And then essentially the same thing for QueryParams but obviously the path is different
From what I understand a lot of RESTful APIs ignore any kind of request body with a DELETE request, or treat it as a PUT or POST. Is there any way around this? Basically I have a database that contains several objects and I need to delete one based on a unique identifier, to be passed in by the client. If there is no way around this is there some other way that I could do it?
Its possible that I'm missing something obvious here as I've only been using Jersey for a few weeks now and up to this point had never even heard of the concept of RESTful services.
You can send information to a RESTful service as either headers, path param, query param or message body.
If all the values go through as expected there is no problem with jax-rs/jersey. Now you need to debug the code and fix your implementation. jax-rs will only match a DELETE call with the DELETE http method you have implemented.
It is then your responsibility to actually perform a DELETE operation of the resource within the method. jax-rs or jersey is not going to complain if you do not DELETE or if you do some other action within the method.
I created a page called api.aspx
I am displaying my output using a JSON serialize.
Each type of function is based on query string. For example:
If Request.QueryString("action") = "getMember"
Then from there, I do a
Request.QueryString("memberid")
For the variable I required for my function to work. Lastly the output is displayed as JSON.
In REST, there should be no verbs in the URI, especially no things such as ?action=getMember. The core REST principle is HATEOAS - which says that the state of the application is represented by the state of its resources.
Furthermore, the application should use a uniform interface - that is, use the standard HTTP methods POST, GET, PUT and DELETE for CRUD operations respectively.
So, such a member resource from your example should be accessed via GET request to a URI such as:
http://example.com/api/member/{memberId}
Where {memberId} is path parameter that represents the memberId.
I would say it's not restful since the url scheme is depending on query string parameters. For a real restful API the url's shoud be http://asite/member/2 and not be named.
If you want to make your api truly restful, use the URL routing from ASP.NET MVC, it's possible to integrate with webforms.
I have a Java client that calls a RESTEasy (JAX-RS) Java server. It is possible that some of my users may have a newer version of the client than the server.
That client may call a resource on the server that contains query parameters that the server does not know about. Is it possible to detect this on the server side and return an error?
I understand that if the client calls a URL that has not been implemented yet on the server, the client will get a 404 error, but what happens if the client passes in a query parameter that is not implemented (e.g.: ?sort_by=last_name)?
Is it possible to detect this on the server side and return an error?
Yes, you can do it. I think the easiest way is to use #Context UriInfo. You can obtain all query parameters by calling getQueryParameters() method. So you know if there are any unknown parameters and you can return error.
but what happens if the client passes in a query parameter that is not implemented
If you implement no special support of handling "unknown" parameters, the resource will be called and the parameter will be silently ignored.
Personally I think that it's better to ignore the unknown parameters. If you just ignore them, it may help to make the API backward compatible.
You should definitely check out the JAX-RS filters (org.apache.cxf.jaxrs.ext.RequestHandler) to intercept, validate, manipulate request, e.g. for security or validatng query parameters.
If you declared all your parameters using annotations you can parse the web.xml file for the resource class names (see possible regex below) and use the full qualified class names to access the declared annotations for methods (like javax.ws.rs.GET) and method parameters (like javax.ws.rs.QueryParam) to scan all available web service resources - this way you don't have to manually add all resource classes to your filter.
Store this information in static variables so you just have to parse this stuff the first time you hit your filter.
In your filter you can access the org.apache.cxf.message.Message for the incoming request. The query string is easy to access - if you also want to validate form parameters and multipart names, you have to reas the message content and write it back to the message (this gets a bit nasty since you have to deal with multipart boundaries etc).
To 'index' the resources I just take the HTTP method and append the path (which is then used as key to access the declared parameters.
You can use the ServletContext to read the web.xml file. For extracting the resource classes this regex might be helpful
String webxml = readInputStreamAsString(context.getResourceAsStream("WEB-INF/web.xml"));
Pattern serviceClassesPattern = Pattern.compile("<param-name>jaxrs.serviceClasses</param-name>.*?<param-value>(.*?)</param-value>", Pattern.DOTALL | Pattern.MULTILINE);
Im developeing a RESTful Service in which Processes can be executed and proivde a resulting calculation. For this i have modeled the process itself as a Resource (Example: /processes/translate). I want to execute the process by sending a GET request with appended Input Parameter as Query Parameter (Example: /processes/translate?input1=xxxx&input2=xxxxx).
Each process has different Input Parameter which are defined during the process creation in the backend. My Question is how should i document or describe which inputs are needed to execute a process in machine readable form. For Example in XML.
Until now ive integrated atom:link elements in the Representation. i thought that maybe including XFORM could be a soluttion?
Best Regards
Andre
I would not model this with a GET. While it's the easier solution, it's also (IMO) the least RESTful. I would have clients POST a document describing what they want you to translate and your service sends them back a URI where their answer can be found (some translations might take a while).
Example (ommiting a lot of HTTP headers/context)
POST /processes/translate
Content-Type: application/xml
...
<translation-request>
<input1 type="type1">....</input1>
<input2 type="type5">....</input2>
</translation-request>
Response:
200 OK
Content-Location: /processes/translate/jobs/1234
....
That's always an interesting question. We have a project called RESTx (http://restx.org), with which you can create RESTful web services very easily. You can write custom component code in either Java or Python and then create RESTful resources by sending parameter sets to the server, which are then stored. Each parameter set gets its own URI, though, so you can always just run the code with those parameters by accessing the new parameter set's URI.
Importantly, the entire RESTful API, is automatically created. RESTx examines the component code and then assembles the API description. We decided to describe parameters in a way that is human as well as machine readable. You can see examples of what that looks like in a browser or in plain JSON.
I'm the lead developer on that, so please feel free to contact me about any questions you might have.