I have set up API Gateway v2 HTTP to SQS. It was fine. However, I would like to send optional headers from the HTTP request to SQS.
As far as I understand from the documentation, it is not possible to send the all headers to SQS, but only the ones that are defined explicitly. E.g. the following works fine:
{"UserAgent": {"DataType": "String", "StringValue": "${request.header.user-agent}"}, "anotherheader": {"DataType": "String", "StringValue": "${request.header.anotherheader}"}}
But it requires that both User-Agent and AnotherHeader headers are present in the HTTP request. What if AnotherHeader header is optional. Is it possible to somehow define a default value? Because otherwise API Gateway returns Bad Request.
Related
A little preface
I am making a REST server (FastApi) that will act as a REST gateway to other network devices. This server currently has a general purpose REST "forwarding" endpoint /query/{device_host} that takes the following body:
{
method: "GET", # Can be any REST method
headers: { ... }, # Dictionary of header key-val strings
url: "http://<device_host>?<optional_queries>", # Url to send request to
body: { ... } # JSON Dictionary of arbitrary shape
}
and passes it to a REST Session Manager object that deals with sending/receiving the request/response.
Dilemma
I want to make routes that send pre-defined REST queries, sort of like shortcuts to the general endpoint above. For example a POST REST request to /query/{device_host}/configure_logs could, by default, add the following to the request body if not already specified:
{
method: "POST",
headers: { "Content-Type": "application/json" },
url: "http://<device_host>/setup/log_server",
body: { "server_port": 10101, "server_url": "10.10.10.10" }
}
and pass it to the REST Session Manager and that is that, which is one approach.
However I figure, instead of passing it directly to the REST Session Manager, I could internally redirect the request(w/ status code 308 I believe as I want to pass the body along) to the general endpoint /query/{device_host}. Some pros/cons to this approach that I can think of:
Pros
Decreases code duplication (which really would only be calling the REST Session Manager)
Decreases required body argument as there are now defaults
Any middleware around the general REST endpoint will affect the pre-defined REST endpoints
Collecting metrics around the forwarded REST requests is easier as we really only need to monitor the general REST endpoint
Cons
Additional latency (unsure if internal redirects add much latency)
Additional I/O
I am also unsure if redirects are, in general, recommended for a REST microservice.
Is there any reason I should prefer one approach over the other?
I am using AWS API Gateway and I want to set my Integration type to http. I have the integrated url as https:// xxxxxx.com which takes a header "apikey". I am not expecting the end user to pass the header rather I want to set the apikey to some constant value.
I see that there is a way to force the user to make him pass the header(by making header required under the Method Request section. However, I want to set it to default.
For example in all the requests which are internally calling the URL inside the API gateway should pass the header value as "12345".
You can add/remove/override headers with an Integration Request Mapping Template.
In the API Gateway console, chose the relevant api/resourece/method. Go to Integration Request > Mapping Templates and chose your Content-Type (if requests are going to be received without a Content-Type header, set the Content-Type for the mapping template to application/json, which is the default behaviour).
Then in the actual mapping template add the following:
{
#set($context.requestOverride.header.apikey= "testMe")
}
This will add (or overwrite if it already exists) a header called apikey with the value "testMe" to all http requests downstream.
If you take this route, then you will need to also map over any other headers, path parameters, query parameters or body that you wish to pass through.
You could loop through the headers and query parameters like this.
## First set the header you are adding
#set($context.requestOverride.header.apikey= "testMe")
## Loop through all incoming headers and set them for downstream request
#foreach($param in $input.params().header.keySet())
#set($context.requestOverride.header[$param]= $input.params().header.get($param))
#if($foreach.hasNext) #end
#end
## Loop through all incoming query parameters and set them for downstream request
#foreach($param in $input.params().querystring.keySet())
#set($context.requestOverride.querystring[$param]= $input.params().querystring.get($param))
#if($foreach.hasNext) #end
#end
As you need to ensure that the header apikey is set to a default value, you should set the override for apikey before looping through the rest of the headers as only the first override will take effect.
The relevant AWS documentation can be found here.
The other alternative would be to point your API Gateway at a Lambda and make the call from the Lambda instead.
Firstly thanks to #KMO for his help. The following is the solution:-
Enable Http Proxy Integration.
Add the headers apikey=xxxx and Accept-Encoding=identity under the same Integration
Request -> Http Headers.
Under Settings -> Binary Media Types set the following as separate Binary Media Types
'*', */*. I mean as two different lines.This step is needed to resolve the Gzip action while returning the response.
Add the Query parameter country in the URL Query String Parameters section.
In the Integration Request map the country parameter to ctry by adding the value under mapped from as method.request.querystring.country. This will ensure that the query parameter country you passed in the main URL will be fed to the downstream url as parameter ctry.
The advantage of this apporoach is that, even if you override the header apikey, the one set under the Http Headers will take the precedence.
Note - I am very much new to all this. Apologies if anything is unclear.
My overriding aim is to pull out MOT history data for a large batch of vehicles from the DVLA API. I understand that this can be done using Postman, which I am using (on a 64-bit Windows laptop if at all relevant).
The DVLA provide the following instructions
====================================================
Getting started
All API interfaces are implemented as restful APIs and accessed over https.
To access API you will need an API key that uniquely identifies the source of the request. DVSA will give you an API key if it approves your application.
You should keep your API key secure, as DVSA manages throttling and quotas at an API key level.
Each request must have the following mandatory fields in the header:
Accept: application/json+v6
x-api-key:
Content-type field confirms that the response type is in JSON format, and the x-api-key field serves your API key to identify the source of the request.
Technical resources
Access the API at https://beta.check-mot.service.gov.uk/
This root URL will change when the service moves from beta to live.
These 4 endpoints equate to the 4 methods of using the API:
/trade/vehicles/mot-tests?registration={registration}
‘Registration’ is the vehicle registration number.
===================================================
In order to test that this is possible, I am entering the following single request into the bar in Postman, selecting "POST" and hitting "SEND"
https://beta.check-mot.service.gov.uk/trade/vehicles/mot-tests?Content-type=application/json&x-api-key=ABCDEFGH®istration=MYREG
n.b. no inverted commas or other punctuation surrounds the actual values for ABCDEFH or MYREG
Expected result: Some sort of JSON with MOT history for this vehicle
Actual result: {"message": "Missing Authentication Token"}{"message": "Missing Authentication Token"}
I am unclear on:
- whether I should be using POST
what the +v6 after the application is necessary (other documentation leaves it out)
Why "Accept" and "Content-type" appear to be used interchangeably in the documentation
Whether the ordering of the parameters matters
Whether this can be equally tested by simply pasting the url into a browser
Thanks for any help
Reading through the Documentation found here:
https://dvsa.github.io/mot-history-api-documentation/
It mentions that those fields should be added as Headers:
Each request must have the following mandatory fields in
the header:
- Accept: application/json+v6
- x-api-key: <your api key>
There are example cURL requests on the site to help you with creating the request.
If you use Postman's Import feature within the app (found in the top right), you can add this cURL request in the Paste Raw Text tab.
curl -H "Accept: application/json+v6" -H "x-api-key: <your_api_key>" https://beta.check-mot.service.gov.uk/trade/vehicles/mot-tests\?registration=ZZ99ABC
This will give you an example request of what it should look like. From here, you will be able to add in your own API Token and send the request.
If you are using Postman, you can use the Authorization tab right under the request to give the required mandatory header fields. Select Header from Add to drop down. You can also add additional headers information using the next tab named Headers. (for example, the accept-headers).
Edit:
Authorization:
Headers Tab
Normally, you should be getting the authorization token when you register to the site in question(x-api-key here).You need to figure out the value of that token from the initial call's response headers. According to the document which Danny shared, you will be getting x-api-key from them once they approve your request.
Edit:
Alternatively, you can use import feature as Danny suggested in his answer. At the end of the day, you need to add the values as headers instead of query parameters.
For anyone using Python with the MOT history api and getting the same error message, try GET:
import requests
url = f'https://beta.check-mot.service.gov.uk/trade/vehicles/mot-tests?registration={plate_number}'
payload = {}
headers = {
'Accept': 'application/json+v6',
'x-api-key': 'your-api-key'}
response = requests.get(url, headers=headers, data=payload)
data = response.json()
model = data[0]['model'] # get the vehicle model for example
print(model)
I'm using OWASP ZAP to test our API. We have a couple of POST endpoints which use an API Token and a shared secret for authentication and validating the request.
Some parameters of the request body are concatenated and hashed using the shared secret. This value is inserted into the request header.
How can I programatically generate this signature using OWASP ZAP?
Request Header
Content-Type: "application/json"
Accept: "application/json"
API-Key: {API_KEY}
Signature: {hash(field_one + field_two + field_three + SHARED_SECRET)}
Request Body
{
"field_one": "abc",
"field_two": "123",
"field_three": "xyz"
}
The SHARED_SECRET is the password that is stored locally by the client and used to hash the three fields from the request.
It is stored on the server along with the API-Key so that requests can be identified and validated.
Use an HTTP Sender Script. Create it in the ZAP UI so that you can test it as you're writing it. First make sure you are just detecting the requests you want to change, then extract the field values you need and finally generate the hash. Keep testing at each stage to make sure its doing what you need. And if you need specific help theres always the zaproxy-scripts group.
I'm try to access message hub instance by using kafka rest API. when i'm sending post request through Postman Client and giving Content-Type as application/vnd.kafka.json.v1+json it's giving me the following response
{
"error_code": 415,
"message": "HTTP 415 Unsupported Media Type"
}
could anyone please help me with that?.
I'm also passing api key in X-Auth-Token Header.
the accepted content-types when producing messages are
"application/vnd.kafka.binary.v1+json"
"application/vnd.kafka.v1+json"
"application/vnd.kafka+json"
"application/json"
but in all cases, the Message Hub REST service expects the content to be formatted as
"application/vnd.kafka.binary.v1+json"
i.e. the value of "value" to be base64-encoded.
E.g. something like
{"records":[{"value":"YmxhaGJsYWg="}]}