I have an API Gateway integration for a method/resource which works when I call it from the API but not when I actually call it:
$ aws apigateway test-invoke-method --rest-api-id $REST_API_ID \
--resource-id $RESOURCE_ID --http-method GET | jq -r .log,.body
This works out fine and I get the following output:
Tue May 16 17:46:42 UTC 2017 : Starting execution for request: test-invoke-request
Tue May 16 17:46:42 UTC 2017 : HTTP Method: GET, Resource Path: /status.json
Tue May 16 17:46:42 UTC 2017 : Method request path: {}
Tue May 16 17:46:42 UTC 2017 : Method request query string: {}
Tue May 16 17:46:42 UTC 2017 : Method request headers: {}
Tue May 16 17:46:42 UTC 2017 : Method request body before transformations:
Tue May 16 17:46:42 UTC 2017 : Endpoint response body before transformations:
Tue May 16 17:46:42 UTC 2017 : Endpoint response headers: {}
Tue May 16 17:46:42 UTC 2017 : Method response body after transformations: { "statusCode": 200 }
Tue May 16 17:46:42 UTC 2017 : Method response headers: {Content-Type=application/json}
Tue May 16 17:46:42 UTC 2017 : Successfully completed execution
Tue May 16 17:46:42 UTC 2017 : Method completed with status: 200
{ "statusCode": 200 }
However, I cannot access this at my URL, which is api.naftuli.wtf/v1/status.json. I have stages defined at glhf, stable, and v1, so by replacing that, you will see different responses. I just simply want a dummy response that returns a 200 JSON blob.
My Terraform for the resources is here as a Gist. Hopefully this fully shows my API Gateway configuration.
If I test invoke this from the CLI or from the web console, I get back what is expected. However, if I curl this from my deployed API at api.naftuli.wtf, I don't get anything nice:
$ for stage in glhf stable v1 ; do
> url="https://api.naftuli.wtf/${stage}/status.json"
> echo "${url}:"
> curl -i -H 'Content-Type: application/json' \
> https://api.naftuli.wtf/${stage}/status.json
> echo -e '\n
> done
https://api.naftuli.wtf/glhf/status.json:
HTTP/1.1 500 Internal Server Error
Content-Type: application/json
Content-Length: 36
Connection: keep-alive
Date: Tue, 16 May 2017 21:41:38 GMT
x-amzn-RequestId: 712ba52b-3a80-11e7-9fec-b79b62d3bf7f
X-Cache: Error from cloudfront
Via: 1.1 da7a5d0ed7f424609000879e43743066.cloudfront.net (CloudFront)
X-Amz-Cf-Id: hBwlbPCP9n2rlz53I-Qb9KoffHB_FoxUCZUaJYNnU3XhCWuMpQTP1Q==
{"message": "Internal server error"}
https://api.naftuli.wtf/stable/status.json:
HTTP/1.1 403 Forbidden
Content-Type: application/json
Content-Length: 23
Connection: keep-alive
Date: Tue, 16 May 2017 21:41:38 GMT
x-amzn-RequestId: 71561066-3a80-11e7-9b00-6700be628328
x-amzn-ErrorType: ForbiddenException
X-Cache: Error from cloudfront
Via: 1.1 0c146399837c7d36c1f0f9d2636f8cf8.cloudfront.net (CloudFront)
X-Amz-Cf-Id: ITX765xD8s4sNuOdXaJ2kPvqPo-w_dsQK3Sq_No130FAHxFuoVhO8w==
{"message":"Forbidden"}
https://api.naftuli.wtf/v1/status.json:
HTTP/1.1 500 Internal Server Error
Content-Type: application/json
Content-Length: 36
Connection: keep-alive
Date: Tue, 16 May 2017 21:41:39 GMT
x-amzn-RequestId: 7185fa99-3a80-11e7-a3b1-2f9e659fc361
X-Cache: Error from cloudfront
Via: 1.1 586f1a150b4ba39f3a668b8055d4d5ea.cloudfront.net (CloudFront)
X-Amz-Cf-Id: dvnOa1s-YlwLSNzBfVyx5tSL6XrjFJM4_fES7MyTofykB3ReU5R1fg==
{"message": "Internal server error"}
My understanding of stages were that they were additional path prefixes to the base path under which all API resources were available. If I had a stage called v1 with a path of /v1, I'd expect that an API Gateway resource for status.json will be basically mapped under /v1, yielding /v1/status.json.
I may be misunderstanding how API Gateway base path mappings and stages work, but CloudWatch tells me that the call is at least happening, though failing for some obscure reason:
21:41:39(c5be3842-6af4-4725-a34f-d6eea8042d17) Verifying Usage Plan for request: c5be3842-6af4-4725-a34f-d6eea8042d17. API Key: API Stage: tcips69qx2/prod_v1
21:41:39(c5be3842-6af4-4725-a34f-d6eea8042d17) API Key authorized because method 'GET /status.json' does not require API Key. Request will not contribute to throttle or quota limits
21:41:39(c5be3842-6af4-4725-a34f-d6eea8042d17) Usage Plan check succeeded for API Key and API Stage tcips69qx2/prod_v1
21:41:39(c5be3842-6af4-4725-a34f-d6eea8042d17) Starting execution for request: c5be3842-6af4-4725-a34f-d6eea8042d17
21:41:39(c5be3842-6af4-4725-a34f-d6eea8042d17) HTTP Method: GET, Resource Path: /v1/status.json
21:41:39(c5be3842-6af4-4725-a34f-d6eea8042d17) Execution failed due to configuration error: statusCode should be an integer which defined in request template
21:41:39(c5be3842-6af4-4725-a34f-d6eea8042d17) Method completed with status: 500
Apparently only traffic across the V1 stage is getting through to CloudWatch logs. I have a misconfiguration somewhere and I can't seem to find it.
Can you try and change you request template in the integration request setup to this:
{
"statusCode": 200
}
API Gateway looks for the status code to return in the response in your integration request template. The response is generated by the mapping template in the integration response. I can see from your terraform setup that you are loading the output json file in the integration request template. This is content API Gateway does not expect.
With Mock Integration Amazon API Gateway there are 2 common reasons for 500 Internal Server error.
Check the mapping template in Integration Request and ensure that you are passing statusCode as an integer to the MOCK Integration endpoint.
{
"statusCode": <Integer_Status_code>
}
Note: Make sure that status code is passed as integer not string.
Correct : 200 Incorrect : "200"
Mock Integrations do not support the binary content. If the API is enabled with bixnary support and has application/json or */* set as binaryMediaTypes, MOCK Integration endpoints would throw a 500 Internal server error when trying to transform the content.
A workaround is to update the contentHandling property of the MOCK Integration to CONVERT_TO_TEXT
Read more here :- https://cloudnamaste.com/500-internal-server-error-mock-integration/
For my case, when I deploy the lambda with serverless framework, the OPTIONS returns 200 when called. However, when I configure manually on AWS API Gateway, it returns 500 Internal Server Error.
When I check the execution log of API gateway, it said
(ee0a42d9-2cfc-4788-8679-00fbd7938cf1) Method request body before transformations: [Binary Data]
(ee0a42d9-2cfc-4788-8679-00fbd7938cf1) Execution failed due to configuration error: Unable to transform request
(ee0a42d9-2cfc-4788-8679-00fbd7938cf1) Gateway response body:
{
"message": "Internal server error"
}
After you create the Resources and OPTIONS methods, select the OPTIONS method then
At Method Execution --> Integration Request --> expand the Mapping Templates, choose "When no template matches the request Content-Type header". Add or select "application/json" in the Content-Type. Click the "application/json", and in the Generate template, paste
{statusCode: 200} without any double quote. Note, if the {"statusCode": 200} exists but with double quote, remove it to be the same as above. Then Save it
At Method Execution --> Integration Response --> expand the 200 response status --> Mapping Template --> Add or select "application/json" in the Content-Type. Make sure that the Generate template box is empty. Then Save it
At Method Execution --> Method Response --> expand the 200 response. In Response Headers for 200, add three headers: Access-Control-Allow-Headers, Access-Control-Allow-Methods, and Access-Control-Allow-Origin. Leave the Response Body for 200 to be empty
Action --> Enable CORS for the Resource
Action --> Deploy API
You have at least two distinct problems with your configuration.
First, one of your three base path mappings doesn't match the way you're trying to invoke your API. Note that the base paths don't have to be the same as your stage names, but they can be if you desire. Since your base path mappings include base paths and stage names, API Gateway is expecting the invoke path to include a base path mapping and not a stage, so it is interpreting the [glhf stable v1] portion of your path as a base path and looking for the corresponding base path mapping entry to determine the API and stage to use. This works fine for the v1 and glhf base paths which return 500 (indicating a different problem). The stable base path (in https://api.naftuli.wtf/stable/status.json) returns a 403 Forbidden because there is no base path of "stable" defined for the domain name api.naftuli.wtf. The stable stage is mapped to the "latest" base path, so calling https://api.naftuli.wtf/latest/status.json should be the way to call the stable stage. This doesn't currently work, and I don't know why. If you tell me what region your running this in, I can look-up the config and do more digging.
The second problem is indicated by the following entry from your CloudWatch logs:
Execution failed due to configuration error: statusCode should be an integer which defined in request template
Can you check that your integration request template (in the file your reference in "${file("${path.module}/files/status.json")}") contains "statusCode: 200" as a top level attribute.
I also found it surprising that you're using the same file for a request template and a response template.
Having had the identical errors I found what helped me solve this issue was to delete my OPTIONS request definition in the AWS Console. I then followed the console's "Enable CORS" form which created a new OPTIONS method.
I subsequently ran terraform plan and looked at the diff between my OPTIONS defintion and theirs. Given that the AWS Console created OPTIONS method worked I applied the changes.
Using terraform 0.12 or greater makes this possible as the terraform plan output detail is more fine grained.
I was doing this in CloudFormation.
It took me a while to get it and the accepted answer here was extremely helpful, but a little vague, so adding some more info.
Stefano Buliani's answer, in CloudFormation YAML, looks like:
RequestTemplates:
application/json: |
{ statusCode: 200 }
What was especially weird here was apparently, the fix was simply to create a deployment using the AWS CLI for each of the stages. Apparently, Terraform was not updating or re-kicking deployments on changes, and so my changes never got out.
I had a similar problem and eventually figured out that my client was using a different content type than I expected. I had foolishly assumed it would use application/json, but it was some custom json thing. In my setup, API Gateway is logging to cloudwatch, which is where I found the content type it received from the client. Once I updated the content type in the request template of the mock integration, things worked as expected.
I'm trying to make requests on the Visual Recognition API and getting the response error below:
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 28 Jun 2016 13:53:07 GMT
Content-Type: application/json
Content-Length: 67
Connection: close
Cache-Control: no-cache
Access-Control-Allow-Origin: *
{
"status": "ERROR",
"statusInfo": "invalid-api-key"
}
I've already tried to recreate my account and the Visual Recognition Service many times this week, and the API Key provided by Bluemix is always invalid.
This is the same problem raised in IBM Watson visual recognition - invalid API key and Watson Visual Recognition - error 401. The Visual Recognition service is currently experiencing problems recognizing keys; the development team is working to resolve the problem. There is additional discussion of this issue on the IBM developerWorks Answers forum. For issues with specific keys, you can open a Bluemix support ticket.
I am building a Flask-based REST API and using Flask-JWT to handle JWT auth. I also want to use the built in roles management with Flask-Security. However, Flask-Security's #roles_required() decorator assumes I am showing a Flask view when it fails.
Here is my token endpoint (which is working as I want):
$ http POST localhost:5000/auth/token username='test' password='test'
HTTP/1.0 200 OK
Content-Length: 192
Content-Type: application/json
Date: Sun, 08 Nov 2015 17:45:46 GMT
Server: Werkzeug/0.10.4 Python/3.5.0
{
"access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE0NDcwMDQ3NDYsIm5iZiI6MTQ0NzAwNDc0NiwiZXhwIjoxNDQ3MDA1MDQ2LCJpZGVudGl0eSI6MX0.RFIeaLuvJNM9fDjFYFQ7sh_WaDVU-_aM7e46tVJzlBQ"
}
Here is a successful response to a resource that does not have any role requirement (using only #jwt_required) This is also working as I want:
$http GET localhost:5000/protected Authorization:'JWT eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE0NDcwMDQ3NDYsIm5iZiI6MTQ0NzAwNDc0NiwiZXhwIjoxNDQ3MDA1MDQ2LCJpZGVudGl0eSI6MX0.RFIeaLuvJNM9fDjFYFQ7sh_WaDVU-_aM7e46tVJzlBQ'
HTTP/1.0 200 OK
Content-Length: 25
Content-Type: text/html; charset=utf-8
Date: Sun, 08 Nov 2015 17:46:24 GMT
Server: Werkzeug/0.10.4 Python/3.5.0
<models.User[email=test]>
When I do the same for a resource that has roles required (such as admin in this example), it seems to assume I have a page to display such as /login which I do not since it is a headless REST API:
$ http GET localhost:5000/admin Authorization:'JWT eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE0NDcwMDQ3NDYsIm5iZiI6MTQ0NzAwNDc0NiwiZXhwIjoxNDQ3MDA1MDQ2LCJpZGVudGl0eSI6MX0.RFIeaLuvJNM9fDjFYFQ7sh_WaDVU-_aM7e46tVJzlBQ'
HTTP/1.0 302 FOUND
Content-Length: 209
Content-Type: text/html; charset=utf-8
Date: Sun, 08 Nov 2015 17:46:43 GMT
Location: http://localhost:5000/
Server: Werkzeug/0.10.4 Python/3.5.0
Set-Cookie: session=eyJfZmxhc2hlcyI6W3siIHQiOlsiZXJyb3IiLCJZb3UgZG8gbm90IGhhdmUgcGVybWlzc2lvbiB0byB2aWV3IHRoaXMgcmVzb3VyY2UuIl19XX0.CSEcAw.pjwXLeSWUsORXR-OU5AfFvq6ESg; HttpOnly; Path=/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: /. If not click the link.
I know Flask-Security uses Flask-Principal behind the scene for its roles management (#roles_required, etc.) and it ties into the RoleMixin and UserMixin for the datastore which is very nice. However, if there is no way to get Flask-Security to just allow the resource through without using my JWT header, then maybe the best bet is to build my own decorators which uses Flask-Principal to manage the roles.
Does anyone have any experience with this? The idea is that the entire front end can and will be built in whatever language we need and that means it may not be Flask's templates/views which is what Flask-Security appears to be doing.
Thank you for any insight anyone can provide!
Instead of a redirect, you would want to respond with an HTTP status code 403.
Your best bet is indeed to create your own decorator to manage the roles, and move away from using Flask-Security entirely.
The author of Flask-Security has mentioned that there are better ways to secure APIs, and it makes even more sense as the library is not maintained.
Flask-JWT or Flask-JWT-Extended are perfect candidates for this task. The former would require a bit more boilerplate to get things going. There is a stale PR suggesting an API to support roles, that you could use to create your own decorator if you decide to go with Flask-JWT.
The Flask-JWT-Extended docs suggest a simpler solution that might fit your case. You should follow the custom decorators section of the documentation for the full example, but here's the decorator in the nutshell:
from functools import wraps
from flask import jsonify
from flask_jwt_extended import (
verify_jwt_in_request, get_jwt_claims
)
def admin_required(fn):
#wraps(fn)
def wrapper(*args, **kwargs):
verify_jwt_in_request()
claims = get_jwt_claims()
if claims['roles'] != 'admin':
return jsonify(msg='Admins only!'), 403
else:
return fn(*args, **kwargs)
return wrapper
This code looks for a roles claim in the JWT, and returns a 403 response if it is not admin.
I was trying to use it for Single sing on. I was able to successfully secure my web services/pages and want to extend it to my REST APIs (services) as well. I am using CAS 4.0.0.
I followed this link http://jasig.github.io/cas/4.0.0/protocol/REST-Protocol.html and change the web.xml as suggested and added the following jar to my CAS deployment WEB-INF/lib folder:
cas-server-integration-restlet-4.0.0.jar
org.restlet.jar
org.restlet.ext.spring.jar
org.restlet.ext.servlet.jar
The Call to the "v1/tickets" is reaching the restlet-frame work but I am getting a 404 error to my rest client
Status Code: 404 Not Found
Accept-Ranges: bytes
Content-Length: 439
Content-Type: text/html;charset=UTF-8
Date: Thu, 06 Nov 2014 13:12:46 GMT
Server: Restlet-Framework/2.2.2
Can you Please help me identify the issue with the set-up or point me to a detailed documentation.
Disclaimer: I'm the Chairman of CAS and founder of CAS in the cloud (https://www.casinthecloud.com).
The endpoint is in lower-case: can you try: /v1/tickets?
Can someone help me out? I am not sure if this is an Intuit system error OR an error on my part. I am able to discoverAndAddAccounts and everything else no problem. However when I try to updateInstitutionLogin I am getting a 503 DownStreamServiceUnavailable response.
FYI: I get the same response if I leave out refresh=true.
Request:
https://financialdatafeed.platform.intuit.com/v1/logins/136**?refresh=true
<InstitutionLogin xmlns="http://schema.intuit.com/platform/fdatafeed/institutionlogin/v1">
<credentials>
<credential><name>passcode</name><value>THEPASSWORD</value></credential>
<credential><name>TAX_AGGR_ENABLED</name><value>FALSE</value></credential>
<credential><name>onlineID</name><value>THEUSERNAME</value></credential>
</credentials>
</InstitutionLogin>
Response:
HTTP/1.1 503 Code:DownStreamServiceUnavailable,Type:SYSTEM
Date: Mon, 23 Dec 2013 12:15:12 GMT
intuit_tid: gw-f4645f18-eac1-4518-b8b4-41756eab406d
Content-Type: text/plain
Content-Length: 0
Connection: close
We found the root cause of this issue being that this Login does not have any Accounts under it. This is causing our system to produce the error that you are seeing. Please have the user re-discover the account to be able to use the system.