How do I log the duration of a request, served with cloudfront?
With a usual apache, I would add %D or %T to the access log configuration - but this seems not to work with cloudfront.
I used this as the reference documentation: http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html
CloudFront does not provide that information in the access logs. You may want to post to this as a feature request in the AWS CloudFront forum.
Related
I am working on creating a new request in AWS API Gateway. I am having issues with a 404 not found on the URL request.
The request (had to create fake one for the question):
GET https://hello.stackoverflow.com/services/misc/myroute/v1/swagger.json
I created a route in API Gateway ANY /services/misc/myroute/{proxy+}
I attached the route to a Load Balancer Listener integration
I set up the listener rule in the Load Balancer:
IF Path is /services/misc* Then Forward to Target
IF Requests otherwise not routed Then Forward to Default
Created logs for this system in the AWS API Gateway: Monitor -> Logging -> Set Log Destination
Set variables for the log format using the $context variables, Context Variables
Ex Log:
{ "requestId":"QWRHQKWFHWAFZ=",
"routeKey":"ANY /services/misc/myroute/{proxy+}",
"path":"/services/misc/myroute/v1/swagger.json",
"domain":"hello.stackoverflow.com",
"domain_prefix":"hello",
"httpMethod":"GET", "status":"404","protocol":"HTTP/1.1", "endpoint":-" }
One final check I have done to make sure its completing its "route" was see the requests in the monitoring and seeing the 4xx come from this ALB listener.
I can send the request via localhost and get a response with the json body
GET https://localhost:8080/v1/swagger.json --> Status 200 OK with body filled
In my quest to solve the issue, it has lead me to many older (2019) stack overflow questions that seem to be outdated with the AWS Console, same with the AWS documentation. See links below...
AWS API Gateway Method request path parameter not working
AWS API Gateway 404 page not found error when invoking endpoint url
AWS API Gateway Method request path parameter not working
With this being my first project in the AWS cloud space, I am not sure where else to turn. My guess would be the authentication headers from the API Gateway are being lost, but not sure where I can see this loss happening.
From my understanding of how the AWS Request Flow goes, I created this diagram:
I’m currently experimenting on Metaflow. I followed the documentation and was able to deploy an aws setup with the given cloud formation template.
My question is why is that I’m always getting a:
message: "Missing Authentication Token"
when I access METAFLOW_SERVICE_URL in the browser, even if I made sure that the APIBasicAuth was set to false during the creation of cloudformation?
Shouldn’t this setting make the metadata/metaflow service accessible without the authentication/api key?
How can I resolve this? Or is this expected? That is, I cannot really view the metadata/metaflow service url via browser?
Thanks in advance
This was resolved under this github issue.
You still need to set the x-api-key header if you are trying to access the service url via the browser. To get the api-key you can go to the aws console
Api Gateway -> Api Keys -> show api key
Alternatively you can use the metaflow client in the sagemaker notebook which should be automatically setup for you via the template.
Also worth mentioning that there are two sets of endpoints: The one provided by the api gateway (which you seem to be hitting) and the one provided by the service itself. The api gateway forwards the requests the the service endpoints but needs the x-api-key to be set in the header. You can probably try hitting the service endpoints directly since you disabled auth.
I'm trying to host a static site in S3 with ability to handle some dynamic content using Lambda/Api Gateway. Can't seem to be able to do that.
I want URLs to look like this:
example.com/index.html
example.com/images/*
example.com/css/*
example.com/api/* -> API Gateway
Also, when redirecting I'd like to keep the example.com as a root domain. I tried RoutingRules in S3, but redirects from the client. I need this to be transparent from the user, like proxying requests.
While Bob's answer is pretty neat for public websites and is simple but if you are looking for other alternates which can work for internal sites or don't want to use CDN, you can try following options.
Option 1 -
This is most common option people prefer. You just configure 2 different DNS hosts for static vs api.(Assuming you enable proper CORS for *.example.com)
example.com(S3) --> S3 static content
api.example.com(APIGateway) --> Lambda
Option 2 -
Example.com(APIGateway) --> /apigLambda -->Lambda
Example.com(APIGateway) --> /* --> S3 Bucket/S3 File.
API Gateway Configuration -
API Gateway S3 Backend Proxy -
Example API Urls -
https://xxx.execute-api.us-east-1.amazonaws.com/dev/apigLambda
https://xxx.execute-api.us-east-1.amazonaws.com/dev/myfilename.css
Reference -
https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-s3.html
Note - In above reference Url, the bucket name is being accepted in Url Path but my example hides bucket name so users have no idea of S3 bucket name when they see API Gateway Url.
Option 3 -
As per your comment just use {proxy+} as resource for proxying S3 to support sub-folders calls but as you suggested, making just pass-through proxy doesn't give much options to transform HTTP response body which I believe still ok since you know your website content files.
You can configure this by putting a CloudFront distribution in front of both the API Gateway API and the S3 bucket for static content. This would also allow you to take advantage of CloudFront's edge caching.
They say that we must check our privacy policy with curl. I tested this with Google's privacy policy to see what the fuss is about with http vs https. I believe the issue is that they are not following the redirects with -L. What should I do?
curl http://policies.google.com/privacy returns nothing
curl https://policies.google.com/privacy
returns the full privacy policy
curl -L http://policies.google.com/privacy returns the full privacy policy
We sent a note to Facebook and they fixed it. There really wasn't anything for us to do once the helpful people here pointed out the site was flagged.
Bluemix availability monitoring provides scripting support for Selenium only. Is there a way, I can have my shell or bash script that does following
- Builds a url
- Calls url using curl
- Process the response
My current urls are protected by Bluemix IAM. To call url, I need to pass access token in the header. The access token expires every hour which makes it impossible to use Bluemix availability monitoring service.
Currently there is only selenium script support.
There are plans for javascript (not shell) for REST APIs, but there are no firm dates when that might be available.
IAM tokens that expire every hour will make it difficult to monitor regardless of choice of monitoring mechanism. I am not familiar enough with IAM to provide much advice. Perhaps a second process outside the monitoring can refresh the accessToken?
Some who use Bluemix Availability Monitoring use APIConnect. Perhaps those tokens are less ephemeral, but I'm not sure how to map APIConnect access to IAM.