I have REST API in API Gateway. Cloudwatch logging is enabled for each stage and logging level is "Full request and Response logs". Because I use metrics also "Enable Detailed CloudWatch Metrics" is enabled in stages Logs/tracing tab.
When I look at Cloudwatch Log streams under right Log group I easily can follow what happens with each request. I have used Cloudwatch logs insights queries to search for 4xx and 5xx errors and then investigate them in log streams.
What i'm missing is how I can query by requests http method or full request in Logs Insights? ie. I'd like to find all HTTP Delete methods requests, or requests to certain resource path "myapi/foo/bar/".
Because there is so much log rows in logs it would be handy to query only for certain requests in logs insights and then drill down in Logs streams
Is there more fields in Logs insights I haven't found yet, or what I'm missing here that isn't obvious after reading documentation about API gateway logging?
Related
Confluent has something called Audit Logs which are written to internal topics (if configured) and which log access to Kafka resources such as clients writing or reading from particular topics. That's all great, however, there are components in a Confluent/Kafka setup such as Confluent Control Center which (should) have a RBAC access set up for users to log in and use.
It is possible to set it up and have users log in with username and password but I am having difficulties in locating where exactly Confluent provides logs of successful/unsuccessful login attempts. If I set the Control Center logs to DEBUG, I can see the HTTP communication and the password lookup for when a user tries to login but I don't see an option for admin review and control of such events. Audit Logs are apparently only for Kafka resources. Is there no other option other than building up a custom solution, scraping DEBUG logs?
The list of auditable events is documented here. Confluent Control Center login events are unfortunately not one of them.
I am working on creating a new request in AWS API Gateway. I am having issues with a 404 not found on the URL request.
The request (had to create fake one for the question):
GET https://hello.stackoverflow.com/services/misc/myroute/v1/swagger.json
I created a route in API Gateway ANY /services/misc/myroute/{proxy+}
I attached the route to a Load Balancer Listener integration
I set up the listener rule in the Load Balancer:
IF Path is /services/misc* Then Forward to Target
IF Requests otherwise not routed Then Forward to Default
Created logs for this system in the AWS API Gateway: Monitor -> Logging -> Set Log Destination
Set variables for the log format using the $context variables, Context Variables
Ex Log:
{ "requestId":"QWRHQKWFHWAFZ=",
"routeKey":"ANY /services/misc/myroute/{proxy+}",
"path":"/services/misc/myroute/v1/swagger.json",
"domain":"hello.stackoverflow.com",
"domain_prefix":"hello",
"httpMethod":"GET", "status":"404","protocol":"HTTP/1.1", "endpoint":-" }
One final check I have done to make sure its completing its "route" was see the requests in the monitoring and seeing the 4xx come from this ALB listener.
I can send the request via localhost and get a response with the json body
GET https://localhost:8080/v1/swagger.json --> Status 200 OK with body filled
In my quest to solve the issue, it has lead me to many older (2019) stack overflow questions that seem to be outdated with the AWS Console, same with the AWS documentation. See links below...
AWS API Gateway Method request path parameter not working
AWS API Gateway 404 page not found error when invoking endpoint url
AWS API Gateway Method request path parameter not working
With this being my first project in the AWS cloud space, I am not sure where else to turn. My guess would be the authentication headers from the API Gateway are being lost, but not sure where I can see this loss happening.
From my understanding of how the AWS Request Flow goes, I created this diagram:
When setting up logging for the API Gateway, the log created follows this default pattern :
API-Gateway-Execution-Logs_awsnameofservice/stage
The awsnamesofservice part is like 'fhsdydfn7*' an aws equivilent for the name of your service. Is there a way to rename this, so it becomes something more user friendly?
Thanks
For execution logs for REST and WebSocket APIs, there is no way for you to configure this today.
For access logs you can provide the log group when you configure the logs by specifying the destinationArn.
https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-logging.html
Is it possible to get access to events generated by User Account and Authentication (UAA) server in the context of Swisscom Application Cloud?
It is essential for me, to be able to have an audit trail of actions executed by authorised operators through the API (that would include cli and portal).
What I am looking for is an alternative of AWS CloudTrail for IAM module, that you can turn on for specific VPCs / regions there.
I have found this in the CF documentation (https://docs.cloudfoundry.org/loggregator/cc-uaa-logging.html) but that (as far as I understand it) requires infrastructure level access.
Thanks a lot for any hints.
We can't expose UAA logs to individual customers since it contains probably sensitive information about other users or the platform.
You should be able to retrieve the logs of your application in the application logs (which you can send to a syslog drain, i.e. the ELK/Elasticsearch service).
All API interactions should be covered by this log stream, according to the documentation:
Users make API calls to request changes in app state. Cloud Controller, the Cloud Foundry component responsible for the API, logs the actions that Cloud Controller takes in response.
For example:
2016-06-14T14:10:05.36-0700 [API/0] OUT Updated app with guid cdabc600-0b73-48e1-b7d2-26af2c63f933 ({"name"=>"spring-music", "instances"=>1, "memory"=>512, "environment_json"=>"PRIVATE DATA HIDDEN"})
From https://docs.cloudfoundry.org/devguide/deploy-apps/streaming-logs.html
Goal: I have an Angular 5 client that I wish to centralise logging to AWS CloudWatch so that I can view all logs (from the web client, API, lambdas, DynamoDB) in one place.
Solution considered:
(1) Create an API on the AWS Api Gateway with the CloudWatch Logs (AWS Service)
(2) Angular 5 client will post all logging to this API.
Problem:
I am having difficulty configuring the AWS Api Gateway correctly. The documentation on this is particularly sparse. I have created a simple test API with the following configuration.
Basic API configuration
This is a POST method on the API. The TestAPIRole has permission to PutEventLogs to CloudWatch.
I ran a test with the following data:
Test message
And I get the following response:
Response from test message
I think I haven't configure the API right but I am not sure what else to try. Has anyone tried to configure AWS API Gateway to post log events to Cloudwatch?
Has anyone tried to configure AWS API Gateway to post log events to Cloudwatch?
We have an error-logger API (API Gateway + Lambda) we use at work. At the minimum, it's just a simple POST endpoint that receives what the frontend (ours is in React) error handler sends. The Lambda simply writes to CloudWatch and we aggregate them through ElasticSearch.