I want to know how Azure WAF identifies whether the request is from bots.
I don't want to let WAF kill some reasonable requests.
Related
Context: We are hosting an online shop that needs to track customer behaviour. To achieve this tracking we have integrated several tracking events based on the customer journey in our shop. Based on the GDPR requirements in Europe we are forced to send the tracking events to infrastructure that is controlled by us as a company. Sending data via the Google Analytics Tag Manager directly to Google Servers is forbidden by the GDPR law. Sidenote: To simplify this question, I intentionally leave out all stuff regards user consent management.
Problem statement: We have the need that each client sends every tracking event directly from the browser to a Pub/Sub endpoint. Now, my question is how a best practise for a proper security would look like.
Current proposal: The Pub/Sub endpoint doesn't require an authentication --> AllUsers have been granted Pub/Sub Publisher permission. In addition I've created an API-KEY that is restricted to
the Pub/Sub API only
to specific HTTP referrers (basically the domain our webshop operates)
Are there other strategies that could be applied? Is the current proposal a valid (aka secure) way to go?
Giving pub/sub publisher access to allUsers are not recommended. Create service account and give publisher access to that and send messages using that service account.
I want to know about webhook (what is webhook). What is the application of webhook (a real world scenario). Besides, what are the differences between webhook & API?
An API is a standardised way of communicating with a service. You've tagged REST in your question so I'll focus on RESTful APIs using HTTP but it is important to know that API is a very generic term.
In the REST world everything is a resource and you use the HTTP methods to define what action you want to take on or apply to that resource. For example, to list all the users on GitHub you would send a GET request to https://api.github.com/users. The URL (specifically the /users part) defines what resource you are interested in. Here the resource is a collection of all the users. There's other methods you can use; such as PUT to create or update a resource. To learn more about the different methods you can read the HTTP specification.
Webhooks are often used in conjunction with APIs but they are focused on events. They allow a service to send out 'notifications' when an event happens or some condition is met.
GitHub is again a good example of what webhooks are used for. Say I'm building a service which sends out an email every time someone leaves a comment on an issue in GitHub. I could use the GitHub API (like above) to list all of the comments on an issue and then check if there have been any new comments since the last time I checked. I can then just repeat this request every few seconds. This is known as polling. The issue here is that most of the time I'm checking the result is not going to change. This is going to be a waste of resources.
Webooks allow for Event-Driven Programming. Instead of randomly checking I can instruct GitHub to send my service a HTTP request every time a comment is added: aka a webhook. In this architecture I only have to send a request to GitHub's API when I know for sure that a new comment has been left.
Overall, you cannot really compare APIs and webhooks. The link between them is simply that webhooks send requests to APIs.
I want to use Dialogflow for my enterprise usage. So want to know whether Dialog flow will be able to hit Non public URLs?
Since Dialogflow is a service hosted by Google, fulfillment requests specified by Webhook URLs must be able to be reached by Dialogflow for them to be invoked. In addition, the webhook endpoints must expose themselves using SSL/TLS and must be associated with a non-self-signed certificate. When a request is made from Dialogflow, dialogflow can provide authentication credentials to ensure that it is indeed Dialogflow that is making the request.
One pattern for your usage is to expose the Webhooks to the Internet and only allow connections from the Google IP address range and also require authentication (known only to Dialogflow). This would go a long way in preventing malicious access to your Webhook.
An alternative would be to define your Webhook as a GCP hosted endpoint and then you would own the routing back to your internal system from there. That could use a variety of technologies beyond HTTP including Pub/Sub. For example, when Dialogflow invokes the Webhook, a GCP application could be called that posts a message to PubSub. Your Enterprise application could be a subscriber and be notified that it has work to do. It does work and responds with a new message which is received by your GCP hosted Webhook that then returns the response to Dialogflow. As such, there is no surface area for an attacker to try and penetrate.
I was wondering if there is a good way to send a message with a topic to a service bus queue via HTTP Post in postman for example.
I red something about Sas-key encryption but, lets say I would like to expose the url to someone for them to send my service bus messages, how do I do that the simplest way for them so to speak?
I just want them to have a url not crating a program to generate w token for it..
I know the Service Bus has a URL linked to it but I cant seem to send anything to it...
Is this possible?
I just want them to have a url not crating a program to generate w token for it..
From the Azure Service Bus send message API, we could know that Authorization header is required. If want to let someone to use just with a url. In my opinion is that we need to implement it ourself.
We could develop a Rest API service then we could give a rest api url to somebody who want to use. We could get some demo code about how to create topic and send message from the azure document.
I'm building a service with API Gateway + Lambda that tracks email link clicks. The links inside the email lead to my endpoint, which gathers the click info and redirects to another URL. However, I'm detecting that in some cases, some software automatically clicks most of the links, probably to prevent phishing, and the usual suspect here is an antivirus. Since I'm targeting only real user clicks, I want to discard them, but didn't find anything weird in the request headers. How would you check that the request comes from a non user?
In API Gateway settings, you can turn on CloudWatch logs to see all the request headers. Specifically, you can use $context and $input variables to log context variables like user-agent, source-ip or log all the headers.
If the bots are using exact same user-agent and set of headers, I do not see a way to distinguish them at API Gateway side.