I'm advertising on Facebook Ads
I'm getting lots of hits on my server with IPs from Facebook but they are not using the typical user agent "facebookexternalhit". The user agent looks like a legit user.
When you check the IP, it's registered under Facebook
So it's hard to detect bots and filter them out. Any idea why they do this and how to detect their IPs?
There's a list of IPs under Facebook Dev area, but that particular IP is not there either
An example:
173.252.95.13 - - [24/Mar/2022:02:19:08 +0000] "GET /in1?utm_source=FB&utm_campaign=C1&fbclid=IwAR0OnUl6Ih4zT-jRhZKnO4Z8xgRcAVN36Z9rNE6WaLtugI0nnnRn4c4Ujyk HTTP/1.1" 200 25186 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36 Edg/99.0.1150.46"
Related
I have a sap trial account and I am developing a sap ui5 app with spring boot backend on cloud foundry. I also have an app router used for authentication/authorization and for access to my app. I have a simple page, index.html, which has a button that calls my spring backend ("/hello") and a logout button on it ("/my/logout").
All the links I give in this post are accessed only through approuter (/index.html, /hello and /debug are all relative paths to "https://approuter-p***trial.cfapps.eu10.hana.ondemand.com").
Here is the chain of events that I am doing and that has me puzzled:
After I login, I can call the spring backend by pushing my button and it returns 200 (OK).
After I logout by pressing the logout button (I am still on the same page, no redirect), I try calling the spring backend again by pressing the button and it returns 401 (OK). I can repeat this many times, it gives 401 (OK).
I try accessing the backend by calling the direct link from another tab in the same browser(https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/hello - it's the same link that gets called when I push my spring button) and I get a 200 - NOK!, I did not login in again...
I go back to the tab that had index.html opened and call the spring backend again by pressing my button - now it returns 200 also (NOK!)
I tried accessing the spring backend link from POSTMAN, no headers/cookies added - I get the same server response I do by calling it from index.html, by the spring button (same situations as above, sometimes 401, sometimes 200 if backend previously accessed directly in browser).
I also checked the token received via a "/debug" backend that prints it, and it seems that after I logout, any call to the spring backend directly from the browser (not from index.html or POSTMAN) receives a new JWT Token, which seems to be valid. The only difference I noticed was that the 200 responses come with the header "X-Frame-Options: DENY" and the 401 responses come with the header "X-Frame-Options: SAMEORIGIN". This was tested with Chrome and IE - both having the same behaviour.
Any ideas why this is happening? My xs-app.json (used by my-xsuaa service and conseqeuntly by approuter) looks like this:
{
"welcomeFile": "index.html",
"authenticationMethod" : "route",
"routes": [{
"source": "/",
"target": "/",
"destination": "app-destination-hello"
}],
"logout": {
"logoutEndpoint": "/my/logout"
},
"destinations": {
"app-destination-hello": {
"logoutPath": "/myrest/logout",
"logoutMethod": "GET"
}
}
}
Thanks!
I will attach some traces from the approuter, maybe it helps. These are the steps I took with the corresponding logs (did not skip any log entry between steps):
I press logout button from index.html:
2018-08-31T15:37:50.63+0300 [APP/PROC/WEB/0] OUT #2.0#2018 08 31 12:37:50:632#+00:00#ERROR#/Handler#########-UTzTjYczINpXSxkJY_QXoWlfSF3dk0O######jlhzhep3#PLAIN##GET request to /my/logout completed with status 401 - Authentication required#
2018-08-31T15:37:50.63+0300 [RTR/7] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:37:50.632+0000] "GET /my/logout HTTP/1.1" 401 0 12 "https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/index.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"bb472547-cf0f-407d-6ccb-87d2dd42d4e7" response_time:0.002667543 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"56fc90749079fa0f" x_b3_spanid:"56fc90749079fa0f" x_b3_parentspanid:"-"
2018-08-31T15:37:50.63+0300 [RTR/7] OUT
I press the spring button from index.html (the endpoint name is now /hello2, but it's the same as before) and I receive a 401, which is OK:
2018-08-31T15:37:57.57+0300 [APP/PROC/WEB/0] OUT #2.0#2018 08 31 12:37:57:575#+00:00#ERROR#/Handler#########utCeM-9IrqFW7oEOGkqMtIoUmwgTpZsv######jlhzhk1z#PLAIN##GET request to /hello2 completed with status 401 - Authentication required#
2018-08-31T15:37:57.58+0300 [RTR/4] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:37:57.583+0000] "GET /hello2 HTTP/1.1" 401 0 12 "https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/index.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"600551f2-7054-4d08-68aa-75d88f2584e2" response_time:0.003231487 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"25c2df6333a965cf" x_b3_spanid:"25c2df6333a965cf" x_b3_parentspanid:"-"
2018-08-31T15:37:57.58+0300 [RTR/4] OUT
I open the approuter/hello2 link from another browser tab and I receive a 200 (NOK):
2018-08-31T15:38:00.79+0300 [APP/PROC/WEB/0] OUT #2.0#2018 08 31 12:38:00:797#+00:00#INFO#/Auth/OAuth2#########pPyrBRKTJWcfwzc8yP8EExMmQbCfW992######jlhzhmjg#PLAIN##sending page with client-side redirect to https://p***trial.authentication.eu10.hana.ondemand.com/oauth/authorize?response_type=code&client_id=sb-firstapp3cris!t5664&redirect_uri=https%3A%2F%2Fapprouter-p***trial.cfapps.eu10.hana.ondemand.com%2Flogin%2Fcallback#
2018-08-31T15:38:00.80+0300 [RTR/5] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:38:00.801+0000] "GET /hello2 HTTP/1.1" 200 0 483 "https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/hello2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"5e18d1e8-f7e6-4b21-6f6a-2a7ea386c5cc" response_time:0.003697001 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"89e2fc6581f0c47b" x_b3_spanid:"89e2fc6581f0c47b" x_b3_parentspanid:"-"
2018-08-31T15:38:00.80+0300 [RTR/5] OUT
somehow a login is called automatically (I see a call in the index.html tab, but it is done when I press the logout button, also no reidrect page to ask me for username/password):
2018-08-31T15:38:39.46+0300 [RTR/5] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:38:39.158+0000] "GET /login/callback?code=UL6niGFX3T HTTP/1.1" 302 0 0 "https://p***trial.authentication.eu10.hana.ondemand.com/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"307eee0f-123f-46f6-7102-9ef370d66ab5" response_time:0.307257726 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"a9a073166c04301f" x_b3_spanid:"a9a073166c04301f" x_b3_parentspanid:"-"
2018-08-31T15:38:39.46+0300 [RTR/5] OUT
I press my spring button from approuter/index.html and now I receive 200 response here also:
2018-08-31T15:38:39.70+0300 [RTR/1] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:38:39.542+0000] "GET /hello2 HTTP/1.1" 200 0 85 "p***trial.authentication.eu10.hana.ondemand.com/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"b6400e34-87c0-47ea-4335-46d9c9348802" response_time:0.158048741 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"eb6dde02e73d9c38" x_b3_spanid:"eb6dde02e73d9c38" x_b3_parentspanid:"-"
2018-08-31T15:38:39.70+0300 [RTR/1] OUT
2018-08-31T15:38:52.91+0300 [RTR/0] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:38:52.897+0000] "GET /hello2 HTTP/1.1" 200 0 85 "approuter-p***trial.cfapps.eu10.hana.ondemand.com/index.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"a29c59b9-ba64-4249-74ac-dbf9cf6b1255" response_time:0.014440749 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"b16f601b3276909a" x_b3_spanid:"b16f601b3276909a" x_b3_parentspanid:"-"
2018-08-31T15:38:52.91+0300 [RTR/0] OUT
Given your xs-app.json and the approuter URL I infer that the logout is located behind:
https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/my/logout.
Are you calling this endpoint to logout from the approuter? This should invalidate all your sessions as well as the JWT temporarily hold within the memory of the approuter.
Furthermore, when you access your Spring backend directly you should always receive 4xx error codes because no client actually ever has any JWT to access the spring-protected backend. In addition, the access to the Spring-based backend should be independent from any cookies or sessions because a JWT is used which by design is a stateless protocol. The only session cookie maintained is between your client and the approuter.
For more details on this, please refer to the Extending SAP S/4HANA Book, Chapter 5.
Trying to make a request from web App on Pythonanywhere (paid account) with a Flask. This works as needed on local PC, but receives an error on Pythonanywhere.
#app.route('/test/')
def test():
url_cookies='https://www.phoenixcontact.com/online/portal/ru/pxc/offcontext/'
headers = {'authority': 'www.phoenixcontact.com', 'method': 'GET','user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'}
requestCookies = requests.get(url_cookies,headers=headers)
return 'Test'+requestCookies.text
pythonanywhere has a only allowed limited webpages to be scrapped with the free version. You can remove this restriction by upgrading to one of their premium plans.
You might consider using FreeHosts like OpenShift for hosting your app.
I have a page protected with a closed user group. In one AEM instance, if an unauthenticated user tries to access the page, it redirects to the login form, as expected.
In another AEM instance, the user accessing the protected page doesn't get redirected to the login form. Instead the result is a 404.
Here's the access.log for the working instance:
10.10.10.10 - - 22/Nov/2016:11:59:01 -0600 "GET /protected-page.html HTTP/1.1" 200 352 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"
10.10.10.10 - admin 22/Nov/2016:11:59:01 -0600 "GET /system/sling/cqform/defaultlogin.html?resource=%2Fprotected-page.html&$$login$$=%24%24login%24%24 HTTP/1.1" 200 2267 "https://domain.com/protected-page.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"
Here's the access.log for the not working instance:
10.10.10.11 - anonymous 22/Nov/2016:12:03:22 -0600 "GET /protected-page.html HTTP/1.1" 404 467 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"
The difference appears to be that the user ID for the working request is -, whereas in the broken one it's anonymous. I think this means that the broken AEM thinks that the user has already been authenticated and just doesn't have access.
Any idea why this is happening and how I can configure the second instance like the first one?
Make sure that anonymous user has permissions to see the CUG page. You get 404 because the user cannot see that resource at all.
it sometimes returned :
{"error_code"=>"2000", "error_message"=>"The specified feature has been temporarily disabled for this application", "state"=>"d022a262e11b1a8bebe4bccb86319514e870fb6657b6aee8"}
from facebook when using facebook login
apache_log:
85.135.128.63 - - [08/May/2013:19:07:40 +0200] "GET /users/auth/facebook HTTP/1.1" 302 250 "http://www.itolar.com/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
85.135.128.63 - - [08/May/2013:19:07:41 +0200] "GET /users/auth/facebook/callback?error_code=2000&error_message=The+specified+feature+has+been+temporarily+disabled+for+this+application&state=d022a262e11b1a8bebe4bccb86319514e870fb6657b6aee8 HTTP/1.1" 500 728 "http://www.itolar.com/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
But when i tried two different accounts to do same thing, everything works fine
I am using omniauth-facebook on Ruby on Rails
Do anybody know whats wrong?
It sounds like you might be hitting a rate limit. Try making less frequent calls and/or optimizing them (ie via batch calls). Also check App Insights -> Activity & Errors.
I'm experiencing a weird thing in an apps I'm developping.
The tab is loading the canvas. In the apps parameter I've filled the right path to both canvas and tabs.
Here is the apache log for the app admina account while loading the tab:
172.16.109.9 - - [01/Sep/2011:15:48:30 +0200] "GET / HTTP/1.1" 200 1593 "http://static.ak.facebook.com/platform/page_proxy.php?v=4" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.865.0 Safari/535.2"
I tried with another account, tester of the apps, he load the right file. Here is the apache log :
172.16.109.9 - - [01/Sep/2011:15:52:59 +0200] "POST /tab HTTP/1.1" 200 1314 "_https://s-static.ak.facebook.com/platform/page_proxy.php?v=4" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.1) Gecko/20100101 Firefox/6.0.1"
Is this a known Facebook glitch ?
Thanks everyone :)
What do you mean by "loading the canvas"?
Is the user redirected to your canvas version or the canvas content is shown inside the page tab, instead of the tab content?
There are some things that come into my mind, but I would like to know what is the problem exactly