Why does an application using the Bluemix Single Sign On service return a 404 when it is horizontally scaled? - single-sign-on

I have a simple Node.js application using express 4.12 and express-session 1.11.2 using the Single Sign On (SSO) service for Bluemix. It works fine when there is a single application instance, but when the application is scaled to two instances either manually or with auto-scaling, it returns a 404 after the user provides their credentials.
I used cf logs to check the logs for the application and in both cases the application callback is being invoked as expected after authentication:
working (one application instance):
2015-07-02T10:11:38.45-0700 [RTR] OUT ssolab-20150601tor.mybluemix.net - [02/07/2015:17:11:38 +0000] "GET /auth/sso/callback?scope=openid&code=JaoWziZSyRNNwrBDECRiWBSNHSl0CF HTTP/1.1" 302 0 68 "https://bestssoever-2zgs9sg44w-cge7.iam.ibmcloud.com/idaas/mtfim/sps/default/oidc/consent" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" 75.126.70.43:34370 x_forwarded_for:"-" vcap_request_id:ec975656-70ae-44d8-4071-5b2d3b92e041 response_time:0.185715429 app_id:8b9f5b57-fbbd-4387-9cda-3737ad1f02ad
not working (two application instances):
2015-07-02T10:21:38.64-0700 [RTR] OUT ssolab-20150601tor.mybluemix.net - [02/07/2015:17:21:38 +0000] "GET /auth/sso/callback?scope=openid&code=wR2FhOFxZnowIIggOqyytf9DzNmwSq HTTP/1.1" 404 0 83 "https://bestssoever-2zgs9sg44w-cge7.iam.ibmcloud.com/idaas/mtfim/sps/default/oidc/consent" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" 75.126.70.43:39545 x_forwarded_for:"-" vcap_request_id:f1b75539-b02e-4217-4e3a-b3f6f1fc1c67 response_time:0.259509247 app_id:8b9f5b57-fbbd-4387-9cda-3737ad1f02ad
I would expect a different code parameter in the GET requests to the callback, so that is ok.
To get more debugging information, I added a console.log statement that is run before the user is directed to the SSO service for authentication. When processing in the callback, the passport openid connect node module (strategy.js) also logs output to the console. When all is working correctly, the output looks like:
2015-07-06T16:22:40.99-0700 [RTR] OUT ssolab-20150601tor.mybluemix.net - [06/07/2015:23:22:40 +0000] "GET /hello HTTP/1.1" 302 0 68 "https://ssolab-20150601tor.mybluemix.net/" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)" 75.126.70.46:15290 x_forwarded_for:"-" vcap_request_id:fe3c942d-7ae2-4500-6274-b1ecf95f24e5 response_time:0.009553663 app_id:8b9f5b57-fbbd-4387-9cda-3737ad1f02ad
2015-07-06T16:22:41.01-0700 [App/0] OUT Request by unauthenticated user
followed by:
2015-07-06T16:22:54.67-0700 [RTR] OUT ssolab-20150601tor.mybluemix.net - [06/07/2015:23:22:54 +0000] "GET /auth/sso/callback?scope=openid&code=QkXlFtOEpBkNBJdo4tSvQA6qUlw7Q7 HTTP/1.1" 302 0 68 "https://bestssoever-2zgs9sg44w-cge7.iam.ibmcloud.com/idaas/mtfim/sps/default/oidc/consent" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)" 75.126.70.46:29117 x_forwarded_for:"-" vcap_request_id:ca85eb7a-8b54-4bc1-43a7-2cff279de499 response_time:0.223288172 app_id:8b9f5b57-fbbd-4387-9cda-3737ad1f02ad
2015-07-06T16:22:54.68-0700 [App/0] OUT TOKEN
With two application instances it looks like the initial application traffic is going to one instance:
2015-07-06T16:37:17.68-0700 [RTR] OUT ssolab-20150601tor.mybluemix.net - [06/07/2015:23:37:17 +0000] "GET /hello HTTP/1.1" 302 0 68 "https://ssolab-20150601tor.mybluemix.net/" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)" 75.126.70.42:24841 x_forwarded_for:"-" vcap_request_id:491b5a52-653d-4fdc-6364-f604b3da395b response_time:0.008679282 app_id:8b9f5b57-fbbd-4387-9cda-3737ad1f02ad
2015-07-06T16:37:17.68-0700 [App/0] OUT Request by unauthenticated user
whereas the callback invocation seems to be handled by the other instance:
2015-07-06T16:37:51.01-0700 [RTR] OUT ssolab-20150601tor.mybluemix.net - [06/07/2015:23:37:50 +0000] "GET /auth/sso/callback?scope=openid&code=ylZKV3IPWMZgo2MNvwt9JxJzjU2Lsj HTTP/1.1" 404 0 83 "https://bestssoever-2zgs9sg44w-cge7.iam.ibmcloud.com/idaas/mtfim/sps/default/oidc/consent" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)" 75.126.70.43:48635 x_forwarded_for:"-" vcap_request_id:44b7893c-b185-4651-5384-7957a598bc20 response_time:0.273983071 app_id:8b9f5b57-fbbd-4387-9cda-3737ad1f02ad
2015-07-06T16:37:51.00-0700 [App/1] OUT TOKEN

The cause for the 404 is that the express-session middleware used by Passport with Express has a default store that is in memory and doesn't scale past a single process (see warning at: https://github.com/expressjs/session )
In the failing debug log snippet, before the authentication is delegated to SSO, Passport uses a session object managed by express-session in App/0. However, when the callback is invoked, the request is handled by App/1, and express-session in that instance doesn't have visibility to the original session.
I tested this hypothesis by implementing an alternate (and persistent) session store for express-session using connect-redis and the Redis Cloud service for Bluemix.
After configuring express-session to persist sessions to this store, the 404 errors went away when more than one instance of the application was active.

Related

sap 4 hana router logout not working properly

I have a sap trial account and I am developing a sap ui5 app with spring boot backend on cloud foundry. I also have an app router used for authentication/authorization and for access to my app. I have a simple page, index.html, which has a button that calls my spring backend ("/hello") and a logout button on it ("/my/logout").
All the links I give in this post are accessed only through approuter (/index.html, /hello and /debug are all relative paths to "https://approuter-p***trial.cfapps.eu10.hana.ondemand.com").
Here is the chain of events that I am doing and that has me puzzled:
After I login, I can call the spring backend by pushing my button and it returns 200 (OK).
After I logout by pressing the logout button (I am still on the same page, no redirect), I try calling the spring backend again by pressing the button and it returns 401 (OK). I can repeat this many times, it gives 401 (OK).
I try accessing the backend by calling the direct link from another tab in the same browser(https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/hello - it's the same link that gets called when I push my spring button) and I get a 200 - NOK!, I did not login in again...
I go back to the tab that had index.html opened and call the spring backend again by pressing my button - now it returns 200 also (NOK!)
I tried accessing the spring backend link from POSTMAN, no headers/cookies added - I get the same server response I do by calling it from index.html, by the spring button (same situations as above, sometimes 401, sometimes 200 if backend previously accessed directly in browser).
I also checked the token received via a "/debug" backend that prints it, and it seems that after I logout, any call to the spring backend directly from the browser (not from index.html or POSTMAN) receives a new JWT Token, which seems to be valid. The only difference I noticed was that the 200 responses come with the header "X-Frame-Options: DENY" and the 401 responses come with the header "X-Frame-Options: SAMEORIGIN". This was tested with Chrome and IE - both having the same behaviour.
Any ideas why this is happening? My xs-app.json (used by my-xsuaa service and conseqeuntly by approuter) looks like this:
{
"welcomeFile": "index.html",
"authenticationMethod" : "route",
"routes": [{
"source": "/",
"target": "/",
"destination": "app-destination-hello"
}],
"logout": {
"logoutEndpoint": "/my/logout"
},
"destinations": {
"app-destination-hello": {
"logoutPath": "/myrest/logout",
"logoutMethod": "GET"
}
}
}
Thanks!
I will attach some traces from the approuter, maybe it helps. These are the steps I took with the corresponding logs (did not skip any log entry between steps):
I press logout button from index.html:
2018-08-31T15:37:50.63+0300 [APP/PROC/WEB/0] OUT #2.0#2018 08 31 12:37:50:632#+00:00#ERROR#/Handler#########-UTzTjYczINpXSxkJY_QXoWlfSF3dk0O######jlhzhep3#PLAIN##GET request to /my/logout completed with status 401 - Authentication required#
2018-08-31T15:37:50.63+0300 [RTR/7] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:37:50.632+0000] "GET /my/logout HTTP/1.1" 401 0 12 "https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/index.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"bb472547-cf0f-407d-6ccb-87d2dd42d4e7" response_time:0.002667543 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"56fc90749079fa0f" x_b3_spanid:"56fc90749079fa0f" x_b3_parentspanid:"-"
2018-08-31T15:37:50.63+0300 [RTR/7] OUT
I press the spring button from index.html (the endpoint name is now /hello2, but it's the same as before) and I receive a 401, which is OK:
2018-08-31T15:37:57.57+0300 [APP/PROC/WEB/0] OUT #2.0#2018 08 31 12:37:57:575#+00:00#ERROR#/Handler#########utCeM-9IrqFW7oEOGkqMtIoUmwgTpZsv######jlhzhk1z#PLAIN##GET request to /hello2 completed with status 401 - Authentication required#
2018-08-31T15:37:57.58+0300 [RTR/4] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:37:57.583+0000] "GET /hello2 HTTP/1.1" 401 0 12 "https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/index.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"600551f2-7054-4d08-68aa-75d88f2584e2" response_time:0.003231487 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"25c2df6333a965cf" x_b3_spanid:"25c2df6333a965cf" x_b3_parentspanid:"-"
2018-08-31T15:37:57.58+0300 [RTR/4] OUT
I open the approuter/hello2 link from another browser tab and I receive a 200 (NOK):
2018-08-31T15:38:00.79+0300 [APP/PROC/WEB/0] OUT #2.0#2018 08 31 12:38:00:797#+00:00#INFO#/Auth/OAuth2#########pPyrBRKTJWcfwzc8yP8EExMmQbCfW992######jlhzhmjg#PLAIN##sending page with client-side redirect to https://p***trial.authentication.eu10.hana.ondemand.com/oauth/authorize?response_type=code&client_id=sb-firstapp3cris!t5664&redirect_uri=https%3A%2F%2Fapprouter-p***trial.cfapps.eu10.hana.ondemand.com%2Flogin%2Fcallback#
2018-08-31T15:38:00.80+0300 [RTR/5] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:38:00.801+0000] "GET /hello2 HTTP/1.1" 200 0 483 "https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/hello2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"5e18d1e8-f7e6-4b21-6f6a-2a7ea386c5cc" response_time:0.003697001 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"89e2fc6581f0c47b" x_b3_spanid:"89e2fc6581f0c47b" x_b3_parentspanid:"-"
2018-08-31T15:38:00.80+0300 [RTR/5] OUT
somehow a login is called automatically (I see a call in the index.html tab, but it is done when I press the logout button, also no reidrect page to ask me for username/password):
2018-08-31T15:38:39.46+0300 [RTR/5] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:38:39.158+0000] "GET /login/callback?code=UL6niGFX3T HTTP/1.1" 302 0 0 "https://p***trial.authentication.eu10.hana.ondemand.com/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"307eee0f-123f-46f6-7102-9ef370d66ab5" response_time:0.307257726 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"a9a073166c04301f" x_b3_spanid:"a9a073166c04301f" x_b3_parentspanid:"-"
2018-08-31T15:38:39.46+0300 [RTR/5] OUT
I press my spring button from approuter/index.html and now I receive 200 response here also:
2018-08-31T15:38:39.70+0300 [RTR/1] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:38:39.542+0000] "GET /hello2 HTTP/1.1" 200 0 85 "p***trial.authentication.eu10.hana.ondemand.com/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"b6400e34-87c0-47ea-4335-46d9c9348802" response_time:0.158048741 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"eb6dde02e73d9c38" x_b3_spanid:"eb6dde02e73d9c38" x_b3_parentspanid:"-"
2018-08-31T15:38:39.70+0300 [RTR/1] OUT
2018-08-31T15:38:52.91+0300 [RTR/0] OUT approuter-p***trial.cfapps.eu10.hana.ondemand.com - [2018-08-31T12:38:52.897+0000] "GET /hello2 HTTP/1.1" 200 0 85 "approuter-p***trial.cfapps.eu10.hana.ondemand.com/index.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "10.0.137.79:61038" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"a29c59b9-ba64-4249-74ac-dbf9cf6b1255" response_time:0.014440749 app_id:"9ceaf8d0-7ee7-4fb2-84d0-05df07af7c01" app_index:"0" x_b3_traceid:"b16f601b3276909a" x_b3_spanid:"b16f601b3276909a" x_b3_parentspanid:"-"
2018-08-31T15:38:52.91+0300 [RTR/0] OUT
Given your xs-app.json and the approuter URL I infer that the logout is located behind:
https://approuter-p***trial.cfapps.eu10.hana.ondemand.com/my/logout.
Are you calling this endpoint to logout from the approuter? This should invalidate all your sessions as well as the JWT temporarily hold within the memory of the approuter.
Furthermore, when you access your Spring backend directly you should always receive 4xx error codes because no client actually ever has any JWT to access the spring-protected backend. In addition, the access to the Spring-based backend should be independent from any cookies or sessions because a JWT is used which by design is a stateless protocol. The only session cookie maintained is between your client and the approuter.
For more details on this, please refer to the Extending SAP S/4HANA Book, Chapter 5.

Different redirect behavior between 2 AEM instances

I have a page protected with a closed user group. In one AEM instance, if an unauthenticated user tries to access the page, it redirects to the login form, as expected.
In another AEM instance, the user accessing the protected page doesn't get redirected to the login form. Instead the result is a 404.
Here's the access.log for the working instance:
10.10.10.10 - - 22/Nov/2016:11:59:01 -0600 "GET /protected-page.html HTTP/1.1" 200 352 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"
10.10.10.10 - admin 22/Nov/2016:11:59:01 -0600 "GET /system/sling/cqform/defaultlogin.html?resource=%2Fprotected-page.html&$$login$$=%24%24login%24%24 HTTP/1.1" 200 2267 "https://domain.com/protected-page.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"
Here's the access.log for the not working instance:
10.10.10.11 - anonymous 22/Nov/2016:12:03:22 -0600 "GET /protected-page.html HTTP/1.1" 404 467 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"
The difference appears to be that the user ID for the working request is -, whereas in the broken one it's anonymous. I think this means that the broken AEM thinks that the user has already been authenticated and just doesn't have access.
Any idea why this is happening and how I can configure the second instance like the first one?
Make sure that anonymous user has permissions to see the CUG page. You get 404 because the user cannot see that resource at all.

Dispatcher not showing results from an AJAX call to an OSGi bundle

In my project, we have a component that includes a JavaScript file & in that file, we are making an AJAX call to a Servlet (defined in an OSGi bundle).
When the package is installed in the Publish instance (along with the OSGi bundle), I'm able to see results after I click on a link which is bound to the AJAX call.
When accessing the same page through the Dispatcher however, the page is getting displayed but the link which should show content from the OSGi bundle is not working. The same link is working fine when accessed directly via the Publish instance
Updated:
The access.log in Dispatcher (Apache Web Server)
Success log of dispatcher
domain - - [11/Jul/2014:15:25:11 +0530] "GET /content/sample.html HTTP/1.1" 200 25805
Failure log on one of the links on the above page
domain - - [11/Jul/2014:15:25:12 +0530] "GET /bin/servlet/SampleServlet?action=GET_SAMPLE_USAGE HTTP/1.1" 404 230
It is not finding GET_SAMPLE_USAGE servlet, but the same is already available in the OSGi bundle and is working perfectly fine via the Publish instance (logs below).
Publisher access.log
domain - admin 11/Jul/2014:15:24:31 +0530 "GET /content/sample.html HTTP/1.1" 200 25805 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0"
domain - admin 11/Jul/2014:15:24:31 +0530 "GET /bin/servlet/SampleServlet?action=GET_SAMPLE_USAGE HTTP/1.1" 200 - [this is not a link] ( "domain:4503/content/sample.html" ) "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0"

Sometimes "The specified feature has been temporarily disabled for this application"

it sometimes returned :
{"error_code"=>"2000", "error_message"=>"The specified feature has been temporarily disabled for this application", "state"=>"d022a262e11b1a8bebe4bccb86319514e870fb6657b6aee8"}
from facebook when using facebook login
apache_log:
85.135.128.63 - - [08/May/2013:19:07:40 +0200] "GET /users/auth/facebook HTTP/1.1" 302 250 "http://www.itolar.com/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
85.135.128.63 - - [08/May/2013:19:07:41 +0200] "GET /users/auth/facebook/callback?error_code=2000&error_message=The+specified+feature+has+been+temporarily+disabled+for+this+application&state=d022a262e11b1a8bebe4bccb86319514e870fb6657b6aee8 HTTP/1.1" 500 728 "http://www.itolar.com/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
But when i tried two different accounts to do same thing, everything works fine
I am using omniauth-facebook on Ruby on Rails
Do anybody know whats wrong?
It sounds like you might be hitting a rate limit. Try making less frequent calls and/or optimizing them (ie via batch calls). Also check App Insights -> Activity & Errors.

Facebook load the canvas in the tab with some accounts

I'm experiencing a weird thing in an apps I'm developping.
The tab is loading the canvas. In the apps parameter I've filled the right path to both canvas and tabs.
Here is the apache log for the app admina account while loading the tab:
172.16.109.9 - - [01/Sep/2011:15:48:30 +0200] "GET / HTTP/1.1" 200 1593 "http://static.ak.facebook.com/platform/page_proxy.php?v=4" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.865.0 Safari/535.2"
I tried with another account, tester of the apps, he load the right file. Here is the apache log :
172.16.109.9 - - [01/Sep/2011:15:52:59 +0200] "POST /tab HTTP/1.1" 200 1314 "_https://s-static.ak.facebook.com/platform/page_proxy.php?v=4" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.1) Gecko/20100101 Firefox/6.0.1"
Is this a known Facebook glitch ?
Thanks everyone :)
What do you mean by "loading the canvas"?
Is the user redirected to your canvas version or the canvas content is shown inside the page tab, instead of the tab content?
There are some things that come into my mind, but I would like to know what is the problem exactly