Puppeteer / Chrome DevTools: retrieve certificate for intermediate server in a redirect chain - google-chrome-devtools

tl;dr
Is there a way to fetch certificates for all intermediates servers involved in a redirect chain and not only the final server?
Details
Lets say we have a URL (e.g., https://fastlane.ci) that redirects (HTTP 3xx) to another
one (e.g., https://github.com/fastlane/ci). Using the following code:
const session = await page.target().createCDPSession()
await session.send('Network.enable')
await page.goto('https://fastlane.ci')
await session.send('Network.getCertificate', { origin: 'https://github.com' })
I can get the x.509 cert provided by github.com. If I change origin to https://fastlane.ci, the result would be empty. I can, however, see in SecurityDetails that for the initial response (HTTP redirect) the certificate is retrieved and parsed somewhere in the background.
I need raw certificates. How can I get them over the DevTools API?

Related

Persisting user state in sveltekit

I'm trying to hook up a Strapi backend to a SvelteKit frontend, and stuck on how to persist user login state so that everything doesn't just reset on refresh, or when navigating to a new page. I've tried:
Storing the jwt and user object issued by Strapi in localStorage and initializing the Svelte store with it. Seemed like I was getting close, but a) I couldn't do export const user = writable(localStorage.user) because that code was running in the browser, and I couldn't wrap it in an if (browser) {...} because import and export can only appear at the top level. Also tried a function in hooks.js to read the contents of localStorage and update the store, but it seems that functions getting called from there run on the server, even if it's the same function that works to access localStorage on login... and plus b) from what I gather, storing jwt's in localStorage is insecure.
Storing the jwt and user object in an http only cookie. Cookies and http headers seem really confusing, and I had a hard time manipulating them to store the jwt and put it in each header. But I think what really stumped me was essentially the same SSR issue of never knowing essentially how to successfully interface between the client and server. I.e. if (browser) {...} never seemed to work, or I couldn't get it to, anyway. (Happy to provide more code details on what I tried here if needed. It's a mess, but it's saved in git.)
I know this is a thing every app that has users needs to do, so I'm sure there's a way to do it in SvelteKit. But I can't find anything online that explains it, and I can't figure it out from the official docs either.
So am I missing something easy? (Probably.) Or is there a tricky way to do this?
For a SvelteKit SPA authenticating to Strapi, here's the happy path flow I would use:
SvelteKit page /routes/login.svelte collects the username (identifier) and password and a button's on:click handler posts those values via fetch to your own /routes/auth.js:post() endpoint. Why? Because now your server's endpoint is handling the login on behalf of the user so you can set an httpOnly cookie in the response.
In your auth.js endpoint post() method, you will want to do a few things:
Use fetch to post identifier and password to Strapi authentication (http://localhost:1337/auth/local) to get the JWT (response.body.jwt)
Use fetch to get user info from Strapi (http://localhost:1337/users/me). In the get, add a header called 'Authentication' with a value of 'Bearer ' + the JWT you just received from Strapi in the previous step
Return the user info from the response from Strapi and pass it back to your client in the response.body
Set a header httpOnly cookie with the value of the JWT.
Back on the client in the last part of the login button's on:click handler, take the user info from res.json() and put it in a writeable store called user or in the SvelteKit session store...
import { session } from '$app/stores'
...
const loginClickHandler = async () => {
...
const fromEndpoint = await res.json()
session.set({ user: fromEndpoint.user })
}
At this point, you have the user information persisted client-side and the JWT as an httpOnly cookie that can't be read or modified by client-side JavaScript code. Each request you make to your own server's pages or endpoints will send the JWT cookie along.
If you want to logout, call an endpoint on your server (/auth/logout) that sets the existing jwt cookie to have an Expires based on the current date/time:
response.headers['Set-Cookie'] = `jwt=; Path=/; HttpOnly; Expires=${new Date().toUTCString()}`
You would also want to clear the user object in your store (or the session store).
The main takeaway for the above example is that your client would never directly talk to Strapi. Strapi's API would be called only by your SvelteKit server's endpoints. The httpOnly jwt cookie representing your session with Strapi would be included in every request to your server's endpoints to use/validate with Strapi's API (or delete if expired or the user logged out).
There are lots of other approaches but I prefer this one for security reasons.

How to read server set cookie in angular 2 application, and how to send same cookie in request from angular 2 application?

Requirement : Our application needs to support same user opening our web application as separated session.
The problem is not how to use cookies in angular 2, but how can sever get cookie from HTTPServletRequest object when angular 2 application makes a rest call to server.
Implementation: Server side restful application has one filter to set user's browser session in cookie and then in HttpServletResponse. Angular client is making one call upon application bootstrap, which is going through server filter to set user's browser session in cookie.
Problem statement: angular client is making first rest call which goes through server filter to set the browser session cookie. When i open chrome developer tool, i do see that rest api response has "set-cookie" which has cookie set by server, but when i open the application tag in developer tool, i do not see any cookie set.
After that if I make any other rest call through angular application, it does not send the cookie in either request or request headers. Now, our application rest api depends on this cookie value to be present in HttpServletRequest and now it is failing.
Can someone please guide me here? I must have done something wrong on angular 2 application side, which i am not able to catch.
I have tried passing "withCredentials =true", but no change.
Another thing I noticed, if i make "GET" request, then i do see cookie in request header, but for "POST" request, I do not see anything for cookie.
Please advice.
server side code to set cookie
String uniqueId = RandomStringUtils.randomAlphanumeric(32);
Cookie userSessionCookie = new Cookie("userSessionId", uniqueId);
if (getDefaultDomain() != null) {
userSessionCookie.setDomain(getDefaultDomain());
}
httpServletResponse.addCookie(userSessionCookie); httpServletResponse.addHeader("Access-Control-Allow-Credenti‌​als", "true"); httpServletResponse.addHeader("access-control-allow-methods"‌​, "GET, POST, PUT, PATCH, DELETE, OPTIONS");
httpServletResponse.addHeader("Access-Control-Allow-Headers"‌​, "Content-Type, token,withCredentials");
angular 2 post request which expects server to get cookie from HttpServletRequest
renderFF() {
//prepare renderFInput object
var fcRenderInput = {};
let headers = new Headers({ 'Content-Type': 'application/json' });
let options = new RequestOptions({ headers: headers, withCredentials: true
});
this._http.post('/api/v1/render/feature/fc',fcRenderI‌​nput,options)
.subscribe((res) => {
console.log(res.json());
});
}
Just a suggestion if this is about only one browser and multiple tabs, in this case you can use the local storage while setting some flag in it. Also when you try to open the same application in the new tab. you check if the flag is there and user is trying to open the same web application in some other tab of the same browser. You also need to delete the local storage you had set after some point.
I hope if you can get some trick to solve this issue :)

Get location fragment with Fetch API redirect response

I am trying to get the redirect response location fragment of a fetch API request. But I can't figure how to access it, if possible.
The context is that I am doing an OpenID Connect request in implicit flow, for a WebRTC Identity Proxy assertion generation.
OIDC specs define the answer of the request as:
When using the Implicit Flow, all response parameters are added to the
fragment component of the Redirection URI
HTTP/1.1 302 Found
Location: https://client.example.org/cb#
access_token=SlAV32hkKG
...
So I'm making the request with fetch set in manual mode. But the response is then an opaque-redirect filtered response, which hides the location header. (https://fetch.spec.whatwg.org/#concept-filtered-response-opaque-redirect)
Other mode for fetch are error and follow which would not help. While XHR automatically follows the redirect so would not help either. I may be missing something from the fetch API, but it seems to be something hidden on purpose.
Could someone gives me a way to access this information (or a confirmation it's impossible) ?
Is there any alternative to fetch and XHR to make this request, which would allow to access the redirect location header?
Since XHR automatically / opaquely follows redirects (in the event you're using the whatwg-fetch polyfill for example), one possible solution is to check the response.url of the fetch resolution, to see if it matches a redirect location that you expect.
This only helps if the possible redirect locations are limited or match some pattern --- for instance, if you could expect at any time to be redirect to /login:
function fetchMiddleware(response) {
const a = document.createElement('a');
a.href = response.url;
if (a.pathname === '/login') {
// ...
} else {
return response;
}
}
fetch(`/api`)
.then(fetchMiddleware)
.then(function (response) {
// ...
});
fetch isn't able to polyfill the entire standard. Some notable differences include:
Inability to set the redirect mode.
See David Graham comment on the Disable follow redirect:
This is a nice addition to the Fetch API, but we won't be able to polyfill it with XMLHttpRequest. The browser navigates all redirects before returning a result, so there is no opportunity to interrupt the redirect flow.
My Solution:
1). First solution: we are sending 200 status and redirect url(in the http header) from the server and client is redirecting based on that.
2). Second solution: Server could also redirect to with 301 and redirect url. I think, This is the best solution(i.e if we consider SEO).

How to secure a Jersey REST call in this case

The problem i am facing is that clicking on F12 on Chrome Browser , i could see all the Rest Calls which are made to fetch the data
For example , one of the REST API call is
(When clicked on the above link , it fetches the data )
This is my front code consists of Jquery
function displaymarketupdates() {
var updatedon = "";
var html = '';
var t = "",
$.ajax({
type: "GET",
url: e,
crossDomain: !0,
dataType: "json",
timeout: 17e3,
async: !0,
cacheResults: !1,
cache: !1,
contentType: "application/json",
charset: "utf-8",
beforeSend: function() {
$(".loadingWrapformarketupdates").show()
},
complete: function() {
$(".loadingWrapformarketupdates").hide()
},
success: function(response) {
},
error: function(t, e, a) {
$(".loadingWrapformarketupdates").hide()
}
}).done(function() {
})
}
And this is my service
#Path("/fetchallvalues")
public class FetchAllValues {
public FetchAllValues() {}
private final static Logger logger = Logger.getLogger(FetchAllValues.class);
#GET#Produces("text/plain")
public Response Fetch_all_values() {
PreparedStatement fetch_all_pstmt = null;
ResultSet fetch_all_Rset = null;
Connection dbConnection = null;
ResponseBuilder builder = Response.status(Status.NOT_FOUND);
final JSONArray fetch_array = new JSONArray();
final String inputsql = "select * from all_values";
try {
dbConnection = DBConnection.getDBConnection();
fetch_all_pstmt = dbConnection.prepareStatement(inputsql);
fetch_all_Rset = fetch_all_pstmt.executeQuery();
while (fetch_all_Rset.next()) {
====
}
Response.status(Status.OK);
builder = Response.ok(fetch_array.toString());
} catch (Exception e) {
e.printStackTrace();
logger.error("Error description", e);
} finally {
try {
DBConnection.close(fetch_all_pstmt, fetch_all_Rset);
} catch (Exception e) {
logger.error("Error description", e);
}
try {
DBConnection.close(dbConnection);
} catch (Exception e) {
logger.error("Error description", e);
}
}
return builder.build();
}
}
Could you please let me know how to secure the REST CALL in this case
You cannot hide an URL from a Browser's network monitoring. It is meant to be displayed so that it can be inferred that what is happening when you hit a button or click something.
Securing a REST Jersey call is a totally different thing. That means you do not want people to see your data that you are going to pass. As correctly mentioned by Martingreber that if you call this URL on HTTPS that may help you encrypt data that you send across the servers. Or securing a REST call actually means you provide some kind of authentication to it . Like Basic , Hashing like MD5, Token based Authentication like JWT.
The only thing that you can do to hide explicit details from your browser that runs your JavaScript is minify your script . But still your URL remains exposed as many times as it is called by someone who fiddles with the F12 key on Chrome to see what's going on. One more thing can be if you are concerned about your main service call, and don't want to expose that , then just PROXY it using some service, which you are already doing . But by no means, you can avoid your URL being getting displayed, when someone calls it.
In your case fetchAllValues service is fetching the data and exposing it to anybody on the web who clicks it, but that can be prevented if you authenticate the service, like the minute i click that URL, it asks me for a password! Then i cannot access it. A very simple way to authenticate this service would to call a Filter or an Interceptor just before the request to ask for username and password like credentials.
I hope you got the point. Hope this helps :)
You will always be able to see the URL that is being processed. Still, you could obfuscate the Service Endpoint to hide the purpose of the service itself, e.g. #Path("/XYZ")instead of #Path("fetchallvalues")
If you want to hide the data that is being transmitted between the client and the server, so noone can read it, simply use https. Depending on your webserver (Jetty, Tomcat) you will have to configure it differently, still you will need a ssl certificate for your domain, which you can get here for example: https://letsencrypt.org
If you want to secure your webservice, so it can't be used by anyone, but only by specific users, you might want to give Spring Security a try: User authentication on a Jersey REST service
This is a problem that needs some smart hacks to fix it.
In the hyperlinked stackoverflow page, you will get an example of how to make a SOAP request from client side JavaScript.
SOAP request from JavaScript
Now here's the plan:
In the server side, we have a random number generator, which generates a random number in short intervals, say 5 minutes.
The random number generator will be exposed as a SOAP service and it will produce the random number generated.
From the client side, we will invoke the SOAP random generator service (refering to the stackoverflow page mentioned above) and get the generated random number as the response. We will invoke the service from a JS function which will be fired when your page is loaded (onLoad). So, now we have the random number at the client side.
Then, we pass the random number as a path param in the GET request URL of the REST call and fire the GET request.
In the server side, once the Rest GET request is received, we check if the number received as path param is the same number that is generated in the server side.
If the numbers match, then we give the required response, else do not send the response.
Here we are trying to introduce an unique key, which is the random number generated at the server side. This unique key, when passed as the path param of the Rest GET request URL, serves as an identity of the origin of the Rest GET call. For someone who wants to invoke the Rest Api by referring to the Network Tab of the Chrome Dev console, will not get the unique key for a long time ( as it is refreshed/regenerated after every 5 minutes). Thus the hacker will not be able to use the Rest Api for a long duration. Also, since we are transporting the unique key (the random number) from the server to client side using SOAP, it is not possible for the hacker to get it from the Chrome's developer console.
Hope this approach helps!
Unfortunately, there is nothing you can do to prevent the client from inspecting the requested URL. But you always can require credentials to access your API endpoints.
Authentication in REST APIs
In REST applications, each request from the client to the server must contain all the necessary information to be understood by the server. With it, you are not depending on any session context stored on the server and you do not break the REST stateless constraint, defined by Roy Thomas Fielding in his dissertation:
5.1.3 Stateless
[...] communication must be stateless in nature [...], such that each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client. [...]
When accessing protected resources (endpoints that require authentication), every request must contain all necessary data to be properly authenticated/authorized. And authentication data should belong to the standard HTTP Authorization header. From the RFC 7235:
4.2. Authorization
The Authorization header field allows a user agent to authenticate
itself with an origin server -- usually, but not necessarily, after
receiving a 401 (Unauthorized) response. Its value consists of
credentials containing the authentication information of the user
agent for the realm of the resource being requested. [...]
In other words, the authentication will be performed for each request.
Basic authentication
The Basic Authentication scheme, defined in the RFC 7617, is a good start for securing a REST API:
2. The 'Basic' Authentication Scheme
The Basic authentication scheme is based on the model that the client
needs to authenticate itself with a user-id and a password for each
protection space ("realm"). [...] The server will service the request only if it can validate
the user-id and password for the protection space applying to the
requested resource.
[...]
To receive authorization, the client
obtains the user-id and password from the user,
constructs the user-pass by concatenating the user-id, a single
colon (":") character, and the password,
encodes the user-pass into an octet sequence,
and obtains the basic-credentials by encoding this octet sequence
using Base64 into a sequence of US-ASCII
characters.
[...]
If the user agent wishes to send the user-id "Aladdin" and password
"open sesame", it would use the following header field:
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
[...]
Token-based authentication
If you don't want to send the username and the password over the wire for every request, you could consider using a token-based authentication. In this approach, you exchange your hard credentials (username and password) for a token which the client must send to the server in each request:
The client sends their credentials (username and password) to the server.
The server authenticates the credentials and generates a token.
The server stores the previously generated token in some storage along with the user identifier and an expiration date.
The server sends the generated token to the client.
In every request, the client sends the token to the server.
The server, in each request, extracts the token from the incoming request. With the token, the server looks up the user details to perform authentication and authorization.
If the token is valid, the server accepts the request.
If the token is invalid, the server refuses the request.
The server can provide an endpoint to refresh tokens.
Again, the authentication must be performed for every request.
The token can be opaque (which reveals no details other than the value itself, like a random string) or can be self-contained (like JSON Web Token).
Random String: A token can be issued by generating a random string and persisting it to a database with an expiration date and with a user identifier associated to it.
JSON Web Token (JWT): Defined by the RFC 7519, it's a standard method for representing claims securely between two parties. JWT is a self-contained token and enables you to store a user identifier, an expiration date and whatever you want (but don't store passwords) in a payload, which is a JSON encoded as Base64. The payload can be read by the client and the integrity of the token can be easily checked by verifying its signature on the server. You won't need to persist JWT tokens if you don't need to track them. Althought, by persisting the tokens, you will have the possibility of invalidating and revoking the access of them. To find some great resources to work with JWT, have a look at http://jwt.io.
In a token-based authentication, tokens are your credentials. So the tokens should be sent to the server in the standard HTTP Authorization header as described above.
Once you are using Jersey, you could have a look at this answer for more details on how to implement a token-based authentication in Jersey.
HTTPS
When sending sensitive data over the wire, your best friend is HTTPS and it protects your application against the man-in-the-middle attack.
To use HTTPS, you need a certificate issued by a certificate authority such as Let’s Encrypt, that claims to be a free, automated, and open certificate authority.

Logging a user out when using HTTP Basic authentication

I want users to be able to log in via HTTP Basic authentication modes.
The problem is that I also want them to be able to log out again - weirdly browsers just don't seem to support that.
This is considered to be a social-hacking risk - user leaves their machine unlocked and their browser open and someone else can easily visit the site as them. Note that just closing the browser-tab is not enough to reset the token, so it could be an easy thing for users to miss.
So I've come up with a workaround, but it's a total cludge:
1) Redirect them to a Logoff page
2) On that page fire a script to ajax load another page with dummy credentials:
$j.ajax({
url: '<%:Url.Action("LogOff401", new { id = random })%>',
type: 'POST',
username: '<%:random%>',
password: '<%:random%>',
success: function () { alert('logged off'); }
});
3) That should always return 401 the first time (to force the new credentials to be passed) and then only accept the dummy credentials:
[AcceptVerbs(HttpVerbs.Post)]
public ActionResult LogOff401(string id)
{
// if we've been passed HTTP authorisation
string httpAuth = this.Request.Headers["Authorization"];
if (!string.IsNullOrEmpty(httpAuth) &&
httpAuth.StartsWith("basic", StringComparison.OrdinalIgnoreCase))
{
// build the string we expect - don't allow regular users to pass
byte[] enc = Encoding.UTF8.GetBytes(id + ':' + id);
string expected = "basic " + Convert.ToBase64String(enc);
if (string.Equals(httpAuth, expected, StringComparison.OrdinalIgnoreCase))
{
return Content("You are logged out.");
}
}
// return a request for an HTTP basic auth token, this will cause XmlHttp to pass the new header
this.Response.StatusCode = 401;
this.Response.StatusDescription = "Unauthorized";
this.Response.AppendHeader("WWW-Authenticate", "basic realm=\"My Realm\"");
return Content("Force AJAX component to sent header");
}
4) Now the random string credentials have been accepted and cached by the browser instead. When they visit another page it will try to use them, fail, and then prompt for the right ones.
Note that my code examples are using jQuery and ASP.Net MVC, but the same thing should be possible with any technology stack.
There's another way to do this in IE6 and above:
document.execCommand("ClearAuthenticationCache");
However that clears all authentication - they log out of my site and they're logged out of their e-mail too. So that's out.
Is there any better way to do this?
I've seen other questions on this, but they're 2 years old - is there any better way now in IE9, FX4, Chrome etc?
If there is no better way to do this can this cludge be relied upon? Is there any way to make it more robust?
The short anser is:
There is no reliable procedure for achieving a "logoff" using
HTTP Basic or Digest authentication given current implemenations of basic auth.
Such authentication works by having the client add an Authorization header
to the request.
If for a certain resource the server is not satisfied with the credentials provided (e.g. if there are none), it will responde with a
"401 Unauthorized" status code and request authentication. For that purpose it will provide a WWW-Authenticate header with the response.
A client need not wait for a server requesting authentication.
It may simply provide an Authorization header based on some local
assumptions (e.g. cached information from the last successful attempt).
While your outlined approach on "clearing" out authentication info has a good chance of working with a wide range of clients (namely widespread browsers),
there is absolutely no guarantee that a nother client might be "smarter" and
simply discriminate proper authentication data for your "logout" page and any other pages of the target site.
You will recognize a similar "problem" with using client side certificate based authentication.
As long as there is no explicit support from clients you might fight on lost ground.
So, if "logoff" is a concern, move over to any session based authentication.
If you have access to the implementation of authentication on the server side you might be able implementing a functionality that will disregard authentication information presented with Authorization header (if still identical to what has been presented during current "session) on request of your application level code (or provide some "timout" after which any credentials will be re-requested), so that the client will ask the user for providing "new" credentials (performing a new login).