I have a DockerFile based on Varnish 7.0 alpine, I have a custom vcl file to handle JWT authentication. We pass the JWT as a Bearer in the header.
I am based on this example: https://feryn.eu/blog/validating-json-web-tokens-in-varnish/
set req.http.tmpPayload = regsub(req.http.x-token,"[^\.]+\.([^\.]+)\.[^\.]+$","\1");
set req.http.tmpHeader = regsub(req.http.x-token,"([^\.]+)\.[^\.]+\.[^\.]+","\1");
set req.http.tmpRequestSig = regsub(req.http.x-token,"^[^\.]+\.[^\.]+\.([^\.]+)$","\1");
set req.http.tmpCorrectSig = digest.base64url_nopad_hex(digest.hmac_sha256(std.fileread("/jwt/privateKey.pem"), req.http.tmpHeader + "." + req.http.tmpPayload));
std.log("req sign " + req.http.tmpRequestSig);
std.log("calc sign " + req.http.tmpCorrectSig);
if(req.http.tmpRequestSig != req.http.tmpCorrectSig) {
std.log("invalid signature match");
return(synth(403, "Invalid JWT signature"));
}
My problem is that tmpCorrectSig is empty, I don't know if I can load from a file, since my file contains new lines and other caracteres ?
For information, this Vmod is doing what I want: https://code.uplex.de/uplex-varnish/libvmod-crypto, but I can't install it on my Arm M1 pro architecture, I spent so much time trying...
Can I achieve what I want?
I have a valid solution that leverages the libvmod-crypto. The VCL supports both HS256 and RS256.
These are the commands I used to generated the certificates:
cd /etc/varnish
ssh-keygen -t rsa -b 4096 -m PEM -f jwtRS256.key
openssl rsa -in jwtRS256.key -pubout -outform PEM -out jwtRS256.key.pub
I use https://jwt.io/ to generate a token and paste in the values from my certificates to encrypt the signature.
The VCL code
This is the VCL code that will extract the JWT from the token cookie:
vcl 4.1;
import blob;
import digest;
import crypto;
import std;
sub vcl_init {
new v = crypto.verifier(sha256,std.fileread("/etc/varnish/jwtRS256.key.pub"));
}
sub vcl_recv {
call jwt;
}
sub jwt {
if(req.http.cookie ~ "^([^;]+;[ ]*)*token=[^\.]+\.[^\.]+\.[^\.]+([ ]*;[^;]+)*$") {
set req.http.x-token = ";" + req.http.Cookie;
set req.http.x-token = regsuball(req.http.x-token, "; +", ";");
set req.http.x-token = regsuball(req.http.x-token, ";(token)=","; \1=");
set req.http.x-token = regsuball(req.http.x-token, ";[^ ][^;]*", "");
set req.http.x-token = regsuball(req.http.x-token, "^[; ]+|[; ]+$", "");
set req.http.tmpHeader = regsub(req.http.x-token,"token=([^\.]+)\.[^\.]+\.[^\.]+","\1");
set req.http.tmpTyp = regsub(digest.base64url_decode(req.http.tmpHeader),{"^.*?"typ"\s*:\s*"(\w+)".*?$"},"\1");
set req.http.tmpAlg = regsub(digest.base64url_decode(req.http.tmpHeader),{"^.*?"alg"\s*:\s*"(\w+)".*?$"},"\1");
if(req.http.tmpTyp != "JWT") {
return(synth(400, "Token is not a JWT: " + req.http.tmpHeader));
}
if(req.http.tmpAlg != "HS256" && req.http.tmpAlg != "RS256") {
return(synth(400, "Token does not use a HS256 or RS256 algorithm"));
}
set req.http.tmpPayload = regsub(req.http.x-token,"token=[^\.]+\.([^\.]+)\.[^\.]+$","\1");
set req.http.tmpRequestSig = regsub(req.http.x-token,"^[^\.]+\.[^\.]+\.([^\.]+)$","\1");
if(req.http.tempAlg == "HS256") {
set req.http.tmpCorrectSig = digest.base64url_nopad_hex(digest.hmac_sha256("SlowWebSitesSuck",req.http.tmpHeader + "." + req.http.tmpPayload));
if(req.http.tmpRequestSig != req.http.tmpCorrectSig) {
return(synth(403, "Invalid HS256 JWT signature"));
}
} else {
if (! v.update(req.http.tmpHeader + "." + req.http.tmpPayload)) {
return (synth(500, "vmod_crypto error"));
}
if (! v.valid(blob.decode(decoding=BASE64URLNOPAD, encoded=req.http.tmpRequestSig))) {
return(synth(403, "Invalid RS256 JWT signature"));
}
}
set req.http.tmpPayload = digest.base64url_decode(req.http.tmpPayload);
set req.http.X-Login = regsub(req.http.tmpPayload,{"^.*?"login"\s*:\s*(\w+).*?$"},"\1");
set req.http.X-Username = regsub(req.http.tmpPayload,{"^.*?"sub"\s*:\s*"(\w+)".*?$"},"\1");
unset req.http.tmpHeader;
unset req.http.tmpTyp;
unset req.http.tmpAlg;
unset req.http.tmpPayload;
unset req.http.tmpRequestSig;
unset req.http.tmpCorrectSig;
unset req.http.tmpPayload;
}
}
Installing libvmod-crypto
libvmod-crypto is required to use RS256, which is not supported by libvmod-digest.
Unfortunately I'm getting an error when running the ./configure script:
./configure: line 12829: syntax error: unexpected newline (expecting ")")
I'll talk to the maintainer of the VMOD and see if we can figure out someway to fix this. If this is an urgent matter, I suggest you use a non-Alpine Docker container for the time being.
Firstly, the configure error was caused by a missing -dev package, see the gitlab issue (the reference is in a comment, but I think it should be more prominent).
The main issue in the original question is that digest.hmac_sha256() can not be used to verify RS256 signatures. A JWT RS256 signature is a SHA256 hash of the subject encrypted with an RSA private key, which can then be verified by decrypting with the RSA public key and checking the signature. This is what crypto.verifier(sha256, ...) does.
In this regard, Thijs' previous answer is already correct.
Yet the code which is circulating and has been referenced here it nothing I would endorse. Among other issues, a fundamental problem is that regular expressions are used to (pretend to) parse JSON, which is simply not correct.
I use a better implementation for long, but just did not get around to publishing it. So now is the time, I guess.
I have just added VCL snippets from production code for JWT parsing and validation.
The example is used like so with the jwt directory in vcl_path:
include "jwt/jwt.vcl";
include "jwt/rsa_keys.vcl";
sub vcl_recv {
jwt.set(YOUR_JWT); # replace YOUR_JWT with an actual variable/header/function
call recv_jwt_validate;
# do things with jwt_payload.extract(".scope")
}
Here, the scope claim contains the data that we are actually interested in for further processing, if you want to use other claims, just rename .scope or add another jwt_payload.expect(CLAIM, ...) and then use jwt_payload.extract(CLAIM).
This example uses some vmods, which we developed and maintain in particular with JWT in mind, though not exclusively:
crypto (use gitlab mirror for issues) for RS signatures (mostly RS256)
frozen (use gitlab mirror for issues) for JSON parsing
Additionally, we use
re2 (use gitlab mirror for issues) to efficiently split the JWT into the three parts (header, payload, signature)
and taskvar from objvar (gitlab) for proper variables.
One could do without these two vmods (re2 could be replaced by the re vmod or even regsub and taskvar with headers), but they make the code more efficient and cleaner.
blobdigest (gitlab) is not contained in the example, but can be used to validate HS signtures (e.g. HS256).
Related
I used this tutorial and created "put" endpoint successfully.
https://sanderknape.com/2017/10/creating-a-serverless-api-using-aws-api-gateway-and-dynamodb/
When I follow this advice, I get authroization required error..
Using your favorite REST client, try to PUT an item into DynamoDB
using your API Gateway URL.
python is my favorite client:
import requests
api_url = "https://0pg2858koj.execute-api.us-east-1.amazonaws.com/tds"
PARAMS = {"name": "test", "favorite_movie":"asdsf"}
r = requests.put(url=api_url, params=PARAMS)
the response is 403
My test from console is successful, but not able to put a record from python.
The first step you can take to resolve the problem is to investigate the information returned by AWS in the 403 response. It will provide a header, x-amzn-ErrorType and error message with information about the concrete error. You can test it with curl in verbose mode (-v) or with your Python code. Please, review the relevant documentation to obtain a detailed enumeration of all the possible error reasons.
In any case, looking at your code, it is very likely that you did not provide the necessary authentication or authorization information to AWS.
The kind of information that you must provide depends on which mechanism you configured to access your REST API in API Gateway.
If, for instance, you configured IAM based authentication, you need to set up your Python code to generate an Authorization header with an AWS Signature derived from your user access key ID and associated secret key. The AWS documentation provides an example of use with Postman.
The AWS documentation also provides several examples of how to use python and requests to perform this kind of authorization.
Consider, for instance, this example for posting information to DynamoDB:
# Copyright 2010-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# This file is licensed under the Apache License, Version 2.0 (the "License").
# You may not use this file except in compliance with the License. A copy of the
# License is located at
#
# http://aws.amazon.com/apache2.0/
#
# This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
# OF ANY KIND, either express or implied. See the License for the specific
# language governing permissions and limitations under the License.
# AWS Version 4 signing example
# DynamoDB API (CreateTable)
# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a POST request and passes request parameters
# in the body (payload) of the request. Auth information is passed in
# an Authorization header.
import sys, os, base64, datetime, hashlib, hmac
import requests # pip install requests
# ************* REQUEST VALUES *************
method = 'POST'
service = 'dynamodb'
host = 'dynamodb.us-west-2.amazonaws.com'
region = 'us-west-2'
endpoint = 'https://dynamodb.us-west-2.amazonaws.com/'
# POST requests use a content type header. For DynamoDB,
# the content is JSON.
content_type = 'application/x-amz-json-1.0'
# DynamoDB requires an x-amz-target header that has this format:
# DynamoDB_<API version>.<operationName>
amz_target = 'DynamoDB_20120810.CreateTable'
# Request parameters for CreateTable--passed in a JSON block.
request_parameters = '{'
request_parameters += '"KeySchema": [{"KeyType": "HASH","AttributeName": "Id"}],'
request_parameters += '"TableName": "TestTable","AttributeDefinitions": [{"AttributeName": "Id","AttributeType": "S"}],'
request_parameters += '"ProvisionedThroughput": {"WriteCapacityUnits": 5,"ReadCapacityUnits": 5}'
request_parameters += '}'
# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()
def getSignatureKey(key, date_stamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), date_stamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = os.environ.get('AWS_ACCESS_KEY_ID')
secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY')
if access_key is None or secret_key is None:
print('No access key is available.')
sys.exit()
# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
date_stamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope
# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
# Step 1 is to define the verb (GET, POST, etc.)--already done.
# Step 2: Create canonical URI--the part of the URI from domain to query
# string (use '/' if no path)
canonical_uri = '/'
## Step 3: Create the canonical query string. In this example, request
# parameters are passed in the body of the request and the query string
# is blank.
canonical_querystring = ''
# Step 4: Create the canonical headers. Header names must be trimmed
# and lowercase, and sorted in code point order from low to high.
# Note that there is a trailing \n.
canonical_headers = 'content-type:' + content_type + '\n' + 'host:' + host + '\n' + 'x-amz-date:' + amz_date + '\n' + 'x-amz-target:' + amz_target + '\n'
# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers include those that you want to be included in the
# hash of the request. "Host" and "x-amz-date" are always required.
# For DynamoDB, content-type and x-amz-target are also required.
signed_headers = 'content-type;host;x-amz-date;x-amz-target'
# Step 6: Create payload hash. In this example, the payload (body of
# the request) contains the request parameters.
payload_hash = hashlib.sha256(request_parameters.encode('utf-8')).hexdigest()
# Step 7: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = date_stamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' + amz_date + '\n' + credential_scope + '\n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, date_stamp, region, service)
# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()
# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# Put the signature information in a header named Authorization.
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
# For DynamoDB, the request can include any headers, but MUST include "host", "x-amz-date",
# "x-amz-target", "content-type", and "Authorization". Except for the authorization
# header, the headers must be included in the canonical_headers and signed_headers values, as
# noted earlier. Order here is not significant.
# # Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'Content-Type':content_type,
'X-Amz-Date':amz_date,
'X-Amz-Target':amz_target,
'Authorization':authorization_header}
# ************* SEND THE REQUEST *************
print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + endpoint)
r = requests.post(endpoint, data=request_parameters, headers=headers)
print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)
I think it could be easily adapted to your needs.
In the console, everything works fine because when you invoke your REST endpoints in API Gateway, you are connected to a user who is already authenticated and authorized to access these REST endpoints.
I am stuck, I am trying to sign a Json Web Token for Docusign. https://developers.docusign.com/esign-rest-api/guides/authentication/oauth2-jsonwebtoken Docusign just provides a RSA private and public key hash. That's it. The JWT must signed using RS256.
I found a JWT module https://www.powershellgallery.com/packages/JWT/1.1.0 but that requires that I have the certificate installed. But all I have is the key hash.
Levering some other code I found I was able to create a JWT token , although with the wrong algorithm. https://www.reddit.com/r/PowerShell/comments/8bc3rb/generate_jwt_json_web_token_in_powershell/
I've been trying modify it to use the RSACryto provider by creating a new object but ive been unsuccessful.
I tried to create a new object and see if I can some how import the key so that I can sign the token. But I cant seem to be able to do that.
$rsa = New-Object -TypeName System.Security.Cryptography.RSACryptoServiceProvider
$keyhash = "-----BEGIN RSA PRIVATE KEY-----
XXXXXX
-----END RSA PRIVATE KEY-----"
$blob = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($keyhash))
I tried to use the ImportCspBlob method but it requires that the string is converted to bytes but I cant seem to do that either.
Im not sure if I am even approaching this int he correct fashion. Ive been getting errors of either
Exception calling "ImportCspBlob" with "1" argument(s): "Bad Version of provider.
or
Cannot convert value to type "System.Byte". Error: "Input string was not in a correct format."
EDIT:
I have a work around using Node.js, although id still like to see if it is possible to do what I am trying to do natively in Powershell . This work aroun might be useful for some as there does not seem to be many references for using Powershell and Docusign API.
I found a node.JS script here that creates a JWT Token using the RS256 algorithm. https://github.com/BlitzkriegSoftware/NodejwtRSA ,
I stripped out all the extra stuff so the output to the console is only the token, and added the relevant scope to the "payload data" and under the sign options updated the sub, aud, and iss. The my RSA Private key was stored locally on the system in a file.
nodejs script - My modified version below
'use strict';
const path = require('path');
const fs = require('fs');
// https://github.com/auth0/node-jsonwebtoken
var jwt = require('jsonwebtoken');
// Private Key (must read as utf8)
var privateKey = fs.readFileSync('./rsatest.pk','utf8');
// Sample claims payload with user defined fields (this can be anything, but briefer is better):
var payload = { };
// Populate with fields and data
payload.scope = "signature impersonation";
// Values for the rfc7519 fields
var iss = "XXXXX-XXXX-XXX-XXX-XXX";
var sub = "XXXXX-XXXX-XXX-XXX-XXX";
var aud = "account-d.docusign.com";
// Expiration timespan: https://github.com/auth0/node-jsonwebtoken#token-expiration-exp-claim
var exp = "1h";
// JWT Token Options, see: https://tools.ietf.org/html/rfc7519#section-4.1 for the meaning of these
// Notice the `algorithm: "RS256"` which goes with public/private keys
var signOptions = {
issuer : iss,
subject: sub,
audience: aud,
expiresIn: exp,
algorithm: "RS256"
};
var token = jwt.sign(payload, privateKey, signOptions);
console.log(token)
process.exitCode = 0;
I called it from Powershell and feed the access token back into my script so i can then get my access token and start making my API calls.
#get the JWT token
$token = & node C:\temp\nodejwt.js
# Generate Header for API calls.
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Content-Type", "application/x-www-form-urlencoded")
$body ="grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion=$token"
$authuri = "https://account-d.docusign.com/oauth/token"
#send the JWT and get the access token
$accesstoken = Invoke-RestMethod -Method post -Uri $authuri -Headers $headers -Body $body -Verbose
$getheaders = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$getheaders.Add("Content-Type", "application/json")
$getheaders.Add('Authorization','Bearer ' + $accesstoken.access_token)
$geturi = "https://account-d.docusign.com/oauth/userinfo"
#use the access token to make api calls
Invoke-RestMethod -Method get -Uri $geturi -Headers $getheaders
I also had the same problem and was stuck.I used Python to get JWT. Be sure to install PyJWT and cryptography library.
$ pip install PyJWT
$ pip install pyjwt[crypto]
import time
import jwt
from datetime import datetime, timedelta
private_key= b'-----BEGIN PRIVATE KEY-----
`-----END PRIVATE KEY-----\n'
encoded_jwt = jwt.encode({
"iss":"<service account e-mail>",
"scope":"",
"aud":"",
"exp":int(time.time())+3600,
"iat":int(time.time())
}, private_key, algorithm='RS256')
encoded_jwt
I am trying to setup token based authentication on Fastly CDN with Varnish VCL and using this sample VCL snippet to generate and validate JWT tokens here -
sub vcl_recv {
#FASTLY recv
if (req.request != "HEAD" && req.request != "GET" && req.request != "FASTLYPURGE") {
return(pass);
}
// Generate synth
if(req.url ~ "generate") {
error 901;
}
// Validate token
if(req.url ~ "validate") {
// Ensure token exists and parse into regex
if (req.http.X-JWT !~ "^([a-zA-Z0-9\-_]+)?\.([a-zA-Z0-9\-_]+)?\.([a-zA-Z0-9\-_]+)?$") {
// Forbidden
error 403 "Forbidden";
}
// Extract token header, payload and signature
set req.http.X-JWT-Header = re.group.1;
set req.http.X-JWT-Payload = re.group.2;
set req.http.X-JWT-Signature = digest.base64url_nopad_decode(re.group.3);
set req.http.X-JWT-Valid-Signature = digest.hmac_sha256("SupSecretStr",
req.http.X-JWT-Header "." req.http.X-JWT-Payload);
// Validate signature
if(digest.secure_is_equal(req.http.X-JWT-Signature, req.http.X-JWT-Valid-Signature)) {
// Decode payload
set req.http.X-JWT-Payload = digest.base64url_nopad_decode(req.http.X-JWT-Payload);
set req.http.X-JWT-Expires = regsub(req.http.X-JWT-Payload, {"^.*?"exp"\s*?:\s*?([0-9]+).*?$"}, "\1");
// Validate expiration
if (time.is_after(now, std.integer2time(std.atoi(req.http.X-JWT-Expires)))) {
// Unauthorized
synthetic {"{"sign":""} req.http.X-JWT-Signature {"","header":""} req.http.X-JWT-Header {"","payload":""} req.http.X-JWT-Payload {"","valid": ""} req.http.X-JWT-Valid-Signature {""}"};
return(deliver);
}
// OK
synthetic {"{"header2":""} req.http.X-JWT-Header {"","payload":""} req.http.X-JWT-Payload {"","sign":""} req.http.X-JWT-Signature {"","valid": ""} req.http.X-JWT-Valid-Signature {""}"};
return(deliver);
} else {
// Forbidden
synthetic {"{"header3":""} req.http.X-JWT-Header {"","payload":""} req.http.X-JWT-Payload {"","sign":""} req.http.X-JWT-Signature {"","valid": ""} req.http.X-JWT-Valid-Signature {""}"};
return(deliver);
}
}
return(lookup);
}
sub vcl_error {
#FASTLY error
// Generate JWT token
if (obj.status == 901) {
set obj.status = 200;
set obj.response = "OK";
set obj.http.Content-Type = "application/json";
set obj.http.X-UUID = randomstr(8, "0123456789abcdef") "-" randomstr(4, "0123456789abcdef") "-4" randomstr(3, "0123456789abcdef") "-" randomstr(1, "89ab") randomstr(3, "0123456789abcdef") "-" randomstr(12, "0123456789abcdef");
set obj.http.X-JWT-Issued = now.sec;
set obj.http.X-JWT-Expires = strftime({"%s"}, time.add(now, 3600s));
set obj.http.X-JWT-Header = digest.base64url_nopad({"{"alg":"HS256","typ":"JWT""}{"}"});
set obj.http.X-JWT-Payload = digest.base64url_nopad({"{"sub":""} obj.http.X-UUID {"","exp":"} obj.http.X-JWT-Expires {","iat":"} obj.http.X-JWT-Issued {","iss":"Fastly""}{"}"});
set obj.http.X-JWT-Signature = digest.base64url_nopad(digest.hmac_sha256("SupSecretStr", obj.http.X-JWT-Header "." obj.http.X-JWT-Payload));
set obj.http.X-JWT = obj.http.X-JWT-Header "." obj.http.X-JWT-Payload "." obj.http.X-JWT-Signature;
unset obj.http.X-UUID;
unset obj.http.X-JWT-Issued;
unset obj.http.X-JWT-Expires;
unset obj.http.X-JWT-Header;
unset obj.http.X-JWT-payload;
unset obj.http.X-JWT-Signature;
synthetic {"{"payload":""} obj.http.X-JWT-Payload {"","header":""} obj.http.X-JWT-Header {"","sign":""} obj.http.X-JWT-Signatre {"","token": ""} obj.http.X-JWT {""}"};
return(deliver);
}
// Valid token
if (obj.status == 902) {
set obj.status = 200;
set obj.response = "OK";
set obj.http.Content-Type = "application/json";
synthetic {"{ "token": ""} req.http.X-JWT {"" }"};
return(deliver);
}
}
Now, when I am trying to compile this it returns -
Syntax error: Unterminated _short-string_
at: (input Line 106 Pos 197)
synthetic {"{"sign":""} req.http.X-JWT-Signature {"","header":""} req.http.X-JWT-Header {"","payload":""} req.http.X-JWT-Payload {"","valid": ""} req.http.X-JWT-Valid-Signature {""}"};
Looks like i am not somehow escaping the values correctly here during the synthetic block.
The only reason I am trying to do add this synthetic block in the vcl_recv subroutine is because I want to test how the digest is generating the JWT token and validating it and with that I wanted to create similar JWT tokens on server side in Node.Js so i was trying to output the different intermediate parts of the token for debugging.
I am not quite familiar with the Varnish syntax and semantics but still I looked for help finding any docs regarding this schedule subroutine but found none so far.
So, can anyone help out in how to fix this and have the vcl_recv, vcl_error interpolate different intermediate values in the json response.
I tried to use some of the Node.Js based base64 url decode libraries to decode the returned token parts and was able to decode the header and payload part but the signature part I am not able to generate from the Node.Js. So, can anyone suggest what is the equivalent of base64url_nopad() in node.js or any javascript libraries ?
For the hmac_256 encryption part we are trying to use the crypto library and creating an hmac like crypto.createHmac('sha256', 'SupSecretStr').update().digest('hex'); but all base64 encode url libraries in js i think return padded urls which is why the base64 encoded part of this hmac 256 digest doesn't match the one generated with varnish
My syntax coloring tool is telling me pretty much the same thing as the error message: you screwed up your quotes :-)
Your last block {""}"}; is opening quotes ({"), immediately closing them ("}), then you are opening simple quotes " and the newline arrives before you close them.
To fix, just put a space between after the final quote of the json: {"" }"};
Is it possible to set x509 v3 extended attribute for subjectAltName on a CSR?
I am able to successfully generate a CSR, and then pass that to a CA to sign.
The CA is able to call functions like X509->setDomain("bob.com","*.bob.com","asdf.org"); and they appear in the final cert without issue (note: i am doing the sign, reload, set extensions, resign workaround for phpseclib).
The CSR process calling the same functions $X509->setExtension("id-ce-subjectAltName",array("names","here") ) or $X509->setDomain("domain1","domain2"); does not appear to set some X509 extended attributes in the CSR itself (openssl can), I have only been able to add SAN by the CA after re-import before the second signing.
Thanks!
Edit: I created an example CSR (sign, reload, x509v3 extensions, resign workaround) and it seems possible to set SOME extended attributes (CA: false, key usage, etc.) but NOT the subjectAltName info:
-----BEGIN CERTIFICATE REQUEST-----
MIIC1TCCAb8CAQAwVDELMAkGA1UEBgwCU1MxEDAOBgNVBAgMB09ic2N1cmUxDzAN
BgNVBAcMBlNlY3VyZTEQMA4GA1UECgwHYm9iY29ycDEQMA4GA1UEAwwHYm9iLmNv
bTCCASAwCwYJKoZIhvcNAQEBA4IBDwAwggEKAoIBAQCunNrjxEOILsESZ1osUkT3
zSeAHlzNiCBQnc/Xf+oW7Ir7wKfbHkV10cM583mw3Zy8rlCT/lUq0H+f3Uoc5/FA
dsYWhatlJRlTjv+yjSsxyB9i/hZ/KliP3Ix2O+Pq1wZIWfvk40hmCHHSB6YDtqt0
vXqUTIhH3SfyLtK9nd/6WG8bgIq9jgL6xvF3h44ynEwkuOnHt4a9WrflGX4KKcwo
OM98M/TAntDgSXBEYoLxenIbl3ypa7gtghVHHls3QOSay5QM87K3PJ3kSWVlZ3tN
tZGfbFdLiS+3MH8G3ujSX8XYeBnUj5jXi4SzpzQ9o5pyArL8DD8kC/Q9P/el1aZb
AgMBAAGgQDA+BgkqhkiG9w0BCQ4xMTAvMAwGA1UdEwEB/wQCMAAwFgYDVR0lAQH/
BAwwCgYIKwYBBQUHAwEwBwYDVR0RBAAwCwYJKoZIhvcNAQELA4IBAQAUIhgLf6LH
WwhncU+yrNy6yHPHjFbipehUaS/Wa6FBbANpLyEdslqNwFD2FyXofSQVCB0L6VnH
NNXHcfZqWOT8+Xd0yNmfr/w+vg+s1yY4S0VAG6mxGsqwdIJqgXm2jaw0rMpzRs/k
wfOpJ+gyRTrQz9VJrn4xC4Uv6zTGNj56qZehDievW74SGISbGzj7AGmCxpp5/qZD
LN4Ls3wZ7I4TfuYZDh7qWuFwEAEEv40JPF2qO3VitvyAYrKg3bnUlFzQrOsfs4Ot
xzihLwkFEobih1bOEhzTnSv+lMckfw0DQ5Eb8mtFTC+/KOqEMMU5Hq1fm3B9Bkgs
FmqQZk4QlerY
-----END CERTIFICATE REQUEST-----
I figured it out, the SignCSR function in phpseclib doesnt respsect the setDomain function, or any content in the $this->domains array however it IS possible to manually encode and force set the subject alt names yourself:
$OPTIONS["altnames"] = array(
"bob.com",
"10.123.123.123",
"*.bob.com",
"asdf.fdsa",
);
$ALTNAMES = $this->altnames($OPTIONS["altnames"]);
if ( count($ALTNAMES) )
{
$CSR->setExtension("id-ce-subjectAltName" , $ALTNAMES );
}
protected function altnames($ALTNAMES)
{
//Sort names and IPs into two different arrays
$DOMAINS = array(); $IPS = array();
foreach ($ALTNAMES as $ALTNAME)
{
if ( filter_var($ALTNAME, FILTER_VALIDATE_IP) )
{
array_push($IPS,$ALTNAME);
}else{
array_push($DOMAINS,$ALTNAME);
}
}
// Create our altname array for the subjectAltName parser
$RETURN = array();
foreach ($DOMAINS as $DOMAIN) { array_push($RETURN, array("dNSName" => $DOMAIN ) ); }
foreach ($IPS as $IP) { array_push($RETURN, array("iPAddress" => $IP ) ); }
return $RETURN;
}
Now, the one caveat I found (that is a feature issue for phpseclib to consider) is the lack of IPv6 support. If you provide a valid IPv6 address in the altnames array (2620:153::1234) it will translate to IP Address: 0.0.0.0 because of the use of IP2LONG and LONG2IP embedded in the X509 code.
While not a showstopper for my use case, it is a minor annoyance and I have experimented with using a custom IP2BIN and BIN2IP function replacing the encode and decode logic for an IP in the library and not met with much success.
Example CSR generated with my code above:
-----BEGIN CERTIFICATE REQUEST-----
MIIC/DCCAeYCAQAwVDELMAkGA1UEBgwCU1MxEDAOBgNVBAgMB09ic2N1cmUxDzAN
BgNVBAcMBlNlY3VyZTEQMA4GA1UECgwHYm9iY29ycDEQMA4GA1UEAwwHYm9iLmNv
bTCCASAwCwYJKoZIhvcNAQEBA4IBDwAwggEKAoIBAQDOGLdjEr2IyW27e08hmRN+
Bcu4uOWAWEvxvY3+5pHdBQQEXhww5YQuEvpmgbKtav0j7aqFPDYNSXTv+aQNe9fq
cP3nZmKAWU+qbQMjWxwV9mEJOlWI214v7C8lLbMvBlny141J7KTvv1TGGLCBH/V2
EnQSdJzGDwXmJ2k0iChlQ7zl1TlonamYX9gefzp3N/DDp6kNhuPSX9zRorYIp5CC
WEIRmDdegwxHACrNu0K4xwuPjTRJf0oUkRsfBuDxqvBalQ0bzd/23fiQ51MEVla3
fUWL/+b2SKOlvgfu0XbfZ+Qx21DeyRQpqIWnv0gR4AM8qltxUgRjZloUfK5IQ8rp
AgMBAAGgZzBlBgkqhkiG9w0BCQ4xWDBWMAwGA1UdEwEB/wQCMAAwFgYDVR0lAQH/
BAwwCgYIKwYBBQUHAwEwLgYDVR0RBCcwJYIHYm9iLmNvbYIJKi5ib2IuY29tgglh
c2RmLmZkc2GHBAp7e3swCwYJKoZIhvcNAQELA4IBAQAf1Vlt09ZBhVpxlmi+n93u
Gm8fNrd5afeIzWj0h4dFGJSOg5T8SkfL5txk2C9tQEdayQWB1kllx5rIqXAPe6gz
kbcjJn8l2IB3khIoKYmylmtV8Yo0Fl/Xba1oLCAsixbK+UxiSLgXqMryz9DBy40s
5oYXpy5JOnqL7BRC7b+Lk/chw7CcncPZI4rei6HM8WATymTySdrPoQegvBj0VIar
qHBZrMV9lsjTREJ9hvA/FycA/PNlP9y8N+eTF9SBrnTi8ix+v+Iirc43xeD2EVLG
6uqXecClji6OEOKcdDsH0D0HD1PMFmKB0FWvq71dt7eVIHkTPwTLFG2XAjfn6Fb8
-----END CERTIFICATE REQUEST-----
Decodes to:
Certificate Request:
Data:
Version: 0 (0x0)
Subject: C=SS, ST=Obscure, L=Secure, O=bobcorp, CN=bob.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:ce:18:b7:63:12:bd:88:c9:6d:bb:7b:4f:21:99:
13:7e:05:cb:b8:b8:e5:80:58:4b:f1:bd:8d:fe:e6:
91:dd:05:04:04:5e:1c:30:e5:84:2e:12:fa:66:81:
b2:ad:6a:fd:23:ed:aa:85:3c:36:0d:49:74:ef:f9:
a4:0d:7b:d7:ea:70:fd:e7:66:62:80:59:4f:aa:6d:
03:23:5b:1c:15:f6:61:09:3a:55:88:db:5e:2f:ec:
2f:25:2d:b3:2f:06:59:f2:d7:8d:49:ec:a4:ef:bf:
54:c6:18:b0:81:1f:f5:76:12:74:12:74:9c:c6:0f:
05:e6:27:69:34:88:28:65:43:bc:e5:d5:39:68:9d:
a9:98:5f:d8:1e:7f:3a:77:37:f0:c3:a7:a9:0d:86:
e3:d2:5f:dc:d1:a2:b6:08:a7:90:82:58:42:11:98:
37:5e:83:0c:47:00:2a:cd:bb:42:b8:c7:0b:8f:8d:
34:49:7f:4a:14:91:1b:1f:06:e0:f1:aa:f0:5a:95:
0d:1b:cd:df:f6:dd:f8:90:e7:53:04:56:56:b7:7d:
45:8b:ff:e6:f6:48:a3:a5:be:07:ee:d1:76:df:67:
e4:31:db:50:de:c9:14:29:a8:85:a7:bf:48:11:e0:
03:3c:aa:5b:71:52:04:63:66:5a:14:7c:ae:48:43:
ca:e9
Exponent: 65537 (0x10001)
Attributes:
Requested Extensions:
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Extended Key Usage: critical
TLS Web Server Authentication
X509v3 Subject Alternative Name:
DNS:bob.com, DNS:*.bob.com, DNS:asdf.fdsa, IP Address:10.123.123.123
Signature Algorithm: sha256WithRSAEncryption
1f:d5:59:6d:d3:d6:41:85:5a:71:96:68:be:9f:dd:ee:1a:6f:
1f:36:b7:79:69:f7:88:cd:68:f4:87:87:45:18:94:8e:83:94:
fc:4a:47:cb:e6:dc:64:d8:2f:6d:40:47:5a:c9:05:81:d6:49:
65:c7:9a:c8:a9:70:0f:7b:a8:33:91:b7:23:26:7f:25:d8:80:
77:92:12:28:29:89:b2:96:6b:55:f1:8a:34:16:5f:d7:6d:ad:
68:2c:20:2c:8b:16:ca:f9:4c:62:48:b8:17:a8:ca:f2:cf:d0:
c1:cb:8d:2c:e6:86:17:a7:2e:49:3a:7a:8b:ec:14:42:ed:bf:
8b:93:f7:21:c3:b0:9c:9d:c3:d9:23:8a:de:8b:a1:cc:f1:60:
13:ca:64:f2:49:da:cf:a1:07:a0:bc:18:f4:54:86:ab:a8:70:
59:ac:c5:7d:96:c8:d3:44:42:7d:86:f0:3f:17:27:00:fc:f3:
65:3f:dc:bc:37:e7:93:17:d4:81:ae:74:e2:f2:2c:7e:bf:e2:
22:ad:ce:37:c5:e0:f6:11:52:c6:ea:ea:97:79:c0:a5:8e:2e:
8e:10:e2:9c:74:3b:07:d0:3d:07:0f:53:cc:16:62:81:d0:55:
af:ab:bd:5d:b7:b7:95:20:79:13:3f:04:cb:14:6d:97:02:37:
e7:e8:56:fc
OK, I think i found a "feature" in phpseclib's X509 signing function:
function sign($issuer, $subject, $signatureAlgorithm = 'sha1WithRSAEncryption')
If you sign the CSR, the code in the sign function claims to copy the X509 v3 extended attributes, however by the time it executes this line:
$csrexts = $subject->getAttribute('pkcs-9-at-extensionRequest', 0);
The result is always blank (an empty array), the attributes in $THIS (new) cert were overwritten somewhere in the previous ~30 lines of code.
I moved the csrexts line to the top of the Sign function, and attributes are now being copied correctly into generated certificates with X509 extended attributes in the CSR (that had to be manually calculated and set by my previous answer)
I will use sourceforge to try and post a bug request to save others the struggle hopefully.
OK, so I've got this to work using an example from the PHPSecLib documentation and playing around with it for a good few hours. Perhaps this is something added post February, but in any case, here is how to generate a CSR with an Subject Alternative Name:
$privKey = new Crypt_RSA();
extract($privKey->createKey());
$privKey->loadKey($privatekey);
$x509 = new File_X509();
$x509->setPrivateKey($privKey);
$x509->setDNProp('id-at-organizationName', 'Company');
$x509->setDNProp('id-at-organizationalUnitName', 'CompanyDepartment');
$x509->setDNProp('id-at-commonName', 'myurl.com');
$x509->setDNProp('id-at-localityName', $this->application);
$x509->loadCSR($x509->saveCSR($x509->signCSR()));
// Set extension request.
$x509->setExtension("id-ce-subjectAltName", [
['dNSName' => 'myalternativename.com', 'iPAddress' => 127.0.0.1]
]);
echo $x509->saveCSR($x509->signCSR());
That should print out the CSR with myalternativename.com as the SAN.
The bug still actual in 2020.
Cause of this bug is need for a valid $x509->currentCert array. The setExtension() function has no effect with empty currentCert array.
The currentCert array structure is equal to result of signCSR() function. Therefore you can just initialize it as following: $x509->currentCert = $x509->signCSR();
Now you can call setExtension() function.
Here is full example how to generate CSR with SAN attribute:
use \phpseclib\Crypt\RSA;
use \phpseclib\File\X509;
$privateKey = new RSA();
$privateKey->loadKey(file_get_contents(__DIR__ . '/private.key'));
$x509 = new X509();
$x509->setPrivateKey($privateKey);
$x509->setDNProp('commonname', 'example.com');
$x509->setDNProp('emailaddress', 'admin#mail.com');
$x509->currentCert = $x509->signCSR(); // Important!
$x509->setExtension('id-ce-subjectAltName', [
['iPAddress' => '127.0.0.1'],
['dNSName' => 'www.example.com'],
]);
$csr = $x509->signCSR();
file_put_contents(__DIR__ . '/domain.csr', $x509->saveCSR($csr));
echo "OK\n";
I am using the current version of rest client builder plugin. I tested out the uri via curl:
curl --user username:password https://localhost:8085/rest/api/latest/plan.json?os_authType=basic
I get the expected json in return. When I try to translate this to grails using the plugin like this:
RestBuilder rb = new RestBuilder()
def response = rb.get("https://localhost:8085/rest/api/latest/plan.json?os_authType=basic"){
auth 'username', 'password'
}
response.json instanceof JSONObject
I get this error:
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Why does it work in curl and not with the plugin? How do I get this to work?
Thanks!
You need to add the root certificate to the store of the trusted ones.
http://docs.oracle.com/javase/tutorial/security/toolsign/rstep2.html
Import the Certificate as a Trusted Certificate
Before you can grant the signed code permission to read a specified file, you need to import Susan's certificate as a trusted certificate in your keystore.
Suppose that you have received from Susan
the signed JAR file sCount.jar, which contains the Count.class file, and
the file Example.cer, which contains the public key certificate for the public key corresponding to the private key used to sign the JAR file.
Even though you created these files and they haven't actually been transported anywhere, you can simulate being someone other than the creater and sender, Susan. Pretend that you are now Ray. Acting as Ray, you will create a keystore named exampleraystore and will use it to import the certificate into an entry with an alias of susan.
A keystore is created whenever you use a keytool command specifying a keystore that doesn't yet exist. Thus we can create the exampleraystore and import the certificate via a single keytool command. Do the following in your command window.
Go to the directory containing the public key certificate file Example.cer. (You should actually already be there, since this lesson assumes that you stay in a single directory throughout.)
Type the following command on one line:
keytool -import -alias susan
-file Example.cer -keystore exampleraystore
Since the keystore doesn't yet exist, it will be created, and you will be prompted for a keystore password; type whatever password you want.
The keytool command will print out the certificate information and ask you to verify it, for example, by comparing the displayed certificate fingerprints with those obtained from another (trusted) source of information. (Each fingerprint is a relatively short number that uniquely and reliably identifies the certificate.) For example, in the real world you might call up Susan and ask her what the fingerprints should be. She can get the fingerprints of the Example.cer file she created by executing the command
keytool -printcert -file Example.cer
If the fingerprints she sees are the same as the ones reported to you by keytool, the certificate has not been modified in transit. In that case you let keytool proceed with placing a trusted certificate entry in the keystore. The entry contains the public key certificate data from the file Example.cer and is assigned the alias susan.
You can just disable SSL check for RestBuilder.
See an example of code:
static Scheme disableSSLCheck() {
def sslContext = SSLContext.getInstance("SSL")
sslContext.init(null, [new X509TrustManager() {
public void checkClientTrusted(X509Certificate[] certs, String authType) {}
public void checkServerTrusted(X509Certificate[] certs, String authType) {}
#Override
X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0]
}
}] as TrustManager[], new SecureRandom())
def sf = new SSLSocketFactory(sslContext, SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER)
def httpsScheme = new Scheme("https", sf, 443)
httpsScheme
}
And register this Scheme to the RestClient:
Scheme httpsScheme = disableSSLCheck()
restClient.client.connectionManager.schemeRegistry.register(httpsScheme)
Mb too late but have a look here.
https://gist.github.com/thomastaylor312/80fcb016020e4115aa64320b98fb0017
I do have it as separate method in my Integration test
def static disableSSLCheck() {
def nullTrustManager = [
checkClientTrusted: { chain, authType -> },
checkServerTrusted: { chain, authType -> },
getAcceptedIssuers: { null }
]
def nullHostnameVerifier = [
verify: { hostname, session -> true }
]
SSLContext sc = SSLContext.getInstance("SSL")
sc.init(null, [nullTrustManager as X509TrustManager] as TrustManager[], null)
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory())
HttpsURLConnection.setDefaultHostnameVerifier(nullHostnameVerifier as HostnameVerifier)
}
And then just
void "test authentication"(){
given:
String url = "j_spring_security_check"
MultiValueMap<String, String> form = new LinkedMultiValueMap<String, String>()
form.add("grant_type", "password")
form.add("j_username", "vadim#ondeviceresearch.com")
form.add("j_password", "notSecure")
form.add("_spring_security_remember_me", "true")
//TODO SET username and pass
//todo get token back
disableSSLCheck()
when:
RestResponse response = rest.post(host + url){
accept("application/json")
contentType("application/x-www-form-urlencoded")
body(form)
}
response
then:
response.status == 200
}