I am trying to generated a signed URL to an object stored on Google Cloud Storage (GCS).
Attempt 1: try the API using the API Explorer
For this, I am trying to sign the blob/object as defined in the following:
GET
<expiration time>
/<bucket name>/<object/blob name>
I first tried Google's serviceAccounts.signBlob API as discussed in the following page:
https://cloud.google.com/iam/reference/rest/v1/projects.serviceAccounts/signBlob
A base64-encoded string.
Note, as mentioned in the API documentation on the above-linked page, I pass a base64 representation of the blob I want to sign, to the API.
The API's response has the following structure where it contains the signedBlob key:
{
"keyId": "...",
"signedBlob": "..."
}
then I generated a signed URL using the obtained signed blob as the following:
encoded_signedBlob = base64.b64encode(signedBlob)
signed_url = "https://storage.googleapis.com/{}/{}?" \
"GoogleAccessId={}&" \
"Expires={}&" \
"Signature={}".format(
bucket_name, blob_name,
service_account_email,
expiration,
encoded_signedBlob)
and when I paste that signed URL in the browser to download the blob, I get the following error:
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
</Message>
<StringToSign>GET <bucket name> <blob/object name></StringToSign>
</Error>
Attempt 2: try python libraries
Then I tried to implement it in python as the following, but still getting the same error.
# -------------
# Part 1: obtain access token using the authorization flow discussed at:
# https://cloud.google.com/iam/docs/creating-short-lived-service-account-credentials
# -------------
client_service_account = "..."
access_token = build(
serviceName='iamcredentials',
version='v1',
http=http
).projects().serviceAccounts().generateAccessToken(
name="projects/{}/serviceAccounts/{}".format(
'-',
service_account_email),
body=body
).execute()["accessToken"]
credentials = AccessTokenCredentials(access_token, "MyAgent/1.0", None)
# -------------
# Part 2: sign the blob
# -------------
service = discovery.build('iam', 'v1', credentials=credentials)
name = 'projects/.../serviceAccounts/...'
encoded = base64.b64encode(blob)
sign_blob_request_body = {"bytesToSign": encoded}
request = service.projects().serviceAccounts().signBlob(name=name, body=sign_blob_request_body)
response = request.execute()
keyId = response["keyId"]
signedBlob = response["signature"]
# -------------
# Part 3: generate signed URL
# -------------
encoded_signedBlob = base64.b64encode(signedBlob)
signed_url = "https://storage.googleapis.com/{}/{}?" \
"GoogleAccessId={}&" \
"Expires={}&" \
"Signature={}".format(
bucket_name, blob_name,
service_account_email,
expiration,
encoded_signedBlob)
You could take a look to the implementation of this using the client libraries, you could give it a try, remember to include google-cloud-storage in your requirements.txt and a service account as specified in the code sample for python
Related
I'm building an oauth2 client with Flask and Authlib. My code to register the oauth is:
google = oauth.register(
name='google',
client_id='',
client_secret="",
access_token_url="https://accounts.google.com/o/oauth2/token",
access_token_params=None,
authorize_url="https://accounts.google.com/o/oauth2/auth",
authorize_params=None,
api_base_url="https://www.googleapis.com/oauth2/v1/",
client_kwargs={'scope': 'openid email'},
server_metadata_url="https://accounts.google.com/.well-known/openid-configuration",
)
And my /authorize endpoint looks like this:
#app.route('/authorize')
def authorize():
google = oauth.create_client('google')
token = google.authorize_access_token()
resp = google.get('userinfo')
resp.raise_for_status()
userinfo = resp.json()
return str(userinfo)
But I am getting the error
authlib.jose.errors.InvalidClaimError: invalid_claim: Invalid claim "iss"
I had this issue and removing the openid value from scope fixed it. I guess my google config didn't accomodate it,
I used this tutorial and created "put" endpoint successfully.
https://sanderknape.com/2017/10/creating-a-serverless-api-using-aws-api-gateway-and-dynamodb/
When I follow this advice, I get authroization required error..
Using your favorite REST client, try to PUT an item into DynamoDB
using your API Gateway URL.
python is my favorite client:
import requests
api_url = "https://0pg2858koj.execute-api.us-east-1.amazonaws.com/tds"
PARAMS = {"name": "test", "favorite_movie":"asdsf"}
r = requests.put(url=api_url, params=PARAMS)
the response is 403
My test from console is successful, but not able to put a record from python.
The first step you can take to resolve the problem is to investigate the information returned by AWS in the 403 response. It will provide a header, x-amzn-ErrorType and error message with information about the concrete error. You can test it with curl in verbose mode (-v) or with your Python code. Please, review the relevant documentation to obtain a detailed enumeration of all the possible error reasons.
In any case, looking at your code, it is very likely that you did not provide the necessary authentication or authorization information to AWS.
The kind of information that you must provide depends on which mechanism you configured to access your REST API in API Gateway.
If, for instance, you configured IAM based authentication, you need to set up your Python code to generate an Authorization header with an AWS Signature derived from your user access key ID and associated secret key. The AWS documentation provides an example of use with Postman.
The AWS documentation also provides several examples of how to use python and requests to perform this kind of authorization.
Consider, for instance, this example for posting information to DynamoDB:
# Copyright 2010-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# This file is licensed under the Apache License, Version 2.0 (the "License").
# You may not use this file except in compliance with the License. A copy of the
# License is located at
#
# http://aws.amazon.com/apache2.0/
#
# This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
# OF ANY KIND, either express or implied. See the License for the specific
# language governing permissions and limitations under the License.
# AWS Version 4 signing example
# DynamoDB API (CreateTable)
# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a POST request and passes request parameters
# in the body (payload) of the request. Auth information is passed in
# an Authorization header.
import sys, os, base64, datetime, hashlib, hmac
import requests # pip install requests
# ************* REQUEST VALUES *************
method = 'POST'
service = 'dynamodb'
host = 'dynamodb.us-west-2.amazonaws.com'
region = 'us-west-2'
endpoint = 'https://dynamodb.us-west-2.amazonaws.com/'
# POST requests use a content type header. For DynamoDB,
# the content is JSON.
content_type = 'application/x-amz-json-1.0'
# DynamoDB requires an x-amz-target header that has this format:
# DynamoDB_<API version>.<operationName>
amz_target = 'DynamoDB_20120810.CreateTable'
# Request parameters for CreateTable--passed in a JSON block.
request_parameters = '{'
request_parameters += '"KeySchema": [{"KeyType": "HASH","AttributeName": "Id"}],'
request_parameters += '"TableName": "TestTable","AttributeDefinitions": [{"AttributeName": "Id","AttributeType": "S"}],'
request_parameters += '"ProvisionedThroughput": {"WriteCapacityUnits": 5,"ReadCapacityUnits": 5}'
request_parameters += '}'
# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()
def getSignatureKey(key, date_stamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), date_stamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = os.environ.get('AWS_ACCESS_KEY_ID')
secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY')
if access_key is None or secret_key is None:
print('No access key is available.')
sys.exit()
# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
date_stamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope
# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
# Step 1 is to define the verb (GET, POST, etc.)--already done.
# Step 2: Create canonical URI--the part of the URI from domain to query
# string (use '/' if no path)
canonical_uri = '/'
## Step 3: Create the canonical query string. In this example, request
# parameters are passed in the body of the request and the query string
# is blank.
canonical_querystring = ''
# Step 4: Create the canonical headers. Header names must be trimmed
# and lowercase, and sorted in code point order from low to high.
# Note that there is a trailing \n.
canonical_headers = 'content-type:' + content_type + '\n' + 'host:' + host + '\n' + 'x-amz-date:' + amz_date + '\n' + 'x-amz-target:' + amz_target + '\n'
# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers include those that you want to be included in the
# hash of the request. "Host" and "x-amz-date" are always required.
# For DynamoDB, content-type and x-amz-target are also required.
signed_headers = 'content-type;host;x-amz-date;x-amz-target'
# Step 6: Create payload hash. In this example, the payload (body of
# the request) contains the request parameters.
payload_hash = hashlib.sha256(request_parameters.encode('utf-8')).hexdigest()
# Step 7: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = date_stamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' + amz_date + '\n' + credential_scope + '\n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, date_stamp, region, service)
# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()
# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# Put the signature information in a header named Authorization.
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
# For DynamoDB, the request can include any headers, but MUST include "host", "x-amz-date",
# "x-amz-target", "content-type", and "Authorization". Except for the authorization
# header, the headers must be included in the canonical_headers and signed_headers values, as
# noted earlier. Order here is not significant.
# # Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'Content-Type':content_type,
'X-Amz-Date':amz_date,
'X-Amz-Target':amz_target,
'Authorization':authorization_header}
# ************* SEND THE REQUEST *************
print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + endpoint)
r = requests.post(endpoint, data=request_parameters, headers=headers)
print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)
I think it could be easily adapted to your needs.
In the console, everything works fine because when you invoke your REST endpoints in API Gateway, you are connected to a user who is already authenticated and authorized to access these REST endpoints.
I am facing this issue connecting with Facebook's API using httr package, while testing on 'me' node I came along the following problem.
I was under the impression that me node does not require special permissions.
Testing on the browser with 'https://graph.facebook.com/me' gave the same results, it would be great if some one could provide an explanation.
# Define keys
app_id = 'my_app_id'
app_secret = 'my_app_secret'
# Define the app
fb_app <- oauth_app(appname = "facebook",
key = app_id,
secret = app_secret)
# Get OAuth user access token
fb_token <- oauth2.0_token(oauth_endpoints("facebook"),
fb_app,
scope = 'public_profile',
type = "application/x-www-form-urlencoded",
cache = TRUE)
response <- GET("https://graph.facebook.com",
path = "/me",
config = config(token = fb_token))
# Show content returned
content(response)
$error
$error$message
[1] "An active access token must be used to query information about the current user."
$error$type
[1] "OAuthException"
$error$code
[1] 2500
$error$fbtrace_id
[1] "ARRnb93rZHmWLlXK_MMJlfi"
Noting that I have signed in using the app.
I am stuck, I am trying to sign a Json Web Token for Docusign. https://developers.docusign.com/esign-rest-api/guides/authentication/oauth2-jsonwebtoken Docusign just provides a RSA private and public key hash. That's it. The JWT must signed using RS256.
I found a JWT module https://www.powershellgallery.com/packages/JWT/1.1.0 but that requires that I have the certificate installed. But all I have is the key hash.
Levering some other code I found I was able to create a JWT token , although with the wrong algorithm. https://www.reddit.com/r/PowerShell/comments/8bc3rb/generate_jwt_json_web_token_in_powershell/
I've been trying modify it to use the RSACryto provider by creating a new object but ive been unsuccessful.
I tried to create a new object and see if I can some how import the key so that I can sign the token. But I cant seem to be able to do that.
$rsa = New-Object -TypeName System.Security.Cryptography.RSACryptoServiceProvider
$keyhash = "-----BEGIN RSA PRIVATE KEY-----
XXXXXX
-----END RSA PRIVATE KEY-----"
$blob = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($keyhash))
I tried to use the ImportCspBlob method but it requires that the string is converted to bytes but I cant seem to do that either.
Im not sure if I am even approaching this int he correct fashion. Ive been getting errors of either
Exception calling "ImportCspBlob" with "1" argument(s): "Bad Version of provider.
or
Cannot convert value to type "System.Byte". Error: "Input string was not in a correct format."
EDIT:
I have a work around using Node.js, although id still like to see if it is possible to do what I am trying to do natively in Powershell . This work aroun might be useful for some as there does not seem to be many references for using Powershell and Docusign API.
I found a node.JS script here that creates a JWT Token using the RS256 algorithm. https://github.com/BlitzkriegSoftware/NodejwtRSA ,
I stripped out all the extra stuff so the output to the console is only the token, and added the relevant scope to the "payload data" and under the sign options updated the sub, aud, and iss. The my RSA Private key was stored locally on the system in a file.
nodejs script - My modified version below
'use strict';
const path = require('path');
const fs = require('fs');
// https://github.com/auth0/node-jsonwebtoken
var jwt = require('jsonwebtoken');
// Private Key (must read as utf8)
var privateKey = fs.readFileSync('./rsatest.pk','utf8');
// Sample claims payload with user defined fields (this can be anything, but briefer is better):
var payload = { };
// Populate with fields and data
payload.scope = "signature impersonation";
// Values for the rfc7519 fields
var iss = "XXXXX-XXXX-XXX-XXX-XXX";
var sub = "XXXXX-XXXX-XXX-XXX-XXX";
var aud = "account-d.docusign.com";
// Expiration timespan: https://github.com/auth0/node-jsonwebtoken#token-expiration-exp-claim
var exp = "1h";
// JWT Token Options, see: https://tools.ietf.org/html/rfc7519#section-4.1 for the meaning of these
// Notice the `algorithm: "RS256"` which goes with public/private keys
var signOptions = {
issuer : iss,
subject: sub,
audience: aud,
expiresIn: exp,
algorithm: "RS256"
};
var token = jwt.sign(payload, privateKey, signOptions);
console.log(token)
process.exitCode = 0;
I called it from Powershell and feed the access token back into my script so i can then get my access token and start making my API calls.
#get the JWT token
$token = & node C:\temp\nodejwt.js
# Generate Header for API calls.
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Content-Type", "application/x-www-form-urlencoded")
$body ="grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion=$token"
$authuri = "https://account-d.docusign.com/oauth/token"
#send the JWT and get the access token
$accesstoken = Invoke-RestMethod -Method post -Uri $authuri -Headers $headers -Body $body -Verbose
$getheaders = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$getheaders.Add("Content-Type", "application/json")
$getheaders.Add('Authorization','Bearer ' + $accesstoken.access_token)
$geturi = "https://account-d.docusign.com/oauth/userinfo"
#use the access token to make api calls
Invoke-RestMethod -Method get -Uri $geturi -Headers $getheaders
I also had the same problem and was stuck.I used Python to get JWT. Be sure to install PyJWT and cryptography library.
$ pip install PyJWT
$ pip install pyjwt[crypto]
import time
import jwt
from datetime import datetime, timedelta
private_key= b'-----BEGIN PRIVATE KEY-----
`-----END PRIVATE KEY-----\n'
encoded_jwt = jwt.encode({
"iss":"<service account e-mail>",
"scope":"",
"aud":"",
"exp":int(time.time())+3600,
"iat":int(time.time())
}, private_key, algorithm='RS256')
encoded_jwt
I'm trying to use gspread with python-social-auth, I followed the sample describe on documentation and I've created this class to use as credential store:
class Credentials(object):
def __init__ (self, access_token=None, scope=None):
self.access_token = access_token
self.scope=scope
def refresh (self, http):
# get new access_token
# this only gets called if access_token is None
pass
And at my code I've used:
import gspread
credentials = Credentials(access_token=user.social_auth.get_access_token(),
scope=['https://spreadsheets.google.com/feeds'])
sh = gspread.authorize(credentials)
And asking any response from API using:
sh.get_spreadsheets_feed()
This error appears on the console:
*** RequestError: (401, '401: <HTML>\n<HEAD>\n<TITLE>Token invalid - AuthSub token has wrong scope</TITLE>\n</HEAD>\n<BODY BGCOLOR="#FFFFFF" TEXT="#000000">\n<H1>Token invalid - AuthSub token has wrong scope</H1>\n<H2>Error 401</H2>\n</BODY>\n</HTML>\n')
I have defined the scope at my settings and this is working well, for example trying to get contacts from Google Contacts
SOCIAL_AUTH_GOOGLE_OAUTH2_SCOPE = [
'https://www.googleapis.com/auth/contacts.readonly',
'https://www.googleapis.com/auth/spreadsheets.readonly'
]
Any ideas?
You have to add these scopes also to Python Social Auth settings:
SOCIAL_AUTH_GOOGLE_OAUTH2_SCOPE = [
...
'https://www.googleapis.com/auth/spreadsheets.readonly',
'https://spreadsheets.google.com/feeds',
'https://www.googleapis.com/auth/drive.file'
]