Authorization for pkrange query on CosmosDB REST API - powershell

For a testing scenario, I'd like to run a Stored Procedure against each partition in a CosmosDB database, and I'm attempting to get the list of partition ranges from CosmosDB using PowerShell.
I'm getting a 401 - Unauthorized response for this query, but other queries on the same collection work fine - for example executing a procedure works.
The code I'm using to query the ranges is:
Add-Type -AssemblyName System.Web
# Configure as required
$accountName = ""
$connectionKey = ""
$collectionName = ""
$databaseName = ""
function GetKey([System.String]$Verb = '',[System.String]$ResourceId = '',
[System.String]$ResourceType = '',[System.String]$Date = '',
[System.String]$masterKey = '')
{
$keyBytes = [System.Convert]::FromBase64String($masterKey)
$text = #($Verb.ToLowerInvariant() + "`n" +
$ResourceType.ToLowerInvariant() + "`n" + $ResourceId + "`n" +
$Date.ToLowerInvariant() + "`n" + "" + "`n")
$body =[Text.Encoding]::UTF8.GetBytes($text)
$hmacsha = new-object -TypeName System.Security.Cryptography.HMACSHA256 -ArgumentList (,$keyBytes)
$hash = $hmacsha.ComputeHash($body)
$signature = [System.Convert]::ToBase64String($hash)
[System.Web.HttpUtility]::UrlEncode($('type=master&ver=1.0&sig=' + $signature))
}
function GetUTDate() {
$date = get-date
$date = $date.ToUniversalTime();
return $date.ToString("r", [System.Globalization.CultureInfo]::InvariantCulture);
}
function BuildHeaders([string]$action = "get",[string]$resType, [string]$resourceId){
$authz = GetKey -Verb $action -ResourceType $resType -ResourceId $resourceId -Date $apiDate -masterKey $connectionKey
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", $authz)
$headers.Add("x-ms-version", '2017-02-22')
$headers.Add("x-ms-date", $apiDate)
$headers.Add("Cache-Control", 'no-cache')
$headers.Add("Accept", 'application/json')
$headers.Add("Content-Type", 'application/json')
$headers
}
function GetPartitionKeys(){
$pkranges = "dbs/" + $databaseName + "/colls/" + $collectionName + "/pkranges"
$headers = BuildHeaders -action Get -resType colls -resourceId $pkranges
$uri = $rootUri + "/" + $pkranges
write-host "Calling" $uri
write-host($headers|Out-String)
$response = Invoke-RestMethod $uri -Method Get -Headers $headers
}
$rootUri = "https://" + $accountName + ".documents.azure.com"
GetPartitionKeys
I think the problem is to do with the Resource Type when I'm building the Auth header but the CosmosDB REST documentation doesn't have much information about querying this resource. At present I'm getting the following output:
Calling https://my-account-name.documents.azure.com/dbs/my-db-name/colls/my-coll-name/pkranges
Invoke-RestMethod : The remote server returned an error: (401) Unauthorized.

When computing the has for the authorization header the ResourceType value should be "pkranges".
The ResourceLink would be "dbs/my-db-name/colls/my-coll-name" and the URL you request is: https://my-account-name.documents.azure.com/dbs/my-db-name/colls/my-coll-name/pkranges

Paul- Stored procedures can be executed only against a partition key, not against individual partition key ranges.

Related

Unable to Get Blob using powershell

I have been trying to GET Blob using the script below but it gives an error highlighted below. I am using the script from this link and the $signatureString is used according to the this link. In a way, what i understand from technet that the authorization header should have the following
Convert the PS data to JSON
$json = $authHeader | ConvertTo-Json
$method = "GET"
$headerDate = (get-date -format r).ToString()
$headers = #{"x-ms-version"="$headerDate"}
$StorageAccountName = ""
$StorageContainerName = "users"
$StorageAccountKey = ""
$Url = "https://$StorageAccountName.blob.core.windows.net/$StorageContainerName/******/*****.rdp"
$body = "Hello world"
$xmsdate = (get-date -format r).ToString()
$headers.Add("x-ms-date",$xmsdate)
$contentLength = $body.Length
$headers.Add("Content-Length","$contentLength")
$headers.Add("x-ms-blob-type","BlockBlob")
$contentType = "application/json"
#$headers.Add("Content-Type","$contentType")
$signatureString = "$method$([char]10)$([char]10)$([char]10)$contentLength$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)"
#Add CanonicalizedHeaders
$signatureString += "x-ms-blob-type:" + $headers["x-ms-blob-type"] + "$([char]10)"
$signatureString += "x-ms-date:" + $headers["x-ms-date"] + "$([char]10)"
$signatureString += "x-ms-version:" + $headers["x-ms-version"] + "$([char]10)"
#Add CanonicalizedResource
$uri = New-Object System.Uri -ArgumentList $url
$signatureString += "/" + $StorageAccountName + $uri.AbsolutePath + $([char]10) + "restype:container"
$dataToMac = [System.Text.Encoding]::UTF8.GetBytes($signatureString)
$accountKeyBytes = [System.Convert]::FromBase64String($StorageAccountKey)
$hmac = new-object System.Security.Cryptography.HMACSHA256((,$accountKeyBytes))
$signature = [System.Convert]::ToBase64String($hmac.ComputeHash($dataToMac))
$headers.Add("Authorization", "SharedKey " + $StorageAccountName + ":" + $signature);
write-host -fore green $signatureString
write-host -fore green $headers
Invoke-RestMethod -Uri $Url -Method $method -headers $headers
Please someone assist me on this.
I am sorry! yes it was not that descriptive. here is the screenshot
InvalidHeaderValue-Incorrect format
I am actually trying to access azure blob storage using powerShell. today, i tried other script to do the same but I get a different error, please see the script and screenshot of that error message below.
$storageAccount = ""
$accesskey = ""
$version = "2017-04-17"
$resource = "tets"
$table_url = "https://$storageAccount.blob.core.windows.net/$resource"
$GMTTime = (Get-Date).ToUniversalTime().toString('R')
$stringToSign = "$GMTTime`n/$storageAccount/$resource"
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accesskey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($stringToSign))
$signature = [Convert]::ToBase64String($signature)
$headers = #{
'x-ms-date' = $GMTTime
Authorization = "SharedKeyLite " + $storageAccount + ":" + $signature
"x-ms-version" = $version
Accept = "application/json;odata=fullmetadata"
}
$item = Invoke-RestMethod -Method GET -Uri $table_url -Headers $headers -ContentType application/json
return $item.value
Here is the error message I get.
AuthenticationFailedServer
Consider using the Azure PowerShell modules to access blobs rather than raw API requests. All of the authentication details should be handled for you already.

cosmos db resource type for building auth header to create UDF using REST

I am trying to create an UDF in cosmos db using REST call from powershell. I am getting unauthorized for the generated auth header. I am using the below methods to build auth header and make REST call for UDF creation.
I am using resource type as udf while building auth header.
function GetKey([System.String]$Verb = '',[System.String]$ResourceId = '',
[System.String]$ResourceType = '',[System.String]$Date = '',[System.String]$masterKey = '') {
$keyBytes = [System.Convert]::FromBase64String($masterKey)
$text = #($Verb.ToLowerInvariant() + "`n" + $ResourceType.ToLowerInvariant() + "`n" + $ResourceId + "`n" + $Date.ToLowerInvariant() + "`n" + "`n")
$body =[Text.Encoding]::UTF8.GetBytes($text)
$hmacsha = new-object -TypeName System.Security.Cryptography.HMACSHA256 -ArgumentList (,$keyBytes)
$hash = $hmacsha.ComputeHash($body)
$signature = [System.Convert]::ToBase64String($hash)
[System.Web.HttpUtility]::UrlEncode($('type=master&ver=1.0&sig=' + $signature))
}
function BuildHeaders([string]$action = "get",[string]$resType, [string]$resourceId){
$authz = GetKey -Verb $action -ResourceType $resType -ResourceId $resourceId -Date $apiDate -masterKey $connectionKey
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", $authz)
$headers.Add("x-ms-version", '2015-12-16')
$headers.Add("x-ms-date", $apiDate)
$headers
}
function CreateUDF([string]$dbname,[string]$collectionName){
$uri = $rootUri + "/" + $dbname + "/colls/" + $collectionName + "/udfs"
$headers = BuildHeaders -action Post -resType udfs -resourceId $dbname
$response = Invoke-RestMethod -Uri $uri -Method POST -Headers $headers -Body $udfCreationInput
$response
Write-Host ("Create Udf Response is " + $Response)
}
Not sure on what is wrong here while building auth header. Please help me on getting this working.
I used the following script to generate the authorization token for creating UDF via Rest API:
$Verb="POST"
$ResourceType="udfs"
$ResourceId="dbs/brucedb/colls/brucecoll"
$Date=[System.DateTime]::UtcNow.ToString("r")
$text = #($Verb.ToLowerInvariant() + "`n" + $ResourceType.ToLowerInvariant() + "`n" + $ResourceId + "`n" + $Date.ToLowerInvariant() + "`n" + "`n")
$masterKey="<your-master-key>"
$keyBytes = [System.Convert]::FromBase64String($masterKey)
$body =[Text.Encoding]::UTF8.GetBytes($text)
$hmacsha = new-object -TypeName System.Security.Cryptography.HMACSHA256 -ArgumentList (,$keyBytes)
$hash = $hmacsha.ComputeHash($body)
$signature = [System.Convert]::ToBase64String($hash)
Add-Type -AssemblyName System.Web
$token=[System.Web.HttpUtility]::UrlEncode($('type=master&ver=1.0&sig=' + $signature))
Write-Host ("the token is:" + $token)
TEST:
Moreover, you could follow Create User Defined Function, Access control in the Azure Cosmos DB SQL API and this github C# Rest sample.

Getting 404 on Powershell Query of Cosmos DB

I'm using this script from the download in this link.
https://gallery.technet.microsoft.com/scriptcenter/How-to-query-Azure-Cosmos-0a9aa517
However for some reason I am getting a 404 response.
I copy-pasted the url of the db directly. Putting in a fake URL gives me a "could not be resolved" error so I know the location exists.
Based on the Azure CosmosDB API documentation here:
https://learn.microsoft.com/en-us/rest/api/documentdb/databases
the $databaseID is user-set and just has to be unique, so I've set it to be the same as the db name and assigned that to the url.
Changing it to be different still gives me the same 404 response message (below).
Edit: Removed original commenting intro for readability
Powershell Script:
# add necessary assembly
#
Add-Type -AssemblyName System.Web
# generate authorization key
Function Generate-MasterKeyAuthorizationSignature
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$verb,
[Parameter(Mandatory=$true)][String]$resourceLink,
[Parameter(Mandatory=$true)][String]$resourceType,
[Parameter(Mandatory=$true)][String]$dateTime,
[Parameter(Mandatory=$true)][String]$key,
[Parameter(Mandatory=$true)][String]$keyType,
[Parameter(Mandatory=$true)][String]$tokenVersion
)
$hmacSha256 = New-Object System.Security.Cryptography.HMACSHA256
$hmacSha256.Key = [System.Convert]::FromBase64String($key)
$payLoad = "$($verb.ToLowerInvariant())`n$($resourceType.ToLowerInvariant())`n$resourceLink`n$($dateTime.ToLowerInvariant())`n`n"
$hashPayLoad = $hmacSha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($payLoad))
$signature = [System.Convert]::ToBase64String($hashPayLoad);
[System.Web.HttpUtility]::UrlEncode("type=$keyType&ver=$tokenVersion&sig=$signature")
}
# query
Function Query-CosmosDb
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$EndPoint,
[Parameter(Mandatory=$true)][String]$DataBaseId,
[Parameter(Mandatory=$true)][String]$CollectionId,
[Parameter(Mandatory=$true)][String]$MasterKey,
[Parameter(Mandatory=$true)][String]$Query
)
$Verb = "POST"
$ResourceType = "docs";
$ResourceLink = "dbs/$DatabaseId/colls/$CollectionId"
$dateTime = [DateTime]::UtcNow.ToString("r")
$authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
$queryJson = #{query=$Query} | ConvertTo-Json
$header = #{authorization=$authHeader;"x-ms-documentdb-isquery"="True";"x-ms-version"="2017-02-22";"x-ms-date"=$dateTime}
$contentType= "application/json "# The original said "application/query+json", I tried both
$queryUri = "$EndPoint$ResourceLink/docs"
$result = Invoke-RestMethod -Method $Verb -ContentType $contentType -Uri $queryUri -Headers $header -Body $queryJson
$result | ConvertTo-Json -Depth 10
}
# fill the target cosmos database endpoint uri, database id, collection id and masterkey
$DatabaseName = "" # name goes here
$MasterKey = "" #key goes here
$CollectionId = "transientUsers"
$DatabaseId = $DatabaseName
$CosmosDBEndPoint = "https://$DatabaseId.documents.azure.com:443/"
# query string
$Query = "SELECT * FROM transientUsers"
# execute
Query-CosmosDb -EndPoint $CosmosDBEndPoint -DataBaseId $DataBaseId -CollectionId $CollectionId -MasterKey $MasterKey -Query $Query
Error I'm getting:
Invoke-RestMethod : The remote server returned an error: (404) Not Found.
At D:\querycosmos\PowerShell\QueryCosmosDB.ps1:69 char:12
+ ... $result = Invoke-RestMethod -Method $Verb -ContentType $contentType ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I noticed that you used $DatabaseId in two places:
$ResourceLink = "dbs/$DatabaseId/colls/$CollectionId"
and
$CosmosDBEndPoint = "https://$DatabaseId.documents.azure.com:443/"
If $DatabaseId refers to your account name, then you would need to change your $ResourceLink variable and use the name of the database inside your account containing the collection. If however $DatabaseId refers to the database name, then you would need to change $CosmosDBEndPoint and use account name there.

Azure DocumentDB Rest API PowerShell delete collection 401 Unathorized

Need to delete collection in my automation process.
Trying to execute script below. Get operations working fine, but Delete operation failed with "(401) Unathorized" error. It is strange cause Delete collection do not need additional headers. Can someone give a hint, what is wrong?
$accountName = 'someaccountname'
$connectionKey = 'masterkey'
$collectionName = 'mycollection'
$databaseName = 'mydatabase'
function GetKey([System.String]$Verb = '',[System.String]$ResourceId = '',
[System.String]$ResourceType = '',[System.String]$Date = '',[System.String]$masterKey = '') {
$keyBytes = [System.Convert]::FromBase64String($masterKey)
$text = #($Verb.ToLowerInvariant() + "`n" + $ResourceType.ToLowerInvariant() + "`n" + $ResourceId + "`n" + $Date.ToLowerInvariant() + "`n" + "`n")
$body =[Text.Encoding]::UTF8.GetBytes($text)
$hmacsha = new-object -TypeName System.Security.Cryptography.HMACSHA256 -ArgumentList (,$keyBytes)
$hash = $hmacsha.ComputeHash($body)
$signature = [System.Convert]::ToBase64String($hash)
[System.Web.HttpUtility]::UrlEncode($('type=master&ver=1.0&sig=' + $signature))
}
function BuildHeaders([string]$action = "get",[string]$resType, [string]$resourceId){
$authz = GetKey -Verb $action -ResourceType $resType -ResourceId $resourceId -Date $apiDate -masterKey $connectionKey
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", $authz)
$headers.Add("x-ms-version", '2015-12-16')
$headers.Add("x-ms-date", $apiDate)
$headers
}
function GetUTDate() {
$date = get-date
$date = $date.ToUniversalTime();
return $date.ToString("r", [System.Globalization.CultureInfo]::InvariantCulture);
}
function GetDatabases() {
$uri = $rootUri + "/dbs"
$hdr = BuildHeaders -resType dbs
$response = Invoke-RestMethod -Uri $uri -Method Get -Headers $hdr
$response.Databases
Write-Host ("Found " + $Response.Databases.Count + " Database(s)")
}
function GetCollections([string]$dbname){
$uri = $rootUri + "/" + $dbname + "/colls"
$hdr = BuildHeaders -resType colls -resourceId $dbname
$response = Invoke-RestMethod -Uri $uri -Method Get -Headers $hdr
$response.DocumentCollections
Write-Host ("Found " + $Response.DocumentCollections.Count + " DocumentCollection(s)")
}
function DeleteCollection([string]$dbname){
$uri = $rootUri + "/" + $dbname + "/colls" + "/" + $collectionName
$hdrs = BuildHeaders -action DELETE -resType colls -resourceId $collectionName
$response = Invoke-RestMethod -Uri $uri -Method Delete -Headers $hdrs
Write-Host "DELETE $uri"
}
$rootUri = "https://" + $accountName + ".documents.azure.com"
write-host ("Root URI is " + $rootUri)
#validate arguments
$apiDate = GetUTDate
$db = GetDatabases | where { $_.id -eq $databaseName }
if ($db -eq $null) {
write-error "Could not find database in account"
return
}
$dbname = "dbs/" + $databaseName
$collection = GetCollections -dbname $dbname | where { $_.id -eq $collectionName }
if($collection -eq $null){
write-error "Could not find collection in database"
return
}
Write-Host
$Delete = DeleteCollection -dbname $dbname | where { $_.id -eq $collectionName }
Normally, either the Authorization or x-ms-date header is not set (or the Authorization header with an invalid authorization token), 401 unauthorized error will be returned. I use fiddler to capture the request and check the response, and I find both Authorization and x-ms-date header are set, so it seems that the Authorization header is set to an invalid authorization token. Based on your code, I do some changes, and the function could work fine on my side.
function DeleteCollection([string]$dbname){
$uri = $rootUri + "/" + $dbname + "/colls" + "/" + $collectionName
$collectionName = $dbname + "/colls" + "/" + $collectionName
$hdrs = BuildHeaders -action DELETE -resType colls -resourceId $collectionName
#Write-Host "resourceId $collectionName"
$response = Invoke-RestMethod -Uri $uri -Method Delete -Headers $hdrs
Write-Host "resource $collectionName"
Write-Host "DELETE $uri"
}
The $collectionName should be dbs/{yourdbname}/colls/{yourcollectionname}
This can happen if there are pending operations. Either retry until it succeeds or add a delay before deleting. For us, a half second is enough to assure we never see this again.

Azure API SharedKeyLite in Powershell

I'm trying to implement a SharedKeyLite Authorization header function in powershell. This is to connect to Azure Tables REST API. I'm missing something cause I keep getting an error:
Server failed to authenticate
the request. Make sure the value of Authorization header is formed correctly including the signature.
function GenerateHeader($accountName, $accountKey, $action)
{
$xmsdate = get-date
$xmsdate = $xmsdate.ToUniversalTime()
$xmsdate = $xmsdate.toString('r')
$newLine = "`n";
$message = $xmsdate + $newline + "/" + $accountname + "/" + $action;
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accesskey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($message))
$signature = [Convert]::ToBase64String($signature)
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("x-ms-version", "2014-02-14")
$headers.Add("x-ms-date", $xmsdate)
$headers.Add("Authorization", "SharedKeyLite " + $accountName + ":" + $signature)
return $headers
}
UPDATE:
Here's code that calls this function. The $action variable is set to the URI string.
$uriString = "https://$StorageAccountName.table.core.windows.net/Tables"
$headers = GenerateHeader $StorageAccountName $StorageAccountKey "Tables"
Invoke-RestMethod -Uri $uriString -Method $method -Headers $headers -Body $body
And here's the error it throws.
Invoke-RestMethod : AuthenticationFailedServer failed to authenticate the request. Make sure
the value of Authorization header is formed correctly including the
signature. RequestId:4215377d-0002-0044-1a92-94cd56000000
Time:2015-05-22T13:21:53.5205261Z At
C:\Users\Samuel\Source\BaseDataInstall\BaseDataInstall\AzureHelpers.ps1:45
char:2
+ Invoke-RestMethod -Uri $uriString -Method $method -Headers $headers -Body $body
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebExc
eption
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
EDIT: Here's the Example output from the $headers variable outside the function...
Key : x-ms-version
Value : 2014-02-14
Key : x-ms-date
Value : Tue, 26 May 2015 19:30:20 GMT
Key : Authorization
Value : SharedKeyLite <MyStorageName>:lf+ndqhi4OeJhIfLljugT0dfcLbqXDBHwrQJn9Q66HQ=
So this ended up being a simple coding error :(
I feel kinda silly posting this now, but I'm going to post an answer cause I couldn't find a working Authorization builder for Azure in Powershell anywhere. This one does work for Azure tables...
function GenerateHeader($accountName, $accountKey, $action)
{
$xmsdate = get-date
$xmsdate = $xmsdate.ToUniversalTime()
$xmsdate = $xmsdate.toString('R')
$newLine = "`n";
$action = $action.ToLower()
$message = $xmsdate + $newline + "/" + $accountname + "/" + $action;
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accountKey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($message))
$signature = [Convert]::ToBase64String($signature)
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Content-Type", "application/json")
$headers.Add("x-ms-date", $xmsdate)
$headers.Add("Authorization", "SharedKeyLite " + $accountName + ":" + $signature)
return $headers
}
There seem to be a change in the Azure API. I had to disable the "toLower" statement for this to Work.
function GenerateHeader($accountName, $accountKey, $action)
{
$xmsdate = get-date
$xmsdate = $xmsdate.ToUniversalTime()
$xmsdate = $xmsdate.toString('R')
$newLine = "`n";
# $action = $action.ToLower()
$message = $xmsdate + $newline + "/" + $accountname + "/" + $action;
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accountKey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes ($message))
$signature = [Convert]::ToBase64String($signature)
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
# $headers.Add("Content-Type", "application/json")
$headers.Add("x-ms-date", $xmsdate)
$headers.Add("Authorization", "SharedKeyLite " + $accountName + ":" + $signature)
return $headers
}
I hope this helps someone :-)