Need to delete collection in my automation process.
Trying to execute script below. Get operations working fine, but Delete operation failed with "(401) Unathorized" error. It is strange cause Delete collection do not need additional headers. Can someone give a hint, what is wrong?
$accountName = 'someaccountname'
$connectionKey = 'masterkey'
$collectionName = 'mycollection'
$databaseName = 'mydatabase'
function GetKey([System.String]$Verb = '',[System.String]$ResourceId = '',
[System.String]$ResourceType = '',[System.String]$Date = '',[System.String]$masterKey = '') {
$keyBytes = [System.Convert]::FromBase64String($masterKey)
$text = #($Verb.ToLowerInvariant() + "`n" + $ResourceType.ToLowerInvariant() + "`n" + $ResourceId + "`n" + $Date.ToLowerInvariant() + "`n" + "`n")
$body =[Text.Encoding]::UTF8.GetBytes($text)
$hmacsha = new-object -TypeName System.Security.Cryptography.HMACSHA256 -ArgumentList (,$keyBytes)
$hash = $hmacsha.ComputeHash($body)
$signature = [System.Convert]::ToBase64String($hash)
[System.Web.HttpUtility]::UrlEncode($('type=master&ver=1.0&sig=' + $signature))
}
function BuildHeaders([string]$action = "get",[string]$resType, [string]$resourceId){
$authz = GetKey -Verb $action -ResourceType $resType -ResourceId $resourceId -Date $apiDate -masterKey $connectionKey
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", $authz)
$headers.Add("x-ms-version", '2015-12-16')
$headers.Add("x-ms-date", $apiDate)
$headers
}
function GetUTDate() {
$date = get-date
$date = $date.ToUniversalTime();
return $date.ToString("r", [System.Globalization.CultureInfo]::InvariantCulture);
}
function GetDatabases() {
$uri = $rootUri + "/dbs"
$hdr = BuildHeaders -resType dbs
$response = Invoke-RestMethod -Uri $uri -Method Get -Headers $hdr
$response.Databases
Write-Host ("Found " + $Response.Databases.Count + " Database(s)")
}
function GetCollections([string]$dbname){
$uri = $rootUri + "/" + $dbname + "/colls"
$hdr = BuildHeaders -resType colls -resourceId $dbname
$response = Invoke-RestMethod -Uri $uri -Method Get -Headers $hdr
$response.DocumentCollections
Write-Host ("Found " + $Response.DocumentCollections.Count + " DocumentCollection(s)")
}
function DeleteCollection([string]$dbname){
$uri = $rootUri + "/" + $dbname + "/colls" + "/" + $collectionName
$hdrs = BuildHeaders -action DELETE -resType colls -resourceId $collectionName
$response = Invoke-RestMethod -Uri $uri -Method Delete -Headers $hdrs
Write-Host "DELETE $uri"
}
$rootUri = "https://" + $accountName + ".documents.azure.com"
write-host ("Root URI is " + $rootUri)
#validate arguments
$apiDate = GetUTDate
$db = GetDatabases | where { $_.id -eq $databaseName }
if ($db -eq $null) {
write-error "Could not find database in account"
return
}
$dbname = "dbs/" + $databaseName
$collection = GetCollections -dbname $dbname | where { $_.id -eq $collectionName }
if($collection -eq $null){
write-error "Could not find collection in database"
return
}
Write-Host
$Delete = DeleteCollection -dbname $dbname | where { $_.id -eq $collectionName }
Normally, either the Authorization or x-ms-date header is not set (or the Authorization header with an invalid authorization token), 401 unauthorized error will be returned. I use fiddler to capture the request and check the response, and I find both Authorization and x-ms-date header are set, so it seems that the Authorization header is set to an invalid authorization token. Based on your code, I do some changes, and the function could work fine on my side.
function DeleteCollection([string]$dbname){
$uri = $rootUri + "/" + $dbname + "/colls" + "/" + $collectionName
$collectionName = $dbname + "/colls" + "/" + $collectionName
$hdrs = BuildHeaders -action DELETE -resType colls -resourceId $collectionName
#Write-Host "resourceId $collectionName"
$response = Invoke-RestMethod -Uri $uri -Method Delete -Headers $hdrs
Write-Host "resource $collectionName"
Write-Host "DELETE $uri"
}
The $collectionName should be dbs/{yourdbname}/colls/{yourcollectionname}
This can happen if there are pending operations. Either retry until it succeeds or add a delay before deleting. For us, a half second is enough to assure we never see this again.
Related
I have Runbook under Azure automation account that should collect results from Resource graph query and pass it to Log analytics as custom log. I have managed to create a script that works fine.
$customerId = "xxxxxxxxx"
$SharedKey = "xxxxxxxxxxxxxxxx"
$LogType = "MyRecord"
$TimeStampField = ""
#function block
Function Connect-ToAzure {
$connectionName = "AzureRunAsConnection"
$automationAccountName = Get-AutomationVariable -Name 'automationAccountName'
Write-Output "Azure Automation Account Name - $automationAccountName"
$connectionName = "AzureRunAsConnection"
Write-Output "Connection Name - $connectionName"
$servicePrincipalConnection = Get-AutomationConnection -Name $connectionName
Write-Output "Logging in to Azure..."
Connect-AzAccount -ServicePrincipal -Tenant $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint `
-Subscription $servicePrincipalConnection.SubscriptionId
}
Function Build-Signature ($customerId, $sharedKey, $date, $contentLength, $method, $contentType, $resource)
{
$xHeaders = "x-ms-date:" + $date
$stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource
$bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash)
$keyBytes = [Convert]::FromBase64String($sharedKey)
$sha256 = New-Object System.Security.Cryptography.HMACSHA256
$sha256.Key = $keyBytes
$calculatedHash = $sha256.ComputeHash($bytesToHash)
$encodedHash = [Convert]::ToBase64String($calculatedHash)
$authorization = 'SharedKey {0}:{1}' -f $customerId,$encodedHash
return $authorization
}
# Create the function to create and post the request
Function Post-LogAnalyticsData($customerId, $sharedKey, $body, $LogType)
{
$method = "POST"
$contentType = "application/json"
$resource = "/api/logs"
$rfc1123date = [DateTime]::UtcNow.ToString("r")
$contentLength = $body.Length
$signature = Build-Signature `
-customerId $customerId `
-sharedKey $sharedKey `
-date $rfc1123date `
-contentLength $contentLength `
-method $method `
-contentType $contentType `
-resource $resource
$uri = "https://" + $customerId + ".ods.opinsights.azure.com" + $resource + "?api-version=2016-04-01"
$headers = #{
"Authorization" = $signature;
"Log-Type" = $LogType;
"x-ms-date" = $rfc1123date;
"time-generated-field" = $TimeStampField;
}
Write-Output "Sending a request"
$response = Invoke-WebRequest -Uri $uri -Method $method -ContentType $contentType -Headers $headers -Body $body -UseBasicParsing
return $response.StatusCode
Write-Output "Request has been sent"
}
try {
Write-Output "Starting runbook"
$customerId = "xxxxxxxxx"
$SharedKey = "xxxxxxxxxxxxxxxx"
$LogType = "MyRecord"
$Query = #'
resources
| where type == "microsoft.compute/disks"
| extend diskState = properties.diskState
| extend diskSizeGB = properties.diskSizeGB
| where properties['encryptionSettingsCollection'] != "enabled"
| where diskState == "Attached" or diskState == "Reserved"
| extend diskCreationTime = properties.timeCreated
| extend hostVM = split(managedBy,"/")[-1]
| project diskCreationTime, name, resourceGroup, hostVM, diskState
'#
Connect-ToAzure
$Result = (Search-AzGraph -Query $Query -First 1000)
$jsonResult = $Result | ConvertTo-Json
Post-LogAnalyticsData -customerId $customerId -sharedKey $sharedKey -body $jsonResult -logType $LogType
Write-Output "Runbook has been finished"
}
catch {
Write-Error -Message $_
break
}
However, I have an issue with collection of logs. I gather logs only from one subscription.
Can someone please help with code adjustment? How can I gather results from all subscriptions not just from one? I assume it should be foreach ($sub in $subs), but not sure how to do it combined with this graph query.
As the run as account will retire by Sep'23 so I would recommend using managed identities approach in your runbook i.e., configure role assignment for managed identity across multiple subscriptions as explained here and update your Connect-ToAzure function block something like shown below. For more context with regards to it, refer this Azure document.
$automationAccountName = Get-AutomationVariable -Name 'automationAccountName'
Write-Output "Azure Automation Account Name - $automationAccountName"
Disable-AzContextAutosave -Scope Process | Out-Null
Write-Output "Logging in to Azure..."
$AzureContext = (Connect-AzAccount -Identity).context
Also, you may use UseTenantScope parameter in your Search-AzGraph cmdlet to run the query across all available subscriptions in the current tenant.
I have been trying to GET Blob using the script below but it gives an error highlighted below. I am using the script from this link and the $signatureString is used according to the this link. In a way, what i understand from technet that the authorization header should have the following
Convert the PS data to JSON
$json = $authHeader | ConvertTo-Json
$method = "GET"
$headerDate = (get-date -format r).ToString()
$headers = #{"x-ms-version"="$headerDate"}
$StorageAccountName = ""
$StorageContainerName = "users"
$StorageAccountKey = ""
$Url = "https://$StorageAccountName.blob.core.windows.net/$StorageContainerName/******/*****.rdp"
$body = "Hello world"
$xmsdate = (get-date -format r).ToString()
$headers.Add("x-ms-date",$xmsdate)
$contentLength = $body.Length
$headers.Add("Content-Length","$contentLength")
$headers.Add("x-ms-blob-type","BlockBlob")
$contentType = "application/json"
#$headers.Add("Content-Type","$contentType")
$signatureString = "$method$([char]10)$([char]10)$([char]10)$contentLength$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)$([char]10)"
#Add CanonicalizedHeaders
$signatureString += "x-ms-blob-type:" + $headers["x-ms-blob-type"] + "$([char]10)"
$signatureString += "x-ms-date:" + $headers["x-ms-date"] + "$([char]10)"
$signatureString += "x-ms-version:" + $headers["x-ms-version"] + "$([char]10)"
#Add CanonicalizedResource
$uri = New-Object System.Uri -ArgumentList $url
$signatureString += "/" + $StorageAccountName + $uri.AbsolutePath + $([char]10) + "restype:container"
$dataToMac = [System.Text.Encoding]::UTF8.GetBytes($signatureString)
$accountKeyBytes = [System.Convert]::FromBase64String($StorageAccountKey)
$hmac = new-object System.Security.Cryptography.HMACSHA256((,$accountKeyBytes))
$signature = [System.Convert]::ToBase64String($hmac.ComputeHash($dataToMac))
$headers.Add("Authorization", "SharedKey " + $StorageAccountName + ":" + $signature);
write-host -fore green $signatureString
write-host -fore green $headers
Invoke-RestMethod -Uri $Url -Method $method -headers $headers
Please someone assist me on this.
I am sorry! yes it was not that descriptive. here is the screenshot
InvalidHeaderValue-Incorrect format
I am actually trying to access azure blob storage using powerShell. today, i tried other script to do the same but I get a different error, please see the script and screenshot of that error message below.
$storageAccount = ""
$accesskey = ""
$version = "2017-04-17"
$resource = "tets"
$table_url = "https://$storageAccount.blob.core.windows.net/$resource"
$GMTTime = (Get-Date).ToUniversalTime().toString('R')
$stringToSign = "$GMTTime`n/$storageAccount/$resource"
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accesskey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($stringToSign))
$signature = [Convert]::ToBase64String($signature)
$headers = #{
'x-ms-date' = $GMTTime
Authorization = "SharedKeyLite " + $storageAccount + ":" + $signature
"x-ms-version" = $version
Accept = "application/json;odata=fullmetadata"
}
$item = Invoke-RestMethod -Method GET -Uri $table_url -Headers $headers -ContentType application/json
return $item.value
Here is the error message I get.
AuthenticationFailedServer
Consider using the Azure PowerShell modules to access blobs rather than raw API requests. All of the authentication details should be handled for you already.
I am trying to create an UDF in cosmos db using REST call from powershell. I am getting unauthorized for the generated auth header. I am using the below methods to build auth header and make REST call for UDF creation.
I am using resource type as udf while building auth header.
function GetKey([System.String]$Verb = '',[System.String]$ResourceId = '',
[System.String]$ResourceType = '',[System.String]$Date = '',[System.String]$masterKey = '') {
$keyBytes = [System.Convert]::FromBase64String($masterKey)
$text = #($Verb.ToLowerInvariant() + "`n" + $ResourceType.ToLowerInvariant() + "`n" + $ResourceId + "`n" + $Date.ToLowerInvariant() + "`n" + "`n")
$body =[Text.Encoding]::UTF8.GetBytes($text)
$hmacsha = new-object -TypeName System.Security.Cryptography.HMACSHA256 -ArgumentList (,$keyBytes)
$hash = $hmacsha.ComputeHash($body)
$signature = [System.Convert]::ToBase64String($hash)
[System.Web.HttpUtility]::UrlEncode($('type=master&ver=1.0&sig=' + $signature))
}
function BuildHeaders([string]$action = "get",[string]$resType, [string]$resourceId){
$authz = GetKey -Verb $action -ResourceType $resType -ResourceId $resourceId -Date $apiDate -masterKey $connectionKey
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", $authz)
$headers.Add("x-ms-version", '2015-12-16')
$headers.Add("x-ms-date", $apiDate)
$headers
}
function CreateUDF([string]$dbname,[string]$collectionName){
$uri = $rootUri + "/" + $dbname + "/colls/" + $collectionName + "/udfs"
$headers = BuildHeaders -action Post -resType udfs -resourceId $dbname
$response = Invoke-RestMethod -Uri $uri -Method POST -Headers $headers -Body $udfCreationInput
$response
Write-Host ("Create Udf Response is " + $Response)
}
Not sure on what is wrong here while building auth header. Please help me on getting this working.
I used the following script to generate the authorization token for creating UDF via Rest API:
$Verb="POST"
$ResourceType="udfs"
$ResourceId="dbs/brucedb/colls/brucecoll"
$Date=[System.DateTime]::UtcNow.ToString("r")
$text = #($Verb.ToLowerInvariant() + "`n" + $ResourceType.ToLowerInvariant() + "`n" + $ResourceId + "`n" + $Date.ToLowerInvariant() + "`n" + "`n")
$masterKey="<your-master-key>"
$keyBytes = [System.Convert]::FromBase64String($masterKey)
$body =[Text.Encoding]::UTF8.GetBytes($text)
$hmacsha = new-object -TypeName System.Security.Cryptography.HMACSHA256 -ArgumentList (,$keyBytes)
$hash = $hmacsha.ComputeHash($body)
$signature = [System.Convert]::ToBase64String($hash)
Add-Type -AssemblyName System.Web
$token=[System.Web.HttpUtility]::UrlEncode($('type=master&ver=1.0&sig=' + $signature))
Write-Host ("the token is:" + $token)
TEST:
Moreover, you could follow Create User Defined Function, Access control in the Azure Cosmos DB SQL API and this github C# Rest sample.
For a testing scenario, I'd like to run a Stored Procedure against each partition in a CosmosDB database, and I'm attempting to get the list of partition ranges from CosmosDB using PowerShell.
I'm getting a 401 - Unauthorized response for this query, but other queries on the same collection work fine - for example executing a procedure works.
The code I'm using to query the ranges is:
Add-Type -AssemblyName System.Web
# Configure as required
$accountName = ""
$connectionKey = ""
$collectionName = ""
$databaseName = ""
function GetKey([System.String]$Verb = '',[System.String]$ResourceId = '',
[System.String]$ResourceType = '',[System.String]$Date = '',
[System.String]$masterKey = '')
{
$keyBytes = [System.Convert]::FromBase64String($masterKey)
$text = #($Verb.ToLowerInvariant() + "`n" +
$ResourceType.ToLowerInvariant() + "`n" + $ResourceId + "`n" +
$Date.ToLowerInvariant() + "`n" + "" + "`n")
$body =[Text.Encoding]::UTF8.GetBytes($text)
$hmacsha = new-object -TypeName System.Security.Cryptography.HMACSHA256 -ArgumentList (,$keyBytes)
$hash = $hmacsha.ComputeHash($body)
$signature = [System.Convert]::ToBase64String($hash)
[System.Web.HttpUtility]::UrlEncode($('type=master&ver=1.0&sig=' + $signature))
}
function GetUTDate() {
$date = get-date
$date = $date.ToUniversalTime();
return $date.ToString("r", [System.Globalization.CultureInfo]::InvariantCulture);
}
function BuildHeaders([string]$action = "get",[string]$resType, [string]$resourceId){
$authz = GetKey -Verb $action -ResourceType $resType -ResourceId $resourceId -Date $apiDate -masterKey $connectionKey
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", $authz)
$headers.Add("x-ms-version", '2017-02-22')
$headers.Add("x-ms-date", $apiDate)
$headers.Add("Cache-Control", 'no-cache')
$headers.Add("Accept", 'application/json')
$headers.Add("Content-Type", 'application/json')
$headers
}
function GetPartitionKeys(){
$pkranges = "dbs/" + $databaseName + "/colls/" + $collectionName + "/pkranges"
$headers = BuildHeaders -action Get -resType colls -resourceId $pkranges
$uri = $rootUri + "/" + $pkranges
write-host "Calling" $uri
write-host($headers|Out-String)
$response = Invoke-RestMethod $uri -Method Get -Headers $headers
}
$rootUri = "https://" + $accountName + ".documents.azure.com"
GetPartitionKeys
I think the problem is to do with the Resource Type when I'm building the Auth header but the CosmosDB REST documentation doesn't have much information about querying this resource. At present I'm getting the following output:
Calling https://my-account-name.documents.azure.com/dbs/my-db-name/colls/my-coll-name/pkranges
Invoke-RestMethod : The remote server returned an error: (401) Unauthorized.
When computing the has for the authorization header the ResourceType value should be "pkranges".
The ResourceLink would be "dbs/my-db-name/colls/my-coll-name" and the URL you request is: https://my-account-name.documents.azure.com/dbs/my-db-name/colls/my-coll-name/pkranges
Paul- Stored procedures can be executed only against a partition key, not against individual partition key ranges.
I'm trying to implement a SharedKeyLite Authorization header function in powershell. This is to connect to Azure Tables REST API. I'm missing something cause I keep getting an error:
Server failed to authenticate
the request. Make sure the value of Authorization header is formed correctly including the signature.
function GenerateHeader($accountName, $accountKey, $action)
{
$xmsdate = get-date
$xmsdate = $xmsdate.ToUniversalTime()
$xmsdate = $xmsdate.toString('r')
$newLine = "`n";
$message = $xmsdate + $newline + "/" + $accountname + "/" + $action;
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accesskey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($message))
$signature = [Convert]::ToBase64String($signature)
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("x-ms-version", "2014-02-14")
$headers.Add("x-ms-date", $xmsdate)
$headers.Add("Authorization", "SharedKeyLite " + $accountName + ":" + $signature)
return $headers
}
UPDATE:
Here's code that calls this function. The $action variable is set to the URI string.
$uriString = "https://$StorageAccountName.table.core.windows.net/Tables"
$headers = GenerateHeader $StorageAccountName $StorageAccountKey "Tables"
Invoke-RestMethod -Uri $uriString -Method $method -Headers $headers -Body $body
And here's the error it throws.
Invoke-RestMethod : AuthenticationFailedServer failed to authenticate the request. Make sure
the value of Authorization header is formed correctly including the
signature. RequestId:4215377d-0002-0044-1a92-94cd56000000
Time:2015-05-22T13:21:53.5205261Z At
C:\Users\Samuel\Source\BaseDataInstall\BaseDataInstall\AzureHelpers.ps1:45
char:2
+ Invoke-RestMethod -Uri $uriString -Method $method -Headers $headers -Body $body
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebExc
eption
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
EDIT: Here's the Example output from the $headers variable outside the function...
Key : x-ms-version
Value : 2014-02-14
Key : x-ms-date
Value : Tue, 26 May 2015 19:30:20 GMT
Key : Authorization
Value : SharedKeyLite <MyStorageName>:lf+ndqhi4OeJhIfLljugT0dfcLbqXDBHwrQJn9Q66HQ=
So this ended up being a simple coding error :(
I feel kinda silly posting this now, but I'm going to post an answer cause I couldn't find a working Authorization builder for Azure in Powershell anywhere. This one does work for Azure tables...
function GenerateHeader($accountName, $accountKey, $action)
{
$xmsdate = get-date
$xmsdate = $xmsdate.ToUniversalTime()
$xmsdate = $xmsdate.toString('R')
$newLine = "`n";
$action = $action.ToLower()
$message = $xmsdate + $newline + "/" + $accountname + "/" + $action;
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accountKey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($message))
$signature = [Convert]::ToBase64String($signature)
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Content-Type", "application/json")
$headers.Add("x-ms-date", $xmsdate)
$headers.Add("Authorization", "SharedKeyLite " + $accountName + ":" + $signature)
return $headers
}
There seem to be a change in the Azure API. I had to disable the "toLower" statement for this to Work.
function GenerateHeader($accountName, $accountKey, $action)
{
$xmsdate = get-date
$xmsdate = $xmsdate.ToUniversalTime()
$xmsdate = $xmsdate.toString('R')
$newLine = "`n";
# $action = $action.ToLower()
$message = $xmsdate + $newline + "/" + $accountname + "/" + $action;
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accountKey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes ($message))
$signature = [Convert]::ToBase64String($signature)
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
# $headers.Add("Content-Type", "application/json")
$headers.Add("x-ms-date", $xmsdate)
$headers.Add("Authorization", "SharedKeyLite " + $accountName + ":" + $signature)
return $headers
}
I hope this helps someone :-)