Start-AzureStorageBlobCopy error in Automation runbook - powershell

I'm following this guide to automate backups of Linux VMs in Azure Automation. Everything goes well in testing on initial run, but the second and all subsequent tests return the following error:
11/26/2014 11:03:45 AM, Error: Start-AzureStorageBlobCopy : One or more errors occurred.
At LinuxVMBackup:40 char:40
+
+ CategoryInfo : CloseError: (:) [Start-AzureStorageBlobCopy], AggregateException
+ FullyQualifiedErrorId :
AggregateException,Microsoft.WindowsAzure.Commands.Storage.Blob.Cmdlet.StartAzureStorageBlobCopy
Here is the runbook:
workflow LinuxVMBackup
{
param (
[Parameter(Mandatory=$true)]
[PSCredential]
$Cred,
[Parameter(Mandatory=$true)]
[string]
$subName,
[Parameter(Mandatory=$true)]
[string]
$serviceName,
[Parameter(Mandatory=$true)]
[string]
$vmName
)
# Login and get VM
$null = Add-AzureAccount -Credential $Cred
$null = Select-AzureSubscription -SubscriptionName $subName
$vm = Get-AzureVM -ServiceName $serviceName -Name $vmName
# Stop VM
$vm | Stop-AzureVM -StayProvisioned
# Get OS and Data Disks
$vmOSDisk = Get-AzureOSDisk -VM (Get-AzureVM -ServiceName $serviceName -Name $vmName)
$vmDataDisks = Get-AzureDataDisk -VM (Get-AzureVM -ServiceName $serviceName -Name $vmName)
# Get and set storage name
$StorageAccountName = $vmOSDisk.MediaLink.Host.Split('.')[0]
Set-AzureSubscription -CurrentStorageAccount $StorageAccountName -SubscriptionName $subName
$backupContainerName = "backups"
if (!(Get-AzureStorageContainer -Name $backupContainerName -ErrorAction SilentlyContinue)) {
New-AzureStorageContainer -Name $backupContainerName -Permission Off
}
# Snapshot OS Disk
$vmOSBlobName = $vmOSDisk.MediaLink.Segments[-1]
$vmOSContainerName = $vmOSDisk.MediaLink.Segments[-2].Split('/')[0]
#Error on this line
Start-AzureStorageBlobCopy -SrcContainer $vmOSContainerName -SrcBlob $vmOSBlobName -DestContainer $backupContainerName
#Error on this line
Get-AzureStorageBlobCopyState -Container $backupContainerName -Blob $vmOSBlobName -WaitForComplete
#Snapshot Data Disks
ForEach ($vmDataDisk in $vmDataDisks) {
$vmDataBlobName = $vmDataDisk.MediaLink.Segments[-1]
$vmDataContainerName = $vmDataDisk.MediaLink.Segments[-2].Split('/')[0]
Start-AzureStorageBlobCopy -SrcContainer $vmDataContainerName -SrcBlob $vmDataBlobName -DestContainer $backupContainerName -Force
Get-AzureStorageBlobCopyState -Container $backupContainerName -Blob $vmDataBlobName -WaitForComplete
}
#Get blobs in storage and output
$insertedBlobs = Get-AzureStorageBlob -Container $backupContainerName
Write-Output $insertedBlobs
$vm | Start-AzureVM
}
I couldn't find anything online, save an answer on Microsoft forums for a similar error stating that the VM needs to be stopped, so I thought maybe the VM is not fully stopped when Start-AzureStorageBlobCopy is initiated. I tried testing with a stopped VM and still got the error.
Thanks for your help.

Given the error only happens after the first successful copy, what is probably happening is the Start-AzureStorageBlobCopy cmdlet is trying to request user confirmation for the operation, since it would overwrite existing data. Since Azure Automation runs runbooks in a non-interactive session, user confirmation cannot be asked, so the command is failing.
Adding -Force should "force" user confirmation for the cmdlet and fix the issue. You have this parameter on the second instance of Start-AzureStorageBlobCopy, but not the first.
PS - You should put this runbook in our runbook gallery so others can take advantage of it!

Related

Powershell mandatory parameters, I am a newbie to PowerShell scripting

I have task on my ci/cd that calls below powershell script meant to stop a function before other deployment tasks on the same job. The script seem to throwing error even though I have all the variables correctly set. Please help and many thanks in advance..
#####Script#######
Param(
[Parameter(Mandatory)][string] $XxxxFunctionAppName,
[Parameter(Mandatory)][string] $ResourceGroupName
)
#stoping function app
$trap = Get-AzFunctionApp -Name $XxxxFunctionAppName -ResourceGroupName $ResourceGroupName -ErrorAction SilentlyContinue
if($trap.state -eq "Running")
{
Stop-AzFunctionApp -Force
}
####Error message#####
2021-04-06T10:24:43.3332161Z ## Validating Inputs
2021-04-06T10:24:43.3390381Z ## Validating Inputs Complete
2021-04-06T10:24:43.4262841Z ## Initializing Az module
2021-04-06T10:24:43.6949853Z Added TLS 1.2 in session.
2021-04-06T10:24:44.0148107Z ##[command]Import-Module -Name C:\Program Files\WindowsPowerShell\Modules\Az.Accounts\1.9.0\Az.Accounts.psd1 -Global
2021-04-06T10:24:44.9908767Z ##[command]Clear-AzContext -Scope CurrentUser -Force -ErrorAction SilentlyContinue
2021-04-06T10:24:45.5333784Z ##[command]Clear-AzContext -Scope Process
2021-04-06T10:24:45.9446194Z ##[command]Connect-AzAccount -ServicePrincipal -Tenant *** -Credential System.Management.Automation.PSCredential -Environment AzureCloud #processScope
2021-04-06T10:24:46.9272395Z ##[command] Set-AzContext -SubscriptionId XXXXXXXXXXXXXXX -TenantId ***
2021-04-06T10:24:47.4342523Z ## Az module initialization Complete
2021-04-06T10:24:47.4352242Z ## Beginning Script Execution
2021-04-06T10:24:47.4690205Z ##[command]& 'c:\vstsagent\A4\_work\r241\a\_xxxxxxxxxxxx\Scripts\Stopxxxx.ps1' -xxxxFunctionAppName "func-xxxxx-develop-02" -ResourceGroupName "xxxxx-develop-rg"
2021-04-06T10:24:50.7841003Z ##[command]Disconnect-AzAccount -Scope Process -ErrorAction Stop
2021-04-06T10:24:51.2391709Z ##[command]Clear-AzContext -Scope Process -ErrorAction Stop
2021-04-06T10:24:51.8052164Z ##[error]Cannot process command because of one or more missing mandatory parameters: Name ResourceGroupName.
2021-04-06T10:24:51.8527691Z ##[section]Finishing: StopxxxxFunc

Azure Search set ReplicaCount by Automation graphical runbook

Azure Search does not give any scheduled scaling option so I try to make it by Automation account.
I followed AzSearch PowerShell command, but it does not work as I expected.
Set-AzureRmResource with ReplicaCount=2 parameter is not applied. Actually, it does not give any result message. What did I miss?
To reproduce my problem, you can import my runbook file at below link;
https://gist.github.com/YoungjaeKim/5cb66a666a3a864b7379aac0a400da40
Save the text file as AzureSearch-SetReplicaCount.graphrunbook and Import it to Automation account > Add a runbook menu.
Following commenters, I ended up by making PowerShell runbook.
I uploaded powershell source code to below link;
https://gallery.technet.microsoft.com/scriptcenter/Azure-Search-change-c0b49c4c
Let me attach the code as below;
<#
.DESCRIPTION
Scale Azure Search ReplicaCount
AzSearch command reference; https://learn.microsoft.com/en-us/azure/search/search-manage-powershell
.NOTES
AUTHOR: Youngjae Kim
LASTEDIT: June 19, 2017
#>
Param(
[string]$SubscriptionId,
[string]$ResourceGroupName,
[string]$AzSearchResourceName,
[int]$InstanceCount = 1
)
# 1. Acquire Automation account
$connectionName = "AzureRunAsConnection"
try
{
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection=Get-AutomationConnection -Name $connectionName
"Logging in to Azure..."
Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found. You must have Automation account. Reference: https://learn.microsoft.com/en-us/azure/automation/automation-role-based-access-control"
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
# 2. Select subscription
Select-AzureRmSubscription -SubscriptionId $SubscriptionId
# 3. Specify Azure Search Resource
$resource = Get-AzureRmResource `
-ResourceType "Microsoft.Search/searchServices" `
-ResourceGroupName $ResourceGroupName `
-ResourceName $AzSearchResourceName `
-ApiVersion 2015-08-19
Write-Output ($resource)
# 4. Scale your service up
# Note that this will only work if you made a non "free" service
# This command will not return until the operation is finished
Write-Output ("Updating InstanceCount to " + $InstanceCount + ". This can take 15 minutes or more...")
$resource.Properties.ReplicaCount = $InstanceCount
$resource | Set-AzureRmResource -Force -Confirm:$false
# 5. Finish
Write-Output ("End of Process to set InstanceCount = " + $InstanceCount + " for " + $AzSearchResourceName)

Upgrading Tier (Sku) via Set-AzureRmAnalysisServicesServer not working

I want to upscale and downscale my Azure Analysis Services with PowerShell (Automation Runbook), but changing the Tier (Sku) doesn't seem to work. However there are no errors. Any suggestions?
# PowerShell code
# Connect to a connection to get TenantId and SubscriptionId
$Connection = Get-AutomationConnection -Name "AzureRunAsConnection"
$TenantId = $Connection.TenantId
$SubscriptionId = $Connection.SubscriptionId
# Get the service principal credentials connected to the automation account.
$null = $SPCredential = Get-AutomationPSCredential -Name "SSISJoost"
# Login to Azure ($null is to prevent output, since Out-Null doesn't work in Azure)
Write-Output "Login to Azure using automation account 'SSISJoost'."
$null = Login-AzureRmAccount -TenantId $TenantId -SubscriptionId $SubscriptionId -Credential $SPCredential
# Select the correct subscription
Write-Output "Selecting subscription '$($SubscriptionId)'."
$null = Select-AzureRmSubscription -SubscriptionID $SubscriptionId
# Get variable values
$ResourceGroupName = Get-AutomationVariable -Name 'ResourceGroupName'
$AnalysisServerName = Get-AutomationVariable -Name 'AnalysisServerName'
# Get old status (for testing/logging purpose only)
$OldAsSetting = Get-AzureRmAnalysisServicesServer -ResourceGroupName $ResourceGroupName -Name $AnalysisServerName
try
{
# changing tier
Write-Output "Upgrade $($AnalysisServerName) to S1. Current tier: $($OldAsSetting.Sku.Name)"
Set-AzureRmAnalysisServicesServer -ResourceGroupName $ResourceGroupName -Name $AnalysisServerName -Sku "S1"
}
catch
{
Write-Error -Message $_.Exception
throw $_.Exception
}
Write-Output "Done"
# Get new status (for testing/logging purpose only)
$NewAsSetting = Get-AzureRmAnalysisServicesServer -ResourceGroupName $ResourceGroupName -Name $AnalysisServerName
Write-Output "New tier: $($NewAsSetting.Sku.Name)"
using Set-AzureRmAnalysisServicesServer
There was a little bug in the PowerShell AzureRM.AnalysisServices module. It has been fixed in 0.4.0 (Thursday, June 08 2017)
Now the code finally works: http://microsoft-bitools.blogspot.com/2017/06/schedule-upscaledownscale-azure.html

Creating snapshot of VM in Azure

Having followed this article(http://www.coreazure.com/snapshot-vms-in-azure-2/) I am trying to create a snapshot of VM using powershell(PS) from Azure portal. This is the PS script which I have created to take a snapshot
workflow snapshot1
{
$subName = 'XYZ'
$cred = Get-AutomationPSCredential -Name "xyz#xyzgmail.onmicrosoft.com"
Add-AzureAccount -Credential $cred
Set-AzureSubscription -SubscriptionName $subName -CurrentStorageAccount 'storagename'
$serviceName = "a1smallvm"
$vm = Get-AzureVM –ServiceName $serviceName –Name "a1smallvm"
$storageContainer = "backups"
InlineScript {
# Create a storage for putting the backups of OSDisk & DataDisks
New-AzureStorageContainer -Name $Using:storageContainer -Permission off
# Stop VM if running
$Using:vm | Stop-AzureVM -StayProvisioned
$vm = Get-AzureVM –ServiceName $Using:serviceName –Name "a1smallvm"
$vmOSDisk = $vm | Get-AzureOSDisk
$vmDataDisks = $vm | Get-AzureDataDisk
Write-output "OSDisk: $vmOSDisk"
$storageAccountName = $vmOSDisk.MediaLink.Host.Split(‘.’)[0]
Write-output "Data Disk: $vmDataDisks"
Write-output "StorageAccountName: $storageAccountName"
$vmOSBlobName = $vmOSDisk.MediaLink.Segments[-1]
$vmOSContainerName = $vmOSDisk.MediaLink.Segments[-1].Split(‘/’)[0]
Write-output "vmOSBlobName: $vmOSBlobName"
Write-output "vmOSContainerName: $vmOSContainerName"
# Backup the osblob and oscontainer
Start-AzureStorageBlobCopy -SrcContainer $vmOSContainerName -SrcBlob $vmOSBlobName -DestContainer $Using:storageContainer
Get-AzureStorageBlobCopyState -Container $Using:storageContainer -Blob $vmOSBlobName -WaitForComplete
# Backup the dataBlob and dataContainer
ForEach ($vmDataDisk in $vmDataDisks) {
$vmDataBlobName = $vmDataDisk.MediaLink.Segments[-1]
$vmDataContainerName = $vmDataDisk.MediaLink.Segments[-2].Split(‘/’)[0]
Start-AzureStorageBlobCopy -SrcContainer $vmDataContainerName -SrcBlob $vmDataBlobName -DestContainer backups -Force
Get-AzureStorageBlobCopyState -Container backups -Blob $vmDataBlobName -WaitForComplete
}
}
}
The cmdlet
Start-AzureStorageBlobCopy -SrcContainer $vmOSContainerName -SrcBlob $vmOSBlobName -DestContainer $Using:storageContainer
throws an error:
Error: Start-AzureStorageBlobCopy : Container name 'a1smallvm-a1smallvm-2015-08-11.vhd' is invalid. Valid names start and end with a lower case letter or a number and has in between a lower case letter, number or dash with no consecutive dashes and is 3 through 63 characters long.
The container name 'a1smallvm-a1smallvm-2015-08-11.vhd' which I am getting follows the correct naming format but still why it's giving an error saying the name is invalid.
The VM was created from the portal, it's an A1 type of VM, OS is CentOS "OpenLogic 6.5".
Any clue what's wrong?
The following are the outputs from Write-output's
OSDisk: Microsoft.WindowsAzure.Commands.ServiceManagement.Model.OSVirtualHardDisk
Data Disk:
StorageAccountName: portalvhds14510n2y65vnh
vmOSBlobName: a1smallvm-a1smallvm-2015-08-11.vhd
vmOSContainerName: a1smallvm-a1smallvm-2015-08-11.vhd
Correct script:
The storage account name has to be same or we will have to add the context of destination storage account.
$storageAccountName = $vmOSDisk.MediaLink.Host.Split(‘.’)[0]
Set-AzureSubscription -SubscriptionName $subName -CurrentStorageAccount '$storagenameAccountName'
And index for Segments should be -2 not -1
$vmOSContainerName = $vmOSDisk.MediaLink.Segments[-2].Split(‘/’)[0]
I'm no PowerShell expert (and I am sure there are better ways of doing it) but you could do the following:
$vmOSContainerName = $vmOSDisk.MediaLink.AbsolutePath.Split('/')[1]
This will output vhds which is the name of your blob container.

Azure: How to check storage account exists in Azure with Get-AzureStorageAccount

I am building a power shell script to automate the setup of a website environment in Azure. This web uses an account storage. I want to the script not to create the account storage if exists.
I thought that using Get-AzureStorageAccount this way may work but it does not:
Write-Verbose "[Start] creating $Name storage account $Location location"
$storageAcct = Get-AzureStorageAccount –StorageAccountName $Name
if (!$storageAcct)
{
$storageAcct = New-AzureStorageAccount -StorageAccountName $Name -Location $Location -Verbose
if ($storageAcct)
{
Write-Verbose "[Finish] creating $Name storage account in $Location location"
}
else
{
throw "Failed to create a Windows Azure storage account. Failure in New-AzureStorage.ps1"
}
}
else
{
Write-Verbose "$Name storage account in $Location location already exists, skipping creation"
}
The issue is I don't know how to handle the return of Get-AzureStorageAccount.
Thank you very much in advance!
I would suggest using the Test-AzureName cmdlet to determine if it exists. So, something like this.
if (!(Test-AzureName -Storage $Name))
{
Write-Host "Creating Storage Account $Name"
New-AzureStorageAccount -StorageAccountName $Name -Location $Location
}
You can use Test-AzureName for other services too, such as Cloud Services, WebSites, and ServiceBus. It returns True if it exists, False otherwise.
Get-AzureRmStorageAccountNameAvailability -Name "accountname"
Try this:
$Name = "myStorageAccount"
$Location = "myLocation"
Write-Host "[Start] creating $Name storage account $Location location"
try{
Get-AzureStorageAccount –StorageAccountName $Name -ErrorAction Stop | Out-Null
Write-Host "$Name storage account in $Location location already exists, skipping creation"
}
catch{
Write-Host "[Finish] creating $Name storage account in $Location location"
New-AzureStorageAccount -StorageAccountName $Name -Location $Location -Verbose
}
Test-AzureName didn't work with our build agents and we already had a try/catch in code so a second one would require building it out as a function. I opted for that standard get and check if null, use -ErrorAction Ignore to stop it throwing an exception
# Check for storage account and create if not found
$StorageAccount = Get-AzureRmStorageAccount -Name $StorageAccountName -ResourceGroupName $StorageAccountRG -ErrorAction Ignore
if ($StorageAccount -eq $null)
{
New-AzureRmStorageAccount -Location "West Europe" -Name $StorageAccountName -ResourceGroupName $StorageAccountRG -SkuName Standard_LRS -Kind Storage
$StorageAccount = Get-AzureRmStorageAccount -Name $StorageAccountName -ResourceGroupName $StorageAccountRG
}
#Rick Rainey's solution works if you're logged in using Add-AzureAccount. However, Azure and powershell have a conflicting and confusing suite of login accounts (Windows Live versus AD) and login mechanisms (Classic: Add-AzureAccount; Resource manager: Login-AzureRmAccount). Some Azure powershell cmdlets require a specific login; further, some require a specific account type!
To clear through this thicket of complicated, undocumented, and confusing permission issues, we always use an AD account, logging in via Login-AzureRmAccount. We also use Azure resource manager (ARM) resources and cmdlets, following Microsoft's movement to ARM as its recommended and strategic approach. However, #RIck's solution is one which the ARM login doesn't work with. :-( So you need another approach, which is #Darren's (for storage). However, for a generic replacement for Test-AzureName I'd suggest Find-AzureRmResource. In the case of storage
$StorageObject = Find-AzureRmResource -ResourceType "Microsoft.Storage/storageAccounts" | Where-Object {$_.Name -eq $storageName}
if ( !$StorageObject ) {
$storageLocation = (Get-AzureRmResourceGroup -ResourceGroupName $resourceGroup).Location
$storageType = "Standard_LRS"
New-AzureRmStorageAccount -ResourceGroupName $resourceGroup -Name $storageName -Location $storageLocation -Type $storageType
}
You should use the latest Powershell module Az.
if ($(Get-AzStorageAccount -ResourceGroupName $resourceGroupName -Name $storageAccountName) -eq $null)
{
# does not exist
}
With the current Az module for PowerShell Version 7, the Get-AzStorageAccountNameAvailability cmdlet might offer a more efficient solution as it was designed specifically for this task. Here is an example:
# ... declare variables and specify values ...
$checkNameAvail = (Get-AzStorageAccountNameAvailability -Name $storageAccountName) | `
Select-Object NameAvailable
if ($checkNameAvail.NameAvailable)
{
Write-Host 'Account name available! Please wait while your resource is being created'
# Create account. Variables used in this example would have been declared earlier in the script.
$storageAccount = (New-AzStorageAccount -ResourceGroupName $resourceGroupName `
-AccountName $storageAccountName `
-Location $location `
-SkuName $skuType `
-AllowBlobPublicAccess $false -EnableHttpsTrafficOnly $true)
# ...
}
else
{
# This section of the script executes if the name is not available
Write-Host "The name <$storageAccountName> is not available. Suggest a new globally unique name!"
}
The condition above will return False, and execute the else statement because the boolean value returned by the cmdlet is in [0] as shown in the PowerShell command-line test below. The availability information (boolean) can thus be stripped from the object returned by the cmdlet and (as in this example) used as a condition in the rest of the script.
PS C:\> Get-AzStorageAccountNameAvailability -Name testaccount1
NameAvailable Reason Message
------------- ------ -------
False AlreadyExists The storage account named testaccount1 is already taken.
Use the error variable
Get-AzStorageAccount -ResourceGroupName 'RG-QA-TEST' -Name 'staccountfor12334ff' -ErrorVariable ev1 -ErrorAction SilentlyContinue
if ($ev1) {
Write-Host "-------------------------- Creating OEM Storage"
//create storage account
}
I had this challenge when setting up Azure storage accounts for Static website hosting using Powershell in Octopus Deploy.
Here's how I fixed it:
Using the Az module for Azure Powershell I did the following:
# Define Variables
$RESOURCE_GROUP_NAME = my-resource-group
$LOCATION = northeurope
$STORAGE_ACCOUNT_NAME = myapplication
$SKU_NAME = Standard_GRS
$STORAGE_KIND = StorageV2
# Check Storage Account and Create if not Found
$STORAGE_ACCOUNT = Get-AzStorageAccount -ResourceGroupName $RESOURCE_GROUP_NAME -Name $STORAGE_ACCOUNT_NAME -ErrorAction Ignore
if ($STORAGE_ACCOUNT -eq $null) {
Write-Host 'Creating storage account'
New-AzStorageAccount -ResourceGroupName $RESOURCE_GROUP_NAME -AccountName $STORAGE_ACCOUNT_NAME -Location $LOCATION -SkuName $SKU_NAME -Kind $STORAGE_KIND
Write-Host "$STORAGE_ACCOUNT_NAME storage account successfully created"
}
else {
Write-Host "$STORAGE_ACCOUNT_NAME storage account already exists"
}
Note:
-ErrorAction Ignore - This ignores the exception that would arise if the storage account does not exist
Write-Host " " - Double quotes were used to allow for string interpolation since we are connecting strings and variables.
That's all.
I hope this helps