I am doing batch uploads from a csv file to Azure table storage through a Powershell script and i have a command: $table.CloudTable.ExecuteBatch($batchOperation)
for which i'm getting the error mentioned in the header of the question of my post. I believe that "ExecuteBatch" is a method in the old AzureRm module and not the newer Az module which i am using, which is causing it to break. Is there a corresponding method in Az module for "ExecuteBatch"?
According to my test, if we use new Azure PowerShell module Az to manage Azure Table storage, we need to use the SDK Microsoft.Azure.Cosmos.Table.
So if ou want to use ExecuteBatch method, we need to use the command [Microsoft.Azure.Cosmos.Table.TableBatchOperation] $batchOperation = New-Object -TypeName Microsoft.Azure.Cosmos.Table.TableBatchOperation to create TableBatchOperation. For example:
Connect-AzAccount
$ResourceGroupName = "testfun06"
$StorageAccountName="testfun06bf01"
$TableName="People"
$keys=Get-AzStorageAccountKey -ResourceGroupName $ResourceGroupName -Name $StorageAccountName
$ctx = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $keys[0].Value
$table = Get-AzStorageTable -Name $TableName -Context $ctx
$e = New-Object Microsoft.Azure.Cosmos.Table.DynamicTableEntity("Jim","test")
$e1 = New-Object Microsoft.Azure.Cosmos.Table.DynamicTableEntity("Jim","test1")
[Microsoft.Azure.Cosmos.Table.TableBatchOperation] $batchOperation = New-Object -TypeName Microsoft.Azure.Cosmos.Table.TableBatchOperation
$batchOperation.InsertOrMerge($e)
$batchOperation.InsertOrMerge($e1)
$table.CloudTable.ExecuteBatch($batchOperation)
i have this powershell using which I want to create new table in azure storage account.
Param(
[string]$rgName,
[string] $tableName
)
$storcontext= New-AzureStorageContext -ConnectionString '$(MyConnectionString)'
if(!(Get-AzureStorageTable -Name $tableName -Context $storcontext ))
{
New-AzureStorageTable -Name $tableName -Context $storcontext
}
New-AzureStorageTable command works perfect.however i tried adding a check to see if table already exists . but on Get command, powershell throws me saying table does not exist.
What I want to do is check if table exist, if not , then create it.
Is there another way of doing this?
The cmdlet throws an error if the table doesn't exists so you could set the ErrorAction to SilentlyContinue and specify a variable for the error which you could check:
Get-AzureStorageTable -Name $tableName -Context $storcontext -ErrorVariable ev -ErrorAction SilentlyContinue
if ($ev) {
New-AzureStorageTable -Name $tableName -Context $storcontext
}
I am running the below PowerShell Script to deploy csv file data into Azure table storage. But The below parameters are different for different environment in the azure.Suppose the below script can be deployed to any environment but the below parameters will be varied as per the environment.So I want to pass the below parameters to the script while running from the PowerShell task in VSTS.How to accomplish task.Please help me out on this.So
**$subscriptionName = "Tech Enabled Solutions"
$resourceGroupName = "abc"
$storageAccountName = "defghi"
$location = "North Central US, South Central US"
$StorageAccountKey = "12345678"**
PowerShell Script:
function Add-Entity()
{
[CmdletBinding()]
param
(
$table,
[string] $partitionKey,
[string] $RowKey,
[string] $Label_Usage,
[string] $Label_Value,
[string] $Usage_Location,
[string] $subscriptionName,
[string] $resourceGroupName,
[string] $storageAccountName,
[string] $location,
[string] $StorageAccountKey
)
$entity = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.DynamicTableEntity -ArgumentList $partitionKey, $rowKey
$entity.Properties.Add("Label_Value",$Label_Value)
$entity.Properties.Add("Label_Usage",$Label_Usage)
$entity.Properties.Add("Usage_Location",$Usage_Location)
$result = $table.CloudTable.Execute([Microsoft.WindowsAzure.Storage.Table.TableOperation]::InsertOrReplace($entity))
}
$tableName = "sampletable"
# Get a storage context
$ctx = New-AzureStorageContext $StorageAccountName $StorageAccountKey
# Get a reference to the table
$table = Get-AzureStorageTable -Name $tableName -Context $ctx -ErrorAction Ignore
$csv = Import-CSV "d:\a\1\s\DeploymentScripts\sampletable.csv"
ForEach ($line in $csv)
{
Add-Entity -Table $table -partitionKey $line.partitionkey -rowKey $line.RowKey -Label_Usage $line.Label_Usage -Label_Value $line.Label_Value -Usage_Location $line.Usage_Location
}
You need to use the arguments text box to pass your parameters into the script (either inline or script file).
Your script would need to look like this:
param (
[string] $table,
[string] $partitionKey,
[string] $RowKey,
[string] $Label_Usage,
[string] $Label_Value,
[string] $Usage_Location,
[string] $subscriptionName,
[string] $resourceGroupName,
[string] $storageAccountName,
[string] $location,
[string] $StorageAccountKey
)
$entity = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.DynamicTableEntity -ArgumentList $partitionKey, $rowKey
$entity.Properties.Add("Label_Value",$Label_Value)
$entity.Properties.Add("Label_Usage",$Label_Usage)
$entity.Properties.Add("Usage_Location",$Usage_Location)
$result = $table.CloudTable.Execute([Microsoft.WindowsAzure.Storage.Table.TableOperation]::InsertOrReplace($entity))
$tableName = "sampletable"
# Get a storage context
$ctx = New-AzureStorageContext $StorageAccountName $StorageAccountKey
# Get a reference to the table
$table = Get-AzureStorageTable -Name $tableName -Context $ctx -ErrorAction Ignore
$csv = Import-CSV "d:\a\1\s\DeploymentScripts\sampletable.csv"
ForEach ($line in $csv)
{
Add-Entity -Table $table -partitionKey $line.partitionkey -rowKey $line.RowKey -Label_Usage $line.Label_Usage -Label_Value $line.Label_Value -Usage_Location $line.Usage_Location
}
Each of your variables will either need to be defaulted or passed in as arguments. In your example, you would look something like the following in the text box:
-subscriptionName "Tech Enabled Solutions" -$resourceGroupName "abc" -storageAccountName "defghi" -location "North Central US, South Central US" -StorageAccountKey "12345678
The box is expecting you input the arguments exactly as you would if you were calling the PowerShell script from the command line.
Some parameters are not used in your script, such as $subscriptionName, $resourceGroupName, you can check whether they are needed.
Refer to this code to add parameters:
param(
[string] $subscriptionName,
[string] $resourceGroupName,
[string] $storageAccountName,
[string] $location,
[string] $StorageAccountKey
)
function Add-Entity()
{
[CmdletBinding()]
param
(
$table,
[string] $partitionKey,
[string] $RowKey,
[string] $Label_Usage,
[string] $Label_Value,
[string] $Usage_Location
)
$entity = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.DynamicTableEntity -ArgumentList $partitionKey, $rowKey
$entity.Properties.Add("Label_Value",$Label_Value)
$entity.Properties.Add("Label_Usage",$Label_Usage)
$entity.Properties.Add("Usage_Location",$Usage_Location)
$result = $table.CloudTable.Execute([Microsoft.WindowsAzure.Storage.Table.TableOperation]::InsertOrReplace($entity))
}
$tableName = "sampletable"
# Get a storage context
$ctx = New-AzureStorageContext $storageAccountName $StorageAccountKey
# Get a reference to the table
$table = Get-AzureStorageTable -Name $tableName -Context $ctx -ErrorAction Ignore
$csv = Import-CSV "d:\a\1\s\DeploymentScripts\sampletable.csv"
ForEach ($line in $csv)
{
Add-Entity -Table $table -partitionKey $line.partitionkey -rowKey $line.RowKey -Label_Usage $line.Label_Usage -Label_Value $line.Label_Value -Usage_Location $line.Usage_Location
}
Specify the parameters' value in PowerShell task (Arguments input box)
-subscriptionName "Tech Enabled Solutions" -resourceGroupName "abc" -storageAccountName "defghi" -location "North Central US, South Central US" -StorageAccountKey "12345678"
Your script doesn't take any parameters. You have a function in your script that takes parameters. A param block at the top of your script, outside of any functions, will make your script take parameters.
Ex:
param($A)
function Foo {
param($B)
Write-Output $B
}
Foo -B $A
I am running the below script and passing the script parameters for the $fileObj through powershell script using arguments section in VSTS powershell task.I am trying to deploy table data into Azure table storage. I have table data in .csv files and I am trying to deploy those table entities using powershell script and deploying into azure table storage.The below script is not deploying the table entities and failing with error. Could any one please help me out.
I have attached the error log in onedrive location: https://onedrive.live.com/?authkey=%21AEh2aAOnbmuzq9U&cid=5599285D52BD31F3&id=5599285D52BD31F3%21900&parId=root&action=locate
foreach($fo in $fileObj){
Write-Host $fo.filepath
$csv = Import-CSV $fo.filepath
$cArray=$fo.Cols.split(",")
foreach($line in $csv)
{
Write-Host "$($line.partitionkey), $($line.rowKey)"
$entity = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.DynamicTableEntity -ArgumentList $line.partitionkey, $line.rowKey
foreach($c in $cArray){
Write-Host "$c,$($line.$c)"
$entity.Properties.Add($c,$line.$c)
}
$result = $table.CloudTable.Execute([Microsoft.WindowsAzure.Storage.Table.TableOperation]::Insert($entity))
}
}
$subscriptionName = ""
$resourceGroupName = ""
$storageAccountName = ""
$location = ""
# Get the storage key for the storage account
$StorageAccountKey = ""
# Get a storage context
$ctx = New-AzureStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
According to your mentioned log, I find that it seems that your csv column name is not corresponding to your code. And your CSV file format with 2 Colunms named Partitionkey and Rowkey is not correct. Please have a try to use the following demo code and csv file format. It works correctly on myside.
$resourceGroup ="resourceGroup name"
$storageAccount = "storage account Name"
$tableName = "table name"
$storageAccountKey = "storage key"
$ctx = New-AzureStorageContext -StorageAccountName $storageAccount -
StorageAccountKey $storageAccountKey
######### Add removing table and create table code #######
try
{
Write-Host "Start to remove table $tableName, please wait a moment..."
Remove-AzureStorageTable -Name $tableName -Context $ctx -Force # Remove the Azure table
Start-Sleep -Seconds 60 # waiting for removing table, you could change it according to your table
Write-Host "$tableName table has been removed"
}
catch
{
Write-Host "$tableName is not existing"
}
Write-Host "Start to create $tableName table"
New-AzureStorageTable -Name $tableName -Context $ctx # Create new azure storage table
##########Add removing table and create table code ############
$table = Get-AzureStorageTable -Name $tableName -Context $ctx
$csvPath ='csv file path'
$cols = "Label_Usage,Label_Value,Usage_Location" #should be corrensponding to your csv column exclude Partitionkey and RowKey
$csv = Import-Csv -Path $csvPath
$number = 0
[Microsoft.WindowsAzure.Storage.Table.TableBatchOperation]$batchOperation = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.TableBatchOperation
foreach($line in $csv)
{
$number++
$entity = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.DynamicTableEntity -ArgumentList $line.partitionkey, $line.rowKey
$colArray = $cols.split(",")
Write-Host "$($line.partitionkey), $($line.rowKey)" #output partitionkey and rowKey value
foreach($colName in $colArray)
{
Write-Host "$colName,$($line.$colName)" #output column name and value
$entity.Properties.Add($colName,$line.$colName)
}
if($number -le 100)
{
$batchOperation.InsertOrReplace($entity) # Changed code
}
else
{ $number =0
$result = $table.CloudTable.ExecuteBatch($batchOperation)
[Microsoft.WindowsAzure.Storage.Table.TableBatchOperation]$batchOperation = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.TableBatchOperation
}
}
if($batchOperation.Count -ne 0)
{
$result = $table.CloudTable.ExecuteBatch($batchOperation)
}
Note: For batch operation requires records in the CSV file with the same partition key value.
csv file example format
Test Result:
I have some existing data in Azure table storage.So when I deploy csv file ,the latest changes are deploying but the data which is existed in the azure table storage is not overwriting or old data is not deleting.for ex:I have 3 rows of data in azure storage existing,when I deploy csv file which is having 5 rows,the5 rows data is deploying and old data of 3 rows is not deleting.It should be overwrite but its not happening.Please help me. – Subscription Details:
$subscriptionName = "Tech Enabled Solutions"
$resourceGroupName = "abc"
$storageAccountName = "defghi"
$location = "North Central US"
$tableName = "TestTable"
# Get the storage key for the storage account
$storageAccountKey = "12345678990"
# Get a storage context
$ctx = New-AzureStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
$table = Get-AzureStorageTable -Name $tableName -Context $ctx -ErrorAction Ignore
#If the table exists, start deleting its entities.
if ($table -ne $null)
{
$table=Get-AzureStorageTableRowAll -table $table | Remove-AzureStorageTableRow -table $table -Context $ctx
}
The Get-AzureStorageTableRowAll command is in AzureRmStorageTable Module, so install it before call command.
Add this command to your script to install that module:
Install-PackageProvider -Name NuGet -Force -Scope CurrentUser
Install-Module -Name AzureRmStorageTable -Force -Verbose -Scope CurrentUser