I am doing batch uploads from a csv file to Azure table storage through a Powershell script and i have a command: $table.CloudTable.ExecuteBatch($batchOperation)
for which i'm getting the error mentioned in the header of the question of my post. I believe that "ExecuteBatch" is a method in the old AzureRm module and not the newer Az module which i am using, which is causing it to break. Is there a corresponding method in Az module for "ExecuteBatch"?
According to my test, if we use new Azure PowerShell module Az to manage Azure Table storage, we need to use the SDK Microsoft.Azure.Cosmos.Table.
So if ou want to use ExecuteBatch method, we need to use the command [Microsoft.Azure.Cosmos.Table.TableBatchOperation] $batchOperation = New-Object -TypeName Microsoft.Azure.Cosmos.Table.TableBatchOperation to create TableBatchOperation. For example:
Connect-AzAccount
$ResourceGroupName = "testfun06"
$StorageAccountName="testfun06bf01"
$TableName="People"
$keys=Get-AzStorageAccountKey -ResourceGroupName $ResourceGroupName -Name $StorageAccountName
$ctx = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $keys[0].Value
$table = Get-AzStorageTable -Name $TableName -Context $ctx
$e = New-Object Microsoft.Azure.Cosmos.Table.DynamicTableEntity("Jim","test")
$e1 = New-Object Microsoft.Azure.Cosmos.Table.DynamicTableEntity("Jim","test1")
[Microsoft.Azure.Cosmos.Table.TableBatchOperation] $batchOperation = New-Object -TypeName Microsoft.Azure.Cosmos.Table.TableBatchOperation
$batchOperation.InsertOrMerge($e)
$batchOperation.InsertOrMerge($e1)
$table.CloudTable.ExecuteBatch($batchOperation)
Related
I have received the below error while executing my code in azureCLI task of azuredevops:
New-CosmosDbContext : The term 'New-CosmosDbContext' is not recognized as a name of a cmdlet,
what is the step to solve this or any other suggestion for insert a document in cosmosDB Container using azure CLI task in Azure Devops?
$resourceGroupName = 'rg-****'
$accountName = 'cosmosserver-****'
$databaseName = 'DefaultSystemDB'
$OrganisationName = "SampleOrg"
$ContractName = 'SampleContract'
$TypeName = 'SampleType"
$currentDate = Get-Date -Format 'yyyy-MM-dd HH:mm:ss
write-Host $currentDate
$containerName = 'Organisation'
$partitionKey = '/id'
$cosmosDbContext = New-CosmosDbContext -Account $accountName -Database $databaseName -ResourceGroup $resourceGroupName
$Organisationid='$([Guid]::NewGuid().ToString())'
write-Host Organisationid: $Organisationid
$document = #"
{
"Name": '$OrganisationName',
"CreatedOn": '$currentDate',
"id": '$Organisationid'
}
"#
$partitionkey = '$Organisationid'
New-CosmosDbDocument -Context $cosmosDbContext -CollectionId $containerName -DocumentBody $document -PartitionKey $partitionkey
Please install the CosmosDB Module first. The module is not in the hosted agent by default. Please add the following at the beginning of your script. It may need some minutes to install it.
Install-Module -Name CosmosDB -force
Edit:
In your script, you are using the New-CosmosDbContext. So you should install the CosmosDB Module instead of Az.CosmosDB Module . Please refer https://github.com/PlagueHO/CosmosDB#installation.
If you want to use the Az.CosmosDB Module , please use New-AzCosmosDBxxxxxx . See https://learn.microsoft.com/en-us/powershell/module/az.cosmosdb/?view=azps-8.1.0 for more detail.
I am trying to create HDInsight cluster in Azure with Metastore using the Powershell script. But it is throwing BadRequest: RegionCapabilityNotAvailable,Region capability not available for region 'East US' error. But East US is a supported region for the HDInsight cluster. Please find my code below.
$storageAccountResourceGroupName = "hdi-rg"
$storageAccountName = "qwertyhdi"
#$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value
$storageContainer = "qwertyiopasdf-2020-05-03t08-30-23-118z"
# Cluster configuration info
$location = "East US"
$clusterResourceGroupName = "hdi-rg"
$clusterName = "qwertyiopasdf"
$username = "admin"
$password = ConvertTo-SecureString "password" -AsPlainText -Force
$clusterCreds = New-Object System.Management.Automation.PSCredential -ArgumentList ($username, $password)
# Hive metastore info
$hiveSqlServer = "server"
$hiveDb = "db123"
$sqlusername = "qwerty"
$sqlpassword = ConvertTo-SecureString "password" -AsPlainText -Force
$hiveCreds = New-Object System.Management.Automation.PSCredential -ArgumentList ($sqlusername, $sqlpassword)
New-AzStorageAccount `
-ResourceGroupName $storageAccountResourceGroupName `
-Name $storageAccountName `
-Location $location `
-SkuName Standard_LRS `
-Kind StorageV2 `
-EnableHttpsTrafficOnly 1
# Note: Storage account kind BlobStorage cannot be used as primary storage.
$storageAccountKey = (Get-AzStorageAccountKey `
-ResourceGroupName $storageAccountResourceGroupName `
-Name $storageAccountName)[0].Value
$defaultStorageContext = New-AzStorageContext `
-StorageAccountName $storageAccountName `
-StorageAccountKey $storageAccountKey
New-AzStorageContainer `
-Name $storageContainer `
-Context $defaultStorageContext #use the cluster name as the container name
$metastore = New-AzHDInsightClusterConfig | Add-AzHDInsightMetastore -SqlAzureServerName "$hiveSqlServer.database.windows.net" -DatabaseName $hiveDb -Credential $hiveCreds -MetastoreType HiveMetastore
New-AzHDInsightCluster -Location $location -ResourceGroupName $clusterResourceGroupName -ClusterName $clusterName -ClusterType Hadoop -OSType Windows -ClusterSizeInNodes 1 -HttpCredential $clusterCreds -DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" -DefaultStorageAccountKey $storageAccountKey -DefaultStorageContainer $storageContainer -Config $metastore
Is -OSType Windows still valid. I realize "Windows" is listed as an option from the PowerShell specs, but I think "linux" is the only value that will actually work.
You will receive this error message BadRequest: RegionCapabilityNotAvailable,Region capability not available for region 'East US', when underlying compute sku is not available for the selected region in that subscription.
I would request you to check if the resource are available from Azure Portal.
Azure Portal => Select your subscription => Usage + Quotas
Filter with => Microsoft.Compute and Region => East US
If you're unable to find a suitable SKU in that region/zone or an alternative region/zone that meets your business needs, submit a SKU request or Quota increase to Azure Support.
I have some existing data in Azure table storage.So when I deploy csv file ,the latest changes are deploying but the data which is existed in the azure table storage is not overwriting or old data is not deleting.for ex:I have 3 rows of data in azure storage existing,when I deploy csv file which is having 5 rows,the5 rows data is deploying and old data of 3 rows is not deleting.It should be overwrite but its not happening.Please help me. – Subscription Details:
$subscriptionName = "Tech Enabled Solutions"
$resourceGroupName = "abc"
$storageAccountName = "defghi"
$location = "North Central US"
$tableName = "TestTable"
# Get the storage key for the storage account
$storageAccountKey = "12345678990"
# Get a storage context
$ctx = New-AzureStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
$table = Get-AzureStorageTable -Name $tableName -Context $ctx -ErrorAction Ignore
#If the table exists, start deleting its entities.
if ($table -ne $null)
{
$table=Get-AzureStorageTableRowAll -table $table | Remove-AzureStorageTableRow -table $table -Context $ctx
}
The Get-AzureStorageTableRowAll command is in AzureRmStorageTable Module, so install it before call command.
Add this command to your script to install that module:
Install-PackageProvider -Name NuGet -Force -Scope CurrentUser
Install-Module -Name AzureRmStorageTable -Force -Verbose -Scope CurrentUser
I am generating SAS token from PowerShell but when I am trying to access that token from Azure Storage explorer, it is giving problem "Authentication Error. Signature fields not well formed."
here is the full Powershell command :-
Parameter required
$StorageAccountName = 'XXXXXX'
$ResourceGroup = 'remoteaccess'
$ContainerName = "vhds"
Powershell Cmd
$AzStrAct = Get-AzureRmStorageAccount -Name $StorageAccountName -ResourceGroupName $ResourceGroup
$AzStrKey = Get-AzureRmStorageAccountKey -Name $StorageAccountName -ResourceGroupName $ResourceGroup
$AzStrCtx = New-AzureStorageContext $StorageAccountName -StorageAccountKey $AzStrKey[0].Value
Get-AzureStorageContainer -Name $ContainerName -Context $AzStrCtx
$ContainerSASTokenURI = New-AzureStorageContainerSASToken -Name $ContainerName -Permission "rwdl" -StartTime "2017-04-12" -ExpiryTime "2017-04-16" -Context $AzStrCtx -FullUri
Write-Host "The SAS Token of container as below:"
$ContainerSASTokenURI
output
https://XXXXXXX.blob.core.windows.net/vhds?sv=2015-04-05&sr=c&sig=XXXXXXXXXXXXXXXXXXXXXXXX&st=2017-04-1
1T18%3A30%3A00Z&se=2017-04-15T18%3A30%3A00Z&sp=rwdl
I test with your script, and it works for me, my Azure storage explorer version is 0.8.12, I suggest you upgrade Azure storage explorer's version to 0.8.12.
is it possible create new table storage on azure with only use connection string by PowerShell?
Param (
[string]$StorageAccountName,
[string]$StorageAccountKey,
[string]$name
)
Import-Module Azure
$tableName = $name
$accountCredentials = New-Object "Microsoft.WindowsAzure.Storage.Auth.StorageCredentials" $StorageAccountName, $StorageAccountKey
$storageAccount = New-Object "Microsoft.WindowsAzure.Storage.CloudStorageAccount" $accountCredentials, $true
$tableClient = $storageAccount.CreateCloudTableClient()
$table = $tableClient.GetTableReference($tableName)
$table.CreateIfNotExists()
not like this way..
If you are using Azure PowerShell Cmdlets, there is a New-AzureStorageTable that you can use to create a new table.
Sample Code:
$storageContext = New-AzureStorageContext -StorageAccountName "accountname" -StorageAccountKey "accountkey"
New-AzureStorageTable -Name "TableName" -Context $storageContext