The PowerShell function Invoke-AzOperationalInsightsQuery docs, source allows me to pass arbitrary an KQL string to an App Insights resource.
Set-AzContext -Subscription "my-sub"
$workspaceName = "vl-loganalytics-workspace"
$workspaceRG = "vl-loganalytics"
$WorkspaceID = (Get-AzOperationalInsightsWorkspace -Name $workspaceName -ResourceGroupName $workspaceRG).CustomerID
$query = "print current_cluster_endpoint()"
$kqlQuery = Invoke-AzOperationalInsightsQuery -WorkspaceId $WorkspaceID -Query $query
$kqlQuery.Results
HT learningbydoing.cloud
Does an equivalent method exist for querying an Azure Data Explorer cluster directly? No public function is listed in the Az.Kusto module as of version 2.1.0, but perhaps there is a community module or blog post documenting an ad-hoc method for this?
Referencing Kusto .NET client libraries from PowerShell, this is possible with the below helper code after downloading & unzipping the Microsoft.Azure.Kusto.Tools NuGet package
$clusterUrl = 'https://help.kusto.windows.net;Fed=True'
# DatabaseName may be null, empty, or incorrect
$databaseName = 'Samples'
$query = 'print current_cluster_endpoint()'
$packagesRoot = Resolve-Path "tools\net6.0"
[System.Reflection.Assembly]::LoadFrom("$packagesRoot\Kusto.Data.dll")
$kcsb = New-Object Kusto.Data.KustoConnectionStringBuilder ($clusterUrl, $databaseName)
$queryProvider = [Kusto.Data.Net.Client.KustoClientFactory]::CreateCslQueryProvider($kcsb)
$crp = New-Object Kusto.Data.Common.ClientRequestProperties
$crp.ClientRequestId = "MyPowershellScript.ExecuteQuery." + [Guid]::NewGuid().ToString()
$crp.SetOption([Kusto.Data.Common.ClientRequestProperties]::OptionServerTimeout, [TimeSpan]::FromSeconds(30))
$reader = $queryProvider.ExecuteQuery($query, $crp)
$dataTable = [Kusto.Cloud.Platform.Data.ExtendedDataReader]::ToDataSet($reader).Tables[0]
$dataView = New-Object System.Data.DataView($dataTable)
$dataView
Note that $databaseName does not need to correspond to an existing database to establish a connection. This can cause errors if you typo a database name; or it can be helpful if the command you wish to execute does not require a database context.
Related
im struggling with an REST Method to DELETE an Entry via PowerShell from an Azure Storage Account. Im authenticating with an SharedAccessSignature (SAS) (has rights for read, write and delete) to create entries but i dont get this to work to also DELETE entries. Has anyone created an PowerShell script to delete form Azure Storage Account Tables from PowerShell yet and could send me an code snippet on how to do this?
Im not using the PowerShell Module im using the "Invoke-WebRequest" CMDlet. Im new to REST APIs so maybe i just dont have the right idea? For the entry creation im using the URI in the Invoke-WebRequest call to give the SAS Token as authentication but changing the "-Method POST" to "-Method DELETE" does not worked.
Thanks for your help
To delete the table by using REST method, make use of below sample query if helpful:
Instead of using "Invoke-WebRequest", make use of "Invoke-RestMethod" like below
function DeleteTableEntity($TableName,$PartitionKey,$RowKey) {
$resource = "$tableName(PartitionKey='$PartitionKey',RowKey='$Rowkey')"
$table_url = "https://$storageAccount.table.core.windows.net/$resource"
$GMTTime = (Get-Date).ToUniversalTime().toString('R')
$stringToSign = "$GMTTime`n/$storageAccount/$resource"
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accesskey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($stringToSign))
$signature = [Convert]::ToBase64String($signature)
$headers = #{
'x-ms-date' = $GMTTime
Authorization = "SharedKeyLite " + $storageAccount + ":" + $signature
Accept = "application/json;odata=minimalmetadata"
'If-Match' = "*"
}
$item = Invoke-RestMethod -Method DELETE -Uri $table_url -Headers $headers -ContentType application/http
}
For more in detail, please refer below link:
Use Azure Table Storage via PowerShell and the Rest API - GCITS
Otherwise, you can install PowerShell module, and make use of below script like below:
$resourceGroup = 'ResourceGroupName'
$storageAccountName = 'StorageAccountName'
$storageAccount = Get-AzStorageAccount -ResourceGroupName $resourceGroup -Name $storageAccountName
$ctx = $storageAccount.Context
$tables = (Get-AzStorageTable -Context $Ctx).name
ForEach ($table in $tables) {
Remove-AzStorageTable –Name $table –Context $ctx -Force
}
For more in detail, please refer below link:
Delete all tables in Azure storage using powershell | Mike Says Meh.
Trying to build a PowerShell script to connect to Analysis Services Tabular Model and pull the output of DMV queries(eg : SELECT * FROM $System.DBSchema_Tables)
Tried Below, but its fails, it seems there is something wrong with connection string or the way I am trying to connect:
$connectionString = "server=TabularServerName;database='ModelName';trusted_connection=true;";
$CubeQuery = "SELECT * FROM $System.DBSchema_Tables";
#SQL Connection - connection to SQL server
$sqlConnection = new-object System.Data.SqlClient.SqlConnection;
$sqlConnection.ConnectionString = $connectionString;
#SQL Command - set up the SQL call
$sqlCommand = New-Object System.Data.SqlClient.SqlCommand;
$sqlCommand.Connection = $sqlConnection;
$sqlCommand.CommandText = $CubeQuery;
#SQL Adapter - get the results using the SQL Command
$sqlAdapter = new-object System.Data.SqlClient.SqlDataAdapter
$sqlAdapter.SelectCommand = $sqlCommand
$dataSet = new-object System.Data.Dataset
$recordCount = $sqlAdapter.Fill($dataSet)
What are you not just using the SQLPS module or DBA Tools module?
There are of course other modules you can leverage:
Find-Module -Name '*sql*' | Format-Table -AutoSize
Find-Package -Name '*sql*' | Format-Table -AutoSize
Here is the stuff I've passed on to others messing with SQL.
Install the SQL Server PowerShell module
https://learn.microsoft.com/en-us/sql/powershell/download-sql-server-ps-module?view=sql-server-ver15
https://learn.microsoft.com/en-us/powershell/module/sqlps/?view=sqlserver-ps
https://learn.microsoft.com/en-us/sql/powershell/sql-server-powershell?view=sql-server-ver15
Then see:
Connecting PowerShell to SQL Server
As an overview the following is the list of options I will go over in
this article:
SQL Server PowerShell (SQLPS)
SQL Server Management Objects (SMO)
.NET (System.Data.SqlClient)
SQLPS
SQL Server PowerShell SQLPS is a utility that was first released
with SQL Server 2008, you may see this referenced in various ways. It
exists as a (1) utility and (2) as a PS module. The utility and module
are installed with the SQL Server Management tools from SQL Server
2008 and up. There are a few ways of connecting to SQL Server using
this utility, and each one has strengths and weaknesses.
SQLPS.exe
This is a utility that you should be able to open by typing
it in the run prompt (Start > Run). A second option, right-click a
node under Object Explorer, within SQL Server Management Studio
(SSMS), and select “Start PowerShell”. The SQLPS utility’s main access
point is using the provider “SQLSERVER:\” to browse SQL Server like a
file directory. With that, based on the node you open SQLPS from will
place you within that path of the provider. Under each “folder” you
are in for the provider offers properties to read or set, and some
methods to use for administration.
Get-ChildItem SQLSERVER:\SQL\LOCALHOST\SQL12\Databases | foreach { $_.RecoveryModel = "SIMPLE"; $_.Alter() }
SQLPS Module
Importing the SQLPS module into a PS session provides the
same access using the utility does but allows you to operate in the
PS version of the OS you operate under. In SQL Server 2008 and 2008 R2
you will load the SQLPS as a snap-in (Add-PSSnapin), then with SQL
Server 2012 and up it is imported (Import-Module).
# Loading SMO
Add-Type -AssemblyName "Microsoft.SqlServer.Smo,Version=11.0.0.0,Culture=neutral,PublicKeyToken=89845dcd8080cc91"
# Connecting with SMO
$srv = New-Object Microsoft.SqlServer.Management.Smo.Server “localhost\sql12”
$srv.Databases | select name
# .NET Framework
# Create a connection
$sqlConn = New-Object System.Data.SqlClient.SqlConnection
$sqlConn.ConnectionString = “Server=localhost\sql12;Integrated Security=true;Initial Catalog=master”
$sqlConn.Open()
# Create your command (the T-SQL that will be executed)
$sqlcmd = $sqlConn.CreateCommand()
<# or #>
$sqlcmd = New-Object System.Data.SqlClient.SqlCommand
$sqlcmd.Connection = $sqlConn
$query = “SELECT name, database_id FROM sys.databases”
$sqlcmd.CommandText = $query
# Create your data adapter (if you want to retrieve data)
$adp = New-Object System.Data.SqlClient.SqlDataAdapter $sqlcmd
# Create your dataset (the adapter fills this object)
$data = New-Object System.Data.DataSet
$adp.Fill($data) | Out-Null
# Retrieving Your Data
$data.Tables
<# or #>
$data.Tables[0]
Lastly:
USE POWERSHELL TO GET ALL THE MEASURES FROM A 2016 TABULAR CUBE
So, here’s the PowerShell script that will get the measures from a
cube (change the first three variables to fit your environment):
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.AnalysisServices.Tabular");
$tab = "YourSSASserver";
$dbId = "ID_or_DB";
$saveas = "C:\YourFolder\{0}.dax" -f $tab.Replace('\', '_');
$as = New-Object Microsoft.AnalysisServices.Tabular.Server;
$as.Connect($tab);
$db = $as.Databases[$dbId];
# in case you want to search by the name of the cube/db:
# $as.Databases.GetByName("DB Name");
$out = "";
foreach($t in $db.Model.Tables) {
foreach($M in $t.Measures) {
$out += "// Measure defined in table [" + $t.Name + "] //" + "`n";
$out += $M.Name + ":=" + $M.Expression + "`n";
}
}
$as.Disconnect();
$out = $out.Replace("`t"," "); # I prefer spaces over tabs :-)
$out.TrimEnd() | Out-File $saveas;
Below is what worked for me
$connectionString = $connectionString = “Provider=MSOLAP;Data Source=TabularServerName;”
$CubeQuery = 'SELECT * FROM $System.DBSchema_Tables';
$connection = New-Object -TypeName System.Data.OleDb.OleDbConnection
$connection.ConnectionString = $connectionString
$sqlCommand = $connection.CreateCommand()
$sqlCommand.CommandText = $CubeQuery;
#SQL Adapter - get the results using the SQL Command
$sqlAdapter = new-object System.Data.SqlClient.SqlDataAdapter
$sqlAdapter.SelectCommand = $sqlCommand
$dataSet = new-object System.Data.Dataset
$recordCount = $sqlAdapter.Fill($dataSet)```
I am attempting to loop through an invoke-sqlcmd for multiple AzureSQL databases via Azure Automation. The first item in the loop executes, but the all the rest fail with a:
Invoke-Sqlcmd : A network-related or instance-specific error occurred
while establishing a connection to SQL Server. The server was not
found or was not accessible. Verify that the instance name is correct
and that SQL Server is configured to allow remote connections.
(provider: Named Pipes Provider, error: 40 - Could not open a
connection to SQL Server)
I am guessing that I need to close the connection from the first invoke-sqlcmd before executing the next, but have not found a direct method to accomplish that with invoke-sqlcmd. Here is my loop:
param(
# Parameters to Pass to PowerShell Scripts
[parameter(Mandatory=$true)][String] $azureSQLServerName = "myazuresql",
[parameter(Mandatory=$true)][String] $azureSQLCred = "myazureautosqlcred"
)
# DB Name Array
$dbnamearray = #("database1","database2","database3")
$dbnamearray
# Datatable Name
$tabName = "RunbookTable"
#Create Table object
$table = New-Object system.Data.DataTable "$tabName"
#Define Columns
$col1 = New-Object system.Data.DataColumn dbname,([string])
#Add the Columns
$table.columns.add($col1)
# Add Row and Values for dname Column
ForEach ($db in $dbnamearray)
{
$row = $table.NewRow()
$row.dbname = $db
#Add the row to the table
$table.Rows.Add($row)
}
#Display the table
$table | format-table -AutoSize
# Loop through the datatable using the values per column
$table | ForEach-Object {
# Set loop variables as these are easier to pass then $_.
$azureSQLDatabaseName = $_.dbname
# Execute SQL Query Against Azure SQL
$azureSQLServerName = $azureSQLServerName + ".database.windows.net"
$Cred = Get-AutomationPSCredential -Name $azureSQLCred
$SQLOutput = $(Invoke-Sqlcmd -ServerInstance $azureSQLServerName -Username $Cred.UserName -Password $Cred.GetNetworkCredential().Password -Database $azureSQLDatabaseName -Query "SELECT * FROM INFORMATION_SCHEMA.TABLES " -QueryTimeout 65535 -ConnectionTimeout 60 -Verbose) 4>&1
Write-Output $SQLOutput
}
You can try making each connection as a powershell job. This solved a very similar issue I had some time ago. Send-MailMessage closes every 2nd connection when using attachments If you want to read an explanation. Basically, if you're unable to use a .Close() method, you can force connections to close by terminating the entire session for each run. In an ideal world the cmdlet would handle all this for you, but not everything was created perfectly.
# Loop through the datatable using the values per column
$table | ForEach-Object {
# Set loop variables as these are easier to pass then $_.
$azureSQLDatabaseName = $_.dbname
# Execute SQL Query Against Azure SQL
$azureSQLServerName = $azureSQLServerName + ".database.windows.net"
$Cred = Get-AutomationPSCredential -Name $azureSQLCred
# Pass in the needed parameters via -ArgumentList and start the job.
Start-Job -ScriptBlock { Write-Output $(Invoke-Sqlcmd -ServerInstance $args[0] -Username $args[1].UserName -Password $args[1].GetNetworkCredential().Password -Database $args[0] -Query "SELECT * FROM INFORMATION_SCHEMA.TABLES " -QueryTimeout 65535 -ConnectionTimeout 60 -Verbose) 4>&1 } -ArgumentList $azureSQLServerName, $Cred | Wait-Job | Receive-Job
}
This is untested since I do not have a server to connect to, but perhaps with a bit of work you can make something out of it.
I faced the same issue previously while doing something with the database of azure sql. You can try this
1. Create Automation Account
New-AzureRmAutomationAccount -ResourceGroupName $resourceGroupName -Name $automationAccountName -Location $location
2. Set the Automation account to work with
Set-AzureRmAutomationAccount -Name $automationAccountName -ResourceGroupName $resourceGroupName
3. Create / Import a Runbook
Here we already have a runbook ready so we import it. Here's the runbook code
workflow runbookNameValue
{
inlinescript
{
$MasterDatabaseConnection = New-Object System.Data.SqlClient.SqlConnection
$MasterDatabaseConnection.ConnectionString = "ConnectionStringValue"
# Open connection to Master DB
$MasterDatabaseConnection.Open()
# Create command
$MasterDatabaseCommand = New-Object System.Data.SqlClient.SqlCommand
$MasterDatabaseCommand.Connection = $MasterDatabaseConnection
$MasterDatabaseCommand.CommandText = "Exec stored procedure"
# Execute the query
$MasterDatabaseCommand.ExecuteNonQuery()
# Close connection to Master DB
$MasterDatabaseConnection.Close()
}
}
4. Importing
Import-AzureRMAutomationRunbook -Name $runBookName -Path $scriptPath -ResourceGroupName $resourceGroupName -AutomationAccountName $automationAccountName -Type PowerShell
I hope this helps. Instead of using Invoke-Sqlcmd use the $MasterDatabaseCommand.ExecuteNonQuery() like i've provided in the runbook. It will work
It seems that you append .database.windows.net to the server name inside the loop. I guess that's why it works for the first iteration only.
Just move this line:
$azureSQLServerName = $azureSQLServerName + ".database.windows.net"
before this line:
$table | ForEach-Object {
anyone know how to rewrite below code, so it can invoke sql script. For instance, instead of put 'SELECT ##SERVERNAME AS ServerName' in the powershell script, I would like to put it into sql file.
$Path = "D:\AdminStuff\PowerShell\Password\Password.txt"
$uid = 'sa'
$pwd = Get-Content D:\AdminStuff\PowerShell\Password\Password.txt | ConvertTo-SecureString
$pwd.MakeReadOnly()
$creds = New-Object System.Data.SqlClient.SqlCredential($uid,$pwd)
$con = New-Object System.Data.SqlClient.SqlConnection
$con.ConnectionString = "Server=SLB-CLMFZ52;Database=master;"
$con.Credential = $creds
$con.Open()
$sql = "SELECT ##SERVERNAME AS ServerName"
$cmd = New-Object System.Data.SqlClient.SqlCommand($sql,$con)
$rdr = $cmd.ExecuteReader()
while($rdr.Read())
{
$rdr["ServerName"].ToString()
}
$con.Close()
While you can use the SqlCredential, SqlConnection, and SqlCommand .NET classes as you show in your question, there are simpler alternatives. It is a lot less work, for example, to use the Invoke-Sqlcmd cmdlet instead. Invoke-Sqlcmd is essentially the venerable sqlcmd utility with a PowerShell disguise. Thus, assuming your query was in a file myScript.sql, you can just run this--the InputFile parameter provides the means to store your SQL in a separate file as you requested:
$InvokeParams = #{
Server = 'SLB-CLMFZ52'
Database = 'Master'
Username = 'sa'
Password = 'xyz'
InputFile = 'myScript.sql'
}
Invoke-SqlCmd #InvokeParams
Now the obvious problem with that, though, is the password must be in plain text. (Of course, the password in the OP was in plain text in a file, as well. :-) Unfortunately Invoke-Sqlcmd does not work with PowerShell credential objects, which would have made it a lot more secure. The only reasonable workaround to get some security is to use Windows Authentication instead of SQL authentication. Then you could, for example, omit the username and password, and the query will be invoked with secure credentials:
$InvokeParams = #{
Server = 'SLB-CLMFZ52'
Database = 'Master'
InputFile = 'myScript.sql'
}
Invoke-SqlCmd #InvokeParams
To use the cmdlet just Import-Module sqlps. For the basics of Invoke-SqlCmd take a look at TechNet and for a more in-depth treatment, including the vagaries of sqlps, see part 1 of my Practical PowerShell for SQL Server Developers and DBAs article.
What about something simple like this:
$sql=get-content $filename
I'm trying to develop a powershell script to help with AD Group Membership management. We have a handful of large groups (30k-60k+ objects) that we want to update with data from another system.
The script loads the objects that should be in the group from a text file. Each object then has to located in AD using a System.DirectoryServices.DirectorySearcher. After that each object is added to the group membership.
The script spends some 80% of its time looking up each object, is there a bulk way to find objects in AD with powershell?
Thanks!
This is the fast way to query AD that I found in my experience, you need to change the query to find specific objects, in this code you'll find all user/person object in $objRecordSet.
$Ads_Scope_SubTree = 2
$objConnection = new-Object -com "ADODB.Connection"
$objCommand = new-Object -com "ADODB.Command"
$objConnection.Provider = "ADsDSOObject"
$objConnection.Open( "Active Directory Provider")
$objCommand.ActiveConnection = $objConnection
$objCommand.Properties.Item("Page Size").value = 1000
$objCommand.Properties.item("Searchscope").value = $Ads_Scope_SubTree
$objCommand.CommandText = "Select Name From 'LDAP://DC = int, DC= my, DC = local' Where objectCategory = 'Person'"
$objRecordSet = $objCommand.Execute()
$objRecordSet.RecordCount
More info here
You perhaps can try System.DirectoryServices.Protocols (S.DS.P) the native (non managed) version is quite efficient.
Here is a PowerShell starting script :
# ADDP-Connect.PS1
Clear-Host
# Add the needed assemblies
Add-Type -AssemblyName System.DirectoryServices.Protocols
# Connexion
$serverName = "WM2008R2ENT"
$ADDPConnect = New-Object System.DirectoryServices.Protocols.LdapConnection $serverName
$userName = "JPB"
$pwd = "PWD"
$domain = "Dom"
$ADDPConnect.Credential = New-Object system.Net.NetworkCredential -ArgumentList $userName,$pwd,$domain
# Create a searcher
$searchTargetOU = "dc=dom,dc=fr"
$searchFilter = "(samAccountName=user1)"
$searchScope = [System.DirectoryServices.Protocols.SearchScope]::Subtree
$searchAttrList = $null
foreach($user in "user1","user2","user3")
{
$searchFilter = "(samAccountName=$user)"
$searchRequest = New-Object System.DirectoryServices.Protocols.SearchRequest -ArgumentList $searchTargetOU,$searchFilter,$searchScope,$searchAttrList
$searchResponse = $ADDPConnect.SendRequest($searchRequest)
foreach($searchEntries in $searchResponse.Entries)
{
$searchEntries.DistinguishedName
}
}
If you start seeing timeout issues then set the timeout parameter appropriately like shown below
$ADDPConnect = New-Object System.DirectoryServices.Protocols.LdapConnection $serverName
$ADDPConnect.Timeout = "1000"
The below can help if you see timeout issues during execution
$ADDPConnect = New-Object System.DirectoryServices.Protocols.LdapConnection $serverName
$ADDPConnect.Timeout = "1000"