How to update Azure SQL datasource credentials of PowerBI dataset programmatically? - azure-devops

I have an Azure Devops Process which uses a Service Principle to deploy a Power BI report from a .pbix file and updates the data source to point at the production Azure SQL server.
The outcome is that the report and dataset are deployed and the connection is updated, however the CREDENTIALS are blank (Username and Password), so in order to make it usable, someone has to log on to Power BI service, open the Dataset and update the credentials, which means there is a manual step involved in our CI/CD process.
I need help with updating the source credentials for the new data source via code so that this manual process is not needed
Any suggestion will be of great help. Thank You.

Here is a sample PowerShell script, using Microsoft Power BI Cmdlets, which will patch the credentials of the dataset and commented at the end are few rows, which will refresh the dataset after that (uncomment them if needed). Just replace the x-es at the top with the actual values (workspace and dataset name, application ID, etc.).
<#
Patch the credentials of a published report/dataset, so it can be refreshed.
#>
# Fill these ###################################################
$tenantId = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" # Get from Azure AD -> Properties (https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/Properties)
$applictionId = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" # Get Application (client) ID from Azure AD -> App registrations (https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps)
$applicationSecret = "xxxxxxxxxxxxxxxx" # Create it from application's "Certificates & secrets" section
$workspaceName = "xxxxxxxx"
$reportName = "xxxxxxxx" # Actually it is dataset name
$sqlUserName = "xxxxxxxx"
$sqlUserPassword = "xxxxxxxxxx"
################################################################
Import-Module MicrosoftPowerBIMgmt
$SecuredApplicationSecret = ConvertTo-SecureString -String $applicationSecret -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($applictionId, $SecuredApplicationSecret)
$sp = Connect-PowerBIServiceAccount -ServicePrincipal -Tenant $tenantId -Credential $credential
$workspace = Get-PowerBIWorkspace -Name $workspaceName
$dataset = Get-PowerBIDataset -WorkspaceId $workspace.Id -Name $reportName
$workspaceId = $workspace.Id
$datasetId = $dataset.Id
$datasources = Get-PowerBIDatasource -WorkspaceId $workspaceId -DatasetId $datasetId
foreach($datasource in $datasources) {
$gatewayId = $datasource.gatewayId
$datasourceId = $datasource.datasourceId
$datasourePatchUrl = "gateways/$gatewayId/datasources/$datasourceId"
Write-Host "Patching credentials for $datasourceId"
# HTTP request body to patch datasource credentials
$userNameJson = "{""name"":""username"",""value"":""$sqlUserName""}"
$passwordJson = "{""name"":""password"",""value"":""$sqlUserPassword""}"
$patchBody = #{
"credentialDetails" = #{
"credentials" = "{""credentialData"":[ $userNameJson, $passwordJson ]}"
"credentialType" = "Basic"
"encryptedConnection" = "NotEncrypted"
"encryptionAlgorithm" = "None"
"privacyLevel" = "Organizational"
}
}
# Convert body contents to JSON
$patchBodyJson = ConvertTo-Json -InputObject $patchBody -Depth 6 -Compress
# Execute PATCH operation to set datasource credentials
Invoke-PowerBIRestMethod -Method Patch -Url $datasourePatchUrl -Body $patchBodyJson
}
#$datasetRefreshUrl = "groups/$workspaceId/datasets/$datasetId/refreshes"
#Write-Host "Refreshing..."
#Invoke-PowerBIRestMethod -Method Post -Url $datasetRefreshUrl

Related

Calls to AzureAD with AzureFunctions for powerShell do not return values ( not available)

When I make calls to AzureAD calls with Azure Functions Powershell I do not get values back. It works when using it locally with VS Code.
The modules are loaded:
This is what I did:
Host.json:
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[3.3.0, 4.0.0)"
},
"managedDependency": {
"enabled": true
}
requirements.psd1:
'Az' = '9.*'
'ExchangeOnlineManagement' = '3.*'
'AzureAD' = '2.*'
profile.ps1:
Import-Module -name 'AzureAD' -UseWindowsPowerShell
call:
# some prelude
$powerShellUserID = $Env:PowerShellUserID
$powerShellSecret = $Env:PowerShellSecret | ConvertTo-SecureString -AsPlainText -Force
$credential = new-Object System.Management.Automation.PSCredential($powerShellUserID, $powerShellSecret)
Connect-AzureAD -Credential $credential
$values = Get-AzureADGroup -top 10
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $values
what am I doing wrong
I was expecting the following
ObjectId DisplayName Description
00007188-....-422b-b619-.......... M_MSCRM_SQLAccessGroup PROD
0002bebb-....-4451-813d-.......... SAP_PF1_GlobalAdmin
000343da-....-4d23-a1fb-.......... m_IS_ATOS_HWUVN1350_Non-Admin Group for Read-Only access on HWUVN1350
0005337b-....-4306-952d-.......... g_UNITY_PC_MHD_QCD_RW Owner=d30\qcancil ancil, Q. (Qc)
00063e98-....-4204-befa-.......... g_IS_ATOS_DL_R2R_PC-WOM_W ATF:C3417347
0007370f-....-4f31-af84-.......... g_Unity_License_PowerApps SN automation[User]
0008d336-....-4509-8ae0-.......... m_IS_ATOS_SASVN101_TempAdmin Group for temporary administrative access on
000915f9-....-4fb7-a626-.......... G_FC_KLN_R_CARBOSULF administration of access rights performed by loacal IT support
000ae777-....-431f-8fd5-.......... m_IS_ATOS_SC_SSD_Administrators-AN ATF:C2884376
000b1f37-....-4f3b-9fcb-.......... m_IS_ATOS_VDCN022_AppSvc Domain Glocal Group for access of Service Accounts on

PowerShell REST DELETE from Azure Storage Account Table

im struggling with an REST Method to DELETE an Entry via PowerShell from an Azure Storage Account. Im authenticating with an SharedAccessSignature (SAS) (has rights for read, write and delete) to create entries but i dont get this to work to also DELETE entries. Has anyone created an PowerShell script to delete form Azure Storage Account Tables from PowerShell yet and could send me an code snippet on how to do this?
Im not using the PowerShell Module im using the "Invoke-WebRequest" CMDlet. Im new to REST APIs so maybe i just dont have the right idea? For the entry creation im using the URI in the Invoke-WebRequest call to give the SAS Token as authentication but changing the "-Method POST" to "-Method DELETE" does not worked.
Thanks for your help
To delete the table by using REST method, make use of below sample query if helpful:
Instead of using "Invoke-WebRequest", make use of "Invoke-RestMethod" like below
function DeleteTableEntity($TableName,$PartitionKey,$RowKey) {
$resource = "$tableName(PartitionKey='$PartitionKey',RowKey='$Rowkey')"
$table_url = "https://$storageAccount.table.core.windows.net/$resource"
$GMTTime = (Get-Date).ToUniversalTime().toString('R')
$stringToSign = "$GMTTime`n/$storageAccount/$resource"
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accesskey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($stringToSign))
$signature = [Convert]::ToBase64String($signature)
$headers = #{
'x-ms-date' = $GMTTime
Authorization = "SharedKeyLite " + $storageAccount + ":" + $signature
Accept = "application/json;odata=minimalmetadata"
'If-Match' = "*"
}
$item = Invoke-RestMethod -Method DELETE -Uri $table_url -Headers $headers -ContentType application/http
}
For more in detail, please refer below link:
Use Azure Table Storage via PowerShell and the Rest API - GCITS
Otherwise, you can install PowerShell module, and make use of below script like below:
$resourceGroup = 'ResourceGroupName'
$storageAccountName = 'StorageAccountName'
$storageAccount = Get-AzStorageAccount -ResourceGroupName $resourceGroup -Name $storageAccountName
$ctx = $storageAccount.Context
$tables = (Get-AzStorageTable -Context $Ctx).name
ForEach ($table in $tables) {
Remove-AzStorageTable –Name $table –Context $ctx -Force
}
For more in detail, please refer below link:
Delete all tables in Azure storage using powershell | Mike Says Meh.

Configuring powershell core from .net core to provision a one drive

I'm attempting to provision a one drive in a dotnet core app using powershell core. Running powershell I've been able to successfully provision a one drive from the powershell command line following the directions provided below:
https://learn.microsoft.com/en-us/onedrive/pre-provision-accounts
Running it programatically in .net core however it looks like it uses a separate powershell that's bundled into .net core 2.1
I believe the unsuccessful in app runs are due to the powershell bundled with core not being setup correctly, namely the first 3 steps in the link above:
1.Download the latest SharePoint Online Management Shell.
2.Download and install the SharePoint Online Client Components SDK.
3.Connect to SharePoint Online as a global admin or SharePoint admin in Office 365. To learn how, see Getting started with SharePoint Online Management Shell.
How do I set up the powershell that gets run by my application to mirror those steps above?
My code looks like this:
using System.IO;
using System.Management.Automation;
namespace PowerShellApp
{
class Program
{
public static int Main(string[] args)
{
using (PowerShell ps = PowerShell.Create(
{
ps.AddScript(File.ReadAllText(<scriptLocation>))
.Invoke();
}
}
return 0;
}
}
How do I achieve these steps when executing within a .net core application
The powershell script I"m running is below and also within the link above:
<#
.SYNOPSIS
This script adds an entry for each user specified in the input file
into the OneDrive provisioning queue
.DESCRIPTION
This script reads a text file with a line for each user.
Provide the User Principal Name of each user on a new line.
An entry will be made in the OneDrive provisioning queue for each
user up to 200 users.
.EXAMPLE
.\BulkEnqueueOneDriveSite.ps1 -SPOAdminUrl https://contoso- admin.sharepoint.com -InputfilePath C:\users.txt
.PARAMETER SPOAdminUrl
The URL for the SharePoint Admin center
https://contoso-admin.sharepoint.com
.PARAMETER InputFilePath
The path to the input file.
The file must contain 1 to 200 users
C:\users.txt
.NOTES
This script needs to be run by a global or SharePoint administrator in Office 365
This script will prompt for the username and password of the administrator
#>
param
(
#Must be SharePoint Administrator URL
[Parameter(Mandatory = $true)]
[ValidateNotNullOrEmpty()]
[string] $SPOAdminUrl,
[Parameter(Mandatory = $true)]
[ValidateNotNullOrEmpty()]
[string] $InputFilePath
)
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.R untime") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.U serProfiles") | Out-Null
$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SPOAdminUrl)
$Users = Get-Content -Path $InputFilePath
if ($Users.Count -eq 0 -or $Users.Count -gt 200)
{
Write-Host $("Unexpected user count: [{0}]" -f $Users.Count) - ForegroundColor Red
return
}
$web = $ctx.Web
Write-Host "Enter an admin username" -ForegroundColor Green
$username = Read-Host
Write-Host "Enter your password" -ForegroundColor Green
$password = Read-Host -AsSecureString
$ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username,$password )
$ctx.Load($web)
$ctx.ExecuteQuery()
$loader = [Microsoft.SharePoint.Client.UserProfiles.ProfileLoader]::GetProfileLoader($ctx)
$ctx.ExecuteQuery()
$loader.CreatePersonalSiteEnqueueBulk($Users)
$loader.Context.ExecuteQuery()
Write-Host "Script Completed"
I'm afraid SP Online management Shell has dependencies from .Net Framework and will not work with Core (check this).
From the other side that module seemed to be a wrapper on top of their REST API. So if you want to integrate it with Core app, you may try to replace it with HTTP requests. Check this documentation
Also, below is a base powershell script to work with those REST API endpoints. I tested this one on my site:
$baseUrl = "http://sharepoint.com/sites/yourSite/_api"
$cred = Get-Credential
# retreive digest
$r = Invoke-WebRequest -Uri "$baseUrl/contextinfo" -Method Post -Credential $cred -SessionVariable sp
$digest = ([xml]$r.content).GetContextWebInformation.FormDigestvalue
# calling endpoint
$endpoint = "sp.userprofiles.profileloader.getprofileloader/getuserprofile"
$head = #{
"Accept" = "application/json;odata=verbose"
"X-RequestDigest" = $digest
}
$re = Invoke-WebRequest -Uri "$baseUrl/$endpoint" -Headers $head -Method Post -WebSession $sp
Write-Host $re.Content
This is a snippet for createpersonalsiteenqueuebulk, but I can't test it since I'm not domain admin. Hope it will work for you
#--- sample 2 (didn't test it since I'm not domain admin). Might need separate session/digest
$endpoint2 = "https://<domain>-admin.sharepoint.com/_api/sp.userprofiles.profileloader.getprofileloader/createpersonalsiteenqueuebulk"
$head = #{
"Accept" = "application/json;odata=verbose"
"X-RequestDigest" = $digest
}
$body = "{ 'emailIDs': ['usera#domain.onmicrosoft.com', 'userb#domain.onmicrosoft.com'] }"
$re2 = Invoke-WebRequest -Uri "$endpoint2" -Headers $head -body $body -Method Post -WebSession $sp
Write-Host $re2.Content

Dynamic user credential generation with Powershell for Basic Auth Implementation

Requirement: Would like to have basic auth setup using custom php script.
Would like to :
Create user credentials dynamically
Create the cred php file with these credentials updated
Update the username & password to respective for Azure WebApp settings.
[ Note: FTPing the cred and auth files automatically if missing would be in upcoming post ]
Tools & Pre-requisites :
Azure Powershell 4.0+
Windows Powershell ISE
Knowledge of Storage Account ( Name, RG's )
Knowledge of WebApp for which backup is desired ( WebApp Name and RG )
Valid & Active Azure portal / AD login credentials
RESULT : On Azure Portal, The webapp in focus would have 'global_cred' variable key set with credentials generated stored as its value in application settings section as [username:password] format.
#######################
# Function to Generate dynamic 22char random string for password use
# Call custom function to write php constants for basic auth
Function writeFocusDomainCredFile{
param( $configObject, $hash, $farmprojectname )
#setting auth username:password
$configObject.authPasswordKey = (Get-RandomAlphanumericString -length 22 | Tee-Object -variable teeTime )
$hash['global_cred'] = [String]( $configObject.authUsernameKey + ':' + $configObject.authPasswordKey )
$prfilename = ( $configObject.ftpappdirectory + '\cred.php')
writeProjectBasicAuthCredOnNew -filename $prfilename -configObject $configObject
}
##########################
# Function to write the credentials to cred php file for basic auth use
# This file with other dependent files could be automatically ftp'd.
# Would share in another post
Function writeProjectBasicAuthCredOnNew{
param( $filename
,$configObject
)
writeDeployFileFiltersForDomain -ReportFileName $filename
Add-Content $filename ( "<?php" )
Add-Content $filename ("define('WPIZED_AUTH_USER', '" + $configObject.authUsernameKey + "');");
Add-Content $filename ("define('WPIZED_AUTH_PASS', '" + $configObject.authPasswordKey + "');");
}
###################################################################################################
# Configure Below as You prefer or desire #
$properties = #{
'ResourceName' = "AzureAppName";
'myResourceGroupName' = "{App Resource Group Name}";
'mySubscriptionName' = "{subscription name}";
'adminEmail' = "H.Bala#volunteering.com";
'ResourceGroupLocation' = "East US";
'authUsernameKey' = 'HBalaUsername'; #For this post, using fixed username as 'HBalaUsername'
'authPasswordKey' = '';
'PathFormatDate' = Get-Date -UFormat "%Y_%m_%d";
}
$configObject = New-Object –TypeName PSObject –Prop $properties
Write-Output $configObject
#Login cmdlet for active session
Login-AzureRmAccount
Get-AzureRmSubscription –SubscriptionName $configObject.mySubscriptionName | Select-AzureRmSubscription
(Get-AzureRmContext).Subscription
Select-AzureRMSubscription -SubscriptionName $configObject.mySubscriptionName
#Pull the Webapp details and configuration
$webApp = Get-AzureRMWebApp -ResourceGroupName $configObject.myResourceGroupName -Name $configObject.ResourceName
#Pull the Application Listing Environment / Configuration Variables
$appSettingList = $webApp.SiteConfig.AppSettings
$hash = #{}
ForEach ($kvp in $appSettingList) {
$hash[$kvp.Name] = $kvp.Value
}
writeFocusDomainCredFile -configObject $configObject -hash $hash
#[FTP Deploy Logic of this file and other basic auth or files shall cover in seperate topic ]
#[ Only setting the generated Credentials and saving to Application setting focused here ]
Set-AzureRMWebApp -ResourceGroupName $configObject.myResourceGroupName -Name $configObject.ResourceName -AppSettings $hash
Disclaimer: The intention is to share to another newbie who might find this helpful.

How to connect Azure Paas Database using Powershell with intergrated security

How can I connect to an Azure SQLDatabase (Paas Instance) using Powershell without a username & password but rather with Active Directory. I can connect with a username and password when I use invoke-sqlcmd as below
$Servername = "XXXXXX.database.windows.net"
$params = #{
Database = $databasename
ServerInstance = $ServerName
Username = $UserName
Password = $password
'OutputSqlErrors' = $true
Query = $QueryString
}
Invoke-Sqlcmd #params
I need the ability to do it with Active Directory. I have read a few articles on how to connect with active directory as long as you specify it on the connection string (Context of C#) Using ODBC connection or other data sources. Since Powershell is based on .Net; this should be do-able..
$ConnString = "DRIVER=ODBC Driver 13 for SQL Server;SERVER=xxxxx.database.windows.net;DATABASE=DBName;Trusted_Connection=Yes"
$Conn = new-object System.Data.Odbc.OdbcConnection($ConnString)
$conn.open()
$cmd = new-object System.Data.Odbc.OdbcCommand("select getdate() as X", $Conn)
$data = new-object System.Data.Odbc.OdbcDataAdapter($cmd)
$dt = new-object System.Data.DataColumn
$data.Fill($dt)
$Conn.Close()
#`server=Server;database=Database;Authentication=ActiveDirectoryIntegrated;`
The objective of trying to use Active Directory is that, within Paas, only AD Accounts can Add other AD accounts. I want to be able to add other accounts with method rather than logging in the Azure portal and adding an AD account manually.
You could use Azure AD account to login Azure SQL database(Paas) by using Azure Active Directory Authentication. More information please refer to this link.
Note: Local domian AD user does not support this.
You could use following script to login with Azure AD authentication.
#You admin Azure AD user name
$Username = "shuitest#*****.onmicrosoft.com"
$Password = "********"
$Database = "testg"
$Server = 'test.database.windows.net'
$Port = 1433
$cxnString = "Server=tcp:$Server,$Port;Database=$Database;Authentication=Active Directory Password;UID=$UserName;PWD=$Password;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;"
$query = "select count(*) from dbo.Authors"
$cxn = New-Object System.Data.SqlClient.SqlConnection($cxnString)
$cxn.Open()
$cmd = New-Object System.Data.SqlClient.SqlCommand($query, $cxn)
$cmd.CommandTimeout = 120
$cmd.ExecuteNonQuery()
$cxn.Close()
The following sample shows how to invoke Add-AzureAccount without the popup dialog:
$username = "someorgid#orgaccount.com"
$password = "Pa$$w0rd" | ConvertTo-SecureString -AsPlainText -Force
$credential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $username, $password
Add-AzureAccount -Credential $credential
AAD does not / can not parse Kerbs tokens. In order to get integrated auth to work from your desktop, you need to have ADFS (or similar) in your environment. That way your desktop will authenticate against AAD, redirect to ADFS, and your kerbs token will be recognized.
See https://learn.microsoft.com/en-us/azure/sql-database/sql-database-aad-authentication for more information.
this now works for me in the connection string instead of user and pass
Authentication=Active Directory Integrated;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;
Depending on your query you might have to use $cmd.ExecuteScalar() instead of $cmd.ExecuteNonQuery() for example
$query = 'select ##version'
$cxn = New-Object System.Data.SqlClient.SqlConnection($cxnString)
$cxn.Open()
$cmd = New-Object System.Data.SqlClient.SqlCommand($query, $cxn)
$cmd.CommandTimeout = 120
$cmd.ExecuteScalar()
$cxn.Close()