I have an SQL database in Azure with always encryption enabled (master key in Azure key vault) to encrypt the data in a particular table. Azure DB is on version Azure SQL Database v12.
Somehow, I am not able to deploy my dacpac DB project into it using Azure DevOps release pipeline. I am using 'Azure SQL Database deployment' task version 1.*
During the execution of the release pipeline, I am getting an error message “The element MPCMK1 is not supported in Microsoft Azure SQL Database v12.” MPCMK1 is the column master key already created in the target server (stored in key azure vault). Other than the deployment, the functionalities such as encryption/decryption and all working fine. I am not understanding why it says not supported in this version.
I added publish profile and configured to skip deploying column master keys, column encryption keys etc but still I am getting this message on the key that is in the target server.
I removed the key from the target server just for testing, and this error went away. But that doesn't help me as removing the key will require to remove encryptions from the tables and all. But this was just to make sure the error is on the target but not on the source.
Also tried the arguments /p:AllowIncompatiblePlatform=True /p:BlockOnPossibleDataLoss=false but no luck.
It would be great if someone can help me on this.
Related
This relates to the documentation that is available at the link below.
https://learn.microsoft.com/en-us/azure/devops/migrate/migration-overview?view=azure-devops
What would be the minimum role for a user to complete the migration successfully and without any permissions issues? That is my question.
For example, the user must have the what kind of necessary roles and permissions on both the Azure Devops server and the Azure Devops Services.
According to the Data Migration Utility Guide, the user who uses this tool must possess the following:
SQL Server's TFSEXECROLE role, and
Access rights to the TFS collection and configuration databases.
My understanding for example :-
Azure DevOps Server:
If we add the user to the Team Foundation Administrators group on the Azure DevOps Server, does the role fulfill.
Azure Devops Services: If we assign the same user who performs migration as an Azure DevOps Administrator mention in the below image on the Azure devops services, does the below role fulfill.
"Azure Devops Administrator"
Also, it would be useful if you could specify the maximum size limit of the. Dacpac backup file that the Data Migration Tool supports (i.e. the maximum size of the project collection backup) in order for the migration to go properly.
What permissions does the same user that runs the data migration tool need in SQL server to perform the command SqlPackage.exe?
I would thank you in advance for the help. It would help us to understand the better usage of the Data Migration Tool.
Many Thanks..!
Best Regards
I'm trying to deploy a dacpac to an Azure Sql Database with Always encrypted enabled. The Devops agent is running in a self-hosted VM with sqlpackage.exe version 19 with build 16.0.5400.1 installed on it.
I've been able to trace down the issues by adding /diagnostics as an argument to the task and the exception that is raised is:
Unexpected exception executing KeyVault extension 'Object reference not set to an instance of an object.' at Microsoft.SqlServer.Dac.KeyVault.DacKeyVaultAuthenticator.Validate(IList`1 keyVaultUrls, CancellationToken cancelToken)
Anybody have a suggestion on how to solve this?
Please check below points
Microsoft.SqlServer.Dac.KeyVault.DacKeyVaultService Provides a service for discovering and configuring a Microsoft.SqlServer.Dac.KeyVault.KeyVaultAuthenticator to handle key vault access requests.
These requests will occur during deployment if an encrypted table is being altered. It also supports initialization of general key vault support in an application
If you store your column master keys in a key vault and you are using access policies for authorization:
Your application's identity needs the following access policy permissions on the key vault: get, unwrapKey, and verify.
A user managing keys for Always Encrypted needs the following access policy permissions on the key vault: create, get, list, sign, unwrapKey, wrapKey, verify.
SEE Create & store column master keys for Always Encrypted - SQL Server | Microsoft Docs
3.
To publish DAC package if Always Encrypted is set up in the DACPAC
or/and in the target database, you might need some or all of the below
permissions, depending on the differences between the schema in the
DACPAC and the target database schema.
ALTER ANY COLUMN MASTER KEY, ALTER ANY COLUMN ENCRYPTION KEY, VIEW ANY COLUMN > MASTER KEY DEFINITION, VIEW ANY COLUMN ENCRYPTION KEY
DEFINITION
we need to enable that Azure virtual machine check box
References:
Configure column encryption using Always Encrypted with a DAC package - SQL Server | Microsoft Docs
azure-sql-advanced-deployment-part4.
KeyVaultAuthenticator.Validate(IList, CancellationToken) >> Microsoft.SqlServer.Dac.KeyVault Namespace | Microsoft Docs
I managed to find a solution. I downgraded the sqlpackage.exe version. If I understand it correctly apparently version 19 seems to be targeted for SQL Server compatibility level 160 which is shipped with SQL Server 2022. When using version 18 it seems to be working with the current 150 that my Azure DB is set to.
I have created a devops project from azure portal.
I choose an ASP.Net Core application, SQL Database and Virtual Machine option.
I have 2 variables in azure devops release pipeline: dbpassword and vmpassword.
I need to connect to my sql database, and to my vm by hand.
How can i reveal the value of this 2 variables ? They are not shown (asterisks)
Thanks a lot
Assuming from the context clues that you used the Azure DevOps starter apps
There is documentation for gaining access to the SQL database here. You will need to reset the password.
I don't see it explicitly in the docs for starter apps but you can reset the VM password in the portal.
In both cases make sure you update those values back in your pipeline.
If you want to get any asterisks value add simple inline powershell task with:
$dbpassword.ToCharArray() -join " "
Copy and remove spaces, not best sollution, but should do the job
I am trying to configure Azure Key Vault and setup Managed Identities for use in CI/CD pipeline for Azure Dev Ops.
I have looked around in MSDN documentation but I only specific links for use with Azure SQL and we are using Azure SQL Managed Instances.
If I did not make any misunderstand, in fact, you want to use Managed Identity work with Azure SQL Managed Instance? If this, unfortunately to say, the Managed Identity could not work with Azure SQL Managed Instance. Please see this doc: Services that support managed identities for Azure resources. It list all of the Azure services name which support work with the Managed identities in great detail.
You can see for SQL database, it only support the integration with Azure SQL instead of Azure SQL Managed Instance. That's why you only see the doc link for the usage with Azure SQL.
Until now, the Azure SQL managed instance only support two authentication method:
SQL Authentication:
This authentication method uses a username and password.
Azure Active Directory Authentication:
This authentication method uses identities managed by Azure Active
Directory and is supported for managed and integrated domains. Use
Active Directory authentication (integrated security) whenever
possible.
You can refer to this thread: Managed Identity with Azure SQL Managed Instance?. In this thread, out engineer provided some work around if you trying to configure the app with Managed Identity.
We are building an ASP.net application using Entity Framework core with Azure DevOps and targeting IIS with MSSQL server 2016.
Our test server is a local Windows 2016 machine containing both IIS and the SQL server instance.
I have successfully written a deployment workflow enabling continuous deployment on that test server of the code itself but I can't find any way to deploy the database. The first version was targeting asp.net core 2.0 so I could use the command-line to start the application outside of IIS and trigger a database update but when we switched to 2.2 and in-process IIS deployment, we apparently lost that capability.
I would like to integrate the deployment of the database to the server in the deployment pipeline, including the DB creation, but I can't find any way to do so using Azure Devops: I can target an Azure SQL instance but, unless I'm missing something, not a local one.
So:
How can I manually create and populate the database using an ASP.NET core 2.2 in-process application on a machine with no SDK installed?
What do I need to add to the Azure DevOps pipeline to deploy the database to a local MSSQL server database
EDIT:
For deploying on local, I followed below two steps:
1. Create Database Script
You can create normal Create Database script which creates the database in the local database instance.
2. Apply Migrations
Create simple console application which runs the create database script first and then applies the migrations.
myDbContext.Database.Migrate();
That's how I got it working.
Previous Contents about Publishing DB to Azure:
You need "Azure SQL Publish" task.
Pre requisite is you should have Azure SQL Database already created.
Steps:
Step1 : Command To generate migration script in build pipeline
Create a command line task to generate the migration script:
dotnet ef migrations script -i -o %BUILD_ARTIFACTSTAGINGDIRECTORY%\migrate.sql --project EfMigrationApp.Database\EfMigrationApp.Database.csproj --startup-project EfMigrationApp\EfMigrationApp.csproj -i -o %BUILD_ARTIFACTSTAGINGDIRECTORY%\migrate.sql
Step 2: Azure SQL Publish in release pipeline
Action: Publish
Type: SQL script file
Sql script should be as below:
$(System.ArtifactsDirectory)/_$(Build.DefinitionName)/drop/migrate.sql
Refer this article for setting up CI pipeline.
Refer this article for setting up CD pipeline.