I am trying to create a solution with Azure DevOps. I need to create a connection to an Azure SQL Database using an Azure Powershell Task in a Release/Pipeline.
On my machine I was able to achieve this using the following
function new-connection($ServerIsntance, $DatabaseName)
{
$obj = New-Object System.Data.SQLClient.SQLCommand
$obj.Connection = New-Object System.Data.SQLClient.SQLConnection
$obj.Connection.ConnectionString = "Server=tcp:$($ServerInstance),1433;Initial Catalog=$($DatabaseName);Persist Security Info=False;User Id=<username>;Password=<password>;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Authentication=Active Directory Password"
$obj.Connection.Open()
.....
}
However when I use this code in Azure DevOps I get the following exception: ##[error]Exception setting "ConnectionString": "Keyword not supported: 'authentication'."
The database that I am trying to use is an AzureSQL one, so I can not use Integrated Security = true as it is not supported. Do you have an idea of how should I face this issue?
Test with the same script, it seems that the cause of this issue is agent type.
When I use the Windows agent(e.g. vs2017-win2016), the script could work as expected.
But when I test with Linux agent (e.g. ubuntu 16.04), I could get the same issue.
So you could check the agent type (windows or linux).
You could try to use the Windows Microsoft-hosted agent (e.g. vs2017-win2016 , windows-2019) .
https://learn.microsoft.com/en-us/sql/connect/jdbc/connecting-using-azure-active-directory-authentication?view=sql-server-ver15
Does your code include spaces between Active Directory Password? If so try removing them since the doc does not have any spaces included.
Related
My colleagues and I are facing an issue when trying to run my databricks notebook in Azure Data Factory and the error is coming from MLFlow.
The command that is failing is the following:
# Take the parent notebook path to use as path for the experiment
context = json.loads(dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson())
nb_base_path = context['extraContext']['notebook_path'][:-len("00_training_and_validation")]
experiment_path = nb_base_path + 'trainings'
mlflow.set_experiment(experiment_path)
experiment = mlflow.get_experiment_by_name(experiment_path)
experiment_id = experiment.experiment_id
run = mlflow.start_run(experiment_id=experiment_id, run_name=f"run_{datetime.now().strftime('%Y-%m-%d_%H-%M-%S')}")
And the error that is throwing is:
An exception was thrown from a UDF: 'mlflow.exceptions.RestException: INVALID_PARAMETER_VALUE: No experiment ID was specified. An experiment ID must be specified in Databricks Jobs and when logging to the MLflow server from outside the Databricks workspace. If using the Python fluent API, you can set an active experiment under which to create runs by calling mlflow.set_experiment("/path/to/experiment/in/workspace") at the start of your program.', from , line 32.
The pipeline just runs the notebook from ADF, it does not have any other step and the cluster we are using is type 7.3 ML.
Could you please help us?
Thank you in advance!
I think you need to set artifact URI and specify experiment ID (if in the artifact directory has much experiment ID
Reference: https://www.mlflow.org/docs/latest/tracking.html#how-runs-and-artifacts-are-recorded
I'm new to using Azure release pipelines and have been fighting issues trying to deploy a database project to a new Azure SQL database. Currently the pipeline is giving me the following error...
TargetConnectionString argument cannot be used in conjunction with any other Target database arguments
I've tried deploying with and without the TargetConnectionString included in my publish profile. Any suggestions or something else to try? I'm out of ideas.
TargetConnectionString
Specifies a valid SQL Server/Azure connection string to the target database. If this parameter is specified it shall be used exclusively of all other target parameters. (short form /tcs)
So please remove all other TargetXXX arguments.
(if you don't have them can you show what arguments you have inline and in publish profile - of course without data)
To create a directory under and add a web app to IIS, I put this script in Azure DevOps Server Deployment Group job Powershell task:
New-Item -Path "C:\inetpub\wwwroot" -Name "MyNewApp" -ItemType "directory" `
C:\Windows\system32\inetsrv\appcmd.exe add app `
/site.name:"Default Web Site" `
/path:"/MyNewApp" `
/physicalPath:"C:\inetpub\wwwroot\MyNewApp"`
When this task runs it throws an error: (timestamps removed for clarity)
APP object "Default Web Site/MyNewApp" changed
ERROR ( hresult:80090016, message:Failed to commit configuration changes.
Keyset does not exist )
After executing the above code from a Powershell console while logged in as either myself or the service account, the following messages are returned:
APP object "Default Web Site/MyNewApp" added
VDIR object "Default Web Site/MyNewApp" added
I can confirm in IIS Manager that the directory and application was created.
I cannot reproduce the error in a console. It is only when it is running as the Release Agent and initiated through the DevOps Server web interface that this error occurs.
I have tried the following:
Keyset Issue - https://support.microsoft.com/en-us/help/977754/keyset-does-not-exist-error-message-when-you-try-to-change-the-identit for both the Local Service and service accounts.
Up-to Modify permissions on C:\inetpub\wwwroot for the service account
Adding the service account to IIS Manager Permissions
Update 1:
I determined the "Keyset does not exist" error was coming from trying to use the DefaultAppPool application pool as part of the setup. This application pool was not setup with the service account but was still set with default Application Pool Identity.
However, after making this change I have a new error:
APP object "Default Web Site/MyNewApp" changed
ERROR ( hresult:80070057, message:Failed to commit configuration changes.
The parameter is incorrect. )
...
Process 'appcmd.exe' exited with code '87'.
Any guidance is greatly appreciated.
Update 2:
I logged an issue with Microsoft community that is currently being looked at. You can see it here.
You could also look at the .net implementation with the servermanager:
https://learn.microsoft.com/en-us/iis/manage/scripting/how-to-use-microsoftwebadministration
https://learn.microsoft.com/en-us/dotnet/api/microsoft.web.administration.servermanager?view=iis-dotnet
Should not be very hard to use this in powershell.
I'm trying to create a script to set / update appsettings in an Azure web app slot using powershell. Using the examples in Adding an App Settings to existing Azure Web Application using Azure Power Shell it works.
My problem is that I want "Slot setting" to be true. In all the examples I've found and in resources.azure.com, the settings are always name/value pairs, with no property to specify the value as "Slot setting".
Is this even possible to script?
Thanks.
Yes when using the Set-AzureWebsite command you can do
Set-AzureWebsite -SlotStickyAppSettingNames #("setting name")
I struggle to (dynamically) start the Jenkins slave agent from my dedicated slave machine (Windows 2012 R2 server).
The Jenkins master (ver. 1.617 - which I can upgrade if necessary, but not downgrade [before ver. 1.498 no credentials were required]) is on a Windows 2012 R2 server.
Security is enabled and configured via the Active Directory plugin and Project-based Matrix Authorization Strategy.
Because of the Active Directory involved, I cannot simply add a system user to authenticate with (via -jnlpCredentials username:password or -jnlpCredentials username:apitoken). As a workaround I am using my Jenkins service user for that, but I don't like it's API-Token lying around hard-coded in the script.
I am trying to use the alternative -secret secretKey, but that secretKey is randomly created when a slave node is registered on the master.
Since I am using the Azure Slave Plugin, the slave nodes and the associated virtual machines are created for me.
The virtual machines are created from a pre-defined image, that I can change in whatever way necessary.
In this pre-defined image I have a PowerShell script executed on start-up. It is derived from the sample given here. It doesn't have to be PowerShell, any other way would be okay as well.
Set-ExecutionPolicy Unrestricted
# base url to Jenkins master
$jenkinsserverurl = "https://jenkins.mycompany.com/"
# the azure-slave-plugin is creating VMs with names like 'Azure0807150842'
$vmname = (Get-Culture).TextInfo.ToTitleCase($env:computername.tolower())
# authenticate with Jenkins service user + API-token - since we don't know the '-secret'
$apiToken="jenkins_user:1234abcdefab56c7d890de1f2a345b67"
Write-Output "Downloading jenkins slave jar "
# in order to avoid updating it manually for Jenkins master updates
$slaveJarSource = $jenkinsserverurl + "jnlpJars/slave.jar"
$slaveJarLocal = "C:\jenkins_home\slave.jar"
$wc = New-Object System.Net.WebClient
$wc.DownloadFile($slaveJarSource, $slaveJarLocal)
Write-Output "Executing slave process "
$jnlpSource = $jenkinsserverurl+"computer/" + $vmname + "/slave-agent.jnlp"
# expect java.exe in the PATH, and use -noCertificateCheck to skip SSL validation
& java -jar $slaveJarLocal -jnlpCredentials $apiToken -jnlpUrl $jnlpSource -noCertificateCheck
Downloading the JNLP file and reading the contained secret is no option, since I need proper HTTP authentication at the Jenkins master for that as well.
Write-Output "Downloading jenkins slave jnlp "
$jnlpSource = $jenkinsserverurl+"computer/" + $vmname + "/slave-agent.jnlp"
$jnlpLocal = "C:\jenkins_home\slave-agent.jnlp"
$wc = New-Object System.Net.WebClient
$wc.DownloadFile($jnlpSource, $jnlpLocal)
Write-Output "Extracting secret from jenkins slave jnlp "
[xml]$jnlpFile = Get-Content $jnlpLocal
# the first argument in the generated JNLP contains the secret
$secret = Select-Xml "//jnlp/application-desc/argument[1]/text()" $jnlpFile
How can I get my hands on the generated secret (without disabling the security), or
What kind of credentials can I use instead (without using an actual user - such as my own or the Jenkins service user)?
In an ideal world, the plugin creating the slave node and its VM, would login to the created VM and execute a script similar to the one in my question - with the addition of the injected Jenkins server url, VM name, and generated secret. Since that is not the case for the current Azure Slave Plugin version, I am stuck with my workaround script - using my existing Jenkins service user.
I use this to let the plugin create a bigger/faster VM on the fly, which is only used for a daily test run and is shutdown automatically the rest of the time (and therefore causes no costs when unused).
If someone is interested, that is the setup I ended up with:
Generalized Azure VM Image (Windows 2012 R2, with JDK, Maven, Git
installed). Via NSSM, I installed the PowerShell script (which
starts the slave agent) as a Windows Service, to be executed
automatically at machine bootup (same as in the question):
Set-ExecutionPolicy Unrestricted
# base url to Jenkins master
$jenkinsserverurl = "https://jenkins.mycompany.com/"
# the azure-slave-plugin is creating VMs with names like 'Azure0807150842'
$vmname = (Get-Culture).TextInfo.ToTitleCase($env:computername.tolower())
# authenticate with Jenkins service user + API-token - since we don't know the '-secret'
$apiToken="jenkins_user:1234abcdefab56c7d890de1f2a345b67"
Write-Output "Downloading jenkins slave jar "
# in order to avoid updating it manually for Jenkins master updates
$slaveJarSource = $jenkinsserverurl + "jnlpJars/slave.jar"
$slaveJarLocal = "C:\jenkins_home\slave.jar"
$wc = New-Object System.Net.WebClient
$wc.DownloadFile($slaveJarSource, $slaveJarLocal)
Write-Output "Executing slave process "
$jnlpSource = $jenkinsserverurl+"computer/" + $vmname + "/slave-agent.jnlp"
# expect java.exe in the PATH, and use -noCertificateCheck to skip SSL validation
& java -jar $slaveJarLocal -jnlpCredentials $apiToken -jnlpUrl $jnlpSource -noCertificateCheck
Jenkins master with Azure Slave Plugin installed and configured to
use this VM image, with shutdown-on-idle after five minutes.
Jenkins Maven Project (Job) that is configured to only run on a
Azure slave node, checks out my test project from Git, and executes
the jUnit Selenium tests from there.
I've hit the same issue with the Jenkins Openstack plugin.
It seems that they will inject the secret directly in the machine's metadata.
https://github.com/jenkinsci/openstack-cloud-plugin/issues/104
In the meanwhile, I'll use instead the SSH slave, more secure since only the pub key needs to be deployed in the slave.