Does anyone know why a connectionString is required when running ef migration from YAML and not Developer Command Prompt?
I believe it might have something to do with not pulling the connectionString value from the Pipeline Variable.
I am setting the connection string here:
builder.Service.AddDbContext<myContext>(options=>
options.UseSqlServer(builder.Configuration.GetConnectionString("MyDB")));
I have tried my Pipeline Variable's names as
MyDB
ConnectionStrings_MyDB
ConnectionStrings:MyDB
The following YAML gives me the error "Value cannot be null. (Parameter 'connectionString')"
- task: DotNetCoreCLI#2
displayName: Create SQL Scripts
inputs:
command: custom
custom: 'ef '
arguments: migrations script --output $(sqlOutputPath) --idempotent --project $(solution)
However running the following command from Developer Command Prompt executes successfully:
dotnet ef migrations script --output complete.sql --idempotent --project myproject
It turned out that I forgot to add the reference for Azure KeyVault to my project; therefore it was not even looking at keyvault for the ConnectionString. Also the name would need to be ConnectionStrings--MyDB
Related
Is there a way to set default shell options for the entire pipelines file?
In github actions you could do it at the workflow level using the 'defaults > shell' value e.g. bash --noprofile --norc -eo pipefail {0}
and this would affect all shell steps in the entire workflow. As far as I can tell, there is no equivalent option in azure devops and I would be forced to copy/paste for each 'bash' task.
You can add the SHELLOPTS variable for the entire pipeline, which would at least get you the -eo pipefail part:
variables:
SHELLOPTS: errexit:pipefail
It is currently not supported to set default shell options in Azure DevOps. Every bash task represents a new session in one Azure Pipeline workflow.
You could create a new Azure DevOps feature request via : https://developercommunity.visualstudio.com/report?space=21&entry=suggestion
From a devops build pipeline, I'd like to run a bicep file for a deployment into a resource group.
My devops service connection is names '365response-tfssc-dev', as seen below:
My yaml job is as follows:
- job: deployAzure
displayName: deploy bicep to Azure
pool:
vmImage: "ubuntu-latest"
dependsOn: [waitForValidation]
steps:
- task: AzureCLI#2
displayName: Deploy Bicep To Azure
inputs:
azureSubscription: "365response-tfssc-dev"
scriptType: "bash"
scriptLocation: "inlineScript"
inlineScript: |
az deployment group create --resource-group rg-365Response-$(env)-001 \
--template-file '$(System.DefaultWorkingDirectory)\bicep\365Response.main.bicep' \
--parameters '$(System.DefaultWorkingDirectory)\bicep\365Response.parameters.$(env).json' \
If I run this from the terminal window of vs code then it works ok.
When this job runs it gives the following message:
/usr/bin/az account set --subscription 'correct subscription id is
listed here' /usr/bin/bash
/home/vsts/work/_temp/azureclitaskscript1654444101122.sh ERROR:
unrecognized arguments:
ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_SERVICEMANAGEMENTURL=https://m...
this line is very very long
The very long line is as follows:
ERROR: unrecognized arguments: ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_SERVICEMANAGEMENTURL=https://management.core.windows.net/ ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_ENVIRONMENT=AzureCloud ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_ARMMANAGEMENTPORTALURL=https://portal.azure.com/ ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_MANAGEMENTPORTALURL=https://manage.windowsazure.com/ ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_GALLERYURL=https://gallery.azure.com/ ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_SUBSCRIPTIONID=subIdHere ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8={"environment":"AzureCloud","scopeLevel":"Subscription","subscriptionId":"subIdHere","subscriptionName":"dev-001","creationMode":"Manual","environmentUrl":"https://management.azure.com/","galleryUrl":"https://gallery.azure.com/","serviceManagementUrl":"https://management.core.windows.net/","resourceManagerUrl":"https://management.azure.com/","activeDirectoryAuthority":"https://login.microsoftonline.com/","environmentAuthorityUrl":"https://login.windows.net/","graphUrl":"https://graph.windows.net/","managementPortalUrl":"https://manage.windowsazure.com/","armManagementPortalUrl":"https://portal.azure.com/","activeDirectoryServiceEndpointResourceId":"https://management.core.windows.net/","sqlDatabaseDnsSuffix":".database.windows.net","AzureKeyVaultDnsSuffix":"vault.azure.net","AzureKeyVaultServiceEndpointResourceId":"https://vault.azure.net","StorageEndpointSuffix":"core.windows.net","EnableAdfsAuthentication":"false"} ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_SQLDATABASEDNSSUFFIX=.database.windows.net ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_ENVIRONMENTAUTHORITYURL=https://login.windows.net/ ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_CREATIONMODE=Manual ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_AZUREKEYVAULTSERVICEENDPOINTRESOURCEID=https://vault.azure.net ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_SUBSCRIPTIONNAME=dev-001 ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_AZUREKEYVAULTDNSSUFFIX=vault.azure.net ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_SCOPELEVEL=Subscription agent.jobstatus=Succeeded ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_ACTIVEDIRECTORYSERVICEENDPOINTRESOURCEID=https://management.core.windows.net/ ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_GRAPHURL=https://graph.windows.net/ ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_ENVIRONMENTURL=https://management.azure.com/ ENDPOINT_URL_7940768d-1de7-44d9-92bf-05d293639bc8=https://management.azure.com/ ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_ACTIVEDIRECTORYAUTHORITY=https://login.microsoftonline.com/ ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_ENABLEADFSAUTHENTICATION=false ENDPOINT_DATA_7940768d-1de7-44d9-92bf-05d293639bc8_RESOURCEMANAGERURL=https://management.azure.com/ SELENIUM_JAR_PATH=/usr/share/java/selenium-server.jar COMMON_TESTRESULTSDIRECTORY=/home/vsts/work/1/TestResults GOROOT_1_17_X64=/opt/hostedtoolcache/go/1.17.10/x64 CONDA=/usr/share/miniconda SYSTEM_JOBNAME=__default AGENT_RETAINDEFAULTENCODING=false JAVA_HOME_11_X64=/usr/lib/jvm/temurin-11-jdk-amd64 SYSTEM_PIPELINESTARTTIME=2022-06-05 15:48:16+00:00 AZURE_CONFIG_DIR=/home/vsts/work/_temp/.azclitask SYSTEM_TASKINSTANCENAME=AzureCLI AGENT_HOMEDIRECTORY=/home/vsts/agents/2.204.0 AGENT_TEMPDIRECTORY=/home/vsts/work/_temp BUILD_REQUESTEDFOREMAIL=aza.'my email here' VSTS_PROCESS_LOOKUP_ID=vsts_8ec9ddb3-be14-4d39-96fe-b09bdd94b311 SYSTEM_COLLECTIONURI=https://dev.azure.com/idsservicesbeta/ BUILD_DEFINITIONNAME=Scaffolding (1) ENDPOINT_URL_SYSTEMVSSCONNECTION=https://dev.azure.com/idsservicesbeta/ JAVA_HOME=/usr/lib/jvm/temurin-11-jdk-amd64 GRADLE_HOME=/usr/share/gradle-7.4.2 SYSTEM_STAGENAME=deployBicep SYSTEM_JOBPARALLELISMTAG=Private AGENT_OS=Linux BUILD_BUILDURI=vstfs:///Build/Build/1755 AGENT_JOBNAME=deploy bicep to Azure XDG_CONFIG_HOME=/home/vsts/.config DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 BUILD_REPOSITORY_URI=https://idsservicesbeta#dev.azure.com/idsservicesbeta/365-Response/_git/Scaffolding ANT_HOME=/usr/share/ant RESOURCES_TRIGGERINGALIAS= JAVA_HOME_8_X64=/usr/lib/jvm/temurin-8-jdk-amd64 BUILD_DEFINITIONVERSION=1 HOMEBREW_PREFIX=/home/linuxbrew/.linuxbrew RUNNER_TOOLSDIRECTORY=/opt/hostedtoolcache SYSTEM_SERVERTYPE=Hosted AGENT_USEWORKSPACEID=true BUILD_REQUESTEDFORID=08c91bb3-5fb2-6b27-a830-47c6829ed7f8 SYSTEM_JOBIDENTIFIER=deployBicep.deployAzure.__default SYSTEM_ARTIFACTSDIRECTORY=/home/vsts/work/1/a AGENT_VERSION=2.204.0 HOMEBREW_CLEANUP_PERIODIC_FULL_DAYS=3650 BUILD_SOURCEVERSIONAUTHOR=BizTalkers SYSTEM_JOBDISPLAYNAME=deploy bicep to Azure BUILD_REPOSITORY_NAME=Scaffolding BOOTSTRAP_HASKELL_NONINTERACTIVE=1 PWD=/home/vsts/work/1/s PIPX_BIN_DIR=/opt/pipx_bin BUILD_ARTIFACTSTAGINGDIRECTORY=/home/vsts/work/1/a AGENT_ACCEPTTEEEULA=True BUILD_SOURCEBRANCHNAME=main AGENT_UPLOADTIMELINEATTACHMENTSTOBLOB=true TASK_DISPLAYNAME=Deploy Bicep To Azure BUILD_CONTAINERID=27996509 ANDROID_NDK_LATEST_HOME=/usr/local/lib/android/sdk/ndk/23.2.8568313 RESOURCES_TRIGGERINGCATEGORY= POWERSHELL_DISTRIBUTION_CHANNEL=Azure-DevOps-ubuntu20 SYSTEM_STAGEDISPLAYNAME=deployBicep SYSTEM_PLANID=6892c8d0-c78e-4c67-b035-05b3489e50dc SYSTEM_POSTLINESSPEED=500 BUILD_BUILDNUMBER=Deploy Bicep files 1755 DOTNET_MULTILEVEL_LOOKUP=0 BUILD_REPOSITORY_LOCALPATH=/home/vsts/work/1/s VSTS_AGENT_PERFLOG=/home/vsts/perflog HOME=/home/vsts LANG=C.UTF-8 BUILD_REPOSITORY_PROVIDER=TfsGit STATS_KEEPALIVE=false SYSTEM_TIMELINEID=6892c8d0-c78e-4c67-b035-05b3489e50dc SYSTEM_PHASEDISPLAYNAME=deploy bicep to Azure SYSTEM_TASKDEFINITIONSURI=https://dev.azure.com/idsservicesbeta/ BUILD_STAGINGDIRECTORY=/home/vsts/work/1/a SYSTEM_HOSTTYPE=build AGENT_WORKFOLDER=/home/vsts/work SYSTEM_STAGEID=bc4f992b-d3a8-5fa4-4306-364494a1b562 SYSTEM_DEFINITIONID=45 INVOCATION_ID=ddfbd830e49e4577879f4d283f4ac321 INPUT_SCRIPTARGUMENTS= AGENT_DISABLELOGPLUGIN_TESTFILEPUBLISHERPLUGIN=true TF_BUILD=True JAVA_HOME_17_X64=/usr/lib/jvm/temurin-17-jdk-amd64 AGENT_TASKRESTRICTIONSENFORCEMENTMODE=Enabled AGENT_ROOTDIRECTORY=/home/vsts/work SYSTEM_JOBATTEMPT=1 ANDROID_NDK_HOME=/usr/local/lib/android/sdk/ndk-bundle SYSTEM_DEFINITIONNAME=Scaffolding (1) HOMEBREW_NO_AUTO_UPDATE=1 BUILD_BINARIESDIRECTORY=/home/vsts/work/1/b NVM_DIR=/home/vsts/.nvm SGX_AESM_ADDR=1 SYSTEM_PHASEATTEMPT=1 SYSTEM_ENABLEACCESSTOKEN=SecretVariable SYSTEM_TEAMFOUNDATIONSERVERURI=https://dev.azure.com/idsservicesbeta/ SYSTEM_TASKDISPLAYNAME=Deploy Bicep To Azure BUILD_BUILDID=1755 TEMPLATEFILE=bicep/365Response.main.json BUILD_REPOSITORY_ID=92e4e7ea-8e17-425b-ad1c-899f9922bc0f AGENT_NAME=Hosted Agent ANDROID_HOME=/usr/local/lib/android/sdk SYSTEM_JOBPOSITIONINPHASE=1 AGENT_MACHINENAME=fv-az414-868 ACCEPT_EULA=Y SYSTEM_PULLREQUEST_ISFORK=False SYSTEM_JOBTIMEOUT=60 SYSTEM_TEAMPROJECTID=4009b106-170a-496d-9af8-9ec836b38dc3 SYSTEM_COLLECTIONID=b3e27278-2d93-48a2-af86-fa3370179011 USER=vsts SYSTEM_TEAMPROJECT=365-Response HOMEBREW_CELLAR=/home/linuxbrew/.linuxbrew/Cellar BUILD_SOURCEVERSION=715f5872b0f65eade29314d0f30bf57a3f191896 PIPX_HOME=/opt/pipx AGENT_DISABLELOGPLUGIN_TESTRESULTLOGPLUGIN=true SYSTEM_PHASEID=f1ebf77f-30ac-526d-968c-fab23fa199f8 GECKOWEBDRIVER=/usr/local/share/gecko_driver BUILD_REASON=Manual SYSTEM_STAGEATTEMPT=1 CHROMEWEBDRIVER=/usr/local/share/chrome_driver SHLVL=0 SYSTEM=build ANDROID_SDK_ROOT=/usr/local/lib/android/sdk VCPKG_INSTALLATION_ROOT=/usr/local/share/vcpkg HOMEBREW_REPOSITORY=/home/linuxbrew/.linuxbrew/Homebrew ImageVersion=20220529.1 BUILD_SOURCEBRANCH=refs/heads/main AZURE_HTTP_USER_AGENT=VSTS_b3e27278-2d93-48a2-af86-fa3370179011_build_45_0 DOTNET_NOLOGO=1 BUILD_SOURCESDIRECTORY=/home/vsts/work/1/s MSDEPLOY_HTTP_USER_AGENT=VSTS_b3e27278-2d93-48a2-af86-fa3370179011_build_45_0 TASK_SKIPTRANSLATORFORCHECKOUT=False SYSTEM_CULTURE=en-US SYSTEM_WORKFOLDER=/home/vsts/work STATS_PFS=true GRAALVM_11_ROOT=/usr/local/graalvm/graalvm-ce-java11-22.1.0 AGENT_READONLYVARIABLES=true AGENT_ID=8 BUILD_QUEUEDBYID=08c91bb3-5fb2-6b27-a830-47c6829ed7f8 AZURE_EXTENSION_DIR=/opt/az/azcliextensions AGENT_BUILDDIRECTORY=/home/vsts/work/1 BUILD_REQUESTEDFOR=Rob Bowman ANDROID_NDK_ROOT=/usr/local/lib/android/sdk/ndk-bundle CHROME_BIN=/usr/bin/google-chrome AGENT_UPLOADBUILDARTIFACTSTOBLOB=true SYSTEM_DEFAULTWORKINGDIRECTORY=/home/vsts/work/1/s GOROOT_1_18_X64=/opt/hostedtoolcache/go/1.18.2/x64 JOURNAL_STREAM=8:23147 AGENT_OSARCHITECTURE=X64 LEIN_HOME=/usr/local/lib/lein LEIN_JAR=/usr/local/lib/lein/self-installs/leiningen-2.9.8-standalone.jar SYSTEM_ISSCHEDULED=False BUILD_REPOSITORY_GIT_SUBMODULECHECKOUT=False PATH=/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/vsts/.local/bin:/opt/pipx_bin:/home/vsts/.cargo/bin:/home/vsts/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/vsts/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin SYSTEM_JOBID=d562b731-90ac-599c-aa5d-b4e5e0c32cf4 BUILD_QUEUEDBY=Rob Bowman SWIFT_PATH=/usr/share/swift/usr/bin PIPELINE_WORKSPACE=/home/vsts/work/1 ImageOS=ubuntu20 BUILD_SOURCEVERSIONMESSAGE=ubuntu-latest SYSTEM_TEAMFOUNDATIONCOLLECTIONURI=https://dev.azure.com/idsservicesbeta/ AGENT_LOGTOBLOBSTORAGESERVICE=true LOCATION=uksouth SYSTEM_TASKINSTANCEID=44b963b8-127f-5c06-baab-44a1330fee42 AGENT_JOBSTATUS=Succeeded DEBIAN_FRONTEND=noninteractive GIT_TERMINAL_PROMPT=0 AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache SYSTEM_PHASENAME=deployAzure OLDPWD=/home/vsts/work/1/s SYSTEM_TOTALJOBSINPHASE=1 GOROOT_1_16_X64=/opt/hostedtoolcache/go/1.16.15/x64 _=/usr/bin/env-001
Anyone see where I've gone wrong?
Please check if your service connection's Azure AD service principal has access to the Azure subscription you are trying to deploy to. Your error message doesn't really look like that, but the error message is not fully shown in your question. (That "very long" error message can be very important. ;) )
Most likely you should debug the az deployment group create ... script locally with the variable values replaced for yourself manually, and see if you can reproduce the error. That would mean that Azure Pipelines has nothing to do with this, you should just make your deployment instruction work and all will be good.
If #1 is not applicable for you (e.g. your deployment instruction is working totally fine locally but it is still failing in the pipeline), my recommendation is to look into Azure CLI version on the pipelines agent vs. the one you need and maybe add az Azure CLI upgrade/downgrade task to suit your needs.
For example, we have used these 2 steps to update AzureCLI when the MS hosted agent version contained a bug.
- script: sudo apt-get update
- task: AzureCLI#2
inputs:
azureSubscription: ${{ parameters.armServiceConnection }}
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |-
az --version
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
az --version
Problem was using ubuntu image - should have been windows-2022
I need that the post deploy script, part of the database project, to set some specific things depending on the environment it will run.
How pass an environment variable that the script can access?
Here what i'm trying to do.
the yaml file:
- task: SqlDacpacDeploymentOnMachineGroup#0
displayName: Install database
inputs:
TaskType: 'dacpac'
DacpacFile: '**/app-db.dacpac'
TargetMethod: 'server'
ServerName: '(localdb)\MSSQLLocalDB'
DatabaseName: 'app-dev'
AuthScheme: 'windowsAuthentication'
The log:
Starting: Install database
==============================================================================
Task : SQL Server database deploy
Description : Deploy a SQL Server database using DACPAC or SQL scripts
Version : 0.3.23
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/deploy/sql-dacpac-deployment-on-machine-group
==============================================================================
*** Could not deploy package.
Warning SQL72013: The following SqlCmd variables are not defined in the target scripts: env.
Error SQL72014: .Net SqlClient Data Provider: Msg 137, Level 15, State 2, Line 14 Must declare the scalar variable "#env".
Error SQL72045: Script execution error. The executed script:
IF (#env = 'DEV')
BEGIN
...
END
Here the complete solution i didn't find anywhere:
Declare a variable for the dacpac. Inside Visual studio, go to the property project page, tab SQLCMD Variables. Add the env variable like this:
In the sql script, write the variable inside a string, so it won't break the build with a SQL syntax error. It is not a SQL variable. The string '$(env)' will be replaced as a token before it runs:
if ('$(env)' = 'DEV') ...
You can test it still in visual studio by publishing the project.
Finally go the deploy task definition and setup the variable like this:
- task: SqlDacpacDeploymentOnMachineGroup#0
displayName: Install database
inputs:
TaskType: 'dacpac'
DacpacFile: '**/app-db.dacpac'
TargetMethod: 'server'
ServerName: '(localdb)\MSSQLLocalDB'
DatabaseName: 'app-dev'
AuthScheme: 'windowsAuthentication'
AdditionalArguments: /v:env=DEV
I'm trying to use AWS cli within a script section of an Azure pipeline. The script section is in a template file and it's accessed from the main pipeline.
steps:
- bash: |
step_function_state=`aws stepfunctions list-executions --state-machine-arn $(stateMachineArn) --status-filter RUNNING | jq -r '.executions[]|.status' | head -1`
echo "State machine RUNNING status: ${step_function_state}"
# Rest of the script#
displayName: "Test Script"
env:
AWS_ACCESS_KEY_ID: $(AWS_ACCESS_KEY_ID)
AWS_DEFAULT_REGION: $(AWS_DEFAULT_REGION)
AWS_SECRET_ACCESS_KEY: $(AWS_SECRET_ACCESS_KEY)
stateMachineArn, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION are stored in a variable group. When running the pipeline it gives the following error,
An error occurred (UnrecognizedClientException) when calling the ListExecutions operation: The security token included in the request is invalid.
Using the same credentials I am able to run my local CLI and get the results.
I tried printenv command and all the AWS variables are in the environment too. What could I possibly do wrong?
I realized that this issue occurred due to credential mismatch.
After adding the correct credentials (same as local cli) the pipeline CLI also started to work.
Based on the error log it felt like aws_session_token could be an issue but the actual issue was in aws_access_key_id and aws_secret_access_key.
Trying to deploy artifacts ends up with the following error:
The input is not a valid Base-64 string as it contains a non-base 64
character, more than two padding characters, or an illegal character
among the padding characters.
I'm running two scripts before and after I build the app in AppVeyor:
cd $env:APPVEYOR_BUILD_FOLDER\patch;
npm install;
node patch-project-json.js $env:APPVEYOR_BUILD_FOLDER\src\Project1\project.json $env:APPVEYOR_BUILD_VERSION;
node patch-project-json.js $env:APPVEYOR_BUILD_FOLDER\src\Project2\project.json $env:APPVEYOR_BUILD_VERSION;
node patch-project-json.js $env:APPVEYOR_BUILD_FOLDER\src\Project3\project.json $env:APPVEYOR_BUILD_VERSION;
cd $env:APPVEYOR_BUILD_FOLDER
dotnet restore
and
dotnet publish .\src\Project1 --output $env:APPVEYOR_BUILD_FOLDER\deploy\Project1 --configuration Release --no-build;
dotnet publish .\src\Project2 --output $env:APPVEYOR_BUILD_FOLDER\deploy\Project2 --configuration Release --no-build;
dotnet publish .\src\Project3 --output $env:APPVEYOR_BUILD_FOLDER\deploy\Project3 --configuration Release --no-build
As you can see, I am using this to set versions in project.json files based on the $env:APPVEYOR_BUILD_VERSION. I don't know if it's relevant information. So after successful build and publish, I want to upload an artifact to the blob storage.
Turned out I had typos in Deployment settings for Storage access key entry ;)