provide credentials in nuget command line parameters - nuget

We have an nuget server hosted in azure that only allow certain people to create packages.
If I run nuget command line, I get asked to provide username and password. Is there a possibility to provide username and password in command line?
I tried something such as -User or -UserName, but I get Unknown option: '-User'
nuget push -Source "XXX" -ApiKey VSTS -noninteractive mypackage.*.nupkg -User myUsername
Is there a way to provide credentials from command line?

You have to change the NuGet.config you use. That is possible from the command line:
nuget sources add -Name "MyExample" -Source "XXX" -username "Y" -password "Z"
The password will be stored encrypted in the config file and can only be decrypted in the same user context as it was encrypted.

Use PAT (Personal Access Token) as the password. Make sure that PAT has packaging read and write permission.

If you're using an Azure Pipelines build file and trying to push to a private feed, you can use these snippets, just make sure to update $VARIABLES and indentation as-needed:
- task: NuGetAuthenticate#0
displayName: "Auth NuGet"
name: Nuget_Authenticate
inputs:
nuGetServiceConnections: '$PROJECT_SETTINGS_SERVICE_CONNECTION_NAME'
forceReinstallCredentialProvider: true
- task: NuGetCommand#2
name: pushNuGetPkg
displayName: "Push NuGet"
inputs:
command: 'push'
packagesToPush: '$(Build.Repository.Localpath)\Packages\*.nupkg'
nuGetFeedType: 'external'
publishFeedCredentials: '$ARTIFACTS_FEED_NAME'
publishVstsFeed: '$AZURE_ORGANIZATION_NAME/$ARTIFACTS_FEED_NAME'
You may also need to set these env-vars up in your ADO Project's Library -> Defaults section. Make sure the second one is securely masked with the Lock icon.
NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED: true
VSS_NUGET_EXTERNAL_FEED_ENDPOINTS: {"endpointCredentials": [{"endpoint":"https://$AZURE_ORG_NAME.pkgs.visualstudio.com/_packaging/$ARTIFACTS_FEED_NAME/nuget/v3/index.json", "username":"$USERNAME", "password":"$PAT"}]}

For Using PAT(Personal Access Token) As Credential, you can use this code below:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="ChequeNuget" value="http://192.168.11.23:6700/DefaultCollection/Framework/_packaging/ChequeNuget/nuget/v3/index.json" />
</packageSources>
<packageSourceCredentials>
<ChequeNuget>
<add key="Username" value="ChequeNuget" />
<add key="Password" value="AQAAANCMnd8BFdERjHoAwE/Cl+sBAAAAFyfr+GavvkSgtlD+6FbrwQAAAAACAAAAAAAQZgAAAAEAACAAAABHbg6o2T6HHgjyud+purjkcwLoK1OH2tQI3TZc3T6wfgAAAAAOgAAAAAIAACAAAADnEEVK46WCJAwou/6Gb2ZudmbEJoPoFUqqyK32yf1OYUAAAABlnx7sfP37M/Kblf7VTN2lD3kTiKjRdr0GPCtrLBxGE6pXCbnBY/E1vIrP8S2Ci8HkMW6SgnSkvSACfliZWzoDQAAAAI0xvaQX2XsnjOuDExXf+tLzcdsDK9ZJNen/f4oMbWejc/zytb7D+oczJmrmRbvKBLBC9nGlK2/JfVZii39euMU=" />
<add key="ValidAuthenticationTypes" value="basic" />
</ChequeNuget>
</packageSourceCredentials>
</configuration>
For updating password use the command below:
nuget sources update -ValidAuthenticationTypes basic -Name ChequeNuget -UserName "ChequeNuget" -Password [Personal Access Token]

Related

Azure CLI 2.x is not installed on this machine

I'm trying to upload blobs using "az storage blob upload-batch". I got below two exceptions.
##[error]Azure CLI 2.x is not installed on this machine.
##[error]Script failed with error: Error: Unable to locate executable file: 'pwsh'. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also verify the file has a valid extension for an executable file.
task: AzureCLI#2
displayName: 'Upload Files'
inputs:
azureSubscription: 'xxxxxxx'
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
$subscriptionName = $(az account show --query 'name' --output tsv)
az storage blob upload-batch `
--subscription $subscriptionName `
--account-name xxxxxxxxx `
--source "C:\xxx\yyy" `
--destination MyContainerName `
--auth-mode login
Thank you #N MZ , Posting your suggestion as an answer to help other community members.
##[error]Azure CLI 2.x is not installed on this machine.
##[error]Script failed with error: Error: Unable to locate executable file: 'pwsh'. Please verify either the file path exists or the file
can be found within a directory specified by the PATH environment
variable. Also verify the file has a valid extension for an executable
file.
For this above errors we need to install Azure cli
and check running cmd az version if its installed or not .
For More information please refer the below links:
MS DOC:Azure File Copy task
SO THREAD: How to run Azure CLI tasks from an Azure DevOps Pipeline
Microsoft Documentation
Although not the best approach, for a quick test, you could install az cli while running the job steps:
jobs:
- job: my_job
steps:
- script: curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
displayName: 'install az cli'

Devops Proxy Configuration for tasks, not for agent

This my scenario:
Azure DevOps Server Version Dev17.M153.3
Self-hosted Agent Agent.Version 2.177.1
Source Control Type TFVC
The Agent is working well without proxy configuration, the problem are 2 tasks it executes.
After getting the source code, the pipeline have two NuGet tasks:
First the NuGet Tool Installer task
Second the NuGet Task
I get the following error in the first task
##[debug]Agent.Version=2.177.1
##[debug]Agent.ToolsDirectory=C:\Proyectos\Agent2\_work\_tool
##[debug]isExplicit: 5.10.0
##[debug]explicit? true
##[debug]evaluating 1 versions
##[debug]matched: 5.10.0
##[debug]Querying versions list
##[debug]Agent.ProxyUrl=undefined
ERR:connect ETIMEDOUT 152.199.23.209:443
##[debug]task result: Failed
##[debug]Processed: ##vso[task.complete result=Failed;]
Then I configure these Pipeline variables
Agent.ProxyBypassList $(ProxyBypassList)
Agent.ProxyPassword $(ProxyPassword)
Agent.ProxyUrl $(ProxyUrl)
Agent.ProxyUsername $(ProxyUsername)
The values are defined in a Variable Group.
With this configuration, the task works well
##[debug]Agent.Version=2.177.1
##[debug]Agent.ToolsDirectory=C:\Proyectos\Agent2\_work\_tool
##[debug]evaluating 0 versions
##[debug]match not found
##[debug]Querying versions list
##[debug]Agent.ProxyUrl=***
##[debug]Agent.ProxyUsername=***
##[debug]Agent.ProxyPassword=***
##[debug]Agent.ProxyBypassList=***
(node:15928) Warning: Use Cipheriv for counter mode of aes-256-ctr
##[debug]evaluating 53 versions
##[debug]matched: 5.10.0
##[debug]Found the following version from the list: 5.10.0
##[debug]isExplicit: 5.10.0
##[debug]explicit? true
##[debug]Agent.Version=2.177.1
##[debug]Agent.ToolsDirectory=C:\Proyectos\Agent2\_work\_tool
##[debug]checking cache: C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64
##[debug]not found
##[debug]Downloading version: 5.10.0
...
##[debug]Processed: ##vso[task.prependpath]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64
##[debug]set NuGetExeToolPath=C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe
##[debug]Processed: ##vso[task.setvariable variable=NuGetExeToolPath;issecret=false;]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe
Then the problem is traslated to the second tasks. This is the generated YAML of the second tasks:
steps:
- task: NuGetCommand#2
displayName: 'NuGet restore packages.config'
inputs:
restoreSolution: '**/packages.config'
feedsToUse: config
nugetConfigPath: 'xxxxxxxxxx\NuGet.Config'
externalFeedCredentials: 'xxxxxxxxx Basic Authentication'
The external Feed Credentials is a Service Connection to a private feed in another DevOps collection of the same server. I have also tried using a Service Connection with a Token and getting the same result.
This is part of the log of this task
Versión de NuGet detectada: 5.10.0.7240 / 5.10.0+dca1d060f38e1e02f6bfca41e25f081f19fd534b.dca1d060f38e1e02f6bfca41e25f081f19fd534b
##[debug]Quirks for 5.10.0.7240:
##[debug] NoTfsOnPremAuthConfig
##[debug] V2CredentialProvider
##[debug]NuGet.ForceEnableCredentialProvider=undefined
##[debug]Credential provider V1 is disabled in favor of V2 plugin.
##[debug]NuGet_ForceEnableCredentialProviderV2=undefined
##[debug]NuGetTasks.IsHostedTestEnvironment=undefined
##[debug]System.ServerType=OnPremises
##[debug]V2 credential provider is enabled.
##[debug]NuGet.ForceEnableCredentialConfig=undefined
##[debug]NuGetTasks.IsHostedTestEnvironment=undefined
##[debug]System.ServerType=OnPremises
##[debug]Credential config is disabled due to on-prem quirks.
##[debug]Setting up auth
##[debug]Discovered URL prefixes: http://xxxxxxxxx:8080/tfs/xxxxxxxx/
##[debug]NuGetTasks.ExtraUrlPrefixesForTesting=undefined
##[debug]Getting credentials for local feeds
SYSTEMVSSCONNECTION exists true
##[debug]SYSTEMVSSCONNECTION exists true
##[debug]Got auth token
##[debug]externalEndpoints=bf184c95-f3f5-4dcf-be45-9aa22eba92e9
##[debug]bf184c95-f3f5-4dcf-be45-9aa22eba92e9=http://xxxxxxx:8080/tfs/xxxxxxxx/_packaging/xxxxxxxx/nuget/v3/index.json
bf184c95-f3f5-4dcf-be45-9aa22eba92e9 exists true
##[debug]bf184c95-f3f5-4dcf-be45-9aa22eba92e9 exists true
##[debug]bf184c95-f3f5-4dcf-be45-9aa22eba92e9 auth scheme = UsernamePassword
##[debug]adding password auth entry for feed http://xxxxxxxxxxx:8080/tfs/xxxxxxxxxxx/_packaging/xxxxxxxxx/nuget/v3/index.json
##[debug]Setting up sources
##[debug]selectOrConfig=config
##[debug]nugetConfigPath=C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config
##[debug]check path : C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config
##[debug]nugetConfigPath=C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config
##[debug]Absolute path for pathSegments: C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config = C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config
##[debug]build.sourcesDirectory=C:\Proyectos\Agent2\_work\1\s
##[debug]Absolute path for pathSegments: C:\Proyectos\Agent2\_work\1\s = C:\Proyectos\Agent2\_work\1\s
##[debug]nugetConfigPathpath supplied :true
##[debug]Agent.BuildDirectory=C:\Proyectos\Agent2\_work\1
##[debug]build.buildId=210
##[debug]No temp nuget.config auth
##[debug]ConfigFile: C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config
##[debug]Absolute path for pathSegments: C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe = C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe
##[debug]which 'C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe'
##[debug]found: 'C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe'
##[debug]which 'C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe'
##[debug]found: 'C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe'
##[debug]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe arg: restore
##[debug]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe arg: C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\xxxxxxxx\packages.config
##[debug]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe arg: -Verbosity
##[debug]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe arg: Detailed
##[debug]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe arg: -NonInteractive
##[debug]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe arg: -ConfigFile
##[debug]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe arg: C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config
##[debug]Agent.TempDirectory=C:\Proyectos\Agent2\_work\_temp
##[debug]NUGET_PLUGINS_CACHE_PATH set to C:\Proyectos\Agent2\_work\_temp\NuGetPluginsCache
##[debug]V2 credential provider set
##[debug]credProviderPath = C:\Proyectos\Agent2\_work\_tasks\NuGetCommand_333b11bd-d341-40d9-afcf-b32d5ce6f23b\2.153.2\CredentialProviderV2\plugins\netfx\CredentialProvider.Microsoft\CredentialProvider.Microsoft.exe
##[debug]Detected external credentials for:
##[debug]http://xxxxxxxx:8080/tfs/xxxxxxxx/_packaging/xxxxxxxx/nuget/v3/index.json
##[debug]agent.proxyurl=***
##[debug]agent.proxyusername=***
##[debug]agent.proxypassword=***
##[debug]Adding environment variable for NuGet proxy: http://***:***#yyyyyyyyyyy:8080/
##[debug]Agent.ProxyBypassList=***
##[debug]Getting sources from NuGet.config in this location: C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config
##[debug]Adding environment variable for NuGet proxy bypass: xxxxxxxx
##[debug]exec tool: C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe
##[debug]arguments:
##[debug] restore
##[debug] C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\xxxxxxxx\packages.config
##[debug] -Verbosity
##[debug] Detailed
##[debug] -NonInteractive
##[debug] -ConfigFile
##[debug] C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config
[command]C:\Proyectos\Agent2\_work\_tool\NuGet\5.10.0\x64\nuget.exe restore C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\xxxxxxxx\packages.config -Verbosity Detailed -NonInteractive -ConfigFile C:\Proyectos\Agent2\_work\1\s\xxxxxxxx\NuGet.Config
##[debug]Re-evaluate condition on job cancellation for step: 'NuGet restore packages.config'.
##[error]The operation was canceled.
The task hangs and I must cancel it.
Could you help me to find what I missing or doing wrong?
I found the right process to get it working properly.
Install the agent.
Create a file named .proxybypass including the URLs to bypass the proxy in the agent directory:
I use the command echo *urltobypass* > .proxybypass
Configure the agent with proxy parameters configuration:
.\config.cmd --proxyurl *YourProxyUrl* --proxyusername *YourProxyUser* --proxypassword *YourProxyPassword*
With these steps to configure the agent the tasks execute properly.
Run a self-hosted agent behind a web proxy

How to perform XML Element substitution in web.config using Replace Tokens?

We have a Web.config file with the following system.net mailSettings attribute:
<system.net>
<mailSettings>
<smtp configSource="mailSettings.config" />
</mailSettings>
</system.net>
We want to replace this line in the web config: <smtp configSource="mailSettings.config" /> with a pipeline variable whose value will be <smtp from="reports#companyxyz.com" deliveryMethod="Network"> <network enableSsl="true" host="smtp.sendgrid.net" port="587" userName="apiKey" password="12346576fgb" /> </smtp>.
Also because the developer doesn't want to change the web config file to accommodate prefixes and suffices required by the Replace Tokens task, we want to use custom Token Prefix, as such: <mailSettings> and Suffix: </mailSettings>
steps:
- task: qetza.replacetokens.replacetokens-task.replacetokens#4
displayName: 'Replace tokens in **/*.config'
inputs:
tokenPattern: custom
tokenPrefix: '<mailSettings>'
tokenSuffix: '</mailSettings>'
Can the above be accomplished? Also do I place this Replace Tokens task before Azure App Deploy task? or after?
Can the above be accomplished?
Yes. This can be accomplished.
The replace token task uses the Pipeline varible to set the value in Web.config file.
In your case, the varaible name is <smtp configSource="mailSettings.config" />.
And in the replacement process, the prefix and suffix will be replaced at the same time.
So you could try to set the variable:
<smtp configSource="mailSettings.config" /> : <mailSettings> <smtp configSource="AA.config" /></mailSettings>
Here is an example:
Definition:
- task: qetza.replacetokens.replacetokens-task.replacetokens#3
displayName: 'Replace tokens in web.config'
inputs:
rootDirectory: '$(build.sourcesdirectory)'
targetFiles: web.config
tokenPrefix: '<mailSettings>'
tokenSuffix: '</mailSettings>'
Result:
Also do i place this Replace Tokens task before Azure App Deploy task? or after?
You need to add this task before the Azure App Deploy task.
Then the changes will apply to the Azure App Deploy task.
Update:
As Cataster said in the comments:
Here are two key points in Azure App Service Task:
Make sure of is that my Azure App Deploy Task packageForLinux attribute had to be set to $(System.DefaultWorkingDirectory)/Build Artifact Name/ArtifactName
Disable XML transformation option in the Azure App Deploy task the format of the token replaced will be the same as that in the pipeline variable. but if we enable this option, it will format/transfom it to neat XML

Can we replace nuget.config file with command parameters?

I am working on an Azure pipeline to run on a Windows self-hosted agent.
We configured a Artefact feed with an upstream to connect to Nuget.
As we are behind a firewall, it seems the only way to connect to Nuget.
My pipeline was working with this nuget.config file :
<packageSources>
<clear />
<add key="FeedName" value="https://***.pkgs.visualstudio.com/***/_packaging/FeedName/nuget/v3/index.json" />
</packageSources>
And this YAML:
- task: NuGetAuthenticate#0
- task: CmdLine#2
inputs:
script: '"C:\dotnet\dotnet.exe" publish ${{ parameters.solutionToPublishPath }} -c ${{ parameters.buildConfiguration }} -o $(Build.BinariesDirectory)'
The nuget.config file breaks the previous pipelines in TeamCity!!
To keep the old one running while I work on the new one, I am looking for a way to move the information from the nuget.config file to the script.
Is it possible ?
I tried with this:
- task: CmdLine#2
inputs:
script: '"C:\dotnet\dotnet.exe" add "src/project/project.API.csproj" package FeedName -s https://***.pkgs.visualstudio.com/***/_packaging/FeedName/nuget/v3/index.json'
I get this message which for me indicates that it tried to reach Nuget directly and failed, this is why we use a feed.
error: Unable to load the service index for source https://api.nuget.org/v3/index.json.
error: Response status code does not indicate success: 302 (Moved Temporarily).
Thanks for any help
You may check Replace Tokens extension to see whether it helps you. It can replace tokens in files with variable values during pipeline.
I would not call it a solution as I can't move the nuget.config information out of the file into the command line, I'll remove the file to enable Team City to work and put it back when running Azure pipelines. Thanks.
We are overriding Nuget.config in Azure DevOps pipeline script with DotNetCoreCLI#2 and restoreArguments
- task: DotNetCoreCLI#2
displayName: Restore
inputs:
command: 'restore'
projects: |
$(buildProjects)
!$(testProjects)
restoreArguments: --source https://api.nuget.org/v3/index.json --source $(Build.SourcesDirectory)/Nugets

Authenticating with Azure Repos git module sources in an Azure Pipelines build

I'm currently creating a pipeline for Azure DevOps to validate and apply a Terraform configuration to different subscription.
My terraform configuration uses modules, those are "hosted" in other repositories in the same Azure DevOps Project as the terraform configuration.
Sadly, when I try to perform terraform init to fetch those modules, the pipeline task "hang" there waiting for credentials input.
As recommanded in the Pipeline Documentation on Running Git Commands in a script I tried to add a checkout step with the persistCredentials:true attribute.
From what I can see in the log of the task (see bellow), the credentials information are added specifically to the current repo and are not usable for other repos.
The command performed when adding persistCredentials:true
2018-10-22T14:06:54.4347764Z ##[command]git config http.https://my-org#dev.azure.com/my-org/my-project/_git/my-repo.extraheader "AUTHORIZATION: bearer ***"
The output of terraform init task
2018-10-22T14:09:24.1711473Z terraform init -input=false
2018-10-22T14:09:24.2761016Z Initializing modules...
2018-10-22T14:09:24.2783199Z - module.my-module
2018-10-22T14:09:24.2786455Z Getting source "git::https://my-org#dev.azure.com/my-org/my-project/_git/my-module-repo?ref=1.0.2"
How can I setup the git credentials to work for other repositories ?
You have essentially two ways of doing this.
Pre-requisite
Make sure that you read and, depending on your needs, that you apply the Enable scripts to run Git commands section from the "Run Git commands in a script" doc.
Solution #1: dynamically insert the System.AccessToken (or a PAT, but I would not recommend it) at pipeline runtime
You could to this either by:
inserting a replacement token such as __SYSTEM_ACCESSTOKEN__ in your code (as Nilsas suggests) and use some token replacement code or the qetza.replacetokens.replacetokens-task.replacetokens task to insert the value. The disadvantage of this solution is that you would also have to replace the token when you run you terraform locally.
using some code to replace all git::https://dev.azure.com text with git::https://YOUR_ACCESS_TOKEN#dev.azure.com.
I used the second approach by using the following bash task script (it searches terragrunt files but you can adapt to terraform files without much change):
- bash: |
find $(Build.SourcesDirectory)/ -type f -name 'terragrunt.hcl' -exec sed -i 's~git::https://dev.azure.com~git::https://$(System.AccessToken)#dev.azure.com~g' {} \;
Abu Belai offers a PowerShell script to do something similar.
This type of solution does not however work if modules in your terraform modules git repo call themselves modules in another git repo, which was our case.
Solution #2: adding globally the access token in the extraheader of the url of your terraform modules git repos
This way, all the modules' repos, called directly by your code or called indirectly by the called modules' code, will be able to use your access token. I did so by adding the following step before your terraform/terragrunt calls:
- bash: |
git config --global http.https://dev.azure.com/<your-org>/<your-first-repo-project>/_git/<your-first-repo>.extraheader "AUTHORIZATION: bearer $(System.AccessToken)"
git config --global http.https://dev.azure.com/<your-org>/<your-second-repo-project>/_git/<your-second-repo>.extraheader "AUTHORIZATION: bearer $(System.AccessToken)"
You will need to set the extraheader for each of the called git repos.
Beware that you might need to unset the extraheader after your terraform calls if your pipeline sets the extraheader several times on the same worker. This is because git can get confused with multiple extraheader declaration. You do this by adding to following step:
- bash: |
git config --global --unset-all http.https://dev.azure.com/<your-org>/<your-first-repo-project>/_git/<your-first-repo>.extraheader
git config --global --unset-all http.https://dev.azure.com/<your-org>/<your-second-repo-project>/_git/<your-second-repo>.extraheader
I had the same issue, what I ended up doing is tokenizing SYSTEM_ACCESSTOKEN in terraform configuration. I used Tokenzization task in Azure DevOps where __ prefix and suffix is used to identify and replace tokens with actual variables (it is customizable but I find double underscores best for not interfering with any code that I have)
- task: qetza.replacetokens.replacetokens-task.replacetokens#3
displayName: 'Replace tokens'
inputs:
targetFiles: |
**/*.tfvars
**/*.tf
tokenPrefix: '__'
tokenSuffix: '__'
Something like find $(Build.SourcesDirectory)/ -type f -name 'main.tf' -exec sed -i 's~__SYSTEM_ACCESSTOKEN__~$(System.AccessToken)~g' {} \; would also work if you do not have ability to install custom extensions to your DevOps organization.
My terraform main.tf looks like this:
module "app" {
source = "git::https://token:__SYSTEM_ACCESSTOKEN__#dev.azure.com/actualOrgName/actualProjectName/_git/TerraformModules//azure/app-service?ref=__app-service-module-ver__"
....
}
It's not beautiful but it gets the job done. Module source (at the time of writing) does not support variable input from terraform. So what we can do is to use Terrafile it's an open source project helping with keeping up with the modules and different versions of the same module you might use by keeping a simple YAML file next to your code. It seems that it's no longer being actively maintained, however it just works: https://github.com/coretech/terrafile
my example of Terrafile:
app:
source: "https://token:__SYSTEM_ACCESSTOKEN__#dev.azure.com/actualOrgName/actualProjectName/_git/TerraformModules"
version: "feature/handle-twitter"
app-stable:
source: "https://token:__SYSTEM_ACCESSTOKEN__#dev.azure.com/actualOrgName/actualProjectName/_git/TerraformModules"
version: "1.0.5"
Terrafile by default download your modules to ./vendor directory so you can point your module source to something like:
module "app" {
source = "./vendor/modules/app-stable/azure/app_service"
....
}
Now you just have to figure out how to execute terrafile command in the directory where Terrafile is present.
My azure.pipelines.yml example:
- script: curl -L https://github.com/coretech/terrafile/releases/download/v0.6/terrafile_0.6_Linux_x86_64.tar.gz | tar xz -C $(Agent.ToolsDirectory)
displayName: Install Terrafile
- script: |
cd $(Build.Repository.LocalPath)
$(Agent.ToolsDirectory)/terrafile
displayName: Download required modules
I did this
_ado_token.ps1
# used in Azure DevOps to allow terrform to auth with Azure DevOps GIT repos
$tfmodules = Get-ChildItem $PSScriptRoot -Recurse -Filter "*.tf"
foreach ($tfmodule in $tfmodules) {
$content = [System.IO.File]::ReadAllText($tfmodule.FullName).Replace("git::https://myorg#","git::https://" + $env:SYSTEM_ACCESSTOKEN +"#")
[System.IO.File]::WriteAllText($tfmodule.FullName, $content)
}
azure-pipelines.yml
- task: PowerShell#2
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
inputs:
filePath: '_ado_token.ps1'
pwsh: true
displayName: '_ado_token.ps1'
I Solved the issue by creating a Pipeline template that runs a inline powershell script. I then pull in the template as the Pipeline template a "resource" when using any terraform module form a different Repo.
The script will do a recursive search for all the .tf files. Then use regex to update all the module source urls.
I chose REGEX over tokenizing the module url, because this will make sure the modules can be pulled in on a development machine without any changes to the source.
parameters:
- name: terraform_directory
type: string
steps:
- task: PowerShell#2
displayName: Tokenize TF-Module Sources
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
inputs:
targetType: 'inline'
script: |
$regex = "https://*(.+)dev.azure.com"
$tokenized_url = "https://token:$($env:SYSTEM_ACCESSTOKEN)#dev.azure.com"
Write-Host "Recursive Search in ${{ parameters.terraform_directory }}"
$tffiles = Get-ChildItem -Path "${{ parameters.terraform_directory }}" -Filter "*main.tf" -Recurse -Force
Write-Host "Found $($tffiles.Count) files ending with 'main.tf'"
if ($tffiles) { Write-Host $tffiles }
$tffiles | % {
Write-Host "Updating file $($_.FullName)"
$content = Get-Content $_.FullName
Write-Host "Replace Strings: $($content | Select-String -Pattern $regex)"
$content -replace $regex, $tokenized_url | Set-Content $_.FullName -Force
Write-Host "Updated content"
Write-Host (Get-Content $_.FullName)
}
As far as I can see, the best way to do this is exactly the same as with any other Git provider. It is only for Azure DevOps that I have ever come across the extraheader approach. I have always used this, and after not being able to get a satisfactory result with the other suggested approaches, I went back to it:
- script: |
MY_TOKEN=foobar
git config --global url."https://${MY_TOKEN}#dev.azure.com".insteadOf "https://dev.azure.com"
I don't think you can. Usually, you create another build and link to the artifacts from that build to use it in your current definition. That way you don't need to connect to a different Git repository