VSO NuGet Publisher Build Step Fails - azure-devops

I am using Visual Studio Online - Package Manager Preview, along with the new build system. The package manager preview adds a number of build steps, including a "NuGet Publisher" step, which should push packages to the private feed hosted by Visual Studio Online.
Now the documentation is a little out of step here. As is the documentation on auth and personal access tokens. There are some indications that you shouldn't need auth between VSO and the Package Manager as long as you have permissions set up (the build service account has permissions to the service endpoint and to the package manager extension). The actual build step asks you to select from your list of Service Endpoints, so that is what I have attempted.
When I place no credentials on the Service Endpoint, I get the error:
Server Key must be set, set the password on the generic service
When I attempt to place the API key against the Service Endpoint it seems to be discarded on save... and the error changes to:
2015-11-18T08:35:24.5678951Z Invoking nuget with push C:\a\1\s\EventViewer\bin\Release\Project.Name.1.1.12.0.nupkg -s https://example.pkgs.visualstudio.com/DefaultCollection/_packaging/example/nuget/v3/index.json usfusmx4ez6mlfqwpp2abzc7e37denfcp7bxsep2hqij3tp4qwvq on C:\a\1\s\EventViewer\bin\Release\Project.Name.1.1.12.0.nupkg
2015-11-18T08:35:24.5688946Z C:\LR\MMS\Services\Mms\TaskAgentProvisioner\Tools\agents\default\agent\worker\tools\NuGet.exe push C:\a\1\s\EventViewer\bin\Release\Project.Name.1.1.12.0.nupkg -s https://example.pkgs.visualstudio.com/DefaultCollection/_packaging/Example/nuget/v3/index.json usfusmx4ez6mlfqwpp2abzc7e37denfcp7bxsep2hqij3tp4qwvq
2015-11-18T08:35:25.3467312Z Please provide credentials for: https://example.pkgs.visualstudio.com/DefaultCollection/_packaging/Example/nuget/v3/index.json
2015-11-18T08:35:25.3667189Z ##[error]Object reference not set to an instance of an object.
2015-11-18T08:35:25.3677179Z UserName: Password:
2015-11-18T08:35:25.4647059Z ##[error]Unexpected exit code 1 returned from tool NuGet.exe
I have also attempted to use a personal access token to no avail.
How do I get the publish step working?

The in-box NuGet Publish task has two options: "external" feeds and "internal" feeds. External feeds are intended for 3rd party services like NuGet.org, Artifactory, and expect a service connection with an API key.
Internal feeds are those hosted by Team Services. Instead of a service connection, you add the URL of the feed's NuGet endpoint. The build system relies on the Project Collection Build Service (for collection-scoped build definitions) or Project Build Service (for "this project"-scoped build defs) being a Reader or Contributor to the feed. Docs for all that are available here.

UPDATE: It is all fixed now so you can use the standard packaging steps in vNext and they work like a charm.
For the time being, I am substituting the NuGet Publisher step with a PowerShell build step.
This slots into the build after the "NuGet Packager" step and allows me to specify all of the credentials by setting up a package source before pusing the package.
$feedUrl = "https://example.pkgs.visualstudio.com/DefaultCollection/_packaging/Example/nuget/v3/index.json"
$packagePath = $ENV:BUILD_REPOSITORY_LOCALPATH + "\YourOrg.YourComponent." + $ENV:BUILD_BUILDNUMBER + ".nupkg"
Write-Host "Adding package Source"
$addSourceCommand = $ENV:BUILD_REPOSITORY_LOCALPATH + "\nuget sources add -name ""Example"" -source " + $feedUrl + " -username ""your.username"" -password ""yourpassword"""
Invoke-Expression -Command $addSourceCommand
Write-Host "Pushing package to NuGet"
$pushCommand = $ENV:BUILD_REPOSITORY_LOCALPATH + "\nuget push $packagePath -Source " + $feedUrl + " -ApiKey Example"
Invoke-Expression -Command $pushCommand

I landed here because I'm researching/configuring an internal deploy -- where I'm running my own NuGet Server (nuget.server, as opposed to visual studio online). The error was the same (or has similar text):
Object reference not set to an instance of an object
My solution, it turned out, was that the URL wasn't right.
The correct version is: http://server-name/NuGet/api/v2/package
For completeness, I had: http://server-name/NuGet/ which was wrong.

Related

VSTS REST API for creating TFVC Branch

I am trying to automate some of my build related tasks, which includes creating new release branch and build definitions for each release. I use VSTS TFVC for version management. When i am trying to do it with TFS REST API, i couldn't find any API for creating branch (microsoft documentation).
I can see .NET API available to do this; unable to find one as REST API.
No such a REST API to create branch for now, I have submitted a user voice here to suggest the feature, you can go and vote it up to achieve that in future.
As a workaround you can try below ways to create a branch in code or script:
Just as #Shayki mentioned you can use Client Object Model Reference
if you want to manage the Version Control programmatically.
Just use the "CreateBranch()" method in
Microsoft.TeamFoundation.VersionControl.Client.VersionControlServer
class to create a branch.
Besides, you can also use the Branch Command to create a branch.
tf branch olditem newitem [/version:versionspec] [/noget] [/lock:(none|checkin|checkout)] [/noprompt] [/silent] [/checkin] [/comment:("comment"|#commentfile)] [/author:authorname] [/login:username, [password]] [/recursive]
Just as what you see in "Branches" page, there isn't any way to create branch with the Rest API. And mostly, you can only read/get the information with the Version Control API for now.
If you don't want use C#, you can automate the process with Powerhshell:
param(
)
begin
{
# load the required dll's
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.VersionControl.Client")
}
process
{
$server = New-Object Microsoft.TeamFoundation.Client.TeamFoundationServer("http://tfsserver:8080/tfs/DefaultCollection")
$vcServer = $server.GetService([Microsoft.TeamFoundation.VersionControl.Client.VersionControlServer]);
$changesetId = $vcServer.CreateBranch('$/Demo/Code/Main', '$/Demo/Code/Dev/Branch', [Microsoft.TeamFoundation.VersionControl.Client.VersionSpec]::Latest, $null, "New branch from script", $null, $null, $null)
"Branch created with ChangesetID: $changesetId"
}

MSDeploy reporting "could not find file" when using setParamFile

I'm using MSDeploy as part of a custom task in TFS2017 (Update 1) but I can't get the setParamFile option to work. It throws one of two errors. If I specify no path, it says "could not find file" (see trace below). If I specify the path to the file on the build machine, it says invalid path.
I'm sure it's something incredibly simply but it's got me beat and Google is, for once, not being my friend! Any help is greatly appreciated.
I have provided the log unaltered as all this is only on internal servers. Without the setParamFile parameter, it deploys fine without updating the variable connection strings I'm trying to update, obviously.
2017-04-15T22:41:13.5065833Z Executing the powershell script: D:\BuildAgent\tasks\MSDeployPackageMultiSync\1.1.16\MSDeployPackageSync.ps1
2017-04-15T22:41:13.6472111Z packageFile= Find-Files -SearchPattern D:\BuildAgent\_work\194fdc7b9\Data Collection App - Release\drop\_PublishedWebsites\Cabi.FormsAdmin.Api_Package\DCA_API.zip
2017-04-15T22:41:13.6472111Z packageFile= D:\BuildAgent\_work\194fdc7b9\Data Collection App - Release\drop\_PublishedWebsites\Cabi.FormsAdmin.Api_Package\DCA_API.zip
2017-04-15T22:41:13.6784674Z DCA_API.SetParameters.xml
2017-04-15T22:41:13.7097114Z Package= D:\BuildAgent\_work\194fdc7b9\Data Collection App - Release\drop\_PublishedWebsites\Cabi.FormsAdmin.Api_Package\DCA_API.zip
2017-04-15T22:41:13.7097114Z ParamFile= -setParamFile:'DCA_API.SetParameters.xml'
2017-04-15T22:41:13.7097114Z DestinationProvider= auto
2017-04-15T22:41:13.7097114Z DestinationComputer= prerelweb3,prerelweb4
2017-04-15T22:41:13.7097114Z Username=
2017-04-15T22:41:13.7097114Z AdditionalArguments=
2017-04-15T22:41:13.7253453Z Deploying package to prerelweb3,prerelweb4
2017-04-15T22:41:13.7253453Z Deploying to prerelweb3
2017-04-15T22:41:13.7253453Z "C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:package='D:\BuildAgent\_work\194fdc7b9\Data Collection App - Release\drop\_PublishedWebsites\Cabi.FormsAdmin.Api_Package\DCA_API.zip' -setParamFile:'DCA_API.SetParameters.xml' -dest:auto,computerName='prerelweb3',userName='',password='',authType='ntlm',includeAcls='False' -setParam:name='IIS Web Application Name',value='WebServices (Secure)\FormsAdmin' -setParam:kind=DestinationVirtualDirectory,scope='WebServices (Secure)\\FormsAdmin',value='e:\cabi_sites\WebServices_Secure\FormsAdmin' -allowUntrusted
**2017-04-15T22:41:14.1315861Z ##[error]System.Management.Automation.RemoteException: Error: Could not find file ''DCA_API.SetParameters.xml''.**
2017-04-15T22:41:14.1315861Z ##[error]System.Management.Automation.RemoteException: Error count: 1.
2017-04-15T22:41:14.1315861Z Deployment to prerelweb3 complete
2017-04-15T22:41:14.1315861Z Deploying to prerelweb4
2017-04-15T22:41:14.1315861Z "C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:package='D:\BuildAgent\_work\194fdc7b9\Data Collection App - Release\drop\_PublishedWebsites\Cabi.FormsAdmin.Api_Package\DCA_API.zip' -setParamFile:'DCA_API.SetParameters.xml' -dest:auto,computerName='prerelweb4',userName='',password='',authType='ntlm',includeAcls='False' -setParam:name='IIS Web Application Name',value='WebServices (Secure)\FormsAdmin' -setParam:kind=DestinationVirtualDirectory,scope='WebServices (Secure)\\FormsAdmin',value='e:\cabi_sites\WebServices_Secure\FormsAdmin' -allowUntrusted
**2017-04-15T22:41:14.4909648Z ##[error]System.Management.Automation.RemoteException: Error: Could not find file ''DCA_API.SetParameters.xml''.**
2017-04-15T22:41:14.4909648Z ##[error]System.Management.Automation.RemoteException: Error count: 1.
2017-04-15T22:41:14.4909648Z Deployment to prerelweb4 complete
I assume that you are using this task: MSDeployAllTheThings. According to the logs, you are using it from Release. So you need to make sure that the "DCA_API.SetParameters.xml" file is published to artifact drop during the build and is also downloaded correctly during release. And then, you need to enter the absolute path in the Additional Arguments so that the task can find the file. For your scenario, if the "DCA_API.SetParameters.xml" file is placed in the same folder with "DCA_API.zip" file, you need to use this:
-setParamFile:"D:\BuildAgent\_work\194fdc7b9\Data Collection App - Release\drop\_PublishedWebsites\Cabi.FormsAdmin.Api_Package\DCA_API.SetParameters.xml"
I tried with absolute path at my side and didn't see any issue with it.

Unable to publish to Azure web app from a .net core application

I have a hard time to publish to azure website via command line. First of all I tried with VS generated powershell scripts which are getting generated inside .\Properties\PublishProfiles. I referred the answer in the post Publishing ASP.Net Core app to Azure silently fails in command line. I am able to pass only one parameter(which is password) as publish Properties. I tried to execute like below
PS D:\Sajan\SampleApplications\ADNext\DotNETCore\DotNetCoreWebApiSample\src\DotNetCoreWebApiSample\Properties\PublishProfiles> & '.\DotnetCoreWebApiSample - Web Deploy-publish' -pa
ckOutput 'D:\Sajan\SampleApplications\ADNext\DotNETCore\DotNetCoreWebApiSample\src\Output' -pubProfilePath 'DotnetCoreWebApiSample - Web Deploy.pubxml'-publishProperties #{'Passwor
d' = 'klk'} -verbose
The above attempt atleast trying to invoke msdeploy, but did not succeed. I got below output
Publishing with publish method [MSDeploy]
Executing command ["C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe" -source:manifest='C:\Users\579859\AppData\Local\Temp\PublishTemp\obj\Output\SourceManifest.xml' -des
t:manifest='C:\Users\579859\AppData\Local\Temp\PublishTemp\obj\Output\DestinationManifest.xml',ComputerName='https://dotnetcorewebapisample.scm.azurewebsites.net/msdeploy.axd?site
=DotnetCoreWebApiSample',UserName='$DotnetCoreWebApiSample',Password='{PASSWORD-REMOVED-FROM-LOG}',IncludeAcls='False',AuthType='Basic' -verb:sync -enablerule:AppOffline -enableRu
le:DoNotDeleteRule -retryAttempts:20]
Info: Using ID '25f52554-53e4-4a74-8392-df82e2a07806' for connections to the remote server.
I am not sure what is wrong here. I also tried to pass multiple parameters like below, but some syntax error was reported by powershell.
PS D:\Sajan\SampleApplications\ADNext\DotNETCore\DotNetCoreWebApiSample\src\DotNetCoreWebApiSample\Properties\PublishProfiles> & '.\DotnetCoreWebApiSample - Web Deploy-publish' -pa
ckOutput 'D:\Sajan\SampleApplications\ADNext\DotNETCore\DotNetCoreWebApiSample\src\Output' -pubProfilePath 'DotnetCoreWebApiSample - Web Deploy.pubxml'-publishProperties #{'usernam
e' = 'myusername' 'Password' = 'uL''AllowUntrustedCertificate' = false''AuthType' = 'Basic'} -verbose
Finally I tried to use Msdeploy directly like as below
#echo off
dotnet publish ".\DotNetCoreWebApiSample" --framework netcoreapp1.0 --output "D:\Sajan\SampleApplications\ADNext\DotNETCore\DotNetCoreWebApiSample\src\Output" --configuration Release
"C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:contentPath='D:\Sajan\SampleApplications\ADNext\DotNETCore\DotNetCoreWebApiSample\src\Output',-dest:contentPath='DotnetCoreWebApiSample',ComputerName='https://dotnetcorewebapisample.scm.azurewebsites.net/msdeploy.axd',UserName="myUser",Password="L",IncludeAcls="False",AuthType="Basic" -enablerule:AppOffline -enableRule:DoNotDeleteRule -retryAttempts:20 -verbose
Now I am getting the error "Error: A '-dest' argument must be specified with the 'sync' verb." I am totally clueless here. Please note that I have already provided -verb:sync and still msdeploy is complaining the same. can someone help me how to publish a .net core web app with command line to an azure web app. For msdeploy, I referred the post http://asp.net-hacker.rocks/2016/07/04/deploy-aspnetcore-to-azure.html. I am using sdk with version "1.0.0-preview2-003131".
The -source attribute in your MSDeploy call has a comma at the end but no additional flags. Replace the comma before -dest with a space:
-source:contentPath='D:\Sajan\SampleApplications\ADNext\DotNETCore\DotNetCoreWebApiSample\src\Output' -dest:contentPath='D

OctopusDeploy - Every website in the deploy has a different AppPool and Website name; how to deal; no other differences

I'm trying to setup a deploy process that targets 16 web sites each hosting an instance of the same application.
Websites and AppPools are named as such:
appServer1:
app10.site.com
app11.site.com
app12.site.com
app13.site.com
appServer2:
app20.site.com
app21.site.com
app22.site.com
app23.site.com
etc.
etc.
...with each website having a correspondingly named AppPool.
I am desperately trying to determine how to use a single Deploy NuGet Package step to target all of these websites/app pools using variables and a combination of powershell scripts if possible.
I'd like to have a single step where I can variable substitute the website and app pool names. As this is the only difference. I basically need the equivalent of being able to loop the nuget package step passing it a list of website and app pool names. I cannot simply use variables because I can only resolve to the machine level with variable scoping.
Create list of all Website and AppPool names, iterate them passing each value to a Step for execution. ForEach processing step for lack of better words.
I do have the ability to rename the AppPools if need be for a more consistent pattern, but I cannot change the website names
Any ideas would be greatly appreciated.
http://help.octopusdeploy.com/discussions/questions/3481-every-website-in-the-deploy-has-a-different-apppool-and-website-name-how-to-deal-no-other-differences
There's a lot to your question, but I'm going to take a stab at explaining our approach, in hopes of jogging your creative juices.
tl;dr
simply put, use your own powershell scripts to install the web-application. In there you can set the app pool name on a per website basis
For starters, we do do a separate deployment step for each project. The scripts we use will allow you to do all deployments from a single deploy.ps1 (including unique appPool names), but we find that it really helps keep each deployment nice and lean, and easy to manage. Each project get's it's own nupkg and therein contains the predeploy.ps1, deploy.ps1, and postdeploy.ps1 as well as a folder of build/deploy scripts that we've open sourcesd, and a folder of environment config xml files.
A sample of an environment config would be this. The name is simply [envName].xml
<!-- environments\Production.xml -->
<environmentSettings>
<webSites>
<app>
<physicalPathRoot>c:\inetpub</physicalPathRoot>
<physicalFolderPrefix>appname</physicalFolderPrefix>
<siteProtcol>https</siteProtcol>
<siteName>appname.tld</siteName>
<siteHost>appname.tld</siteHost>
<portNumber>443</portNumber>
<appPath>/</appPath>
<appPool>
<name>appname.tld</name>
<!-- valid identityTypes are: [LocalSystem, LocalService, NetworkService, SpecificUser, ApplicationPoolIdentity] -->
<identityType>NetworkService</identityType>
<!-- Set this value to the User the Service will run under in the format DOMAIN\username -->
<!-- If Running as 'NetworkService' then 'NT AUTHORITY\Network Service' is used -->
<userName>NT AUTHORITY\Network Service</userName>
<!-- Leave blank unless using SpecificUser -->
<password></password>
<maxWorkerProcesses>5</maxWorkerProcesses>
</appPool>
</app>
</webSites>
<serverDatabase>
<name>database_name</name>
<connectionString>REPLACED BY OCTOPUS</connectionString>
<providerName>System.Data.SqlClient</providerName>
</serverDatabase>
</environmentSettings>
You can see in the corresponding Get-EnvironmentSettings.ps1 where we load up the config, and then update it with any Octopus variables. This is the trickiest part, because we use dot-Notation to update the paths (case sensitive).
Our octopus variables really only contain information that is secret, as everything else lives in [environment].xml
| Name | Value | Scope
--------------------------------------------------------------------------
| webSites.app.appPool.password | supersecret | Production
So now a typical deployment script simply imports the modules, grab environmentSettings, update config, and install the web app.
# Top of the script, get Octopus environment and version
param(
[string] $version = $OctopusPackageVersion,
[string] $environment = $OctopusEnvironmentName
)
# Make sure a failed deployment actually fails
$ErrorActionPreference = "Stop"
# Import the modules
$currentDir = Split-Path $script:MyInvocation.MyCommand.Path
$moduleDir = "$currentDir\modules"
Import-Module BuildDeployModules
# Grab the environment settings
$environmentSettings = Get-EnvironmentSettings $environment "//environmentSettings"
$databaseSettings = $environmentSettings.serverDatabase
$websiteSettings = $environmentSettings.webSites.app
# update the config
Update-XmlConfigValues $currentDir\website\Web.config "//appSettings/add[#key='databaseName']" $($databaseSettings.name) "value"
Update-XmlConfigValues $currentDir\website\Web.config "//connectionStrings/add[#name='databaseConnection']" $($databaseSettings.connectionString) "connectionString"
Update-XmlConfigValues $currentDir\website\Web.config "//connectionStrings/add[#name='databaseConnection']" $($databaseSettings.providerName) "providerName"
# Install the web application
Install-WebApplication $environment $websiteSettings $version "anonymousAuthentication"
In doing all of this, the web application is installed into IIS with a specific application pool, and appropriate config transforms without relying on any unknowns.
Our nupkg structure looks something like this
appname.1.2.3.4.nupkg
environments
dev.xml
staging.xml
qual.xml
production.xml
modules
[all of our build modules]
website
[all of our website files]
This is super repeatable, easy to maintain, and easy to edit config. Hope it helps

Breaking MsBuild package & deploy into separate MsBuild and MsDeploy commands

I'm having a few problems breaking out an MsBuild package+deploy command into two separate commands. (I need to do this to pass additional parameters to MsDeploy).
The command that works fine looks like this:
msbuild "src\Solution.sln"
/P:Configuration=Deploy-Staging
/P:DeployOnBuild=True
/P:DeployTarget=MSDeployPublish
/P:MsDeployServiceUrl=https://192.168.0.1:8172/MsDeploy.axd
/P:DeployIISAppPath=staging.website.com
/P:AllowUntrustedCertificate=True
/P:MSDeployPublishMethod=WmSvc
/P:CreatePackageOnPublish=True
/P:UserName=staging-deploy
/P:Password=xyz
The separated packaging command looks like this:
msbuild "src\Solution.sln"
/P:Configuration=Deploy-Staging
/P:DeployOnBuild=True
/P:DeployTarget=Package
/P:_PackageTempDir=C:\temp\web
which works fine. But then the MsDeploy portion:
msdeploy
-verb:sync
-allowUntrusted
-usechecksum
-source:manifest=
'src\WebProject\obj\Deploy-Staging\Package\WebProject.SourceManifest.xml'
-dest:auto,ComputerName=
'https://192.168.0.1:8172/MsDeploy.axd?site=staging.website.com',
username='staging-deploy',password='xyz',authType='basic',includeAcls='false'
-enableRule:DoNotDeleteRule
fails, with the following error in WmSvc.log
wmsvc.exe Error: 0 : Attempted to perform an unauthorized operation.
setAcl/C:\temp\web (Read)
ProcessId=15784
ThreadId=31
DateTime=2011-03-30T14:57:02.4867689Z
Timestamp=3802908721815
wmsvc.exe Error: 0 : Not authorized.
Details: No rule was found that could authorize user 'staging-deploy',
provider 'setAcl', operation 'Read', path 'C:\temp\web'.
(and several more Read/Write operations)
Something is clearly going wrong with the paths it's trying to access (as it works fine with the other method) - I'm not sure it's even trying to use the iisApp targeting correctly, and at the moment I don't think the correct web.config's will be deployed either.
I've got this fixed now - I needed a different command to the one the automatically generated .cmd file was using, but comparing the two allowed me to fix it up (thanks #Vishal R. Joshi)
The differences I needed was:
basic authentication
allow untrusted certificates
?site=staging.webserver on the end of the MsBuild.axd path, as with my original command
override the IIS Web App name that is set in the params file
enable the do not delete rule
The winning command is as follows:
msdeploy
-verb:sync
-allowUntrusted
-source:package='src\WebProject\obj\Deploy-Staging\Package\WebProject.zip'
-dest:auto,ComputerName=
'https://192.168.0.1:8172/MsDeploy.axd?site=staging.website.com',
username='staging-deploy',password='xyz',authType='basic',includeAcls='false'
setParamFile:
"src\WebProject\obj\Deploy-Staging\Package\WebProject.SetParameters.xml"
-setParam:name='IIS Web Application Name',value='staging.website.com'
-enableRule:DoNotDeleteRule
-disableLink:AppPoolExtension -disableLink:ContentExtension
-disableLink:CertificateExtension
Hope this helps someone!
Add a delegation rule on the server using inetmgr to allow staging-deploy to carry out set-Acl operations.
Inetmgr -> Click on server node -> Management Service Delegation (in Management) -> Click Add rule to the right -> Choose the template labelled "Set Permissions for Applications" -> Accept defaults and click OK.
This should let you deploy any package or manifest with setAcl as long as the user you are deploying as, has permissions to the site you are deploying to.
You are able to specify the -setParam:name='',value='' flag when calling the MyProject.deploy.cmd file that is created when you generate a Package from a web project. The cmd is a friendly wrapper around msdeploy.exe, so you have no need to specify all the rest of the defaults.
Here's the details: http://evolutionarydeveloper.blogspot.co.uk/2013/05/specifying-environment-variables-at.html