I am starting to delve into Octopus Deploy and setting up my first deployment but I've ran into a bit of a snag. I have tried to be as specific as possible in lieu of finding someone who has experienced this same problem more recently. "This is not a new problem. See -reseach"
The setup
Versions:
TeamCity 10.0.4
Octopus Deploy 3.10.1
My continuous integration stack is comprised of TFS, TeamCity and now Octopus. My deployment process:
TeamCity runs the .NET application build. (Scripts, tests, etc..)
TeamCity Octopus plugin successfully creates the versioned Nuget packages.
A TeamCity build step using "OctopusDeploy: Push packages" pushes the packages to the octopus server.
In Octopus an external feed was added an tested called "Octopus Local Packages" pointing to the default internal octopus package directory: C:\Octopus\Packages.
An octopus project was created with a single step using the "Deploy an IIS Web Site" template, where the package section is setup like this:
The problem
Research
This has happened in the past on versions prior to 3.0.7 where it was fixed. Link to bug thread 1.
Then it started happening again at some point prior to version 3.4.15 where it was fixed. Link to bug thread 2.
Any help, fix, or workaround will be greatly appropriated. If there is a detail that I'm missing I will be more than happy to clarify.
Update
This may very well be a rookie mistake when it comes to Octopus. The error shown in the question is in fact a result of the deployment package not being found. The following are findings through documentation and trial an error:
Octopus Server (built in) is a push feed only. Meaning that packages can be uploaded to this feed but not consumed.
To consume Octopus' server local packages, namely the packages previously pushed to Octopus Server (built in) from say TeamCity, you have to serve those packages yourself.
You can server those packages through Octopus by creating an external feed that points to the server's package default path: C:\Octopus\Packages
Most Importanly, and the fix to the error!
When changes are made to a deployment process step, a new release MUST BE created. Simply redeploying the previous release will try to redeploy with the previous step settings saved in a snapshot and not with the new step settings. This may come a a surprise to TeamCity users like myself based on TeamCity's behaviour.
Related
I'm trying to use TFS 2015.1 on premise to build a CI pipeline for our dev & uat. I've created a vNext CI build, which builds fine. But when I want to add a deploy step for on prem IIS server, I only then see Azure Web Deployment options.
Ideally I wanted to add a step which uses the existing deploy (MS Deploy) profiles, which I'm able to use from VS2015 directly, using 'Publish'. However I see no option to do so.
How can I deploy the latest build to internal dev servers (not Azure)? I would like to use the MS Deploy option, unless there's a better way of doing it?
The fact that their is no option to starts to make me think there's probably a different way to accomplish it!
Thanks.
If you're able to upgrade to TFS 2015.2, web-based Release Management came out with it that works similarly to Build vNext with flexible and open-source tasks. You can also customize tasks.
Here's a link for IIS Web App Deployment from the vso-agent-task's GitHub repo where Microsoft stores updated versions of their tasks that you can download for web-based Build and Release Management.
I'll be publishing a blog about web-based RM with TFS 2015 Update 2 or VSTS on my website in the next few weeks. To give you an idea though, the starting point (for a web application) is a folder in your web project called WebDeploy (no significance - any name will do) that contains a PowerShell DSC script that configures the server, deploys the web files and then replaces any tokenised configs. To give you an idea see this post about how to use DSC to configure servers. (Only covers part of the final script though!) The next steps are:
In the build hub create a Website artifact - containing your web files and DSC script.
In the release hub for an environment use a Windows Machine File Copy task to deploy the artifact to a temp folder on the target node.
Then use a PowerShell on Target Machines task to execute the DSC script. After configuring the server the script copies the web files to their proper location, sorts out config using xReleaseManagement and cleans up the WebDeploy folder.
See this article for general details of the route I'm taking, but watch out as it has some errors eg the firewall instructions are incomplete (file and print sharing through the firewall needs to be enabled).
I can thoroughly recommend the PowerShell DSC route - I've had a few glitches but on the whole it feels very productive and the right way to be going.
I've been researching this for a while but can't find an answer.
I use Bamboo 5.3 with Artifactory plugin 1.6.2. I have a build project that generates a .war and two .zips. I also have a Bamboo Deployment project that creates releases with these three files and deploys to DEV, QA and so on.
For a build project I am able to use the artifactory plugin, that's fine. The problem is that I end up with a lot of artifacts if I publish all the builds. I would like to publish to Artifactory only the files from the releases, so that is happens less often, and that the people would see only the 3-4 releases tries, not the 150 builds.
My issue is that when creating my Deployment tasks (like download, copy, call ssh script...) there is no 'Artifactory Generic Deploy', like in the build project tasks.
I see there is a new Bamboo 5.4 with some improvement around the deployment process, maybe this could help?
Support for deployment task from Bamboo to Artifactory will be available starting with version 1.8.0 of the Artifactory plugin.
Here is the Jira issue.
I faced a similar issue. Hopefully the next release of the artifactory plugin will integrate with deployment projects.
If you are willing to use Maven to broker the deployment, deploy-file can get the job done.
In the deployment project, after your artifact download task add a Maven 3.x task for each artifact you want to send.
You'll need to specify a build JDK and for environment variables I'm using MAVEN_OPTS="-DskipTests=true -XX:MaxPermSize=4096m"
For the actual maven command:
deploy:deploy-file
-Durl=http://${bamboo.artifactory_username}:${bamboo.artifactory_password}#${bamboo.artifactory_url}/artifactory/${bamboo.destinationRepo}
-DrepositoryId=localhost
-Dfile=${bamboo.pathToArtifact}/${bamboo.arftifactName}-${bamboo.majorVersion}.${bamboo.minorVersion}.${bamboo.arftifactExtension}
-DgroupId=${bamboo.arftifactGroup}
-DartifactId=${bamboo.arftifactName}
-Dversion=${bamboo.majorVersion}.${bamboo.minorVersion}
-Dpackaging=${bamboo.arftifactExtension}
-DgeneratePom=true
Hope this helps!
The artifactory API is pretty usable for this purpose. You can deploy directly using curl in a shell script.
See https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API for the details.
Trying to make my life easier, Currently we have 4 developers working in Visual Studio 2012 and we are using TFS 2012 for source control. The project we work on is a multi-tenant web application (single source directory with multiple dbs) that is a mixture of legacy, asp and vb6 com components, coupled with new C# code. We use TFS for source control and for managing User Stories and Bugs. Because of the way our site works it can not be ran or debugged locally only on the server.
Source Control is currently setup with a separate branch for each developer that's working directory is mapped to a shared network path on the dev server that has a web site pointed to it in IIS. Dev01-Dev05 etc. The developers work on projects in their branch test it using their dev website, then check in changes to their own branch and merge those into the trunk. The trunk's work space is mapped to the main dev website so that the developers can test their changes against the other customer's dev domains to test against customizations and variances in functionality based on the specific dbs the are connected to.
Very long explanation but basically each dev has a branch and a site, that are then merged into the trunk with its own site.
In order to deploy our staging server:
I compile the trunk's website via a bat file on the server
Run a windows app I built to query TFS for changesets associated with
specific WorkItems in a certain status, and copy all the files for
those changesets from the publish folder to a deployment folder.
Run another bat file on the server to use RedGate's Deployment Manager
to create a package from those new files
Go to the DM site on our network to create and deploy that release (haven't been able to get the command line tools to work for this, so I have to do it manually)
Run any SQL scripts that have been saved off in Folders that match ticket numbers on each database (10 or so customer dbs) to support the release
I have tried using TFS automated build stuff and never really got it to build the website correctly. Played around with Cruise Control also with little success. Using a mishmash of skunk works projects to do this is very time consuming and unreliable at best.
My perfect scenario would be:
Gated Checkin, Attempt build/publish every time a developer merges into the trunk, rejects and notifies developer if the build fails.
End of the day collect the TFS Items of a certain status and deploys files associated with them to the staging site
Deploy SQL scripts for those TFS items across all the customer dbs in staging
Eventually* run automated regression UI tests, create new WorkItems or emails to devs if failed
Update TFS WorkItems to new state so QA/Customers know their items are ready to test in our staging environment
Send report of what items were deployed successfully
How can I get here so that I am not spending hours preparing and deploying releases to staging and eventually production? Pretty open to potential solutions, things that would be hard to change would be the source control we are using, can't really switch to subversion or something else so we are pretty stuck with TFS.
Thanks
Went back in and started trying to get TFS to build/publish my web solution. I was able to get a build to complete successfully. adding msbuild argument /p:DeployOnBuild=True and setting the msbuild platform to x86 seemed to do the trick on that.
Then I found https://github.com/red-gate/deployment-manager-tfs which gives you a build process template to do the package and deployment using the redgate tools. After playing with that for a bit I finally got it to create, package and deploy my build to our staging environment.
Next up will be to modify the template to run some custom scripts to collect only the correct items to deploy, deploy all the sql files and then to set the workitems to the appropriate statuses after completion.
Really detailed description of your process. Thanks for sharing!
I believe you can set up TFS to have gated check-in on a single branch, which if you can setup on trunk would make sure that the merges built successfully. That could trigger msbuild, if you can get that working or a custom build job.
If you can get that working then you'd be able to use that trunk code as the artifact to send to Deployment Manager. That avoids having to assemble the files for deployment through the TFS change sets, as you'd be confident that the trunk could always build.
Are you using Deployment Manager to deploy the database from source control as well as the application?
That could be a way to further automate the process. SQL Source Control and SQL CI allow you to source control the structure of a database, keep a database up to date on each check-in, and run database unit tests. They also produce database packages for Deployment Manager, so you can deploy a release that contains both the application and the database.
If you want to send me the command you're using in step 4 to deploy the release using Deployment Manager I can help out with that. The commands I use are:
DeploymentManager.exe --create-release --server=http://localhost:81 --project="Project Name" --apiKey=XXXXXXXXXXX--version=1.1
DeploymentManager.exe --deploy-release --server=http://localhost:81 --project="Project Name" --apiKey=XXXXXXXXXXX--version=1.1 --deployto=CI-Environment-Name
That will create a release version 1.1 using the latest available packages for that project. You can optionally specify the package to be used when creating the release with
--packageversion=<package name>=<version>
--packageversion="application=1.5
We are building packages for multiple deployment environments using TeamCity server and OctoPack. The problem is that tentacle agent chooses the latest by number version of the package, so it's the same (latest) package that is deployed on all environments. Here's the summary of our setup:
Environments DEV and STAGE;
Deployment to DEV is triggered from Git "dev" branch;
Deployment to STAGE is triggered from Git "stage" branch;
OctoPack is configured to generate packages MyProduct.1.0.0.dev-%build_counter% for DEV build configuration;
OctoPack is configured to generated packages MyProduct.1.0.0.%build_counter% for STAGE build configuration;
TeamCity is configured to expose OctoPack artefacts (NuGet packages) via its NuGet feed;
Octopus project is configured to deploy packages with NuGet Id MyProduct from TeamCity NuGet feed.
So what happens is that since DEV builds are run more frequently, they have larger %build_counter%, and STAGE doesn't get a chance to get a deployment of its own packages - Octopus tentacle preferes packages with 1.0.0.dev-* suffix.
This must be fairly common scenario, but I haven't found a simple way to solve it.
There are some parts that are not documented here: https://github.com/OctopusDeploy/Octopus-Tools. But if you look at https://github.com/OctopusDeploy/Octopus-Tools/blob/master/source/OctopusTools/Commands/CreateReleaseCommand.cs it is possible to figure out what you can do.
I think the tools is backward compatible, but not 100 % sure about that.
When you are using the octo tools, which I expect that you use, you can set the version (also called releasenumber now) option to specify the release number. If you doesn't specify anything else it will take the latest package so what you want to do is set the packageversion (also called defaultpackageversion now) that should be used for the release.
I think that should do it. If it doesn't, what are you using to create the release?
Example of what we are using from our TeamCity when using octo tools which we have added to the environment path on the build agents:
create-release --server=%conf.OctoServerApi% --project=%conf.OctoProject% --version=%env.OctopusPackageVersion% --deployto=%conf.OctoDeployEnv% --packageversion=%env.OctoPackPackageVersion% --apiKey=%conf.OctoApiKey% --waitfordeployment %conf.OctoExtraParams%
UPDATE:
The documentation for 2.0 is much better: http://docs.octopusdeploy.com/pages/viewpage.action?pageId=360596
Inspired by Tomas Jansson's answer, simply adding the following to Additional command line arguments in the OctopusDeploy: Create release build step (TeamCity v9) worked for me:
--packageversion=%build.number%
We decided to use AMAZON AWS cloud services to host our main application and other tools.
Basically, we have a architecture like that
TESTSERVER: The EC2 instance which our main application is
deployed to. Testers have access to
the application.
SVNSERVER: The EC2 instance hosting our Subversion and
repository.
CISERVER: The EC2 instance that JetBrains TeamCity is installed and
configured.
Right now, I need CISERVER to checkout codes from SVNSERVER, build, if build is successful, unit test it, and after all tests pass, the artifacts of successful build should be deployed to TESTSERVER.
I have completed configuring CISERVER to pull the code, build, test and produce artifacts. But I couldn't manage how to deploy artifacts to TESTSERVER.
Do you have any suggestion or procedure to accomplish this?
Thanks for help.
P.S: I have read this Question and am not satisfied.
Update: There is a deployer plugin for TeamCity which allows to publish artifacts in a number of ways.
Old answer:
Here is a workaround for the issue that TeamCity doesn't have built-in artifacts publishing via FTP:
http://youtrack.jetbrains.net/issue/TW-1558#comment=27-1967
You can
create a configuration which produces build artifacts
create a configuration, which publishes artifacts via FTP
set an artifact dependency in TeamCity from configuration 2 to configuration 1
Use either manual or automatic triggering to run configuration 2 with artifacts produced by configuration 1. This way, your artifacts will be downloaded from build 1 to configuration 2 and published to you FTP host.
Another way is to create an additional build step in TeamCity for configuration 1, which publishes your files via FTP.
Hope this helps,
KIR
What we do for deployment is that the QA people log on to the system and run a script that deploys by pulling from the team city repository whenever they want. They can see in team city (and get an e-mail) if a new build happened, but regardless they just deploy when they want. In terms of how to construct such a script, the team city component involves retrieving the artifact. That is why my answer references getting the artifacts by URL - that is something any reasonable script can do using wget (which has a Windows port as well) or similar tools.
If you want an automated deployment, you can schedule a cron job (or Windows scheduler) to run the script at regular intervals. If nothing changed, it doesn't matter much. I question the wisdom of this given that it may mess up someone testing by restarting the system involved.
The solution of having team city push the changes as they happen is not something that team city does out of the box (as far as I know), but you could roll your own, for example by having something triggered via one of team city's notification methods, such as e-mail. I just question the utility of that. Do you want your system changing at random intervals just because someone happened to check something in? I would think it preferable to actually request the new version.