I've been reading up on how people do continuous delivery with some of the popular toolsets.
Lots of posts (like this one) seem to indicate that a common way of doing things is to use something like capistrano to push software from your builds to your machines, and then chef or puppet to configure anything related to it.
My question is, do people generally push there software directly into a special git repo for binary assets, or can capistrano fetch it out of a maven repo? The maven approach seems most natural to me, but I don't seem to be able to find much information on it - which is what makes me think it's not the approach that people are generally taking.
Basically, I'm slightly confused as there seems to be a gap between the build output (where one would normally publish to a maven repo) - and where the delivery tools expect to find the software you have asked them to deploy (which seems to be a file system, or a git repo)
When it comes to artifacts; I attempt to leverage the jenkins plugin to upload to S3. Here's a link to it.
Basiclly right now, all my ci goes through Jenkins and when I get a complete build I upload it to a bucket and have chef pull the tarball/war/gem from it and install it from there.
Related
I searched the web now for several hours but couldn't get around this:
Is there an easy way to deploy a private repository from Github to a staging/development server on each push (or at least manually)? (Best would be if only FTP-data of development server would be needed for this).
I found this: How can I automatically deploy my app after a git push ( GitHub and node.js)? but this kind of "tutorial" in the best answer stops at the point of what exactly to insert into the build.sh. And what modules are needed for this on the development server? SSH, GIT, Ruby? Maybe this sounds stupid to you, or is a wrong thinking of mine, cause nowhere on the net I found any answer to this.
The problem is, that most time, the server on which the contents of the master branch should be deployed is on a shared hosting server, where you doesn't always have SSH, GIT, Python, Ruby, etc. on which most solutions for deploying from github seem to rely on... :/
http://beanstalkapp.com/ is really great at this, you can just enter FTP-Data and deploy automatically or manually for chosen repositories and branches. So I wondered why I couldn't find a similar easy way to deploy from Github?
Thank you very much in advance!
Jonas
It isn't really clear what type of project you have, but here are a couple of ideas.
If your code is written in a compiled language, then you could:
Have a Jenkins server as mentioned in the other comment
Write a simple script in bash that does a git pull and compile and add a cron job to it.
Use an automation framework like Chef or Puppet which would automatically keep the compiled binary up to date.
If your code is an interpreted language (like HTML & JavaScript), then you could:
Use vagrant for local testing. The biggest reason is that changes are live on your local system. It only takes a git push on your machine and a git pull on the production server to make your changes live globally.
Your best bet is probably going to be #2.
I haven't worked much with Nexus before, so I'm still trying to work out a product lifecycle that works for us.
I want to be able to export certain sets of repository artifacts from nexus into another nexus container. It looks like the only way to do this, as of now, is to pull artifacts as a set of dependency builds and then deploy them to the new repository. This may be what we have to go with, I was just looking for a better approach.
It looks like mirroring or proxying won't give us the fine grain control of export that I need.
I see that I can just copy artifacts out of nexus, but I'm not sure how to tell the new nexus container that it is supposed to manage those files.
What I want to do is be able to put a set of artifacts onto a DVD that can be run as a localized nexus instance for the purpose of installing software at a customer site. It appears that for any customer that will allow a connection back to us for software installs, they can be treated with the same install setup we use for QA. The reason to use a nexus deploy instead of an installer is because we need to be able to roll back a "patch install", as each path/install set would be maintained as a release version. Right now this is all done in custom code, since there doesn't seem to be an installer that handles rollbacks (with backups) after the install has completed.
This would be a non-standard usecase for Nexus and it strikes me that Nexus would be overkill for your requirements. Any web server can act as a Maven repository, once the files are available in the correct format.
For example, why not burn the desired subset of repository files onto DVD and include a copy of Jetty? Jetty which can be launched from the DVD and serve up the local content over HTTP.
I am looking for alternatives to NWDI (Stands for Netweaver Development Infrastructure by SAP) source control system for developing
Java EE Applications. Primarily because:
NWDI is not DVCS : So developers have to be online to do just about anything.
User Interface: Its very difficult to use and train developers on using this system.
Tracking Changes/Generating Reports: Very limited support for this.
For example I cant find out what projects (Files within the project) have been changed in the last 2 weeks.
Code Review: You can do code reviews, it has a good diff utility. But thats about it, there is no way to attach code reviews to a change request.
Branching and Merging are extremely painful.
However the current system has a few handy features:
Automatic Builds: No need to write any build scripts , everything is built in. So when a new repository (we call it track)
is created it automatically configures the build based on the type of components (Supported by the repository) selected on creation.
A Central Build is triggered whenever a developer commits (Activates the changes). Irrespective of the status of the build the changes are now inflicted on the entire team.
Automatic push to Central Test Server: While creating a repository you can define all the servers (Central test, QA, Prod). A developer can push his changes by a
click of a button to Central Test Server. Again everything is built in and there is no need to extend any hooks like you have to do in Mercurial.
I was exploring Mercurial, Kiln and but couldn't find anything helpful. For mercurial Hooks can be used to do the same but I guess some customization effort is required.
Are there any cool DVCS like Mercurial which does the above 2 as well or is it something that I have to customize to make it work?
I don't know of a DVCS proposing everything build-in.
The only alternative (not DVCS, but with some of DVCS characteristics in it) is Rational Team Concert or RTC (free for up to 10 developers).
With a DVCS alone, the usual setup for CI and reviews is:
Git
Gerrit (review)
Jenkins (scheduler)
See "Using Gerrit Git Review with Jenkins CI Server"
Looks like there nothing useful out of the box. I am going to try out Kiln as it appears to be easy to use and try customizing it.
I've never worked with any version control systems before. Now I'm trying to learn Mercurial, but I'm confused (I've already read about 10-15 articles + hginit.com). I don't know how to organize the workflow.
I have a testing server and a production server. I work from my office computer and from my home laptop. I make changes directly on the testing server, and every week or so copy new code to my production server. I also need wiki/issues/etc. pretty much everything bitbucket.org has. I know that's a bad way of doing things.
Is there any tutorial or articles on how to organize the workflow? I'd also appreciate any schemes/sketches describing the process.
Thank you!
[Edit: Changed based on comments]
Using Bitbucket
Once you have created an account.
You should be able to create a repo with an appropriate url. Then you can clone it to create a local repository.
Check out getting started.
See the following to push the updates to BitBucket.
BitBucket comes with very extensive documentation.
Also there are, other useful tools to work with BitBucket:
BitbucketExtension that allows you to use command line for a number of operations.
Using Mercurial Queues and bitbucket.org
Organizing workflow
You will have to evolve a workflow that suits you. In your case, it looks like you have a testing server and production server.
So , you can setup two repositories, one for the testing server ad one for production. You can make push to testing server automatic so that you can test out the changes immediately. You can tag releases that are then pushed to production server.
Your local repo can be used to publish changes to testing server.
You can push the approved changes, tagged to BitBucket repository.
Tipically a deploy in production does not involve just a mere source code update (build) but requires a lot of other important tasks like, for example:
Db scripts
Configuration files (differents from test\production)
Batch to schedule
Executables to move to the correct path
Etc. etc.
In our company we just send an email to a "Release email address" describing the tasks in order, which changeset need to be published (TFS), which SP need to be updated, db scripts and so on.
I believe there's not a magic tool that does these tasks automagically in order, rollback included; but probably there's something better than email that helps to keep track of releases in production.
Do you have any tools to suggest or practices to share?
When multiple tasks are required to support a full project deployment (and that's frequently the case, in my experience), I'd suggest using a build/deployment tool. I've used Ant in the past with great success, but know others who swear by Capistrano, Maven and others.
Using Ant, I wrote a script that would:
Pull the specific revision I wanted from my VCS
Create a tarball of the target directory on the remote machine (in case a rollback was required)
Create a MySQL dump file of the database (also for rollback purposes)
Delete the remote directory and SSH the new content just pulled from the VCS
Perform various other logistical operations (setting file perms, ownership, etc.)
Create a release branch on the VCS itself
Create a tag with the appropriate version information so I always had a snapshot of the code base at that moment of deployment.
Hope that helps some. I've written a few blog posts about this that may (or may not) be useful. They're dated now, but the general information should still be solid enough.
Introductory thoughts
Details of how I use Ant for deploying--including scripts
You might be interested in the Team Foundation Build Recipes Website, that showcases some build scripts developed using SDC Tasks Library and the MSBuildTasks library
How about something like SVN? You can put all of your code in a repository, then when you are ready to release from production bring your stuff over from test. Then you'll have very specific revisions with information on what happened. SVN keeps track of all of it.