Chef workflow for new cookbooks - workflow

Question

While building my own cookbooks I find myself constantly within this cycle:
Change cookbook on my local computer
Upload modified cookbook to chef server
Run chef-client on remote machine
Repeat
Since I am new to chef, I repeat that cycle extremely often, but I find rather cumbersome uploading and downloading so frequently from the chef server.
How experienced chef users ease this cycle? I learnt Chef with the free hosted solution, but I am not sure if I should be better off using just Chef-solo and move back later to the hosted version once I have many servers and more experience with chef.
Is there maybe a workflow where I can quickly try changes to my cookbooks directly on my remote machine while using hosted Chef but without uploading them?

Answer 1

I do all my development locally on my laptop, using Vagrant. When the cookbook is ready, it gets pushed to it's new Git repository home and integrated into my production chef server infrastructure.
The following example runs nginx on a virtualbox image.
Example
Install vagrant plugins (only needs to be done once):
vagrant plugin install vagrant-omnibus
vagrant plugin install vagrant-berkshelf
vagrant plugin install vagrant-chef-zero
Use Berkshelf to generate a new application cookbook.
$ berks cookbook demo
create demo/files/default
create demo/templates/default
create demo/attributes
create demo/definitions
create demo/libraries
create demo/providers
create demo/recipes
create demo/resources
create demo/recipes/default.rb
create demo/metadata.rb
create demo/LICENSE
create demo/README.md
create demo/Berksfile
create demo/Thorfile
create demo/chefignore
create demo/.gitignore
run git init from "./demo"
create demo/Gemfile
create demo/Vagrantfile
Edit the following files (Details below):
Vagrantfile <-- Controls vagrant's operation
metadata.rb <-- List community cookbook dependencies
recipes/default.rb <-- Calls to community cookbook recipes and LWRPs
Running vagrant will startup a virtual machine that is provisioned using chef-client. The chef zero plugin will run a local embedded instance of chef-server. The berkself plugin is used to automatically load the cookbook dependencies.
vagrant up
The following command will rerun chef-client (following edits):
vagrant provision
Finally, the really big advantage of doing development, using local virtualization, is that you can tear everything down and build it again from scratch:
vagrant destroy -f && vagrant up
Vagrantfile
Controls vagrant's operation. In this case I'm only starting a single VM, provisioned using chef-client:
Vagrant.require_plugin "vagrant-omnibus"
Vagrant.require_plugin "vagrant-berkshelf"
Vagrant.require_plugin "vagrant-chef-zero"
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# Box config
config.vm.box = "saucy64"
config.vm.box_url = "http://cloud-images.ubuntu.com/vagrant/saucy/current/saucy-server-cloudimg-amd64-vagrant-disk1.box"
# Virtualbox config
config.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--memory", 1024]
end
# Networking config
config.vm.network "private_network", ip: "10.0.0.10"
# Plugin config
config.omnibus.chef_version = :latest
config.chef_zero.enabled = true
config.berkshelf.enabled = true
# Chef config
config.vm.provision :chef_client do |chef|
chef.add_recipe "demo"
end
end
The omnibus plugin is responsible for installing the desired version of chef. The Berkshelf plugin will download cookbook dependencies and when combined with chef-zero will upload cookbooks during each provision run.
metadata.rb
Add nginx as a cookbook dependency:
name 'demo'
maintainer 'YOUR_NAME'
maintainer_email 'YOUR_EMAIL'
license 'All rights reserved'
description 'Installs/Configures demo'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '0.1.0'
depends "nginx"
The Berksfile is preconfigured to load the cookbook dependencies, listed in the metadata file, from the community cookbook repository.
site :opscode
metadata
recipes/default.rb
Run the default nginx recipe
#
# Cookbook Name:: demo
# Recipe:: default
#
# Copyright (C) 2014 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#
include_recipe "nginx"

Answer 2

I go through that loop most of the time during my testings. But I also use Chef solo when I need to try several minor changes in a recipe. If you have a test machine with a local copy of your chef repo then you can try making changes there and running chef solo to see the results. This way there's no uploading.