Vagrant and Chef-solo
Vagrant is a very useful tool to build development environments, where it provides tools to build virtual machines that contain everything you need to get started with building software. Consider, for a moment, working on a team that builds software and relies on a service-oriented architecture (SOA), and this software is composed of a number of different services. In order for it to work, you may be required to install and configure all of the dependent services to even begin working on a part of the system; this could be a time-consuming and error-prone exercise for even seasoned developers. Now imagine that all you had to do was download a configuration file and execute vagrant
to do it for you—this is the world of Vagrant.
One of the interesting facets of Vagrant is that it has support to provision new instances using a number of different mechanisms. Currently, this list includes 10 or so different tools, but the most interesting two are Chef-solo and Chef client. By now, you should be comfortable with how you might provision a virtual machine using the Chef client; it's not much different than provisioning an EC2 instance or a dedicated server. However, we haven't discussed using Chef-solo much yet, so this is a good time to learn more about it.
Installing Vagrant
Historically, Vagrant was installed via RubyGems; this is no longer the case, and if you have an older version installed as a gem, it is recommended that you remove it before installing Vagrant. Installers for all supported platforms (OS X, Windows, and Linux) are available at the following URL:
http://www.vagrantup.com/downloads
If you are new to Vagrant, then in addition to installing Vagrant, you will want to install VirtualBox for simplicity, as Vagrant has built-in support for VirtualBox. Vagrant does support other providers such as VMWare and AWS, but it requires plugins that are not distributed with the core Vagrant installation in order for them to work.
Once you have installed Vagrant and VirtualBox, then you can continue on with the following examples.
Provisioning a new host with Vagrant
Provisioning a new virtual instance requires that you build a Vagrant configuration file called Vagrantfile
. This file serves two purposes: to denote that the directory is a Vagrant project (similar to how a Makefile
indicates a project that is built with Make), and to describe the virtual machine that is being run, including how to provision it, what operating system to use, where to find the virtual image, and so on. Because this is just a plain text file, you can include it along with any auxiliary files required to build the image such as cookbooks, recipes, JSON files, installers, and so on, and commit it to the source control for others to use.
In order to begin, you will want to create a directory that will house your new Vagrant project. On Unix-like systems, we would bootstrap our project similarly to the following command:
mkdir -p ~/vagrant/chef_solo cd ~/vagrant/chef_solo
Windows hosts will be the same except for different paths and changes in methods of directory creation. Once this step is complete, you will need to create a skeleton configuration located in ~/vagrant/chef_solo/Vagrantfile
. This file can be generated using vagrant init
, but we would not want to use the contents of the generated file; so, we will skip that step and manually construct our Vagrantfile
instead (with a simple one-line configuration that uses a base image of Ubuntu 13.10). Your Vagrantfile
should look like the following code:
Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" end
Here, "2"
is the API version, which is currently Version 2.0 as of this writing, and the configured base image (or box
) is the Ubuntu Trusty (14.04) 64-bit image. Before you use this base image, it needs to be downloaded to your local machine; you can add it to Vagrant using the box add
command:
vagrant box add ubuntu/trusty64
For future references, if you want to find alternative OS images to use for your Vagrant machines, you should look at Vagrant Cloud (https://vagrantcloud.com/), where you can find a number of other freely available base images to download for use with Vagrant.
Booting your Vagrant image
Once your base image has completed downloading, you will use the vagrant up
command to boot up a new virtual machine. By doing this, you will instruct Vagrant to read the Vagrantfile
and boot a new instance of the base image:
Bringing machine 'default' up with 'virtualbox' provider... ==> default: Importing base box 'ubuntu/trusty64'... ==> default: Matching MAC address for NAT networking... ==> default: Checking if box 'ubuntu/trusty64' is up to date... ==> default: Setting the name of the VM: chef_solo_default_1402875519251_51266 ==> default: Clearing any previously set forwarded ports... ==> default: Clearing any previously set network interfaces... ==> default: Preparing network interfaces based on configuration... default: Adapter 1: nat ==> default: Forwarding ports... default: 22 => 2222 (adapter 1) ==> default: Booting VM... ==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:2222 default: SSH username: vagrant default: SSH auth method: private key default: Warning: Connection timeout. Retrying... ==> default: Machine booted and ready! ==> default: Checking for guest additions in VM... ==> default: Mounting shared folders... default: /vagrant => /Users/jewart/Temp/vagrant/chef_solo
As you can see from the output, Vagrant performed the following things:
- Used the base image
ubuntu/trusty64
- Configured VirtualBox to use a NAT adapter, mapping port
22
to2222
- Started the VM in headless mode (such that you don't see the VirtualBox GUI)
- Created a user,
vagrant
, with a private key for authentication - Mounted a shared folder mapping
/vagrant
on the guest to the Vagrant workspace on the host
Now that you have a running guest, you can control it by running vagrant commands from inside of the vagrant workspace (~/vagrant/chef_solo
); for example, you can SSH into it using the following command:
vagrant ssh
And you can destroy the running instance with the following command:
vagrant destroy
Go ahead and SSH into your new guest and poke around a little bit—you will notice that it looks just like any other Ubuntu 14.04 host. Once you are done, use destroy
to destroy it so that you can look at how to provision your Vagrant image using Chef-solo. It's important to know that if you use destroy
on your guest, changes to your Vagrant image are not persisted; so, any changes you have made inside it will not be saved and will not exist the next time you use vagrant up
to start the VM.
Combining Vagrant with Chef-solo
In our previous example, our Vagrantfile
simply declared that our guest relied on the ubuntu/trusty64
image as the base image via the config.vm.box
property. Next, we will look at how to extend our configuration file to use the Chef-solo provisioner to install some software on our guest host. Here, we will use Chef-solo to install PostgreSQL, Python, and a web application inside of the guest.
You will probably notice that the configuration sections in the Vagrantfile
look sort of like resources in Chef—this is because they both leverage Ruby blocks to configure their resources. So with Vagrant, in order to specify the provisioning mechanism being used, the config.vm.provision
option is set to the desired tool. Here, we will use Chef-solo, which is named "chef_solo"
; so, we will extend our Vagrantfile
to indicate this:
Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "chef_solo" do |chef| # ... Chef specific settings block end end
For the most part, Chef-solo operates a lot like the traditional client-server mode of chef-client. The primary differences result from the fact that Chef-solo does not interact with a central Chef server and therefore lacks support for the following:
- Node data storage
- Search indexes
- Centralized distribution of cookbooks
- A centralized API that interacts with and integrates infrastructure components
- Authentication or authorization
- Persistent attributes
As a result, if you are writing recipes to be used with Chef-solo, you will be unable to rely on search for nodes, roles, or other data and may need to modify the way you find data for your recipes. You can still load data from data bags for complex data, but they will not be centrally located; rather, they will be located in a number of JSON files that contain the required data.
There are a number of options available for the Chef-solo provisioner in Vagrant. For the most up-to-date documentation of Vagrant, be sure to visit the official Vagrant documentation site at http://docs.vagrantup.com/v2/.
Most of the configuration options are ways to provide paths to various Chef resources such as cookbooks, data bags, environments, recipes, and roles. Any paths specified are relative to the Vagrant workspace root (where the Vagrantfile
is located); this is because these are mounted in the guest under /vagrant
and are the only way to get data into the host during the bootstrap phase. The ones we will be using are:
cookbooks_path
: This consists of a single string or an array of paths to the location where cookbooks are stored. The default location iscookbooks
.data_bags_path
: This consists of a path to data bags' JSON files. The default path isempty
.roles_path
: This consists of an array or a single string of paths where roles are defined in JSON. The default value isempty
.
In our case, we will be reusing our example cookbooks from the earlier chapter. You can fetch them from GitHub at http://github.com/johnewart/chef_cookbook_files; either download the ZIP file or clone them using Git locally. Once you have done that, copy cookbooks
, roles
, and data_bags
from the archive to your Vagrant workspace. These will be the resources that you will use for your Vagrant image as well. In order to tell Vagrant's Chef-solo provider how to find these, we will update our Vagrantfile
again to include the following configuration:
Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "chef_solo" do |chef| chef.cookbooks_path = "cookbooks" chef.roles_path = "roles" chef.data_bags_path = "data_bags" end end
Inside of the provision
block, we have a Chef object that effectively represents a Chef client run. This object has a number of methods (such as the path settings we already saw), one of which is the add_recipe
method. This allows us to manually build our run list without requiring roles or data bags and can be used, as shown in the following example, to install the PostgreSQL server with no special configuration:
Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "chef_solo" do |chef| chef.cookbooks_path = "cookbooks" chef.roles_path = "roles" chef.data_bags_path = "data_bags" # Build run list chef.add_recipe "postgresql::server" end end
This will tell Vagrant that we want to use our defined directories to load our resources, and we want to add the postgresql::server
recipe to our run list. Because cookbooks are by default expected to be in [vagrant root]/cookbooks
, we can shorten this example as shown in the following code, as we are not yet using roles or data bags:
Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "chef_solo" do |chef| chef.add_recipe "postgresql::server" end end
As you are already aware by now, we may want to perform more complex configuration of our hosts. Let's take a look at how to use both roles and data bags as well as our cookbooks to deploy our Python web application into our Vagrant guest similar to how we deployed it to EC2:
Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "chef_solo" do |chef| chef.cookbooks_path = "cookbooks" chef.roles_path = "roles" chef.data_bags_path = "data_bags" # Build run list chef.add_role("base_server") chef.add_role("postgresql_server") chef.add_role("web_server") end end
Just like the cookbooks
path, the roles path is relative to the project root if a relative path is given.
Additional configuration data for Chef attributes can be passed into Chef-solo. This is done by setting the json
property with a Ruby hash (dictionary-like object), which is converted to JSON and passed into Chef:
Vagrant.configure("2") do |config| config.vm.provision "chef_solo" do |chef| # ... chef.json = { "apache" => { "listen_address" => "0.0.0.0" } } end end
Hashes, arrays, and so on can be used with the JSON configuration object. Basically, anything that can be turned cleanly into JSON works.