One of problems when using chef as automation tool for provisioning is manage dependencies of upstream cookbooks (as consequence of reuse cookbooks). In typical scenario a cookbook A depends on B which in turn depends on C and D.
During development berkshelf can be used to manage dependencies tree of a single cookbook, which is quite handy. With berkshelf we can pass our cookbook to others, who can then test the cookbook using exact versions of its direct and indirect dependencies, which save others from headache due to version's mismatch.
Unfortunately there is no such facility during actual deployment, which is no arguably critical to success for large scale deployment of chef across multiple teams who constantly add, modify, reuse chef cookbooks.
When deploying a cookbook in production, we want that chef run exact versions of the cookbook and all of its dependencies as they were tested (predictability is obviously important in production).
So far chef-client and chef-server fail to meet this critical requirement. Using
berks apply ENVIRONMENT
to specify version of in environment can solve the problem only partly. This is because people usually put more than one cookbooks into the run list of single node (either directly or via role). This may cause version conflict if different cookbooks in run list depends on different version of the same upstream cookbook (the problem that is known as dependency islands). In addition there is no effective way to enforce that we use the same version of ruby, chef-client, gems that are referenced by some of the cookbooks in the dependency tree in production as in test. I myself had a problem with different version of chef-client in different environments.
The solution
One quite elegant solution for the problem mentioned is using combination of bundler, chef-solo and single top level environment cookbook.
We use bundler to enforce version of ruby, berkshelf, chef-solo. The bundler Gemfile is e.g.
source "https://rubygems.org"
gem "chef"
gem "berkshelf"
ruby "2.0.0"
To avoid dependency islands we create a single top level cookbook per node, that so called environment cookbook is managed by berkshelf, which enforces same version of all cookbook's dependencies in both test and production environment. To provision a node, we pull the desire version of top level cookbook from e.g. a web server or git sever to the node, then runbundle install --deployment
bundle exec chef-solo -c config -j solo.json
which is the same as when running an ruby application.
The benefit of this approach is beyond solving dependencies issue, it reduce steps required to deploy a chef cookbook by eliminating chef-server completely and make running cookbook in development and test exact the same as in production.
