Puppet is one project but many products

Reginald Braithwaite has an interesting post on the difference between a project and a product:

Software applications are often defined in advance as masses of features, loosely organized into modules and components. One of the things I do as a matter of course when starting a project is to count the number of products. A product is a piece of software that stands alone and delivers value. In the context of a big project, products are parts of the project that could, all by themselves, be projects.

His central point is that attempting to develop multiple products in parallel is prohibitively difficult:

This is not about maximizing programmer productivity. In fact, it has nothing to do with programmers. Its about a very scarce resource: management attention. Whether you are building a commercial application for an ISV or a client project for a government ministry, there is a very limited ability of the stake-holders to digest the projects progress and contribute with feedback and priorities.

When the management is also the primary developer, as is the case with Puppet, things become even more pointed.

Puppet itself is composed of many products. In the majority of cases, I was able to see that they were such and developed them to be essentially stand-alone, such as the support for configuration, features, and autoloading new classes from disk. However, there are some much larger products within Puppet that I have only recently realized are stand-alone products, and this late realization has caused me no end of pain.

In particular, the language and the resource abstraction layer (RAL) are separate products. I have always known they should be decoupled, but it's more than that -- they should have almost nothing to do with each other. Even further, the transactional system that Puppet uses should itself be a separate product, and should only have the faintest idea of how the RAL works, and should certainly not know a damn thing about the parser.

Going yet further, Puppet's mechanism for providing an abstract, indirect interface to all of its network services (which I've recently been calling the Indirector) should itself be a separate product, and each service should also be a separate product. Interestingly, in at least one case (file serving) a single product consists of components within the RAL and the Indirector.

Fortunately, most of Puppet's products are small enough that a single person can get the majority of the product built in days, not weeks or months. The language and RAL are clear counterexamples, and the first Indirector took, um, forever, I think, but the configuration system was probably around a week, and features were less than a day. Once the products exist, thought, you still have to manage maintenance, refactoring, and further development, and seeing that clean separation makes all of these easier.

Either way, I know I feel a lot more productive these days when I pick a product within Puppet and focus on it for an extended period of time. I've got a lot of new tickets sitting in Trac right now, but I can't afford to swap the Indirector out right now. Once I'm comfortable with it, I'll clean up the tickets and put a release out. Then, the next release will focus on a different product.

I think this article helps to point toward something I was thinking anyway: Individual Puppet releases should each be focused on a single product, although of course they'll all have to include bug fixes in other products. I'll never get ahead unless I start thinking about how to move the needle that is Puppet's functional state, and I think picking a product and hammering exclusively on it is a great way to do so.

Tue, 11 Sep 2007 | Tags: puppet, ruby, product, programming, development