Puppet's Two Abstraction Layers

Today I basically concluded I'd finished with providers, but I was stupid enough to spend a while on the phone with a friend who's kind enough to talk about Puppet with me.

In the course of this discussion, I realized that I've been stupidly conflating two layers within Puppet this whole time. Making providers into a separate layer is a good start toward fixing that, but it's not enough.

See, there are two aspects to the type library within Puppet: The first aspect functions as an abstraction layer above the operating system: I say Create user johnny and it says useradd johnny or niutil -create / /user/johnny, depending on the platform or my preferences or whatever.

The second aspect involves the tool that Puppet is; the transactional configuration management tool. This tool is inherently two-phase: I must first collect the state of any given object, then compare it to the prescribed state, and then either log or fix the problem.

These layers should be entirely decoupled within Puppet, but they are not. There's no good reason for their lack of separation, I just didn't have a clear design in my head when I started oh so long ago. Now that I have providers created for a few types, I realize that these providers should probably handle validation, for instance -- clearly, anyone who ever uses the providers outside of Puppet will want the same validation that Puppet has. And they'll certainly want the same documentation that Puppet has.

But then you walk into these complicated problems -- what stays in the transactional layer? Anything? Is it even possible to define a generic API between these layers, such that they don't have to be coupled? Right now, types and their associated providers share a unique API -- the service type expects a specific set of methods on service providers, and the package type expects a completely different set of methods on the package providers. Is it possible to fix this?

I can't see the separation; I don't know exactly where it sits. It's funny, because this keeps causing flashbacks to organic chemistry lab, where the question was always, "which layer do I keep?" You always had an aqueos layer and a nonpolar layer, and your experimental result was in one layer and the other was waste. In this case, though, it feels like a gradient in which I have to draw a hard line that doesn't really exist.

It seems a lot like 99% of what's currently in what I'll call the transaction layer -- the types -- should be pushed into providers. This includes the docs, the validation, the value specification, etc. Most likely, the transactional layer can be reduced to a single class, rather than one class per type. I expect that the transactional layer will keep most of the metaparams, since scheduling clearly takes place at that layer not at the provider layer, and relationships are also at that level.

This means that the transactional layer's job is basically just to do the three-step: Retrieve, compare, sync. It shouldn't need much more than that.

Ouch. This is nearly a completely rethink of the whole damn system. Fortunately, it should have almost no impact from the outside -- no language changes, and very few changes to what amounts to the external API -- but it's still, um, big.

Sun, 13 Aug 2006 | Tags: puppet, abstraction, design