Old Ideas

I have been thinking about what I can do to make Puppet a bit more obviously useful, because people don't seem to think that there is a real visceral difference between it and cfengine, and the two things that keep popping up are centralization of the output of Puppet (e.g., logs, metrics) and using the semantics of the configuration to make that output easier to manage (e.g., error log messages should know what objects produced them and what services they are affecting).

In somewhat standard fashion for me, I decided to do both at once, and I decided to start with logs since there is much less expected (note I said expected not desired) from logs than from metrics. In particular, I can get away without a database, at least to start.

I had already set up tagging in Puppet, so I figured I just needed to add those tags to the log messages, and then set up another server module to accept the logs. No real problem there.

Before I got a chance to implement this, though, I spent some time talking about it with my friend Andrew Shafer, and I realized that the logs should also store the path from the objects generating them. My objects have long had a concept of a path, but I haven't used it much and so it's not very well characterized. The basic idea is that the path would map directly to hierarchy of the configuration -- e.g., server classes would be analogous to directories in the path -- but because I've basically not used the paths much, that hasn't really been the case.

So, I went to implement what seemed a pretty easy idea, add the path and tags to the log messages. As I went about it, I realized that some refactoring was in order, because the current methods of logging had no way to link the messages with the object creating them. When I went about that refactoring, though, I was reminded that I originally wrote the Log class to accommodate this kind of link.

So, I spend tons of time and cash learning about Web 2.0 stuff, do lots of thinking, and go about starting something that I think will be cool and new, and I find that I planned it out, oh, 3 weeks into development of Puppet.

It's horribly embarrassing to have to reinvent a great idea, but at least I'm on track to do some cool stuff, even if I should have had it working 5 months ago.

I now have this all working, and the additional benefit (although planned, months ago) is that the log messages can easily include the path to the object that generated the message, or just the object name, or none. This should make parsing of log messages much easier, and should move to a database very easy, since the object will be a separate field, instead of sometimes-but-not-always being in the log text itself.