Testing Cached Values
I'm currently in the middle of the largest refactoring effort I've ever done, while simultaneously learning tons about how to be a better developer. I'm constantly feeling a bit overwhelmed, a bit behind the curve, and like someone's going to look at my code one day and say, "hey nimwit, you just pull this string here and suddenly 2/3 of your code just goes away."
However, one thing I've really grokked recently is that if you're fighting your tools too much, you are on the wrong track, and one way in which it seems I'm constantly fighting my tools is the combination of cached values and testing.
For instance, I have an HttpPool class that knows how to set up Net::HTTP instances with all of the SSL information they need. This class caches SSL information like the certificate and key, so that each connection doesn't hit the disk for this info, which is obviously a pretty decent use of the cache. This caching generally takes this simple form:
def ssl_host
unless defined?(@ssl_host)
@ssl_host = Puppet::SSL::Host.new
end
@ssl_host
end
Yes, you could just do @ssl_host ||= Puppet::SSL::Host.new, but I've gotten some weird behaviour out of that in the past, and it also throws a warning for undefined variables.
So anyway, this works fine in unit tests; I test the caching, and I use mocks everywhere else. When I get to integration tests, though, it starts to really hurt, especially since I try not to do much mocking in my integration tests.
For instance, say I've got two unrelated tests that do an ssl connection. They each create some certificates, start a daemon, and try to connect. In this situation, the first one caches the ssl information, and the second one uses the cached values instead of its own new certificate, and you get invalid certificates.
After talking to Rick Bradley on #nashdl on IRC (gotta represent!), I've decided on at least an initial course of action. I'm going to create a module that provides both a caching and cache-clearing interface; anyone using cached values would use this caching interface instead of caching the values themselves, and the module itself would give you a single point of entry for clearing all caches on the system.
My first instinct was to create a Cache class or struct and keep a list of them in the caching module, so they can be cleared as necessary, but my recent work with TTLs has made me realize that time-based concepts of cache dirtiness are much better than actively cleaning.
So now, I'm thinking that the caching module will just have a timestamp, and only cached values created after that time will be valid. Before the Cache struct returns any values, it will always check that time, and it will know whether the value it has is still good or should be discarded.
This keeps us from maintaining global lists of caches, and it also makes clearing caches insanely cheap -- just reset a timestamp. Given that Puppet already sometimes has onerous memory requirements, I also like that it makes the caches themselves more likely to get garbage collected, since only the caching instance ever knows about the actual cache itself.
So my new method would look something like this:
def ssl_host
cache(:ssl_host) { Puppet::SSL::Host.new }
end
That bit with the block is something I just thought of; if we've got a value, it's not called, but if we need a value, it's there for us. Pretty sweet.
Now just to implement it.
Wed, 07 May 2008 | Tags: programming, testing, caching
Jay and I converge on testing
Those four people who have been reading this blog for a while know I've been struggling to think and program like Jay Fields. In particular, he seems to have presented a few rules in the past that don't like to be used together:
- Inline setup code: You apparently should not use before or setup blocks with your code; instead, you should have all of the setup code in every method.
- Mocks instead of Stubs: Here he claims that it's always better to use expects instead of stubs.
- One expectation per test: With only one expectation, you're only testing one feature at a time, and you're guaranteed that each behaviour is tested separately.
Now, let's do a simple combinatorial exercise, and put these three rules together:
- You should have all related code in your test
- You should only use Mocks (expects), not Stubs (stubs)
- You should only have on expectation (expects) per test
It's pretty clear that, like the old saw about programming ("All programs can be reduced to one line of code with a bug in it"), Jay is pointing us toward tests that can largely only be one line of code. Yeah, I know sometimes setup methods don't involve any mocking, but often they do.
And, since your tests can only be one line of code, they can't test very much, which means that all of your methods need to also be one line of code, else they aren't testable. (Yes, I'm being a touch extreme here, but that is where the arrow is pointed, anyway.)
You can see how this would kinda drive me bonkers. Some local dev friends have been trying to help me see the error of my ways, mostly so my code would stop looking like such crap; I've learned a helluva lot in the last 8 months or so, and most of it has actually made my code look more like Jay would recommend. I will say it's blindingly obvious Jay is doing internal development at enterprises, rather than developing as part of a consistent team producing software that is distributed to the wider world.
But one can only go so far, and the three rules above, in combination, are just way too far. I've often kind of sputtered expasperatedly at Jay's posts, especially his announcement of his new testing tool, expectations. Again, I can kind of see where he's going with that, but you've got another thing coming if you think I'm using it, especially given how happy (at least, relative to test/unit) I am with RSpec.
Also, I think it's just stupid having all setup code inline. DRY ("don't repeat yourself") is just as true in your test code as anywhere else, and having a maintainable test code base is, IMO, more important than having your normal code base be maintainable, because tests are kind of unnecessary. If you have good, readable, maintainable tests, then people who contribute will also contribute tests. If your tests are all 50 lines long and have lots of repetition, then 1) you've got 5x the amount of code you should, which is wicked expensive, and 2) you've got so much code no one will look at it. Yay, never getting patches with tests in them. My favorite example of this is Steve Yegge's rant Code's Worst Enemy; he describes his 500k line Java project with no tests, which is a lot of code but much less code than if it had tests. I've experienced in Puppet that test code seems to be much harder to maintain that normal code (although maybe it's just own crap test code, not normal test code), and having 5x test code than normal code would make me just quit writing unit tests entirely.
So, I am absolutely overjoyed to announce that Jay has changed one of his rules: He now recommends stubs over mocks. This is clearly just for setup code and such, but it's a big step. He even goes into using stub_everything, which I find is the only way to build tests that aren't fragile. For instance, say you start with this code:
class MyClass
def go
start()
finish()
end
end
describe MyClass do
before
@me = MyClass.new
@me.stubs(:start)
@me.stubs(:finish)
end
it "should start when going" do
@me.expects(:start)
@me.go
end
it "should finish when going" do
@me.expects(:finish)
@me.go
end
end
Now you find you need a validation method, so you add this test:
it "should validate when going" do
@me.expects(:validate)
@me.go
end
Update: Fixed code to actually call @me.go in the validate test.
Oops. Now your single test passes, but your two old tests break, because you were only stubbing start and finish, instead of using stub_everything. Your setup code needs to be modified to take this new call into account (or, if you're Jay, you need to modify every test in your suite; yay). This comes up constantly. If you specifically mock or stub methods during setup, then you are almost guaranteed to have cascading failures when you expand your code.
Anyway, the point is, if I tried to follow Jay's rules, then the above trivial change -- I add one line of code to a very simple method -- would result in me adding a test for that line, plus at least one line of code in every other test in that suite. Instead, if I use stub_everything, then I add my new test and I'm done. (Well, kind of; notice I'm not actually testing the order of the method calls, which is actually pretty tough.)
My recommendation is to read Jay, since he's clearly thinking and talking about aspects of testing that not many others are, but read him with a skeptical eye, and be willing to say "That's just nuts!" and write your own relatively abstracted test code. And if you're working with people who can't think to look at their setup code when a test fails, then you need to find a different job.
Tue, 06 May 2008 | Tags: ruby, testing, programming, bdd, tdd, jayfields
Caching and REST
(by Luke)
One of the things I was supposed to write about last week was how I'm rethinking some of Puppet's internal caching. This rethinking is a direct result of listening to ThoughtWork's IT Matters Podcast on REST (I've only listened to part 1 so far). I actually listened to the episode three times, because it's only about 20 minutes and I listened to it on a 60 minute bike ride, which worked well because it was so windy that day that I didn't hear the whole thing any of those listenings.
I'll hopefully write later about how this podcast made me rethink how environments are used in fileserving, but for now, I'm going to focus on caching.
Indirection
For a couple of months now, Puppet has had an Indirector module that is basically useful for connecting classes with collections of instances of those classes. The only reason you'd really even bother to use it is if you had multiple collections, and needed to interact with different collections at different times, but you wanted those differences to be transparent.
For instance, when retrieving node information, you just call this code:
Puppet::Node.find("mynode")
Somewhere else, you'll have configured which collection (the word I'm currently using is terminus) this uses, and the Indirector just delegates the find call to the right collection. For nodes, you might be using the exec collection, which calls an external script, turns the resulting YAML into a Node instance, and returns it (or returns nil if nothing was found).
I think the Indirector is pretty cool, and it's certainly simplified a lot of my modeling of interacting with different sources of information. Those who are familiar with REST, at least how it's usually done in the Ruby world, will recognize the find as one of the methods usually used for REST interfaces -- it's mapped to the HTTP verb get. One of the primary design goals of the Indirector was to facilitate REST interfaces, so the methods we're indirecting are, not coincidentally, exactly the methods you'd implement for REST support.
Caching
One of the later additions to the indirection code was support for cache collections. That is, you might have a canonical collection, and then a cache collection for speed or proximity purposes. Following our Node example above, if you were using the exec collection, you'd probably want to have the results cached in the yaml collection, so they were inexpensive to retrieve.
The critical question with any caching system is how to know when the cache is dirty. How do you know if you should use the cached node information or go back to the source?
I expect there are as many answers to this question as there are caching implementations, just about. I had never implemented a caching solution before, and I probably misinterpreted my discussions with Rick Bradley, because I ended choosing a not-very-good system. The current cache invalidation mechanism is based on relative versions: If the version of the cached object is older than the version of the object in the other collection, then your cache is dirty.
What is a version? Well, normally it's just the timestamp of when the instance was created. This might work okay for some systems, but in general, the timestamp ends up being pretty useless. Look at our Node example -- the timestamp of the exec collection is always later, because we retrieve the cache version, then generate a new node using the exec collection, and compare. Duh. The answer's always the same.
Even worse, in most situations the cache doesn't save you any work, because you're pulling fresh data from the original source. If we have to re-execute the external node script to get the latest node version, we haven't saved any effort at all, we've just added a bunch of useless work, which is stupid.
Puppet 0.24.4 "fixed" this problem by saying that the cached node's version was the timestamp of the node's Facts cache. If the facts are updated, then the cache needs to be updated. This seems to mostly work, but it feels like a hack for something that should be easy.
TTL
So, on to the podcast. It was a good podcast in general, and they focused a good bit on caching. At first I found this pretty strange -- why is caching an important design criterion? As they talked, though, I realized that a generalized, simple caching model is useful a lot more places than I would expect, including in Puppet.
There didn't seem to be any disagreement over the best way to handle knowing when a cache is dirty -- they apparently just use time-to-live (TTL) or expiration headers. I think it was the second time listening through that I realized that the vast majority of my caching problems could be fixed with this.
Puppet has a natural TTL for most of its information -- every host runs every half an hour, so if you set a TTL of half an hour (or whatever you're run interval is), then you'll get fresh data once a run, and cached data the rest of the time. In the above Node scenario, the exec collection would set the TTL of the node (so that your external node app could pick its own TTL), or Puppet would have a default TTL equal to the run interval. Then, when Puppet goes to check whether its cache was dirty, it could just compare the TTL against the current time -- no need to hit both collections, and no arbitrary definition of "version".
This actually makes even more sense with the current problem I'm trying to solve. I'm trying to remodel the SSL certificate signing process, and it's gotten pretty messy. With this, though, you just set the TTL of the certificate to its own internal TTL, and you use the local system as the cache the CA server as the ultimate source. If there is a local cert and it's still valid, use it; if there's a local cert but we're past its TTL, then discard it and get a fresh cert; if there's no cert, then get one from the server and cache it locally.
Next Steps
I don't have the whole thing figured out mentally yet, but I'm pretty close. At the least, the next step is to replace the current broken version-based cache with ttl-based caching. The two things I most need to resolve are:
- Who's responsible for the ttl? Is it the indirection (e.g., Node), or the collection (e.g., the external node script)?
- How does the user configure the ttl? Say I want the ttl for my node to be 30 seconds instead of thirty minutes, or I want to invalidate the cached values for all nodes; how would I do that?
Obviously, these two things are linked -- the user needs a complete configuration path from the command line or configuration file to the bit that actually sets the ttl.
For now, fortunately, I don't need to worry about it, because I can just stick with the run interval as the TTL for essentially everything I'm doing. As things get more interesting, though, we're going to want to configure these values, because....
TTL Can Help Provide Change Control
One of my primary goals in moving the catalog compiling process to REST is to enable a decoupling between compiling and applying. In other words, I want people to be able to apply a configuration without recompiling.
Imagine a configuration TTL of a week -- every host recompiles its configuration during some specific maintenance window, like Sunday morning between 2 and 6 am. They still apply their configurations every half an hour, but that's normally just validating that nothing has drifted.
Obviously, this wouldn't be used by most shops -- most people would still want all hosts to recompile every time. But for those shops that are highly worried about change control, or those who want to do rolling upgrades, where they upgrade 10% of a pool of servers at a time, this would help a lot. You take your pool of servers, trigger a recompile on 10%, and once you're confident they're working, you trigger a recompile on another 10%, and so on.
Once you can do that with Puppet, it'll feel almost enterprisey. :)
Wed, 02 Apr 2008 | Tags: programming, thoughtworks, podcast, rest, caching, design, ruby, api, luke
Testing Initialization Code
One of the things I continually struggle with in testing is code that runs during initialization. A lot of times this code is very simple:
def initialize(name, cert, key)
raise Puppet::Error, "Cannot manage the CRL when :cacrl is set to false" if [false, "false"].include?(Puppet[:cacrl])
@name = name
unless read(Puppet[:cacrl])
generate(cert, key)
save(key)
end
end
It's easy to test that first and second line, and it's entirely obvious what that last bit does, but it's fantastically difficult to test, especially if you follow the advice of Jay Fields and try to stick to one expectation per test.
If you do it all in one test, you end up with a relatively long test that covers a single specific version, but it doesn't describe the behaviours very well. You want something like this:
describe "when initializing" do
it "should fail if :cacrl is set to false"
it "should set the name"
it "should read the crl in from disk"
describe "and no crl exists on disk" do
it "should generate a new crl"
it "should save the new crl"
end
end
The only way to do this, though, is to use stub_everything, and then individually test for each method, which is messy.
Even worse, you now have to stub out these methods every time you want to test an instance of the class in any other way. For instance, (as you might have guessed) I'm remodeling our Certificate Revocation List as a class, and I'm going to need to test the actual revocation, along with storage to disk. Each of these are made more complicated by the code in the initialize method.
Why, then, don't I just leave the code out?
Well, I could easily have it lazy evaluate, only running when someone actually asks for the crl. The problem is that I've consistently found that lazy evaluation causes more problems than it saves. I tend to run into permission problems (because the code doesn't evaluate until puppetmasterd is running as puppet, when it sometimes doesn't have the permissions it needs), and it's just very difficult to really control ordering.
Also, it just feels messy to reorganize easy code to make it more testable. There seems to be a postulate in the testing world that code that's difficult to test is bad code, but I defy anyone to argue that the above code is unclear or "bad code", other than just directly saying it's bad because it's hard to test.
I expect that in this case I'll have a generate_and_save method that, well, generates and saves, or maybe a load_or_create method that does this bit. Yay. Because simple code is hard to test, I end up with less simple code, and I still have to use stub_everything.
Tue, 01 Apr 2008 | Tags: ruby, testing, initialization, programming
A bit more DTrace
(This should have been posted a while ago, but I guess I had a problem and it's been sitting uncommitted for a while.)
After pulling apart the skip method in the lexer, so that the various parts are in separate methods, I get this as my count:
Puppet::Parser::Lexer munge_token 56778 358 20335592 Class new 28242 889 25132822 Puppet::Parser::Parser ast 25881 1147 29695496 Fixnum < 1817071 16 30723097 StringScanner check 1829886 26 48732560 String length 3757782 20 78611361 Puppet::Parser::Lexer::TokenList each 56778 6618 375813485 Puppet::Parser::Lexer find_token 56778 6714 381227038 Hash each 84949 4563 387630769 Puppet::Parser::Parser import 9 45754308 411788774 Puppet::Parser::Parser _reduce_132 9 45755009 411795083 Object catch 56018 8086 452970031 Puppet::Parser::Lexer scan 173 2751816 476064309 Racc::Parser _racc_yyparse_c 173 2751907 476080064 Object __send__ 173 2751984 476093248 Racc::Parser yyparse 173 2752322 476151712 Puppet::Parser::Parser parse 173 2752742 476224530 Array collect 331 1446548 478807659 Array each 26303 18476 485983221
The interesting one there is the Lexer.find_token method -- I just created that, and it looks like it's taking 38/48 of the total parse time, which is a helluva lot.
This method is responsible for picking the token to return, and the complicated aspect of the method is that it has to return the longest match, which is currently done by matching each token in turn (skipping those that don't match), and picking the longest match. This is expensive, because it means that every token is iterated over for every returned token, which means it scales at O(N^2), which is bad.
Mon, 28 Jan 2008 | Tags: tools, dtrace, ruby, programming
Ten Challenges by Yegge
In following Steve Yegge's comments on code size, I noticed an old post of his about ten challenges, which is really about books:
These are books that are important to me. Not in the Lewis Carroll or Herman Melville sense; they're not cherished fictional works, or even fictional works that are just thick enough to prop up the couch. For the most part they're technical books. But each of them is a book that I return to regularly as I try to figure out well, how stuff "works".
There are more than ten, obviously, but I decided to cap this list at ten books just to have a shot at finishing this essay before the end of the year.
I've read his first book, GEB, (all the way through, really!) and it's basically the most important book in my life. It helped pull me out of a post-college intellectual funk, and as Yegge says, it's hugely informative about intelligence. It provides great insight on recursion, self-reference, and much more. In fact, I did a talk (note that's a 16MB download) at RubyConf this year that was basically things I've learned from trying to apply this book to programming.
His other books are all directly programming books, which is slightly disappoint, but I've added them all to my Amazon Wish List since I've been recently feeling a bit like I've let my book learning fall away and I need to get back into it. If you're a Puppet user who's been looking for a way to show your appreciation, buying one of these books would be a great way to do so, hint hint.
Yegge's also got a Ten Great Books post, which ends up looking pretty similar but with different actual books.
Mon, 24 Dec 2007 | Tags: programming, books, development
Ohloh is sweet
The more I play with Ohloh, the more I like it. I've added Facter and ldapsh to the site (Puppet was already there), and I've played a bit with how to record contributions and how code is tracked. I have to say, I've been waiting for a site like this for ages. It's very developer-centric, in that it's not necessarily something that a user would want to use to find and download projects, but I'm perfectly comfortable with that. Frankly, though, I think it could quickly overtake sites like Freshmeat and SourceForge, at least partially because they don't seem to have changed in years, even though there are plenty of opportunities for them to get better.
I love that I can get a great idea of what's actually happening in a project, rather than some mystical 'activity' metric, and people can claim their work across multiple projects. I think a site like this could really encourage open source development because people will be able to point to a central metric of what they're doing across the different projects they're on.
You can check my stats on the site, although anyone who follows my work won't exactly be surprised by where my time goes. You can also compare Puppet, Bcfg2, and Cfengine, which I think is interesting (it's especially interesting to check the number of contributers_ to Cfengine).
Fri, 21 Dec 2007 | Tags: development, ohloh, programming, forge
A little more RSpec
I've been using RSpec more thoroughly recently, so I figured it's time produce a post covering some of the things I've learned.
Particularly, thanks to much forcing by Rick, I have now come to realize that RSpec involves writing two very coupled and very important documents: The spec, which describes how your code should behave, and the tests, which are attached to the spec and theoretically prove that your code does behave this way.
This was a big step in realization for me, because I had always thought of my tests as just that: tests. That is, their job was to make sure my code was behaving as I thought it should be behaving. When working with Rick the other day, though, he kept saying my spec was bad and I kept wanting to shout that I understood my tests were bad, let's just fix them.
After enough prodding, though, and some judicous using of RSpec's formatting, I came to see that he views them as separate things, and that I should too.
With RSpec, you can obviously run your tests:
luke@phage(0) $ bin/spec unit/indirector/indirector.rb ......... Finished in 0.010605 seconds 9 examples, 0 failures luke@phage(0) $
But you can also run them and produce a document that looks suspiciously like a spec:
luke@phage(0) $ bin/spec unit/indirector/indirector.rb --format s Puppet::Indirector when available to a model - should provide a way for the model to register an indirection under a name Puppet::Indirector when registering an indirection - should require a name when registering a model - should create an indirection instance to manage each indirecting model - should not allow a model to register under multiple names - should make the indirection available via an accessor Puppet::Indirector when redirecting a model - should give model the ability to lookup a model instance by letting the indirection perform the lookup - should give model the ability to remove model instances from a terminus by letting the indirection remove the instance - should give model the ability to search for model instances by letting the indirection find the matching instances - should give model the ability to store a model instance by letting the indirection store the instance Finished in 0.014107 seconds 9 examples, 0 failures luke@phage(0) $
Note that it's still running the tests here (as shown by the lack of failures at the bottom), it's just producing output differently.
I was surprised to find Rick wanting to read this output again and again, completely ignoring any associated test code. It finally hit me, of course, that if we aren't doing a good job of describing the expected behaviour, then how can we do a good job of testing it?
When we first started working through some code recently, he kept niggling about some code that wasn't in the spec, which kinda miffed me because it was clearly being tested. When I finally realized what he meant about the difference between the spec and the tests, no matter how tightly coupled they are, I realized why it mattered so much to him: There was behaviour in my code that I essentially hadn't documented, even if I was confident I was testing that it was working.
Certainly an interesting realization.
Sun, 23 Sep 2007 | Tags: ruby, rspec, puppet, development, programming, testing, bdd
Reification in Puppet
It's apparently Ruby day in Puppetton (as in, Puppet + Hobbiton, or something like that). I've been trying to add more Ruby blogs to my blogroll, and I guess they're starting to succeed.
Piers Cawley has a post about reification:
However, in OO circles (or maybe just in my head), reification is a good thing. Its the process of taking something abstract and turning it into a real object. Usually, the word gets used for big things like turning an intractable method into an object as a step on the way to refactoring that method. I tend to use it in a slightly broader sense. For me, reification is the process of turning something (a method or a data structure usually) into a full blown object with its own behaviour.
This is something I've only recently started thinking about as a concrete process. I learned to program (according to most definitions) in the perl world, and I relied ridiculously heavily on hashes as my mechanism for managing all of my data. Even when you go OO in perl, you're still relying on hashes, so there isn't as much impetus to reify as you might find in Ruby.
That reliance on hashes has stuck with me over the years, and I'm only just starting to get over it. A heckuva lot of parts of Puppet have been modeled as hashes long past their useful lifetimes, and even when I correctly use a separate class to model something, I'll often stupidly avoid differentiating two uses of a class until it's so painful I can't deny it any longer.
Probably the biggest example of this is that I did not have a parser-specific resource class until relatively recently (sometime in 2007, anyway, I think). Before it existed, I was using the same Hash-like class I use for transferring configurations over the wire. Hey, it existed, it stores parameters, yay. The problem is that the two classes share almost no other functionality. Parser resources have tons of special behaviour -- they get to determine whether a class can override a given parameter, they have to add default values, they can be exported or virtual, and they can model types that are defined in the parser, as opposed to only types built into the RAL.
When I finally couldn't take the pain any more, I created a simple parser resource class. Then, before I knew it, I moved tons of code into that class, and all of the parser structural classes -- mainly the parser, the interpreter, and the scope classes, at the time -- became much easier.
I'm actually going through all of this again right now. As insane as it sounds, until two weeks ago, Puppet did not have a Configuration class to model a specific configuration. Well, really, a configuration is a collection of resources, and it's the resources that matter, right? Except there's so much more than that -- the resources have relationships to each other, and those relationships really matter. More importantly, there's a lot of code related to these configurations. When I created a separate Configuration class, my Interpreter class went from 708 lines to 105. That means that 6/7 of my Interpreter class was actually Configuration code, but because I hadn't reified my configuration class, I didn't realize it.
It's actually stupider than that. When I first created a Configuration class, it only modeled the configuration as it was being compiled; I still returned something that wasn't a stand-alone object that could have code attached. When I finally realized, I started really punching myself in the head. I now have a Compile class that manages interaction between the parser and the configuration, and I have a Configuration class that is the result of a compile. It really is a world of difference.
The way that I tend to think about this process is that I'm converting "else" code into "me" code. That is, the Interpreter previously had a ton of code that interacted with something that wasn't itself, and now essentially all of that code has been moved into the Compile and Configuration classes, and all of that code can pretty much be used as private methods, meaning it's all "me" code now -- it's the classes talking about themselves, instead of a class talking about something "else".
This provides what might be a good way to notice whether it's time to reify -- if you have a lot of code that uses receivers on the methods, then you're mostly interacting with something else. There's a very good chance that "else" should be its own class, and all that code should become methods on that new class.
This is something I'm still working on in Puppet. I've already created a bunch of new classes for the next release (Compile, Configuration, Node, Facts, Checksum) and I'll be shocked if I don't end up with more. The key to these classes is that I'm not adding code -- I'm taking code that already exists but talks about something "else" and making it talk about itself. In the majority of cases, this results in a significant reduction in line count, because I lose all the reference management and argument validation that has to go on when a class has to assume some hostile external force is interacting with it. I even get to make a lot of that code private, because no one needs to care about it any more.
Tue, 11 Sep 2007 | Tags: ruby, puppet, reification, refactoring, oo, design, programming
Puppet is one project but many products
Reginald Braithwaite has an interesting post on the difference between a project and a product:
Software applications are often defined in advance as masses of features, loosely organized into modules and components. One of the things I do as a matter of course when starting a project is to count the number of products. A product is a piece of software that stands alone and delivers value. In the context of a big project, products are parts of the project that could, all by themselves, be projects.
His central point is that attempting to develop multiple products in parallel is prohibitively difficult:
This is not about maximizing programmer productivity. In fact, it has nothing to do with programmers. Its about a very scarce resource: management attention. Whether you are building a commercial application for an ISV or a client project for a government ministry, there is a very limited ability of the stake-holders to digest the projects progress and contribute with feedback and priorities.
When the management is also the primary developer, as is the case with Puppet, things become even more pointed.
Puppet itself is composed of many products. In the majority of cases, I was able to see that they were such and developed them to be essentially stand-alone, such as the support for configuration, features, and autoloading new classes from disk. However, there are some much larger products within Puppet that I have only recently realized are stand-alone products, and this late realization has caused me no end of pain.
In particular, the language and the resource abstraction layer (RAL) are separate products. I have always known they should be decoupled, but it's more than that -- they should have almost nothing to do with each other. Even further, the transactional system that Puppet uses should itself be a separate product, and should only have the faintest idea of how the RAL works, and should certainly not know a damn thing about the parser.
Going yet further, Puppet's mechanism for providing an abstract, indirect interface to all of its network services (which I've recently been calling the Indirector) should itself be a separate product, and each service should also be a separate product. Interestingly, in at least one case (file serving) a single product consists of components within the RAL and the Indirector.
Fortunately, most of Puppet's products are small enough that a single person can get the majority of the product built in days, not weeks or months. The language and RAL are clear counterexamples, and the first Indirector took, um, forever, I think, but the configuration system was probably around a week, and features were less than a day. Once the products exist, thought, you still have to manage maintenance, refactoring, and further development, and seeing that clean separation makes all of these easier.
Either way, I know I feel a lot more productive these days when I pick a product within Puppet and focus on it for an extended period of time. I've got a lot of new tickets sitting in Trac right now, but I can't afford to swap the Indirector out right now. Once I'm comfortable with it, I'll clean up the tickets and put a release out. Then, the next release will focus on a different product.
I think this article helps to point toward something I was thinking anyway: Individual Puppet releases should each be focused on a single product, although of course they'll all have to include bug fixes in other products. I'll never get ahead unless I start thinking about how to move the needle that is Puppet's functional state, and I think picking a product and hammering exclusively on it is a great way to do so.
Tue, 11 Sep 2007 | Tags: puppet, ruby, product, programming, development
Tags
