Thursday, February 17, 2011

Working in the Community

The PostgreSQL community makes decisions by consensus.  The craziness of this approach should be readily apparent.  The United States Senate has come under withering criticism for rules which permit one or a small number of Senators to hold up the business of the chamber to such an extent that it's difficult to get anything done.  Forty-one senators can filibuster just about anything, to the frustration of the fifty-nine who want to have a vote and get on with it; and sometimes just one Senator can make himself or herself a near-complete obstacle to progress.

Our community is not dissimilar, although we lack a formal roster of membership, votes are frequently not as simple as "yes or no", how much any particular person's vote counts depends on both who they are and how well they justify their argument, a quorum to do business depends on the nature of the proposal but rarely exceeds three individuals, and nobody's in charge of counting the votes, so it's perfectly possible for different people to have different ideas about what was or was not decided.  Furthermore, it's hard to imagine a way of fixing these problems that wouldn't be worse than the disease.  A more formal voting process would slow the decision-making process to a crawl, and concentrating too much decision-making power in the hands of a small group of people such as the core team doesn't seem like a good idea, either.  Issues of trust aside, who would have the time to arbitrate every dispute?

Right now, we're in the process of wrapping up the last CommitFest of the 9.1 development cycle.  The CommitFest, according to a development plan published last May, was scheduled to end on February 15.  But it didn't: we haven't finished dealing with all of the patches yet, and we're holding the door open for a few more features that we really want to get into the release.  Of course, we can't keep doing this indefinitely, or the release will never get out the door.  It's a difficult balancing act.  Every day we keep the CommitFest open is one more day that the approximately 20 remaining patches can potentially committed, but it's also one more day until release, and one more day until we can begin meaningful work on PostgreSQL 9.2.  Moreover, deciding to wait, say, an extra two weeks can actually slow things down instead of speeding them up: people decide they have time to go work on something other than PostgreSQL development and come back to finish up their patch later.  It's this very effect that has made the last CommitFest of the 9.1 cycle so large and difficult to handle in the first place.

Fundamentally, our community has conflicting priorities.  We want every patch to get is due share of attention and we want the release to get out on time.  In a company where all the developers were employees who were getting paid to do this work, this might be just barely possible, given a skilled manager and an adequate budget.  We have no budget at all, and the hammer of "I'm the boss - do it or you're fired" is not hanging over anyone's head.  Our ability to get our releases out the door with high-quality features and on a reasonable time frame is dependent on the goodwill of people who get paid the same whether it happens or not.

There's definitely room for improvement, but I think that - given the constraints mentioned above - we're doing pretty well.  It's not clear how much longer this last CommitFest will go on, but I'm hopeful that we'll find a way to wrap it up reasonably soon.  The decision as to when it's time to pull the plug will be made, as always, by consensus.


  1. I think another problem is a lack of reviewers, the kind of reviewers who can do more than test that a patch applies cleanly and that it behaves as expected. Both Tom Lane and yourself do exceptional work in this case, even though you're both spread quite thin. But more people who understand the codebase well and the real effect of changes are needed. The earlier patches can get reviewed, the higher the chance of it being worked on early enough to meet the commitfest deadline.

    I'd love to spend time thoroughly reviewing patches making sure they're up to scratch, but unfortunately I'm not qualified for such a task. The best I can do is review syntactic issues or typos, or check the patch does what it claims to do.

    Yet despite this, the great thing about this process is that if something gets committed, it's because it's been thoroughly discussed, debated, sent back, returned, sent back again, tweaked and reviewed so that virtually no poorly thought-out features make it in.

  2. Wait, since when are we still letting in new patches? That's got to stop. I'm in favor of keeping the commitfest going until March 3 to finish up existing major patches (SQL/MED, synchrep, etc.). But keeping the door open for new patches which are not strictly bugfixes? I don't think anyone consciously agreed to that.