Monday, May 23, 2011

PostgreSQL 9.2 Development: CommitFest Managers Needed

There's an old joke that project managers don't know how to do anything; they only know how to do it better.  Of course, the project manager is the butt of the joke: how can you know how to do something better, if you don't know how to do it in the first place?

Well, as it turns out, it's not all that hard.  Engineers are good at building things: that's what we do.  But just because we are good at building things doesn't mean that we're good at deciding what we ought to build in the first place, or getting it done on time, or keeping track of what the open action items are and which things are on the critical path.  Of course, there are some people who have both sets of skills, but even those people may not be equally good at both tasks, or may not be able to do them both effectively at the same time.

The PostgreSQL development community is, primarily, a community of engineers.  While it's certainly not the case that every reader of pgsql-hackers is a crack developer, many of the most active participants are very knowledgeable C coders.  Others are expert DBAs or power users with some C skills.  The discussions tend to revolve around the best technological fix to any given problem.  These are great conversations, and we're fortunate to have some incredibly talented people working on our code base.  Every release contains code contributions from scores of developers, and those contributions have made PostgreSQL what it is today.

At the same time, as much as we need even more people to contribute technically, that's not the whole story.  Over and over, I have heard contributors - including several very talented committers - that they would like to contribute more time to the project, but that they are not sure what work would be most valuable.

And this is clearly not a theoretical problem.  Our CommitFests, scheduled for a month long, sometimes run over, sometimes quite substantially.  Why does that happen?  Well, sometimes we just don't have enough people to review all of the patches in a month.  We are certainly in critical need of more volunteers, and we skate by in no small part due to the presence of a relatively small number of volunteers who approach the task with exceptional dedication.  But in my experience,  the problem is caused just as much by volunteers who are not quite sure what they ought to be doing.  Which patches need review?  Which patches are ready to be committed?  Which patches need to be looked at by someone with a particular skillset?  What are the aspects of each patch that appear to need further discussion?  A CommitFest may involve 40, or 70, or even 100 patches; keeping track of those patches is not an easy job, even with the CommitFest application to help.

Bridging the gap between willing volunteers and patches in need of work is the role of the CommitFest Manager.  When the CommitFest manager does a good job making sure that authors and reviewers know what work needs to be done, and a good job making sure that no patches fall through the cracks, the CommitFest not only gets finished more quickly, but the outcome is better.  Fewer patches are rejected for lack of time.  More reviewers participate.  Everyone who wants to contribute is given an opportunity to do so.  Making this happen does not necessarily require a huge amount of technical skill, but it sure does take effort.

We need more people with good organizational skills to contribute those skills to PostgreSQL.  CommitFest management is one area where I have seen, first hand, how much of a difference that can make.  If anything, one CommitFest manager is not enough.  Having two or three people keeping track of action items and matching them up with volunteers would be even better.  Nor is CommitFest management the only area in which we need more help.  Making sure bugs get followed up on, keeping the open items list up to date so it's clear what needs to be done before release, even just making sure that every user question on one of our mailing lists gets an answer - these are all valuable contributions.


  1. Never thought of as being based on FreeBSD is developed.

  2. We need to have a clear discussion about what problems we are trying to solve before we solve them. That question was raised at the Dev Meeting and not answered. Since we currently allow endless discussion and questions on patches, its not surprising we take longer than a month to answer the discussions and fix patches. We need more reasonable behaviour; if a topic is important, perhaps lengthy discussions are worthwhile and delays in completing patches are acceptable. PostgreSQL's quality and pace of development has been earned by particular development practices, not by following the rules of Project Management to the letter. We need to have faith that we have more things right than other dev teams and that this shows clearly in our output. IIABDFI. Reason, not rules.