Sunday, December 7, 2014

Picking open source technology: pick the community too

A brief post on picking (open source) technology. When we need something for our project, say, a database, we have a large range of choices. There is old and staid software that may not be hip, but comes with a well known track record and lots of features. There are new shiny frameworks that are blazing trails and publishing impressive benchmarks, written in languages still being standardized.

What do we pick? First of course, our dependency has to meet (most of) our needs. Next, we can look at its current reputation - if a project is known to be a bloated mess, or is well known to crash if you wave at it, we can discount the project for now.

But, even then, we are left with a lot of choice. What I care about these days is the community around a project. Because it turns out that the community is the best predictor of the future of a project. We can’t actually predict the future, but we can be sure that we’ll have new needs for our dependencies. We can also be sure we’ll have questions and discover bugs.

And a healthy community almost guarantees that things will end up well. As a recent example, PowerDNS has recently become involved with a customer where we are helping setup a PowerDNS based anycast environment. For this we needed a BGP implementation. A quick consultation with our community reported that ExaBGP would be a very good choice, and indeed, it offered all the features we needed.

On deployment however, we found two small issues that were holding up our deployment. PowerDNS employee of the month Peter van Dijk worked with the ExaBGP people, fixed the two issues, and both fixes have now been merged by the ExaBGP project. They are happy, we are happy. The next release of ExaBGP won’t just meet our needs, it will suit the needs of many more people.

Last week, a developer reported that the most excellent Valgrind tool found some potential errors in the LMDB project, and I was shocked to see LMDB lead developer tell the reported to ‘learn how to use his tools’, and not address his report in any meaningful way.

I asked Howard Chu to reconsider his stance, given my belief that open source can’t be great without a functioning community, something you don’t build this way. Howard told me that how he treated the reporter was entirely intentional. Shortly afterwards, the following was posted on the LMDB list:

“if you post to this list and your message is deemed off-topic or insufficiently researched, you *will* be chided, mocked, and denigrated. There *is* such a thing as a stupid question. If you don't read what's in front of you, if you ignore the list charter, or the text of the welcome message that is sent to every new subscriber, you will be publicly mocked and made unwelcome.”

This is entirely intentional and by design.”

Now of course, everyone is free to run their project in their own way. But I’m also free to pick my dependencies, and I care about the development community being sane and inclusive. “Bitchslapping” reporters of potential valid issues isn’t that. Formalizing this behaviour most definitely isn’t. I won’t be picking LMDB for any future projects unless this changes.

As a parting gift to LMDB, I worked to document the (powerful) semantics of LMDB, and dragged details out of Howard. This document can be found on If you use LMDB, it may be helpful for you.

I spent this evening working with PowerDNS contributor Ruben Kerkhof to merge several of his fixes for PowerDNS issues. Ruben’s company Tilaa uses PowerDNS, and we are Tilaa customers. I’m truly proud of our community and what we achieve together, and I recommend that every open source project works to foster its community as well.

So in short.. before picking a technology to depend on, investigate how they deal with feature requests, bug reports and questions. What you will learn is the best predictor of how the project will serve you (and vice versa!) over the coming years.


  1. Valid questions get handled. Saying "I'm not sure how this happens" when valgrind explicitly tells you how to find out the details simply has no excuse.

    We all make choices about who we work with and what we work on. I choose to only work with people who pay attention.

  2. Look, it goes like this - someone chose LMDB, and they ran testing on it, they do work on it. Their tool reported an issue that looked scary to them (and to me by the way), and they care enough to share that issue with you. Even if the poster had used the '--track-origins' suggestion, all they would've found was which function did not fill out the data properly. Would that have helped them or you solve the problem? This user person cared about your project, and you reward him with a slap in the face for not caring enough or not being competent enough.

    Similarly, when I tried to document the (convoluted) semantics and lifetimes of the various LMDB objects, you blamed me for not understanding it, when I was in fact trying to help the project. Since if I don't understand it, lots of people won't either.

    But enjoy your project - it has reemphasized for me the need to be critical of a community in addition to that of the (indeed excellent) LMDB technology.

    1. No ,it goes like this - someone took the sample-mdb.txt file, whose sole purpose is to be *read by human eyes* and compared to the sample-bdb.txt file, and tried to compile it and run it, when that was never its purpose. There is no code problem to solve, only a problem with people who don't read. If they cared about what they were doing we wouldn't be having this conversation.

      Frankly, your memory of things differs from mine. I recall patiently taking the time to answer your questions and help clarify the document you were writing. If you at any time felt like I was berating you, that was never my intent.

      Communities run on cooperation. Cooperation begins with respecting the established guidelines. Respect is treated with respect.

      The openldap-devel charter is quite clear. If you're a good developer and want to help write OpenLDAP code, you're welcome to jump in. If you're not actively writing OpenLDAP code, you shouldn't be posting on that list, and you will be shown the door.