This post digs into the most significant frustrations & rewards uncovered by the Q3 2010 tb survey. There’s evidence that we’re in great shape in a number of ways, and there are significant opportunities to improve as well.
next steps
* Comments and insights from any of you about the data and conclusions here as well as things I haven’t uncovered would be great. People are more than welcome to sift through the summary report or raw csv data, as well as the cross tab data about frustrations and areas of contribution.
* I’ll work on getting a better understanding of the “disagree with project direction or decisions” responses, likely in part by gathering more data.
* I’ll talk to the area leads to try and get a better understanding of low-hanging fruit in making the various parts of the project easier to understand.
some context
After taking a high-level overview of the first Thunderbird survey, I’d like to dig into the rewards and frustrations of participating in Thunderbird development. This feels complicated to me, in part because when I review data like this, I notice a couple of things that I tend to stumble upon.
One is that when I read through the various frustrations, it’s hard for me not to feel a somewhat down, because I empathize with how hard it can feel to invest energy into helping and then run up against one of the rough edges of the process.
The other complication I run into reviewing the data is that I find myself semi-regularly losing sight of the reality that because these numbers are just a view into our process and community, they are materially separate from the processes and community themselves. As an example, the number of people who disagree with project directions and directions doesn’t represent a single opinion about one or all of the decisions we’ve made, but is actually a collection of a whole bunch of individual opinions, probably of different strengths, probably about varying sets of individual decisions. So I keep reminding myself of this, and it helps me realize where the information here is really just a starting point and we need to dig in deeper.
frustrations
119 people responded to the question “What are the things you find most unpleasant or frustrating about contributing to Thunderbird? (Check all that apply)“. Here is the percentage of those 119 people (out 137 survey respondents total) who gave the most common answers:
As one can see in the graphic above, the most common frustration people had about contributing to the project was “Other”. I’m not going to dig into the “Other” responses right here, though I hope to find time to dig into them in a future post.
One important thing to note is that while every other question on the survey had 137 or 138 responses, this question only had 119, from which one can presumably infer that 18 or 19 respondents didn’t have any particular frustrations with the process (or else weren’t comfortable talking about them?). Note that (unlike in the graphic above), I’ve redone the math based on the 138 number below, because I think that’s the one that actually counts.
“Too hard to figure out how to effect change in the produce or community” was the most common response, given by 28% of the survey respondents. Cross-tabulation of data for this question a shows there were a non-trivial number of these responses in almost every category of contributor (development, support, localization, etc.). Furthermore, only contributors who had been around longer than 2 years had less than 20% of respondents considering this to be a problem. This leads me to believe that these responses do not characterize a single problem such as “it’s hard to get started”, but, rather, that there are a variety of things in each area of the project that are hard to figure out. So my feeling about the best way to dig into this is to work with each area lead to get a better feeling about what the particular pain points in their area are.
The second most frequent response was “Disagree with project direction or decisions”. Of the people who rated overall enjoyment of contribution “neutral” (26 out of 138) or “unenjoyable” (16 out of 138), 29% and 27% of them (respectively) selected this response. Localization contributors selected this response mildly more frequently than other folks who did not consider localization one of their primary areas of contribution. Among a variety of things not yet clear include how many and what sorts of things people are disagreeing with most, if people perceive decisions they disagree with to be reached reasonably, and how strong the disagreement they feel is. It also seems likely that selection-bias for the entire survey is coming into play here, since people who are sufficiently unhappy tend to be more motivated to respond. At this point, I feel like it’s important to dig into this and collect some more data. I don’t yet have a strong feeling about whether it makes the most sense to do a quick randomly-selected survey, talk to some folks directly, something else, or, most likely, some combination of these things. Input here is welcome.
rewards
When asked “What are the things you find most enjoyable or rewarding about contributing to Thunderbird? (Check all that apply),” the percentage of 137 responses that had the top answers looked like this:
Apologies for the caption lossage; click on the graphic to see a less confusing version.
67% of respondents felt that “having my efforts help millions of people” was one of the most rewarding aspects of contributing to Thunderbird, making it easily the most popular choice. Behind that, 59% felt that helping further the Mozilla Mission was the one of the most rewarding things. 42% selected “being able to solve problems that effect me or my organization personally”. I find this both intriguing and inspiring: I have a ton of respect for this last “scratch my own itch” motivation, and think the fact that the open development model makes it possible is extremely important. That said, I find it particularly impressive and awesome that more contributors are more motivated by having broad impact (whether measured by number of users or the principles of the mission).
Credit: graphics created by SurveyMonkey.
I’ll be digging into this at the same time as Atul. If you’re interested in joining us, let me know!
Posts