Real-time remote code-reviews: shared screens & video chat

I’ve been involved in development processes that involve reviewing code for people who are not nearby for a bunch of years now.  Recently, Alan and I have been experimenting with code-review using video chat & shared screens, and we’ve found some significant advantages.

This works particularly well for large patches, largely because of the very fast feedback loop and almost total lack of context-switching, which can carry very high costs.

This didn’t totally surprise me when I realized it, but another thing did: I started looking forward to large code reviews, rather than dreading them.  My current feeling is that a lot of this is because they suddenly felt like a normal social interaction, and one where I got to learn a bunch about some new code and be helpful to the person working on it as well.  The key part of that last sentence is the “normal social interaction” piece, I think.  Which is to say that hearing someone’s voice, seeing their face, and being able to just talk through issues feels viscerally much much more efficient and fun to me than stuffing the same interaction through a very high-latency ASCII pipe.

Previous to this, when I did code reviews, I could be reasonably quick (mostly) turning around simple reviews, and painfully slow with larger ones.  Being slow on large patches was tremendously painful in the sense that it was surely frustrating and demotivating for the folks who had contributed the large patches, and I often felt overwhelmed by large reviews looming on my to-do list and guilty for not having gotten to them yet.  But now that I tend to look forward to code reviews, my turnaround time is now much happier for everyone involved.

One downside of this mode, however, is that since it requires technological and time coordination between the folks involved, for the most part, it’s hard to include onlookers who are interested in learning from the review.  This is why I’m trying to get into the habit of doing the short stuff the usual way.

I’d be very interested to hear other people’s experiences with real-time remote code-review…

asking for feedback in blog posts

Last week, we released Gladius 0.1, a modular 3D game engine for the web.  As part of the release process, Havi, Pascal, and Erica encouraged us to ask for feedback as part of our blog post.  We were initially concerned that we’d got too much feedback or too much non-useful feedback, given the impact of the Mozilla brand and the broad appeal of video games.

We decided to try a with a technical question that was somewhat broad, but (we hoped) not too broad:

One thing that we’re hoping to get from this release is API feedback about the asset loader. That API is currently entirely callback-based and we’re curious whether game developers find that comfortable, or whether something that uses promises would be preferable.

Fortunately for us, Damon Oehlman offered up some very helpful thoughts, and not only that, he was kind enough pull in some other folks from the twittersphere with significant experience (thanks, Damon!), who offered up a lot of very helpful analysis. The github issue has all the details.

A question for other bloggers: in your experience, what are some of the dos and don’ts of asking for feedback successfully?

 

what I’ve been up to lately: 3D web gaming

I haven’t blogged much lately, because I’ve been busy working with a fantastic group of folks from all parts of Mozilla and the surrounding community. Over on the Labs blog, I’ve just posted more details…

changing my focus

As folks close to the project may be aware, I’ve been focusing less on Thunderbird recently, and today I’m handing off my remaining Thunderbird responsibilities to Mark Banner and the rest of the team. As part of this, there will be a post within the next few days about some changes in the Thunderbird / MailNews module owners and reviewers to better reflect reality going forward.

Starting today, I’ll be working with Pascal Finette to help define a new area of Labs.

My role at Mozilla Messaging was primarily about helping grow the organization, product, community, and culture around Thunderbird. There is now a great team doing a fantastic job evolving and producing Thunderbird, and continuing to drive towards the high-level goals of keeping users in control of their own mail and growing the user-base. With MoMo merging in with the rest of Mozilla, it felt like a good time for me to consider other ways I could help out with the Mozilla Mission.

I’ve been using my coding muscles again recently on a few labs-y things, which has been great fun, and I look forward to doing more of that before long. But an opportunity to help define and shape n new area of labs has come along that is just too exciting for me to pass up. You’ll see and hear more about this before long.

I feel grateful to have had the chance to work with so many dedicated, talented, and fun folks; thank you!

focusTimer: a useful work tool, and some learnings about webapps on the Mac desktop

a simple tool for work and breaks

I’ve been using a simple timer to break my work up into 25 minute chunks with 5 minute breaks on and off for a while.  It’s super helpful as a way to keep myself focused on almost anything, to force myself to think about what I’m doing in terms of the time it will require, and to ensure that I take enough breaks that I avoid premature brain drain.  I learned about it at The Pomodoro Technique website and started off using their timer app.  But it just wasn’t quite flexible enough for my needs.  And all of the other Mac timer apps I tried were weird mixes of overly intrusive or baroque UI.

So I wrote my own, entirely in web-content, in part as an experiment to get a better feel for the current state of the art in running pure web apps on the Mac desktop:

Note that for this first iteration, it signifies that it has expired by turning the window red.  Getting a real notification popup is still to come.

written as a web page

I played with Chromeless, Safari web clips, Fluid.app, and its open-source cousin fluidium.   They all had their quirks; I ultimately found the best general day-to-day user-experience in Fluid.app, largely because it has the tightest OS X integration.  However, they all have the age-old barrier of requiring someone who wants to even play with this to download and install software.

When I mentioned this to Anant, he said that he had been just talking about web-pages as widgets as part of his work on Appetizer, and he proceeded to whip up a restartless add-on called Widgetify which takes a first cut at solving this very problem, and in a cross-platform way, no less!

giving focusTimer a try

If you’re interested in playing with focusTimer using Firefox 4 beta or nightly, all you need to do is:

Presto!  You’ll have a functional focusTimer to try.

For day-to-day use, I recommend checking out the instructions for setting it up with Fluid so that you can get  always-on-top, glued-to-all-Spaces behavior and window transparency to go with that, which adds to the user-experience significantly.

the source, and next steps

The code is on github; feel free to fork, play, and issue pull requests if you come up with interesting changes.  Be warned, though, that the code contains some ugliness.

There are lots of things that could make this significantly nicer; see the roadmap for some of them.  In particular, I expect it will make a good testbed for exploring various features in Chromeless and Appetizer, not to mention HTML desktop notifications.

1:1 discussions with a few significant Tb contributors positive & helpful

Over the last weeks, I’ve been having individual discussions with a sample of people who make significant Thunderbird contributions.  The folks I’ve talked to seemed happy with the overall process, and I learned significant things in the process.  There are also still unanswered questions.

next steps

  • finish the open email discussions
  • work on broadening use of video chat by talking to folks in the add-on community
  • draft questions for the next survey to dig further into frustrations from the last one
  • work with area leads on low-hanging fruit in individual areas (e.g. IRC channel links)

the discussions

As mentioned in my last post about the Thunderbird survey, I wanted to get a deeper understanding of some of the results from our last survey, particularly around the decisions & directions frustrations that were mentioned in some responses.  After talking it over with a few folks, I decided that to counterbalance the very narrow medium of a survey, it made a lot of sense to have 1:1 conversations in order to capture the more human side of what’s going on.

I asked area leads for the names of a couple of people in their area making significant contributions. I sent these folks all mail to arrange conversations. I’ve ended up having 6 conversations, 3 via video chat (Skype), 1 by voice, and 2 by email. The email ones are still in progress. Thanks to everyone who kindly offered their time.

the general feeling

Each person that I’ve spoken to in this group has said they are and seemed happy with their participation in the project, and no one has expressed significant problems with either direction or decision-making.  This is, in and of itself, very good to hear.  That said, it doesn’t invalidate the feelings of people outside the sample who feel differently, and we still don’t have deep understanding about the feelings of survey respondents who listed decisions and directions as being a significant source of frustration.  Chances seem high that this is due to selection bias, and that most folks who have those frustrations aren’t motivated to do larger amounts of work.  Since that might change if those frustrations went away, this bears further digging into.  My current thinking is that trying to dig further into this on the next survey with more specific questions might be the best way forward here.

motivations & frustrations

When I talked with people about what they found motivating and rewarding, I found a number of themes that I expected, including liking the product, giving back to Mozilla, and enjoying technical challenge.  One thing that I hadn’t been expecting, however, was that more than one person mentioned that they were in situations in their own lives that made socializing more complicated, and they found that working with people (both users and Mozilla community folks) to be an enjoyable way to socialize.  I also heard from a couple of people that having more social interactions (particularly IRC) seemed to make things significantly easier on new contributors.  This makes me think there is probably low-hanging fruit around linking to the appropriate IRC channels in a few key places.

People’s frustrations so far have been very individual, and haven’t shown any particular theme that I can see.

other conclusions

Using video chat was very energizing. There’s something very rewarding and fun talking face-to-face with people who care enough about Thunderbird to invest significant amounts of time in it, and it makes the relationship with whomever you’re talking to feel much more human than Bugzilla or IRC ever can. Further, once one gets into the rhythm of video chat (the lag can be somewhat noticeable), it can also be much more efficient and effective for many kinds of conversations. I would strongly encourage anyone who has things they need to talk about with other Mozilla contributors to experiment with video chat a few times.

As with doing the survey, just the act of talking to people more has drummed up a bit more interest in contribution. One of the existing contributors has expressed significant interest in helping out with work on a contribution funnel so that we can do better at getting new people truly integrated into the project.

One of my next steps is to have some individual conversations with folks in the add-on community as well, and I’m hoping that I can recruit a few core contributors besides myself to help out with this.

As usual, comments & thoughts are much appreciated.

November Zen-to-Done habit: Inbox processing

As mentioned in another recent blog post, I’m working (along with Atul and Clint) on some Zen To Done habit changes.  For November, I’m working on processing my Inboxes.  This includes work & personal email, Bugzilla mail, and some work-related mailing lists & blogs.  The hardest (and most beneficial) part, I expect, will be to improve my discipline around processing each message very quickly.  In particular, I expect to repeatedly notice that I’m spending more than a couple of minutes on various messages as I find them in my Inbox.  I’ll then need to let go of those messages, add them to my to-do list and/or calendar, and get back to processing all the rest of the incoming stuff down to Inbox Zero.  I may also have to give up some of the stuff that I try to follow right now, but I’m hoping that can be avoided.

Anyone have experiences or tips with changing this particular habit?

reflections on my first Zen To Done habit: planning

As I mentioned above, I’ve spent this month working on the planning habit of Zen To Done.  I’ve found it really helpful.  The most helpful thing seems to be always having thought up the “Big Rocks” (personal weekly goals) and “Most Important Things” (personal daily goals) and doing at least one first in the day, before email or anything else has the chance to be disruptive and pull me into being reactive.  It’s been helpful both causing me to understand what my priorities are before the day starts, and by making sure I get some big stuff done, regardless of all the the small things that need attending to.

I’ve been hitting slightly less then 2/3 of my daily goals per week, which corresponds to exactly how I want to use those goals: both as a way to cause me to think about important stuff ahead of time, but also being optimistic enough that it forces me to get better at the things I’m executing.  I’ve learned a bunch about myself through this framing.  In particular, at the moment, when I miss them, it’s occasionally because I get sucked into the various small stuff in each day that wants to be dealt with.  This is great, because that used to happen much more frequently before I started this habit, so I’ve already got a victory on my hands. More often, when I miss my daily goals, it’s because I significantly underestimate the amount of time something will take.

For anything that I’ve done repeatedly or is well-understood and scoped, estimation doesn’t seem to be a problem.  However, when a daily goal is something I haven’t done before and haven’t spent a bit of time pre-planning in my head, there’s a _much_ higher chance of me not hitting it, even when I spend the time that I intended at it.  This is making me think that I want to incorporate a habit of drafting at least a very lightweight plan for anything I haven’t done previously before I commit to it as a daily goal.

Anyway, it’s been a super helpful habit to acquire.  My habit for November: processing my Inboxes efficiently and effectively.  I have a bit of trepidation of both maintaining this habit and taking on another, but, hey, how hard can it be.  :-)  I’ll be digging into this at the same time as Atul.  If you’re interested in joining us, let me know!

tb survey, part 2: frustrations & rewards

This post digs into the most significant frustrations & rewards uncovered by the Q3 2010 tb survey.  There’s evidence that we’re in great shape in a number of ways, and there are significant opportunities to improve as well.

next steps

* Comments and insights from any of you about the data and conclusions here as well as things I haven’t uncovered would be great.  People are more than welcome to sift through the summary report or raw csv data, as well as the cross tab data about frustrations and areas of contribution.

* I’ll work on getting a better understanding of the “disagree with project direction or decisions” responses, likely in part by gathering more data.

* I’ll talk to the area leads to try and get a better understanding of low-hanging fruit in making the various parts of the project easier to understand.

some context

After taking a high-level overview of the first Thunderbird survey, I’d like to dig into the rewards and frustrations of participating in Thunderbird development.  This feels complicated to me, in part because when I review data like this, I notice a couple of things that I tend to stumble upon.

One is that when I read through the various frustrations, it’s hard for me not to feel a somewhat down, because I empathize with how hard it can feel to invest energy into helping and then run up against one of the rough edges of the process.

The other complication I run into reviewing the data is that I find myself semi-regularly losing sight of the reality that because these numbers are just a view into our process and community, they are materially separate from the processes and community themselves.  As an example, the number of people who disagree with project directions and directions doesn’t represent a single opinion about one or all of the decisions we’ve made, but is actually a collection of a whole bunch of individual opinions, probably of different strengths, probably about varying sets of individual decisions.  So I keep reminding myself of this, and it helps me realize where the information here is really just a starting point and we need to dig in deeper.

frustrations

119 people responded to the question “What are the things you find most unpleasant or frustrating about contributing to Thunderbird? (Check all that apply)“.   Here is the percentage of those 119 people (out 137 survey respondents total) who gave the most common answers:

frustrations graph

As one can see in the graphic above, the most common frustration people had about contributing to the project was “Other”.  I’m not going to dig into the “Other” responses right here, though I hope to find time to dig into them in a future post.

One important thing to note is that while every other question on the survey had 137 or 138 responses, this question only had 119, from which one can presumably infer that 18 or 19 respondents didn’t have any particular frustrations with the process (or else weren’t comfortable talking about them?).  Note that (unlike in the graphic above), I’ve redone the math based on the 138 number below, because I think that’s the one that actually counts.

“Too hard to figure out how to effect change in the produce or community” was the most common response, given by 28% of the survey respondents. Cross-tabulation of data for this question a shows there were a non-trivial number of these responses in almost every category of contributor (development, support, localization, etc.).  Furthermore, only contributors who had been around longer than 2 years had less than 20% of respondents considering this to be a problem.  This leads me to believe that these responses do not characterize a single problem such as “it’s hard to get started”, but, rather, that there are a variety of things in each area of the project that are hard to figure out.  So my feeling about the best way to dig into this is to work with each area lead to get a better feeling about what the particular pain points in their area are.

The second most frequent response was “Disagree with project direction or decisions”.  Of the people who rated overall enjoyment of contribution “neutral” (26 out of 138) or “unenjoyable” (16 out of 138), 29% and 27% of them (respectively) selected this response.  Localization contributors selected this response mildly more frequently than other folks who did not consider localization one of their primary areas of contribution.  Among a variety of things not yet clear include how many and what sorts of things people are disagreeing with most, if people perceive decisions they disagree with to be reached reasonably, and how strong the disagreement they feel is.  It also seems likely that selection-bias for the entire survey is coming into play here, since people who are sufficiently unhappy tend to be more motivated to respond.  At this point, I feel like it’s important to dig into this and collect some more data.  I don’t yet have a strong feeling about whether it makes the most sense to do a quick randomly-selected survey, talk to some folks directly, something else, or, most likely, some combination of these things.  Input here is welcome.

rewards

When asked “What are the things you find most enjoyable or rewarding about contributing to Thunderbird? (Check all that apply),” the percentage of 137 responses that had the top answers looked like this:

rewards graphic

Apologies for the caption lossage; click on the graphic to see a less confusing version.

67% of respondents felt that “having my efforts help millions of people” was one of the most rewarding aspects of contributing to Thunderbird, making it easily the most popular choice.  Behind that, 59% felt that helping further the Mozilla Mission was the one of the most rewarding things.  42% selected “being able to solve problems that effect me or my organization personally”.  I find this both intriguing and inspiring: I have a ton of respect for this last “scratch my own itch” motivation, and think the fact that the open development model makes it possible is extremely important.  That said, I find it particularly impressive and awesome that more contributors are more motivated by having broad impact (whether measured by number of users or the principles of the mission).

Credit: graphics created by SurveyMonkey.

a portrait of the Tb community as reflected by survey, part 1

Our first Thunderbird contributor survey reflects, among many other things, a community that largely finds contributing enjoyable and shares significant rewards and frustrations around the process.

what’s next

The survey generated a lot of food for thought, and trying to distill it all in a single blog post wouldn’t work very well. Instead, I’m going to split this up into three posts:
  • in this post, I’ll offer a high-level overview of what the survey tells us about the community as a whole.
  • next, I’ll dig into the most common rewards & frustrations around contribution, and what actions I think it makes sense to prioritize around them.  This post will also include links to various reports and raw data.
  • finally, I’ll touch on some of the themes captured in the free form “Other” responses.

who we are today

First off, thanks to everyone who took the time to respond.  We got 138 responses, and every single one helps make the dataset more statistically meaningful and more useful in understanding ourselves as a community and in helping prioritize community and process work.

fairly happy

The largest group of us enjoys contributing: 65% of people who responded said it was “enjoyable” or “very enjoyable”, and only 15% said it was “unenjoyable” or “very unenjoyable”.  I think this is a fine thing, though I’d very much like those numbers to be slanted even more towards the enjoyable end of the scale.  Probably some of this reflects the fact, too, that when people find something sufficiently unenjoyable, they often leave rather than sticking around to be surveyed.

I’d like this question to be on a survey every quarter so that we can use it as something to try and improve against.  Do folks know of any other similar survey data from other open-source projects?  I’d love to get some kind of feel for both what’s possible and what’s typical…

Enjoyable Pie Chart

testers (30%), localizers (28%), developers/coders & reviewers (17%), and supporters (15%)

My guess is that this reflects the fact that the barrier to entry for these things is relatively low, particularly when compared to the higher technical & training barriers of development, design, and documentation.  I think there’s a fair bit of depth underneath this question, and I’d be very interested in seeing folks who work in the various different areas comment on what this post brings up for them.

[Update: After coalescing the developer/coder/reviewer categories and updating the header of the above section, as suggested by Ludo, I suspect the conclusions in the above paragraph are less correct than I realized.  Hmm.]

area of most involvement

growing, and here for the long-haul

14% of contributors say they joined us between one and two years ago; 19% in the last year.  It’s a fine thing to be on a positive trajectory.

65% of respondents have been contributing longer than two years, and it’s fantastic that folks find contribution rewarding enough to stick around for the long term.

More data from both other projects and (at least occasional) future survey questions would make this information easier to contextualize and draw conclusions from.

contribution length

Up next: a closer look at the specific rewards and frustrations around contributing.

Credit: The graphics were output by SurveyGizmo, which we used for the survey.  Thanks, SG!