I don’t recall precisely how I first got involved with Mozilla, but a large part of my early contributions was triaging bugs. I stumbled across Bugzilla at some point, and I really enjoyed finding duplicates and trying to figure out the root causes of bugs. I would watch the “bugs filed today” query, and try to triage as many bugs as I could. It was a fun challenge to find duplicates quickly, and memorize common issues (this, and later IRC- and bonsai-watching habits and general Mozilla addiction are what led to the coining of the name “gavinbot“). Later, as I took on more responsibility in other areas (e.g. development and code review), I stopped being as actively involved in Bugzilla triage itself; the bugmail hasn’t stopped, though 🙂
Recently Tyler posted about his frustrations with the bug triage effort. He raises some good points, but I think one of his premises is faulty: I don’t think that focusing on the number of UNCO bugs is useful. We should focus on outcomes, not on driving a number to 0 for the sake of it.
Are we missing bugs that we should have noticed, but failed to? Maybe, but we’re never going to be 100% on top of every bug, and I don’t think the situation here is as dire as “5000 untriaged bugs!” makes it sound – a very large percentage of those are duplicates, or invalid, or minor issues that we shouldn’t be focusing on. Spending contributor time sorting through many of those bugs probably doesn’t have a very good cost/benefit ratio. Important issues still tend to bubble up pretty effectively, in my experience.
Are people being discouraged from reporting problems to us because they get ignored? This is a problem, certainly (though again, I’m not sure about the degree). We should fix this by having better tools for giving feedback (i.e. not Bugzilla), and there are already successful efforts underway to do that (e.g. input.mozilla.com).
It’s important to step back and look at the bigger picture – driving a Bugzilla query of untriaged bugs to 0 should only be a goal insofar as it helps us achieve the primary goal of producing good software. I contend that it probably isn’t the most effective way to contribute to that goal at the moment.
I think part of tyler’s point is that the process is inherently disrespectful of people trying to help. It is now almost pointless for individual non-developer’s to contribute bug reports. There are bugs that get completely ignored for months.
Yes Gavin, and I tried to make it clear in my post (I have a clarifying post up now), that I was more concerned with how quickly we respond to bugs, than how many we have. Naturally, 13,000 is far far too many, and I think even 6,000 is a bit much (and this is totally a failure on triage’s part). But I more have to see 2,600 bugs go 150 days with no activity. So many bugs just get abandoned, and the only activity is when a triager, trying to keep up, comes along and asks the reporter to retest. So, numbers don’t matter as much to me, as the time between reporting, and the triage response.
Right, “disrespecting” people reporting issues is certainly a problem. I guess the secondary point I was trying to make iis that rather than solving that problem by increasing our efforts responding to bugs, we should instead do it by directing people filing these bugs away from Bugzilla.
http://tylerdowner.wordpress.com/2011/08/29/even-more-clarifications/ might help explain my position a bit better 🙂
Why is input.mozilla.com a better channel for bug reporting than bugzilla? To the casual observer, it seems like input.mozilla.com is primarily a relief valve for users– a way to try to make them feel “appreciated” and convince them that their input is “important to us” while piping their input to /dev/null.
Sure, sometimes somebody will file a bug based on input, but what proportion of the feedback entries ever receive that kind of attention, much less ever have action taken on the relevant bug?
(even then, aren’t the odds of successfully reproducing buggy behavior drastically reduced when you have no communication channel with those experiencing the problem?)
Maybe a lot of users can’t be bothered to properly check bugzilla for existing bugs describing their problem and follow proper bug submission guidelines. But that doesn’t mean that taking that option away from those willing to take those measures is a good idea.
Input not perfect, but it’s better than Bugzilla because it lets us aggregate input more usefully (or at least that’s the idea – to be honest I haven’t been following input.mozilla.com development too closely).
Input is not better for bug reporting, but the reports in question aren’t “bugs” in any useful sense – they were filed as such, because Bugzilla is a communication medium we expose to people, but as I said earlier, they largely consist of duplicate or invalid issues – they’re not useful on their own, but they are useful in the aggregate (if we can capture that data appropriately).
I think there is a user/developer interface missing today. Sumo is great for support. Bugzilla is good for reporting actual bugs. Input is probably great for watching trends, but for sharing an idea it is not; there is a 250 character limit and a suggestion will be buried among thousands of other ideas in hundreds of languages, random characters, incomprehensible rants, etc.
People are passionate about Mozilla apps and have ideas for features and enhancements that are neither bugs per se nor support issues. Some are great, some not so great (IIRC there was a suggestion for adding a Private Browsing feature in Firefox years before it was implemented in Safari even). If you had an idea for a feature that you thought was essential or that you thought could be a killer feature, how would you go about to submit that to Mozilla? E-mail? Blog comment? Bugzilla?
My suggestion is a separate site similar to Dell IdeaStorm, where users can share their ideas and suggestions and other users can vote on said ideas. I think this would fill the gap between Bugzilla and Input with the right back end tools (e.g. marking a suggestion as “in progress” if there is active development on that particular issue, or marking a suggestion as “support issue” and sending it to Sumo).
Hopefully, that would reduce the amount of bugs submitted to Bugzilla. And in particular the amount of NEW bugs that ends up in severity: enhancement never to be read by another human again.
I agree with you about having a tool like IdeaStorm. I thought Input was heading into that direction, which is part of why I suggested it as a better option – but like I said before, I don’t actually have much knowledge about the future plans for Input.
What about something like http://uservoice.com/ ?
Pingback: Tyler o ocenie błędów – stream of bytes
“If you had an idea for a feature that you thought was essential or that you thought could be a killer feature, how would you go about to submit that to Mozilla? E-mail? Blog comment? Bugzilla? ”
While not as user friendly as it could be, we do have a set of forums that were designed to include just these kinds of discussions: newsgroups. It may be a usability gap, but it’s not a “we didn’t care so we didn’t bother making a forum” gap.
A good idea will generally attract a decent group of participants in a newsgroup thread. Usually the next step is a wiki page. We even have templates specifically designed for documenting features now.
“While not as user friendly as it could be, we do have a set of forums that were designed to include just these kinds of discussions: newsgroups.”
As you mentioned, that’s not the most user friendly: few people are familiar with nntp these days and google groups isn’t ideal. But much more importantly, people aren’t directed towards the newsgroups in a helpful manner. People looking for a way to make suggestions or express concerns end up posting to getsatisfaction, to suggestionbox, to mozillazine’s forums, dozens of other random fora, and Bugzilla because they don’t know where to go and they don’t get direction. (The fact that Mozilla doesn’t provide clear guidance about where to post feedback is itself interpreted as a sign of hostility to user feedback.) If they do find http://www.mozilla.org/about/forums/, they are presented with a list of 164 newsgroups to choose from, and if their concerns/suggestions don’t seem to fall under mozilla.support.*, the only one which seems at first glance to be the right forum is dead (mozilla.wishlist, completely dominated by spam). On the whole, the non-support newsgroups seem to an outsider to be made for developers only and not meant as an interface to the public at large.
The situation with newsgroups is bad enough that when you tell people that the newsgroups are the preferred forum this seems to most people to be a way of dismissing them (“everyone whose ideas matter are already subscribed to the relevant newsgroups, so we don’t want to listen to people on this channel”) rather than a way of inviting them to join in discussion elsewhere.
Asa, I know you care. After all, why wouldn’t you? It’s just that there are so many ways to submit ideas to Mozilla it’s hard to know which way to use and it is easy to pick the wrong way. So perhaps adding yet another isn’t the way to go?
If newsgroups is the right way to go, perhaps the “I have an idea for Firefox” link in BMO > Enter a bug >Step 1 should point to a newsgroup gateway?
In any case, the main thing for idea submissions I think is that you feel that somebody actually listens. At the end of the day any tool, be it IdeaStorm, BMO or newsgroups, needs someone at the other end. I think this is the USP for an IdeaStormesque tool – even an upvote from a random stranger might be sufficient. And everyone likes to vote!
Asa: “While not as user friendly as it could be, we do have a set of forums that were designed to include just these kinds of discussions: newsgroups.”
Which newsgroup(s) were you thinking of? Maybe the policy has changed recently, but when discussion about feature ideas that aren’t actually part of the core development plans has come up on dev.apps.firefox, my impression is that people have been asked, politely, to stop discussing things there (admittedly I think that was in the last months of Firefox 4 development, so probably not the best time…).
I don’t expect the number of untriaged bugs to go to 0. However, which bugs are untriaged should be shifting quickly over time so that even if we had 12000 untriaged bugs at any given time, any one bug would spend at most, say, a month as untriaged.
I think disrespecting bug reporters by letting them do work and then ignoring them is worse than telling them up front not to bother. In the case of Firefox, my expectation is that if I (a person working on Core) file a UI bug, it will get ignored among the mass of other bugs filed by the general public unless I poke the right people or put it in the dependency chain of another UI bug that’s already getting the attention of the right people. At least with Firefox UI, I have some idea of whose attention I need to catch in addition to filing the bug.
In the case of Ubuntu, I don’t know whose attention I need to catch, so I’m just a random nobody in their bug tracker. In the interest of not getting moderated out from these blog comments, I don’t even try to put into words how annoyed I am about the way Ubuntu handled https://bugs.launchpad.net/ubuntu/+source/vinagre/+bug/535046 (another person explained the reason behind the problem, I provided a patch that isn’t a proper fix but proves what the problem is, and they resolved it as expired because I didn’t go on the treadmill telling them “Yes, the problem still exists.” with each and every one of their subsequent releases). Since I’ve had this experience over and over with Ubuntu and don’t have positive experiences about Ubuntu bug reporting beyond getting useful insights about how to work around the bug when Canonical doesn’t do anything, I’ve pretty much decided not to report Ubuntu bugs anymore.
I really hope Mozilla weren’t giving that kind of experience to our bug reporters, but we often are. (It depends really on whether the bug initially lands into a component whose owner looks at every incoming bug and sends them onwards to the right people.) Given that Mozilla has the capacity to hire these days, I really wish Mozilla hired people whose job it is to make sure bugs don’t sit languishing under “General”.
I agree with everything you’ve said!
Pingback: Answering some Questions, and Responding to Comments, Part 1 « The life and adventures of Tyler