QA:Triage Policies and Procedures

From Camino Wiki
Revision as of 21:34, 22 August 2006 by Sardisson (talk | contribs) (→‎Common QA Troubleshooting Tips: some utils we need)
Jump to navigation Jump to search

Common QA Troubleshooting Tips

  • [01:06am] smorgan: Clean profile is supposed to be a narrowing down tool, not a magical remover of bugs. Only corrupt/invalid profiles that appear to be caused by users mucking around should be consider ignorable.
  • Clean profile/org.mozilla.camino.plist
  • Console.log
  • http headers (browser and server)
    • curl -I 'foo'
    • the JS bookmarklet to catch server response:
    javascript:document.location.href%20=%20'http://webtools.mozilla.org/web-sniffer/?url='%20+%20escape(document.location.href)
  • http://kb.mozillazine.org/Standard_diagnostic_%28Firefox%29 (we need a Camino version; see User:Sardisson/To_Do)
  • Blocked cookies (either too restrictive a global setting or some domain set to deny; many sites require cookies from multiple seemingly-unconnected domains in order to function).
    • This is sometimes the case when something works in one browser but inexplicably doesn't work in Camino; users often have more restrictive cookie settings in their main browser but don't mess with the cookie prefs in alternate browsers.
  • Move UA spoofing from hidden prefs to this page, to make it clear that it's a QA tool, not something we "endorse" for everyday use like other hidden prefs. If people are smart enough to find the instructions themselves, we're going to hope they're smart enough to use it responsibly.....
  • When CAMINO_PROFILE_DIR-enabled builds become stable builds, we need a mini app/script that you drag Camino onto to have it launch with a temp directory. Then there's no room for accidental data loss while troubleshooting. [smorgan on #camino]
  • [11:40pm] smorgan: Someone should make CaminoConflictCatcher
    [11:41pm] ardissone|in: lol
    [11:41pm] smorgan: Copies the profile to a temp location, launches there, and removes prefs.js lines one by one, relaunching each time
    [11:41pm] smorgan: And it pops up a box when you quit: "Was the bug still there?"
    [11:41pm] smorgan: And then tells you what line it last removed when it said no
    [11:41pm] smorgan: That would be awesome
    [11:42pm] ardissone|in: sounds like a job for Wevah Wonka's Cocoa Factory
    [11:42pm] ardissone|in: or Froodian's Camino Therapy Couch

Searching for Duplicate Bugs

Confirming Bugs

  • Don't confirm RFEs (bugs with severity set to enhancement, or enhancement requests misfiled with some other sort of bug severity, which are very common, particularly if another browser provides such a feature) until there has been discussion among the triage team and, in some cases, with the project lead(s). This helps combat feature creep and keeps Camino from having a kitchen sink.
  • Don't confirm any bug that looks like it has the potential of being a Core bug (anything that has to do with the content area, or with certain app features whose underlying implementation is tightly linked with the Core, e.g. the pop-up blocker, session or global history); see Moving Bugs to Other Products and Components below for more about moving bugs to Core components.
  • Don't confirm bugs that look like they might be resolved DUPLICATE, INVALID, or WONTFIX. Perform a search of old Camino bugs with these resolutions before confirming bugs (especially the latter two; a request or bug may have been WONTFIXed before you began QA work, but that doesn't mean the project lead(s) have changed their opinion). See QA:Bugzilla:Searching For Dupes for more on searching for duplicates, including duplicates of WONTFIXed and INVALID bugs.
  • If the bug has a clean report and is clearly a real bug in the Camino application UI, then by all means confirm the bug. Otherwise, proceed more cautiously, seek additional opinions, etc., before confirming a bug.
  • When you confirm a Camino bug, you should also do the following:
    • Set priority and target milestone (not always; when?)
    • Fix other fields (Camino component and QA contact, branch, mis-filed severity)...
    • CCing developers if appropriate; see their areas of expertise in Development:Reviewing
    • Tagging bugs that only appear in one or more major Mac OS X versions...
      • Add [10.4] or [10.3] to the beginning of the summary when it's clear a bug only appears on 10.4 or 10.3. (During periods of time when we support 3 or more Mac OS X versions, a bug that appears in two or more major Mac OS X versions can have multiple version tags.)
    • Adding keywords
    • Regression windows
    • Adding to [meta] trackers, blocks/depends management
  • WONTFIXing bugs: Within the Mozilla project, WONTFIXing bugs is the purview of the module owner. Within Camino, the unanimous agreement of 3 regular QA/triage team members, at least one of whom must be a Camino QA/Triage Lead or the Team Coordinator (Samuel Sidler), is usually also sufficient to WONTFIX a bug. In cases of disagreement, the bug needs to be referred to the Camino lead/module owner (pinkerton) for further discussion and final decision.

Moving Bugs to Other Products and Components

  • Be sure to check the box to reset the assignee and QA contact to the defaults for the new component
  • Status (UNCO vs. NEW; what's required)
  • Testcases (esp. for Core:Layout and Core:Printing (also for DOM-related stuff); link to bugathon doc)
  • Moving to Tech Evangelism product
    • What qualifies as a TE bug
    • TE summary field format (Item 10)
  • Core:General component, for bugs you're sure are Core bugs but unsure of the proper component
  • Setting and Using Flags
  • Setting Milestone, Priority, and Severity [link to bugzilla page on severity]
  • Adding keywords and status whiteboard info
  • http://www.mozilla.org/quality/help/bugzilla-privilege-guide.html covers a lot of this

Regression Windows

  • Old nightlies on ftp.m.o
  • Older nightlies on archive.m.o
  • Bonsai queries (module Camino vs. directory mozilla/camino), branch vs. trunk

“CLOSEME”

  • Dealing with old UNCOs

Using Talkback

General Mozilla Policies

Retrieved from "http://wiki.caminobrowser.org/index.php?title=QA:Triage_Policies_and_Procedures&oldid=6304"