Difference between revisions of "User:Sardisson/Crash Analysis Workflow"
(→Releases: h,,) |
(→Development Branches: more thoughts) |
||
| Line 11: | Line 11: | ||
==Development Branches== | ==Development Branches== | ||
| − | On a weekly basis, I want to look at the accumulated crash stats for that branch and | + | '''A.''' On a weekly basis, I want to look at the accumulated crash stats for that branch and |
* Quickly see where the top crashes are occurring (crash signatures) | * Quickly see where the top crashes are occurring (crash signatures) | ||
| Line 17: | Line 17: | ||
* From the above screen, access crash stacks for each crash signature | * From the above screen, access crash stacks for each crash signature | ||
* Possibly "compare" crash signatures to trunk and releases to see if they're occurring there, too | * Possibly "compare" crash signatures to trunk and releases to see if they're occurring there, too | ||
| − | * Cross-reference with any existing bugs | + | * Cross-reference with any existing bugs (or recently closed bugs) |
| − | If there are no existing bugs on a major-seeming crash, attempt to reproduce from | + | If there are no existing bugs on a major-seeming crash, attempt to reproduce from URLs and comments and file bug(s). |
| + | |||
| + | '''B.''' If there I file a bug based on crash analysis, particularly one with a large number of incidents but which I was unable to reproduce, I want to | ||
| + | |||
| + | * Continue to monitor the crash signature for new incidents which may provide URLs or comments that allow the bug to become reproducible | ||
| + | * Search across products for incidents if there's reason to believe it might be in shared code | ||
| + | |||
| + | '''C.''' If a bug filed based on crash analysis, or having a crash analysis component, has been marked FIXED, I want to | ||
| + | |||
| + | * Verify that the crash signature disappears (or, in the case of overly-generic crash signatures, declines significantly) in builds following the checkin | ||
| + | * Have easy access to incidents of the crash signature from *only* builds after the check-in of the fix, in order to check whether the incidents appear to be the same crash or another crash whose top frame happens to be the same signature | ||
Revision as of 15:21, 22 May 2007
bsmedberg asked ss and I to write up our thoughts on how we use (or would like to use) crash analysis reporting and make them available for the denizens of mozilla.dev.quality and the socorro server engineering team.
Below is where I am working through my thoughts.
Releases
During the Release Candidate stage and for the first while after a release,
- Get an overview of where the crashes are occurring
- Check
Development Branches
A. On a weekly basis, I want to look at the accumulated crash stats for that branch and
- Quickly see where the top crashes are occurring (crash signatures)
- See which crashes are "increasing"
- From the above screen, access crash stacks for each crash signature
- Possibly "compare" crash signatures to trunk and releases to see if they're occurring there, too
- Cross-reference with any existing bugs (or recently closed bugs)
If there are no existing bugs on a major-seeming crash, attempt to reproduce from URLs and comments and file bug(s).
B. If there I file a bug based on crash analysis, particularly one with a large number of incidents but which I was unable to reproduce, I want to
- Continue to monitor the crash signature for new incidents which may provide URLs or comments that allow the bug to become reproducible
- Search across products for incidents if there's reason to believe it might be in shared code
C. If a bug filed based on crash analysis, or having a crash analysis component, has been marked FIXED, I want to
- Verify that the crash signature disappears (or, in the case of overly-generic crash signatures, declines significantly) in builds following the checkin
- Have easy access to incidents of the crash signature from *only* builds after the check-in of the fix, in order to check whether the incidents appear to be the same crash or another crash whose top frame happens to be the same signature
