Introductions

In this orientation module you will learn how Quality Assurance is done in our community. You will also learn about basic tasks that are easiest to do for new QA Volunteers.

To complete this module, read through the material on this page, including the linked references. There will also be some start-up tasks for you to perform, such as signing up for an account in our defect tracking database.

Your first task is to subscribe to our QA mailing list and to introduce yourself to the other QA volunteers. You can subscribe by sending an email to qa-subscribe@openoffice.apache.org.

Then you can introduce yourself by sending an email to the list. We'd love to hear who you are, where you are from, what your background is, etc. Also as you work through the items on this page, if you have questions or problems, please feel free to ask for help by sending a note to this same list.

Note: In parallel with the QA-specific items on this page, you may want to first review the Level 1 and Level 2 Orientation Modules as well. They have useful background information on The Apache Way, mailing list etiquette, decision making in the project, etc. A quick review is a good idea, especially if you are new to working in Apache-style open source projects.

Now with the introductions out of the way, let's get started with the QA!

The Purpose of QA

Our goal is to maintain and improve the quality of Apache OpenOffice. We primarily accomplish this by finding defects (bugs) in the product before it is released to the general public. The defects are found by a combination of manual and automated tests that we perform on pre-release builds of OpenOffice. We also review and try to reproduce defects reported by end-users and submitted to us.

Since OpenOffice is a core software application for millions of users, the role of QA is critical. Although our programmers are wonderful, talented and very experienced, they do make mistakes. Our task is to find these mistakes, report, prioritize them, and verify the fixes when they are made.

QA is a discipline with many best-practices related to process and methodology, tools, techniques and theory. Although professional QA practitioners are welcome in this project, we are also happy to welcome those with no prior experience, or those who are learning about QA, perhaps as a possible career. Aside from the satisfaction of improving the Apache OpenOffice product, you can learn or practice new skills.

Why Help with QA?

As a volunteer why would you want to help with OpenOffice QA? A few things to consider:

QA Activities

QA activities within the Apache OpenOffice project include:

Skills Wanted

The skills we need on the QA team include:

QA is as much an attitude as it is a skill set. A tester likes solving puzzles, likes a methodical approach, likes the challenge of finding an elegant way to reproducibly break software.

Mailing List

As mentioned above, QA volunteers need to subscribe to our dedicated QA mail list.

The mailing list is for bug reports, quality assurance for release, beta tests, manual test, automated tests, etc. This is where the QA community coordinates its activities.

Subscribe: qa-subscribe@openoffice.apache.org
Post (after subscription): qa@openoffice.apache.org
Unsubscribe: qa-unsubscribe@openoffice.apache.org

Archives:

We use the following special subject tags to identify QA activities:

After you subscribe QA mail list then you can post your topic in the mail group.

Apache OpenOffice Test Builds

Since our job is to find bugs in OpenOffice, we must run pre-release builds that contain many bugs. These bugs could be major or minor. They could include document corruption bugs, crashes, even (in rare cases) bugs that could make your system unstable. So QA volunteers generally try to separate their QA work from their normal desktop activities. You don't want to write your thesis on a test build!

Some volunteers simply use two machines: one dedicated to their testing work. If things become unstable they can then reinstall the operating system and start fresh.

A more sophisticated approach is to use virtual machines, which can work quite well. VirtualBox is a popular virtualization package.

Feel free to ask questions on the QA list about effective ways to manage a test environment.

Once you have your test environment set up, you need to download and install the latest OpenOffice build.

Bug Handling

Apache Bugzilla is where we track defects:

Bugzilla related documents:

Bugzilla related mailing list:

Everyone in QA needs a Bugzilla account, which you can get here. Once you have a Bugzilla account, you should send a note to the QA list asking to be added to the "qa-team" group in Bugzilla. This will give you some additional permissions in Bugzilla. Include your Bugzilla login ID in your request.

Easy QA Task: Confirm New Defect Reports

Most new volunteers start by reviewing incoming defect reports and attempting to "confirm" them. The defect reports are often from users and are often not very clearly written. You learn to "read between the lines" and ask the user clarifying questions in order to turn the raw report into a reproducible defect that the developers can debug and fix.

You can get a list of unconfirmed defect reports with from this query.

Or, if you are just starting out, you can post a note to the QA mailing list and ask for a small batch of reports to be assigned to you. This can be a good first step, since we'll try to give you easier reports to review at first.

Here's what to do to confirm a new defect report:

  1. First, assign the report to yourself by setting your Bugzilla username (your email address) in the "QA Contact" field. If you are not able to add yourself, then send a note to the QA mailing list asking to be added to the "qa-team" group in Bugzilla.
  2. Then read over the report. What are they really saying? Are they reporting a bug? Asking for a new feature? Asking a support question? Just because it is in Bugzilla does not mean that it is a defect report.
  3. Ask yourself: Are they saying that something is working incorrectly? Or are they asking for new functionality? If they are asking for a new feature, change the Issue Type field to "FEATURE". If they are asking for an enhancement of an existing feature, set the Issue Type field to "ENHANCEMENT". You can now set the Status to CONFIRMED and then save your changes. You are done with the report.
  4. Otherwise, search Bugzilla for keywords related to the bug report to see if this is a duplicate defect report. If it is, change the Status to RESOLVED/DUPLICATE and type in the defect number that this report is a duplicate of. You are now done with this report.
    1. If the report is actually reporting a defect, then you have more work to do. Try to reproduce it with the latest release of OpenOffice. If you can reproduce the defect. set the Status to CONFIRMED, set the "Last Confirmation" field to the release that you tested with, and set the Importance field to an appropriate value, depending on the severity of the defect.
    2. If the report is not clear enough to reproduce, enter a comment asking the user for more information and add the "needmoreinfo" to the Keyword field. Be specific about what information you need: more detailed steps, a test document, clarification about what version they are running, etc. When the user responds you will be copied on their response and can continue with this report. But for now you are done with this report.
    3. If your testing cannot reproduce the bug, and you do not need more information from the user, then change the status to RESOLVED/IRREPRODUCIBLE.
    4. If the user was confused, did not understand the functionality or was asking a support question, then point them to the Community Support Forums and mark the issue as RESOLVED/INVALID. Of course, you can help if it is a simple question, but the forums are the better place for a user to find help. Bugzilla is not for support. It is for reporting bugs.

Note: Once you have reviewed a bug report, it should only be left in the UNCONFIRMED state if you are waiting for more information from the user, in which case you should also set the "needmoreinfo" value into the Keyword field. In all other cases you should push the report forward to a new status, either CONFIRMED, RESOLVED/IRREPRODUCIBLE, RESOLVED/INVALID or RESOLVED/DUPLICATE, or by changing the Issue Type to FEATURE or ENHANCEMENT.

It is a judgement call on whether to immediately mark an issue RESOLVED/IRREPRODUCIBLE or to send the user a question and wait for a response. An intermediate approach, when confirming defects report written against older versions of OpenOffice, is to mark the bug as RESOLVED/IRREPRODUCIBLE and at the same time add a comment saying, "I was not able to confirm the bug given these steps in the current version of OpenOffice (AOO 4.1.3), so I'm closing this report. If you are still seeing this problem after upgrading to AOO 4.1.3 please post the details and we can reopen the report".

Finally, think of the confirmation process as the opportunity for the QA Team to improve the value of information we receive from users. We're taking the raw bug reports, sorting through them, eliminating the ones that do not report new bugs, and then passing on the good ones to the programmers. So anything you can do to improve the quality of the incoming defect reports will help. This includes clarifying the steps needed to reproduce the problem, attaching sample documents that you might create to reproduce the problem, improving report titles to make them more accurate/relevant to the real issue, correcting classifications and adjusting defect priorities.

Easy QA Task: Verifying Fixed Defects

Verifying bug fixes (re-testing a bug report after a developer has fixed it) is also very important, since some bug fixes either fail to fix the bug, or cause a new bug.

You can get a list of fixed bugs from this query: All fixed bugs after AOO 3.4 release

Select the bug you want to verify:

  1. First you need to understand which problem the defect report is referring to.
  2. Install the latest build.
  3. Follow the steps to reproduce and check to see if the defect has been fixed.
  4. Add additional comments to the bug with your test result. e.g. build revision number, platform you have been tested etc.
  5. "Test around" the bug to make sure that nothing was broken when fixing it. You will develop and intuition for this as you learn to "think like a bug".
  6. Change defect status to "Verified"

Easy QA Task: Manual Testing

Manual testing gains you further familiarity with QA process, by executing pre-defined test cases and writing up defect reports for any new defects found.

To get more familiar with AOO, now you can try to perform some manual test cases.

Easy QA Task: Test Case Authoring

This is a more advanced topic, but with the above two steps, and learning to "think like a bug", you will be ready for this.

After some practice on test case execution, now you can start writing new test cases.

Useful guide for writing manual test cases:

Module Completion

Once you have done the above, go to our Directory of Volunteers wiki page and add or update your information. Also, add an entry for yourself to the QA Testing Preferences page. Congratulations! Please send a note to qa@openoffice.apache.org so we know.