Page History

...

Author: Tony Lombardo
Created on: 08/05/2010
Last Modified onUpdated: 08/06/2010

Contents

I am a tester, how do I go about testing KFS?

These are the main things you need to know to test (well, almost!). Your Testing Coordinator or QA Lead may have additional instructions for you and as things change during QA. You may also see instructions that modify or change what is state here on our email list service. (For additional information on the Roles and Responsibilities of our QA Team members please see: KC QA Team Roles and Responsibilities).

1. Test Kuali Financial System (KFS)

...

• Using the steps that created the bug, make sure you can reproduce it one more time. It is important you are confident that the bug is reproducible. This will take more time, but we would prefer to empahsize emphasize the quality of the bugs rather than the quantity.
• It is a good idea to write down each step you take as you work. Otherwise you may find it difficult to recreate what you did to cause the bug. If you are following the steps in a test script exactly, it is written down for you so you can just copy it from the test script.

...

• You can use the following link to quickly create a bug issue: Create a bug report (Be sure to log into Jira.)
• Here are step-by-step instructions for creating the bug: How to Enter A bug in Jira.
• Be sure to include the exact steps to recreate the bug.
• If you use the previous link, it will automatically set the issue type to be a "Bug."
• If you want it to be some other issue type, such as an "Improvement," you can re-edit the issue type after you have saved it.
• Note on Exception/Stack Trace Errors: If the application produced an exception error, there will be a big chunk of error text on your screen. Here is a sample of what they look like: KC KFS Exception Error Sample. Please cut and paste the entire error into the bottom of the Description field after your step-by-step instructions for creating the bug. Do not rely on the screen shot to document this error.

...

• When your issue has been fixed, you need to confirm that it really was fixed to as you expected it. This means you need to retest KC KFS for the issue.
• Retest for the bug as soon as possible after you learn of the fix. As a team managing issues and work gets difficult and inefficient if the bugs are not addressed quickly - thank you for helping with this!
• What to do when the bug is fixed or if it is not:
  1. If fixed - close the bug and comment that you retested and found it fixed. In the comment, indicate how you tested if you did not use the exact same steps or data as you outlined in the original bug description.
  2. If bug is not fixed - reopen the bug with a comment saying why you are reopening it. - Please note: If you found an unrelated bug, create a new bug report - don't use the existing bug report for anything new. Continue to monitor for complete fix.
• Remember Testers! You are not done with a bug until you have confirmed it is fixed and you have closed it. Please!

...

Jira is an issue management tool designed principally for software projects. It is built an sold by Atlassian. The Kuali software projects use Jira for managing bugs, tasks and project issues. Each software project may have one or more Jira projects or spaces to manage their issues. To access a list of the the Kuali Jira home page and a list of all Kuali's Jira projects click here: https://test.kuali.org/jira/secure/BrowseProjects.jspa^{Image Removed}.

updated: KHensley, 2010-03-05 Fri

Where should I go to test KFS?

...

The Kuali projects use an issue tracking tool called Jira. People working on the project will also refer refer to these issues as "Jiras." If someone says to you, "Could you please create a Jira for that?" they mean please record the issue using, our web-based, issue tracking software of the same name. Each of the Kuali software projects has serveral Jira projects to keep the bugs and issues organized. The one where we enter bugs for Kuali Coeus is known as the QA Feedback project. It's shorthand name is KRAFDBCK. Each issue in a Jira project is given a unique ID which consists of the short name and a number as in KRAFDBCK-199. If you have that number you can always find your bugs by entering them in the Jira search box at the top right of the screen. Here is s a link to the Jira, "KRA QA Feedback Feedback" project (note the project still uses acronym of KC's old name: "KRA" for "Kuali Research Administration") KFS Feedback project.

A bug stopped me from testing, should I set it to Blocker priority?

• Well of course, it all depends. It is natural to think that if you cannot proceed with testing that you are "blocked" and that this means the priority is Blocker. However, what is a priority in one context is not in another. In the case of "Blocker" we think big - in terms of the application or an entire Module as a whole.
• In KC KFS we we use Blockers for really serious issues with a big impact to testing or the application. If you are asked to test the Sponsor Lookup and you click on the icon and nothing happens, you may well consider yourself blocked. But in terms of the Application and Module, it turns out it is not so severe because you find you can enter the sponsor if you know the ID and you can find the ID in the maintenance document. It maybe critical, but certainly not a blocker.
• Also, please note that setting Blocker priority sends emails to the DMs, QA Manager and QA Leads and everyone "hops to it" with the old Adrenaline rush.

...

• An important module, Proposal or Budget say, in the application is rendered unusable or unavailable. For instance, you cannot search for or open existing Proposals or you cannot create new ones.
• There are seriously lost or corrupted data, such as situations where you cannot save a proposal or budget or data is transmitted to a grants.gov form that would have serious consequences or prevents submission.
• The issue is affecting all users and not just a single or small group of users.
• The application as whole is completely down and is preventing anything from being tested.

What is the backdoor login and when should I use it?

...

• The back door login is a feature that allows you to login as a new person without first logging out
• Testing while logged in using the backdoor feature is definitely not recommended - especially for permissions. Testing this way cannot be considered conclusive and bugs will be sent back to you for retesting
• Please open a new instance of your browser and login with the user that has the permissions you are trying to test.
  7.

I got an exception error/stack trace, can I just insert a screenshot of an exception error in my bug report?

...

• Please include both the text as well as a screenshot of the entire screen (press Alt-PrtScn).
• The disadvantage of screenshots is that they are not searchable and this makes it hard to find them. Screenshots, on the other hand, may convey some information to the developer that the exception will not.
• To include the text, copy the big chunk of error text on your screen and paste it at the end of the step-by-step instructions you provided for creating the bug.
• Here is a sample of what exception errors look like: KC KFS Exception Error Sample.

What's the quickest way to enter a new bug?

The following link will save you some clicks two and directly open a blank bug report screen for the QA Feedback project in Jira: Create Bug Report
Fill in all the fields according to these instructions: KC KFS QA Reporting Issues in JIRA - KRAFDBCKKFSFB

Please remember, that other than this, there are no shortcuts to entering a quality bug report! Please see I am a tester, what do I need to do to test KCKFS?

What should I do with all of the jira@kuali.org email messages I receive?

...

The JIRA bug tracking system will send automatic email messages to the Reporters and Assignees of any JIRA issues, so when you create a new issue you will receive email updates any time that JIRA is modified. The QA process itself updates the status of a JIRA several times as the issue moves through the system, so you can expect to receive several email messages from jira@kuali.org for that issue alone. You might create a rule to filter and move these messages to a different folder so your Inbox doesn't get cluttered. It's important to at least read the most recent email message received for each JIRA, as it will often have comments or questions from the QA team or Developers for the Reporters (testers).

...

To set the Kuali Coeus project as the default the first time you log into Contour:

• Sign into Contour https://contour.kuali.org/^{Image Removed}
• Click on "Change Project" button in upper left
• Select "Kuali Coeus" from the pop-up box

To find the test scripts assigned to you:

• Sign into Contour https://contour.kuali.org/^{Image Removed}
• Click on "Reports" button in upper right
• Under the Global Reports tab, click on "!KC KFS - Testing Assignments & Status (by Team and Tester)"
• Choose your output, HTML or PDF
• Click "Run Report" button at the bottom
• This list is sorted by team, so scroll down to your team to see your list of scripts to test.
• Click on the blue ID link to pull up the script.

...

• On the QA Home page, left hand side is the heading Testing Team Pages, click on our team, "Proposal Workflow and Award Workflow"
• At the bottom of the page is a header, "Proposal Workflow and Award Workflow Issues Requiring Retesting"
• It is your responsibility to retest the script once it appears here.
• Follow the instructions in section(g) of I am a tester, how do I go about testing KCKFS?

How do I user filters to find My Testing Assignments?

You can use the reports or filters in Contour to find your assignments. Most testers find that the filters are easier to use because, unlike the reports, opening a test scripts or test analyis analysis item does not open a new browser session or browser tab (depending on your Browser's settings) each time you click a link. Instead, it opens a new tab in Contour and you can see and work with all your items more easily. The following steps will give you an idea of how to do it. Click the image below for an overview. There is more information in the steps below.

...

As indicated above, there are some more cases of course. If we use QA Workflow this way, an appropriately set list of issue filters will clearly indicate the state of our issues list.
Issue Status - Description Next Step(s) Filter Status Page Category
New - Requires QA Review

• A new issue has been reported. A QA Lead or other designated person, assigns themselves the issue, marks it as in "QA Review" and runs the issue through the Issue Review Process.
  "Requires Review" is the initial default for QA Workflow. "Requires Review" should only be set for new bugs that have never been looked at. If you open and review this field should be set to something else when you are done looking at it. Unless you decide that you are not going to complete the review and you are going to leave it for someone else. Even then, though, it would be better to assign it to the QA Lead who will review the issue and set QA Workflow = QA Review. QA Workflow = Requires Review

Fix For = (blank)
New - Requires QA Review
In QA Review

• Issue is being reviewed by a QA Lead or other QA team member. A QA Lead must review the issue (see QA Review Process described in the QA FAQ) to ensure it is actionable for development. NOTE: The reviewer must make certain that both the Fix Version and the QA Workflow field are set correctly.
  QA Review indicates that the bug is under review by a QA Lead. QA Workflow = QA Review

Fix For = (blank)
OR
Fix For = (a future release)
In QA Review
Subcommittee / SME / FC Review

• Requires SME functional discussion and decision. SME group for the appropriate module reviews issue. This may be to determine if the issue should be included the release currently under development (or a future release) or to resolve some functional questions that require input from the SME group. QA Workflow = Subcommittee / SME / FC Review

Fix For = Unknown
OR
Fix For = Unscheduled Release Subcommittee / SME / FC Review
Ready For Development

• The issue has been reviewed by QA and cleared for development work. The issue is reviewed by someone from development (usually the DM). If it is cleared for work, it is moved to the appropriate Development Project to be assigned to the developer who will work on the issue. If there are questions, it is marked as having "Questions From Development".
  Fix - The issue is ready to be addressed by the developers. QA Workflow = Ready for Development

Fix For = (the release currently in development) Ready for Development
Questions From Development - The review by Development identified questions that must be answered before the issue can be assigned. A QA or functional lead assigns themselves the issue and takes responsibility for resolving any open questions the developer may have. After answering the questions QA Workflow field can be set back to "Ready for Development" if that is appropriate or it may require SME Review as appropriate. QA Workflow = Questions From Development Questions From Development
In Development

• The issue has been accepted by Development for assignment and resolution. These issues reside in the Development project. The developer works to resolve the issue. When work is completed, the issue is set to "Resolved" indicating that the issue is fixed and ready for retesting by the reporter. NOTE: If the developer finds they cannot reproduce the issue at this time, they should assign the issue back to the reporter of the issue for final closing. QA Workflow = Ready For Development In Development
  Hindered - Functional
• There is a functional question about an in-progress issue that is preventing further work. These issues reside in the Development project. A QA or functional resource must address any question delaying work on the issue. Once the assignee is satisfied, they can mark the issue as "Unhindered.". Proceed? = Hindered Functional

Fix For = (the release currently in development) Technical Review (Development, Technical Committee(s))
Resolved Issues - Retest Required

• A fix for issue has been provided and the issue now requires retesting by the person who reported the issue. The issue is retested following the steps as originally written. Responsibility for retesting is usually conducted by the person who found and reported it but may also be conducted by a QA Lead or other designated person. The issue will then be either Closed or Reopened. Status = Resolved

Fix For = (the release currently in development) Resolved Issues - Retest Required
Closed - Fixed

• The fix supplied for the issue was tested and confirmed. The reported issue is no longer present. Nothing to do - end of issue. QA Workflow = QA Review Closed
  Closed - Duplicate
• This issue was already reported. Close any duplicates and leave the best written JIRA Open. Link duplicate JIRAs to open issue. QA Workflow = QA Review Closed
  Closed - Cannot Reproduce
• The issue was closed after it could not be recreated and confirmed. Nothing to do - end of issue. QA Workflow = QA Review Closed
  Closed - Won't Fix
• The issue will not be fixed because it was determined the behavior was "as designed" or some other technical reason. Nothing to do - end of issue. QA Workflow = QA Review Closed
  Reopened
• The issue has been reopened. Depends on where the issue was reopened and who reopens it. If the issue is in the QA project, it will go through the review process. If it is in a development project, it is the responsibility of the assigned developer to respond. (depends where opened)
  Deferred / Postponed
• The issue will not be fixed in the Release currently under development but will be considered for a future release. During QA, it does not matter if you set QA Workflow to QA Review or to one of the other review types, but it should never be left at "Requires Review". Always indicate the next step. Issues that are set to Unscheduled Release may require action in one or more of the additional review types - so set it. Picking one makes the next step clear. You can also set it to QA Review and set the Fix Version if that is clear. You can always review it again update any QA fields. QA Workflow = QA Review
  OR
  QA Workflow = Subcommittee / SME / FC Review

Fix For = (a future release)
OR
Fix For = Unscheduled Release In QA Review
Deferred to Future Release

• The issue will not be fixed in the Release currently under development but will be considered for a future release. After the successful release of the current version under development, these issues must be re-reviewed for inclusion in the subsequent release. QA Workflow = Subcommittee / SME / FC Review

Fix For = (any of the Unreleased Release Versions)
OR
Fix For = Unscheduled Release