Classification of Defects in Software Testing

Defect Tracking Overview

One of the important aspects of the Defect Life cycle on a generic level includes defect tracking.

This is important because test teams open several defects when testing a piece of software which is only multiplied if the particular system under test is complex. In such a scenario, managing these defects and analyzing these defects to drive closure can be a daunting task.

In line with defect maintenance processes, when any tester files a defect- apart from the method/description to reproduce the issue seen, he has to also furnish some categorical information that would aid inaccurate classification of the defect.

This, in turn, would help in efficient defect tracking/maintenance processes and would also form the basis for quicker defect turnaround time.

Software defects by nature

Functional defects

Functional defects are the errors identified in case the behavior of software is not compliant with the functional requirements. Such types of defects are discovered via functional testing.

For example, in one of our recent testing projects, a functional defect was found in an ecommerce website’s search engine. It didn’t return any results when a user typed in a product ID, while it was stated in the requirements that the search could be conducted by both a product’s name and ID.

Performance defects

Performance defects are those bound to software’s speed, stability, response time, and resource consumption, and are discovered during performance testing. An example of a performance defect is a system’s response time being X times longer than that stated in the requirements.

Usability defects

Usability defects make an application inconvenient to use and, thus, hamper a user’s experience with the software. A content layout that is difficult to scan or navigate and an overly complex signup procedure are examples of usability defects.

To identify such defects, test engineers and business analysts (or UX designers) validate software against usability requirements and Web Content Accessibility Guidelines (WCAG) during usability testing.

Compatibility defects

An application with compatibility errors doesn’t show consistent performance on particular types of hardware, operating systems, browsers, and devices or when integrated with certain software or operating under certain network configurations.

Compatibility testing is carried out in order to discover such issues. For instance, testing a mobile app for car insurance claim estimation, we uncovered that it failed to behave according to the requirements on Android 8.0 and 8.1. The defects were related to the changes in font size, content alignment, and scroll bar.

Security defects

Security defects are the weaknesses allowing for a potential security attack. The most frequent security defects in projects we perform security testing for are encryption errors, susceptibility to SQL injections, XSS vulnerabilities, buffer overflows, weak authentication, and logical errors in role-based access.

Software Defects by Severity

Defect Severity determines the defect’s effect on the application. It represents the impact on the business of the client. Defect Severity is totally based on how important functionality is blocked or if that functionality functions incorrectly & accordingly add Defect Severity.

The degree of impact that a defect has on the development or operation of a component or system is known as severity.

What is Defect Severity?

Severity means how severe it is affecting functionality. It is totally related to the quality standard or devotion to standard. The severity type is defined by the Software Tester based on the written test cases and functionality. Severity is associated with quality standards or devotion to standard.

It is related to the technical aspect of the product & decides based on how bad the bug is for the system. The severity of the defect means how big functionality is affecting the product. Also, we can say The Severity status is used to explain how badly the deviation is affecting the build.

Based on Bug Severity the product fixes are done. It is classified into different categories but it can vary on organization, projects, people, or defect tracking tool.

Standard classification of Defect Severity

  • Critical: It affects critical functionality & cannot able to test the application further. Testing cannot be started with any workaround as well.
  • Major: Major functionality not working but able to test application. It has a workaround but not obvious.
  • Minor: Bug in Functionality but in the submodule or one under the other module. The minor feature in the module but has workaround from other modules easily.
  • Trivial: Issues in the location of the object or the look and feel issue.

It is also denoted as S1 for Critical Severity, S2 for Major Severity, S3 for Minor Severity & S4 for Trivial Severity.

Software Defects by Priority

Priority is defined as the order in which the defects should be resolved. The priority status is usually set by the QA team while raising the defect against the dev team mentioning the timeframe to fix the defect. The priority status is set based on the requirements of the end-users.

For example, if the company logo is incorrectly placed on the company’s web page then the priority is high but it is of low severity.

Defect Classification by priority

Defect Priority is classified into the following categories:

  • Immediate: The defect needs to be fixed right now, everything else can wait, the build cannot be released with this defect.
  • Urgent: Needs to be fixed before any other high, medium or low defect should be fixed
  • High: Should be fixed as early as possible
  • Normal: Maybe fixed after the release / in the next release. This is the priority set as default.
  • Low: Fixing can be deferred until all other priority defects are fixed
  • None: This priority should not be used. It could be temporarily set in case a defect requires further discussion after triage, particularly in case is was not yet decided to accept the issue as valid.

Nevertheless, defect priority is a subjective decision so it could vary from the above categories and, to determine the proper priority, the following few points needs to consider:

  • Urgency as Business point of view to fix the defect
  • Impact of the defect on functionality
  • Visibility of the defect
  • Check if resources are available or not to fix this means developer to fix this and Testers to verify the fixes.
  • Main constraint in Availability of Time to fix the defect

Definition of priority

The level of business importance assigned to an item, e.g. defect.

Especially when there is a large of number of defects then management of the defect is taken care based on the Defect Priority of the Defect which helps to minimize the product instability.

Setting the priority on a defect

While it is very important for a reporter to set the right Severity, the Priority should be kept as Normal, by letting Triage to set the proper one.

It is possible for a reporter to set a Priority, though. It can be done to help the Development Team on taking the most relevant issues to be fixed first. For example, if the reported defect is a regression on a Stable release currently experienced by several Customers, the fix must be ready before a regular issue.

It is important to note that Priority may change over time. Triage, on top of setting the priority when receiving a new issue, will periodically review the backlog of open issues to ensure the order of fixing matches the current Product Roadmap.

Difference between Priority and Severity

Priority Severity
  • Defect Priority has defined the order in which the developer should resolve a defect
  • Defect Severity is defined as the degree of impact that a defect has on the operation of the product
  • Priority is categorized into three types
    • Low
    • Medium
    • High
  • Severity is categorized into five types
    • Critical
    • Major
    • Moderate
    • Minor
    • Cosmetic
  • Priority is associated with scheduling
  • Severity is associated with functionality or standards
  • Priority indicates how soon the bug should be fixed
  • Severity indicates the seriousness of the defect on the product functionality
  • Priority of defects is decided in consultation with the manager/client
  • QA engineer determines the severity level of the defect
  • Priority is driven by business value
  • Severity is driven by functionality
  • Its value is subjective and can change over a period of time depending on the change in the project situation
  • Its value is objective and less likely to change
  • High priority and low severity status indicates, defect have to be fixed on immediate bases but does not affect the application
  • High severity and low priority status indicates defect have to be fixed but not on immediate bases
  • Priority status is based on customer requirements
  • Severity status is based on the technical aspect of the product
  • During UAT the development team fix defects based on priority
  • During SIT, the development team will fix defects based on the severity and then priority

Classification of Defects Error wise

  • Comments: Inadequate/ incorrect/ misleading or missing comments in the source code
  • Computational Error: Improper computation of the formulae / improper business validations in code.
  • Data error: Incorrect data population / update in database
  • Database Error: Error in the database schema/Design
  • Missing Design: Design features/approach missed/not documented in the design document and hence does not correspond to requirements
  • Inadequate or sub optimal Design: Design features/approach needs additional inputs for it to be complete Design features described does not provide the best approach (optimal approach) towards the solution required
  • In correct Design: Wrong or inaccurate Design
  • Ambiguous Design: Design feature/approach is not clear to the reviewer. Also includes ambiguous use of words or unclear design features.
  • Boundary Conditions Neglected: Boundary conditions not addressed/incorrect
  • Interface Error: Internal or external to application interfacing error, Incorrect handling of passing parameters, Incorrect alignment, incorrect/misplaced fields/objects, un friendly window/screen positions
  • Logic Error: Missing or Inadequate or irrelevant or ambiguous functionality in source code
  • Message Error: Inadequate/ incorrect/ misleading or missing error messages in source code
  • Navigation Error: Navigation not coded correctly in source code
  • Performance Error: An error related to performance/optimality of the code
  • Missing Requirements: Implicit/Explicit requirements are missed/not documented during requirement phase
  • Inadequate Requirements: Requirement needs additional inputs for to be complete
  • Incorrect Requirements: Wrong or inaccurate requirements
  • Ambiguous Requirements: Requirement is not clear to the reviewer. Also includes ambiguous use of words – e.g. Like, such as, may be, could be, might etc.
  • Sequencing / Timing Error: Error due to incorrect/missing consideration to timeouts and improper/missing sequencing in source code.
  • Standards: Standards not followed like improper exception handling, use of E & D Formats, and project related design/requirements/coding standards
  • System Error: Hardware and Operating System related error, Memory leak
  • Test Plan / Cases Error: Inadequate/ incorrect/ ambiguous or duplicate or missing – Test Plan/ Test Cases & Test Scripts, Incorrect/Incomplete test setup
  • Typographical Error: Spelling / Grammar mistake in documents/source code
  • Variable Declaration Error: Improper declaration / usage of variables, Type mismatch error in source code

Why does correct defect classification matter?

Having learned the basics of defect classification, you will not only make sure that defect handling is assigned to the responsible project teams but also streamline defect prioritization. It, in turn, speeds up defect fixes and increases the overall efficiency of the testing and development processes.

As defect severity and priority levels impact the evaluation of the development process efficiency and may influence payments and penalties in case of outsourced development, it is crucial to define the criteria for assessing the defect severity and priority that work for your project.