Software Testing Process and Quality Assurance

Establishing the SQA Program

2010-01-16T21:58:00.000-08:00

To make SQA program successful, the most important step is to get senior management commit to it. The senior managers must first agree on SQA goals, and agree to resolve major SQA issues between SQA and line management. Without management support, SQA can not be effective.

Creating SQA Plan
An effective SQA program requires forward planning and following through it. The SQA plan specifies its goals, tasks to be performed, and the standards and procedures against which the development work is to be measured.

The IEEE standard for SQA plan preparation contains the following:
1. Purpose
2. Reference Documents
3. Management
4. Documentation
5. Standards, Practices, and Conventions
6. Reviews and Audits
7. Software Configuration Management
8. Problem Reporting and Corrective Action

9. Tools, Techniques, and Methodologies
10.Code Control
11.Media Control
12.Supplier Control
13.Records Collection, Maintenance, and Retention

Many of these topics are relatively clear from their headings, the documentation, standards sections are given more explaination in the following part. The SQA plan section on reviews and audits should describe
both the technical and the managerial reviews and audits to be conducted.

Documentation
The documentation section should describe the documentation to be produced and how it is to be previewed. These include:
1. Software requirement specification, which specifies each software function, performance parameter, interface, or other attribute with sufficient precision to permit its verification.
2. Software Design Description, which describes the major components, databases, and internal interfaces.
3. Software Verification and Validation Plan, which describes the methods used to verify that the requirements are implemented in the design, that the design is implemented in the code, and that the code meets the requirements.
4. Software Verification and Validation Report, which is used to report on the SQA verification and validation activities.
5. User Documentation, which is required for installation, operation, and maintenance of the software.
6. Other, includes software development plan, the software configuration management plan, the standards and procedures manual, together with the planned review methods.

Standards and Procedures
The standards are the criteria to which software products are compared. Procedures are the criteria to which development and control processes are compared. They are closely related. The standards define what should be done; while procedures define how the work is actually to be done, by whom, when and what is done with the results.

The minimum requirement for standards include:
1. Documentation Standards specify form and content for planning, control, and product documentation and provide consistency throughout a project.
2. Design Standards specify the form and content of the design product. They provide rules and methods for translating the software requirements into the software design and for representing it in the design documentation.
3. Code Standards specify the language in which the code is to be written and define any restrictions on use of language features. They define legal language structure, style conventions, rules for data structures, and internal code documentation.

SQA Activities
SQA activities include product evaluation and process monitoring, which ensure the product and the process used in development are correctly carried out and standards are followed. SQA audit, another SQA activity, is a key technique used to perform product evaluation and process monitoring. Production evaluation is to ensure that standards are followed. It assures that clear and achievable standards exist and evaluate compliance of software product with the standards.

Process monitoring is to ensure that appropriate steps to carry out process are being followed. SQA monitors processes by comparing actual steps performed with established procedures.

Audit is a fundamental SQA technique. It looks at a process or product in depth, comparing them with established standards and procedures.

Tailoring SQA to Project
Each project has its specific attributes and SQA program should be tailored to accommodate to the project needs. The characteristics that should be considered are: mission critical of project, schedule and budget, size and complexity of project, size and complexity of project staff organization. The relationship between SQA program and mission critical level is very straight forward. The more critical the project, the more important and formal the SQA should be. The relationship between SQA and budget and schedule is not so intuitive; the tighter the budget and schedule, the more critical it is to have a well planned and effective SQA program. This does not mean that SQA program for project with more resources can be lax. The project organization structure also influence the SQA program. For large or dispersed staff, more formal SQA program is required. A small, centralized development staff can easily inform each other the nonconformance and
helping each other in meeting standards, less formal SQA effort is required.

Responsibilities of SQA

2010-01-16T21:54:00.001-08:00

To achieve its goals, SQA is responsible for:

1. Review all development and quality plans for completeness.
2. Participate as inspection moderators in design and code inspections.
3. Review all test plans for adherence to standards.
4. Review samples of all test results to determine adherence to plans.
5. Periodically audit SCM performance to determine adherence to
standards.
6. Participate in all project phase reviews and write down any
nonconformance.

Goals of SQA

2010-01-16T21:53:00.000-08:00

The software development is a complex process full risks. There are technical risks such as software will not perform as intended or be too hard to operate, modify, and/or maintain; there are programmatic risks such as the project will overrun cost or schedule. The goals of SQA is to reduce these risks by:

Appropriately monitoring the software and the development process.
Ensuring full compliance with standards and procedures for software and process.
Ensuring that inadequacies in product, process, or standards are brought to management's attention so that they can be fixed.

SQA is not responsible for producing quality products or for making quality plans. They are responsible for auditing the quality actions and for alerting management to any deviations.

Software Quality Assurance | SQA

2010-01-16T21:52:00.000-08:00

(SQA) is a planned and systematic approach to ensure that software process and products conforms to the established standards, processes, and procedures. The goals of SQA are to improve software quality by appropriately monitoring both software and the development process to ensure full compliance with the established standards and procedures. The first step to establish an SQA programis to get the top management's agreement on its goal. It then needs to identify SQA issues, to write SQA plan, to establish standards and SQA functions, to implement the SQA plan and to evaluate SQA program. For SQA to be effective, they must have good people and full management support. High quality software product must be able to run correctly and consistently, have few defects (if there are), handle abnormal situation nicely, and need few installation effort. The defects should not affect the normal use of the software, will not do any destructive things to system, and rarely be evident to the users. Before deciding what measures to use, it is essential to consider the objectives of the measurement program. If the measures will be used to manage software development, they should be objective, timely available and controllable. On the other hand, if the measures are to support decisions on product acceptance, they must reasonably represent user needs.

Definition

SQA is the planned and systematic set of activities that ensure that software process and products conform to requirements, standards, and procedures. "Processes" include all activities involved in designing, coding, testing and maintaining; "products" include software, associated data, documentation, and all supporting and reporting paperwork. The role of SQA is to give management the assurance that the officially established process is actually being implemented. It ensures that:

1. An appropriate development methodology is in place.

2. The projects use standards and procedures in their work.

3. Reviews and audits are conducted.

4. Documentation is produced to support maintenance and enhancement.

5. Software Configuration Management is set up to control change.

6. Testing is performed and passed.

Any deficiencies and deviations are identified and brought to management's attention.

Introduction to Six Sigma

2009-08-17T02:31:00.000-07:00

What Is Six Sigma?

Six Sigma is a methodology that aligns core business processes with customer and business requirements; systematically eliminates defects from existing processes, products, and services; or designs new processes, products, and services that reliably and consistently meet customer and business requirements. It essentially boils down to an approach for quantifying how well a business is meeting stakeholder expectations and then applying tactics for ensuring that those expectations are met virtually every time.

A major difference between Six Sigma and other quality programs, such as Total Quality Management (TQM), is that Six Sigma incorporates a control phase with ongoing checks in order to ensure that once improvements are achieved, they are not a one-time or temporary phenomenon, but maintained over time. Six Sigma methodology gives those who use it a structured yet flexible process to follow, a large and expanding tool set to employ, a configuration to clarify roles and responsibilities, and a governance to ensure compliance. Six Sigma can be used to improve existing processes or create a new product or process.

The “sigma” in Six Sigma is the Greek letter that statisticians use to represent the standard deviation of a population: it tells how much variability there is within a group of items (the “population”). The more variation there is, the bigger the standard deviation is. Thus, the sigma level is tied directly to the number of defects: the fewer the defects, the higher the sigma level, and the better the quality.

Each time that a process or product does not meet stakeholdersÙ expectations, it is counted as a defect. To achieve Six Sigma, a process must not produce more than 3.4 defects per million opportunities. To put this in perspective, if you were a publisher and a misspelled word was considered a defect, 99 percent quality would mean that for every 300,000 words that were read by the customers who purchased your books, 3,000 would still be misspelled. Six Sigma strives for near perfection; therefore, reaching Six Sigma quality would mean that for the same 300,000-word opportunity, only 1 word would be misspelled. For training programs, a defect would be anything that did not meet customer requirements. It could be a misspelled word, a simulation that has incorrect information or does not work properly, a hyperlink that is broken, or a course taking too long to complete or costing too much. In short, anything that does not meet a customer requirement is considered a defect.

The Six Sigma philosophy can be captured as a methodology that allows companies to:

1. Consistently meet customer requirements

2. Use data to drive all decision making

3. Do everything with quality

In other words, Six Sigma is a customer-focused, data-driven, measurement-based strategy that allows companies to meet customer requirements virtually every time.

The History of Six Sigma

When many people think of Six Sigma, the first name that comes to mind is Jack Welch. Welch was the CEO of General Electric who championed Six Sigma and in the process made it a household word in corporate America. Welch launched the effort in late 1995 with two hundred projects and intensive training programs, moved to three thousand projects and more training in 1996, and undertook six thousand projects and still more training in 1997. The initiative was a stunning success, delivering far more benefits than Welch had first envisioned. Six Sigma delivered $320 million in productivity gains and profits, more than double WelchÙs original goal of $150 million.

In fact, Six Sigma greatly predates Welch and his experience at GE. A Motorola engineer, Bill Smith, is the individual credited with coining the term Six Sigma. In the mid-1980s, Motorola engineers decided that the traditional quality levels for measuring defects did not provide enough granularity. At that time, it was common practice to measure how many defects occurred for every 1,000 opportunities, but these engineers wanted to measure the defects per 1 million opportunities. Motorola developed this new standard and then created the methodology.

As a measurement standard, however, Six Sigma goes even further back, to Carl Frederick Gauss (1777–1855) the German mathematician who introduced the concept of the normal curve. And as a measurement standard in product variation, it dates to the 1920s when Walter Shewhart (who is credited with combining creative management thinking with statistical analysis) showed that three sigma from the mean (or average) is the point where a process requires correction.

Understanding the history of this methodology and the backgrounds of the individuals who shaped this approach sheds light on the rationale for the methodology . Shewhart spent most of his career in the Bell Telephone Laboratories, where he worked on statistical tools to examine when a corrective action must be applied to a process. Bill Smith was a Motorola engineer who introduced the statistical approach to increasing profitability by decreasing defects. The work of Jack Welch and how he turned GE around, largely as a result of adopting Six Sigma, is well documented. None of these men were just dealing purely in theory; they worked in businesses and were responsible for quantifying their contributions to the organization in a way that the business respected. As a result they understood the need to show business results.

Three concepts are at the core of Six Sigma: the concept of the customer, a defect, and tollgate reviews. These concepts apply to Six Sigma regardless of the model and will be discussed throughout this book.

The Customer

With Six Sigma, everything begins and ends with the customer. According to iSixSigma, a customer is “one who buys or rates our process/product (in terms of requirements), and gives the final verdict on the same” (www.isixsigma.com). Motorola University teaches that there are two types of customer classifications: internal and external.

Internal customers are stakeholders, departments, and employees within the company. They are frequently referred to as process partners. They may use their companyÙs products or services or may be part of the value chain that helps to produce the product. In developing a training program, a process partner might be a subject matter expert or the manager of an employee who will take the training. The requirements of internal customers are frequently referred to as the voice of the business.

External customers are individuals or organizations outside the company. They use or purchase a product or service in its final form and are referred to as end users. They are the reason an organization is in business. If the training group is designing a training program for the accounting department, then the accountants who are taking the training are considered the customers. Whoever is paying for the course development is also considered an external customer. The same is true if the training is being developed for individuals who do not work for the company. The requirements of external customers are referred to as the voice of the customer.

The same rigor that is applied to external customers needs to be applied in understanding internal customer needs. Improvements made for an internal customer ultimately lead to a quantitative improvement for the external customers.

The Six Sigma philosophy holds that these two entities, the internal and the external customers, dictate the requirements for specific products or services and thus quality . The highest level of quality means meeting the expectations of both internal and external customers.

The Defect

Internal and external customers dictate the requirements for a product or service. Not meeting a requirement is considered a defect. To use a sample example from the training world, letÙs assume that as a training manager, you are commissioned by the accounting department to develop a program that teaches employees how to use a new accounting system. One of the requirements is that the class is no longer than one hour. You build the course, but it is one hour and ten minutes long. The course length then becomes defective. LetÙs assume that another requirement is that a specific logo must appear on every page of a PowerPoint presentation. Each time the logo does not appear is considered a defect. If the PowerPoint presentation is 100 pages and the logo is absent from fifteen slides, the defects per million opportunities (DPMO) would be equal to the total defects divided by the total opportunities then multiplied by 1 million:

DPMO = Total defects/total opportunities x ,000,000.

Applying this formula to the training example would be as follows:

(15/100) x 1,000,000 = 150,000 DPMO.

One hundred and fifty thousand defects per million opportunities would translate into a sigma level of less than 2.5.

The number of DPMOs translates into your sigma level: the fewer defects, the higher the sigma level. The performance in the example would yield a sigma level of between 2.0 and 2.5, meaning that it would be meeting critical customer requirements (CCRs) less than 84 percent of the time.

TABLE 2.1: SIGMA LEVEL AND DPMO

Guidelines for the Review Session

2009-03-21T19:55:00.000-07:00

Guidelines for conducting of formal review sessions must be established in advance, distributed to all reviewers, agreed upon, and then followed. The following checklist represents a set of guidelines for formal technical reviews.

The duration of the review meeting should be less than 2 hours.
If necessary, a new session is convened at the earliest the next day.
The moderator may cancel or call off the review meeting in case
- one or more reviewer did not occur or did not sufficiently prepare in advance
- for any reason the moderator is not able to successfully and efficiently conduct the review meeting.
The reason for cancelling should be recorded.
The manager of the software project may not interfere in the review meeting.
He should not participate in the review meeting or take the part of the recorder.
Review the product, not the producer.
- The tone of the meeting should be loose and constructive.
- The moderator should conduct the meeting to ensure that the proper tone and attitude are maintained.
- The author may not defend himself or his product.
Limit the number of participants.
Keep the number of people involved to a necessary minimum (a maximum of five reviewer).
The moderator may not act as a reviewer at the same time.
General style concerns (outside the guidelines) may not be discussed.
Enunciate problem areas, but do not attempt to solve every problem noted.
A review is not a problem solving session. Problem solving should be postponed until after the review meeting.
Every reviewer must have the possibility to adequately present his findings.
Each agreement on a finding has to be recorded in such a way that every reviewer can see the notes.
Findings may not be noted in the form of instructions for the author.
Every single finding has to be weighted as
- crucial error (usability of the product impossible; it has to be corrected before release);
- main error (affects usability of the product; it should be corrected before release);
- besides error (hardly affects usability);
- good (without error).
The author of the product must be passive during the review meeting.
He should only clear up misunderstandings or answer questions.
At the end of the review, all attendees must decide whether to
- accept the product without further modification,
- accept the product provisionally (minor errors have been encountered and must be corrected, but no additional review will be required),
- reject the product due to severe errors (once corrected, another review must be performed).
At the end of the review meeting, all attendees have to sign-off the protocol.
At the end of the review meeting, the author must collect all marked copies of the product to be able to adopt the markings when reworking the product.

CMMI for Small Organizations

2009-03-21T19:34:00.000-07:00

Regardless of an organization’s size, process improvement must use a structured approach to be successful. CMMI includes more information and includes more areas that promote the development of high-quality systems. However, professional judgment in determining how to implement, interpret, and scale the model must be used.

One problem that still surfaces concerning CMMI and small organizations is cost. The costs of implementing CMMI in any organization are high. Costs include training your own organization, assigning personnel at least on a part-time basis to perform process improvement work that is nonbillable to your client, and perhaps hiring external consultants for process improvement and appraisal assistance.

Additionally, the SEI is enforcing conflict-of-interest rules governing who can conduct and participate in SCAMPI appraisals. Basically, those personnel who have been heavily involved in process improvement efforts for an organization (such as writing procedures or overseeing their implementation) cannot serve as Lead Appraisers or as members of the SCAMPI team. This places an additional cost burden on the organization should that organization need to hire or contract with personnel outside of its immediate organization or business unit in order to perform the SCAMPI and procure the additional formal training required for the SCAMPI. More training activities, more certifications and observations to authorize personnel, and the costs of such activities have also increased. Therefore, for those organizations that are small and are struggling just to meet their payroll requirements, the CMMI approach may be beyond your reach. You may have been priced out of the marketplace. While the concepts expressed in the CMMI are still beneficial, a formal implementation and appraisal of them may not be financially feasible for the truly small organization. It can also be said that the CMM is always right for an organization; the CMMI is always right for an organization; SCAMPI/SCE/CBA-IPI is always right for an organization. What is not right for an organization is basing contract awards or contract bid opportunities solely on appraisal results.

Impact Analysis Checklist for Requirements Changes

2008-12-30T00:38:00.000-08:00

Implications of the Proposed Change

* Identify any existing requirements in the baseline that conflict with the proposed change.
* Identify any other pending requirement changes that conflict with the proposed change.
* What are the consequences of not making the change?
* What are possible adverse side effects or other risks of making the proposed change?
* Will the proposed change adversely affect performance requirements or other quality attributes?
* Will the change affect any system component that affects critical properties such as safety and security, or involve a product change that triggers recertification of any kind?
* Is the proposed change feasible within known technical constraints and current staff skills?
* Will the proposed change place unacceptable demands on any computer resources required for the development, test, or operating environments?
* Must any tools be acquired to implement and test the change?
* How will the proposed change affect the sequence, dependencies, effort, or duration of any tasks currently in the project plan?
* Will prototyping or other user input be required to verify the proposed change?
* How much effort that has already been invested in the project will be lost if this change is accepted?
* Will the proposed change cause an increase in product unit cost, such as by increasing third-party product licensing fees?
* Will the change affect any marketing, manufacturing, training, or customer support plans?

System Elements Affected by the Proposed Change

* Identify any user interface changes, additions, or deletions required.
* Identify any changes, additions, or deletions required in reports, databases, or data files.
* Identify the design components that must be created, modified, or deleted.
* Identify hardware components that must be added, altered, or deleted.
* Identify the source code files that must be created, modified, or deleted.
* Identify any changes required in build files.
* Identify existing unit, integration, system, and acceptance test cases that must be modified or deleted.
* Estimate the number of new unit, integration, system, and acceptance test cases that will be required.
* Identify any help screens, user manuals, training materials, or other documentation that must be created or modified.
* Identify any other systems, applications, libraries, or hardware components affected by the change.
* Identify any third party software that must be purchased.
* Identify any impact the proposed change will have on the project’s software project management plan, software quality assurance plan, software configuration management plan, or other plans.
* Quantify any effects the proposed change will have on budgets of scarce resources, such as memory, processing power, network bandwidth, real-time schedule.
* Identify any impact the proposed change will have on fielded systems if the affected component is not perfectly backward compatible.

Download thew complete document:
Impact Analysis Checklist for Requirements Changes (From Scribd)

Contents in the document:
1. Implications of the Proposed Change
2. System Elements Affected by the Proposed Change
3. System Elements Affected by the Proposed Change
4. System Elements Affected by the Proposed Change

SPM - General aspects that should be considered when managing a software project.

2008-12-09T07:31:00.000-08:00

Have all responsibilities been defined?
Are all members of the project sufficiently informed about the project?
Are status reports created regularly?
Is there any trouble between the members of the project?
Are there any standards for project management? Are these standards applied?
Are there any agreements with the customer that are not documented in the project folder?
Is the customer sufficiently informed about the progress of the project?
Are there any problems with the customer's personnel?
Have you informed the customer about your deputy?
Is there a back-up position for every key position of the project?

Software Metrices - Checklist - Software Metrics

2008-12-06T03:24:00.000-08:00

The process metrics questions focus on the degree to which the software engineering process is quantified and measured. Typical metrics concern software quality, the amount of code developed, resources used, and such progress indicators as review coverage, test coverage, and test completion.

Are any metrics defined and applied for measuring the quality of the software product?
Are there any metrics to rate the quality of the software process?
Are quality measures used to identify weak points of the product and the process and to verify quality criteria?
Are measures used for initiating corrective actions, if a measurement value deteriorates or exceeds given boundaries?
Are statistics on software design errors gathered?
Are software staffing profiles maintained of actual staffing versus planned staffing?
Are software problem reports tracked to closure?
Are the cost and effort needed to correct errors measured and recorded?
Are statistics on software code and test errors gathered?
Are profiles maintained of actual versus planned software units designed, over time?
Are profiles maintained of actual versus planned software units completing unit testing, over time?
Are measures used for introducing product and process improvements?
Are all data collected during the project recorded and evaluated?
Are process and product metrics (e.g. productivity and effort) used to plan following projects?
Are there any mechanisms to regularly review whether the defined metrics are still adequate and representative?
Do you plan for measurement activities?
Are productivity and effort measured and recorded for each phase of the project?
Are profiles of software size maintained for each software configuration item?
Are profiles of software complexity maintained for each software item?
Are profiles maintained of actual versus planned software units integrated, over time?
Are target computer memory utilization estimates and actuals tracked?
Are target computer throughput utilization estimates and actuals tracked?
Are design and code review coverages measured and recorded?
Is test coverage measured and recorded for each module tested?

Acceptance Testing - Checklist - Acceptance Testing

2008-12-06T03:12:00.000-08:00

Test Preparation

Has the plan for acceptance testing been submitted?
Have all possible interactions been described?
Has the method of handling problems detected during acceptance testing and their disposition been agreed between you and the customer?
Have you defined the testing procedure, e.g. benchmark test?
Have you designed test cases to discover contradictions between the software product and the requirements, if existent?
Have you established test cases to review if timing constraints are met by the system?
Are all input data required for testing available?
Is it possible to automatically document the test runs?
Have the customer specific constraints been considered?
Have you defined acceptance criteria (e.g. performance, portability, throughput, etc.) on which the completion of the acceptance test will be judged?

Test Execution and Evaluation

Has the acceptance test been performed according to the test plan?
Have the users reviewed the test results?
Are the services provided by the system conform to user requirements stated before?
Have the users judged about acceptability according to the predetermined criteria?
Has the user signed-off on output?
Have all steps of the test run been documented?

Guidelines for Test Case/ Test Plan/ Requirements/ Project Plan Review Planning and Preparation

2008-12-06T03:10:00.000-08:00

Review Planning

Allocate resources and time schedule for reviews.
For reviews to be effective, they should be scheduled as tasks during the software engineering process. You should plan for up to 20 % of the effort spent on creating the documents to be reviewed.
In addition, time should be scheduled for the possible modifications that will occur as the result of a review. You should plan for up to 10 % of the effort spent on creating the documents to be reviewed (in case of one rework).
Conduct meaningful training for all reviewers.
The training should stress both process-related issues and the human psychological side of reviews.
Choose a moderator for the review session.
- The moderator must have the analytic expertise to be able to follow the review discussions.
- The moderator should not directly be involved in the project.
Determine the aspects for which the result should be surveyed by the review team.
The aspects conform to
- experiences from past products,
- the previous progression of the software project and
- the presumed strength and weakness of the author.
You should choose those aspects which conceal the highest risks!
The reviewers should be selected depending on the chosen aspects.
- Each aspect should be reviewed by at least two qualified reviewers.
- Each reviewer may examine more than one aspect.
- Human aspects should also be considered when assembling the review team.
If the required reviewers are not available in your project you should ask for experts in other projects.
Develop a checklist for each product that is likely to be reviewed.
A checklist helps the moderator to structure the review meeting and helps each reviewer to focus on important issues. A set of representative review checklists is presented in the following documents.

Review Initialization

After the product to be reviewed is available, the producer has to inform the project manager.
The manager must decide (maybe in cooperation with the moderator) whether the result is worth to be reviewed.
After this, the moderator should initialize the review process. He must provide the reviewers with all necessary information:
- the product to be reviewed
- reference documents, e.g. documents the result is based on and guidelines that had to be respected
- checklists to be answered when reviewing the product
The product should be supplied together with the invitation, the additional documents should be referenced uniquely.
Plan for a presentation of the product and its application domain and/or an introduction into the importance and purpose of technical reviews
- in case the reviewers are not very familiar with the peripherie of the product or
- they have only little experience in technical reviews.
The presentation should take between half an hour and one hour.

Review Preparation

The moderator must make sure that all reviewers are in possession of all required information.
The moderator must ask the reviewers if the time given for preparation is sufficient.
If there is no choice for thorough preparation, the moderator has to cancel the review process.
All reviewers have to examine the product according to the allocated aspects.
Each finding may either be
- marked in the document (in case of slightly errors) or
- formulated to be presented in the review session (in case of weightily errors).
The author may not modify the product during the review preparation.

VERIFICATION AND VALIDATION

2008-12-03T09:07:00.000-08:00

VERIFICATION AND VALIDATION
The following IEEE definitions apply to this SQA plan:
Verification: The process of determining whether or not the products of a given stage of the software development life cycle fulfill the requirements established during the previous stage.
Validation: The process of evaluating software at the end of the software development process to ensure compliance with software requirements.

VERIFICATION
Best practices in SQA identify the following activities as part of requirements verification:
• Evaluate requirements and relationships for correctness, consistency, completeness, accuracy, readability and testability.
• Assess how well the requirements document satisfies the high level requirements described in the project plan.
• Assess the criticality of requirements to identify key performance or critical areas of software.
• Produce a traceability matrix tracing all requirements back to high-level requirements in the project plan and forward to software design elements and downstream test case elements.
The first two activities are handled by the review cycles and DCS documents described in the following chapters. The last two activities are handled by the development of a Requirements Traceability Matrix, as described in the following section.

REQUIREMENTS TRACEABILITY
The linkages between test cases, their parent design elements, and their parent requirements are maintained through a technique termed Requirements Traceability. In essence, it is necessary to be able to trace the linkage from a test case all the way through to a grandparent goal.

Requirements traceability is managed through the establishment and maintenance of a Requirements Traceability Matrix (RTM). The RTM is first established during the Requirements stage and is one of the formal deliverables for that stage. The RTM is then updated during each of the subsequent stages and reviewed to ensure that all elements have appropriate linkages (there are no "orphans").

A RTM can best be envisioned as an outline. As with any outline, parent elements are on the left and child elements move successively to the right.

(Click on the below image to see a clear view)

Verification of requirements traceability is required for the requirements, design, and test stages of the SDLC.

VALIDATION
In validation, the software is tested in a formal, structured manner intended to provide complete coverage of all requirements. This is accomplished through the execution of a series of test cases, each of which is traceable to parent requirements as described above.

The test cases are created during the development stage and incorporated into the project test plan, which is one of the deliverables of the development stage. The test cases include criteria for compliance with all requirements, performance at boundaries, and under stress conditions. Test cases are run against the software during the integration & test stage, where they may be modified before being incorporated into the final acceptance plan. The Software Testing chapter of this document describes this process in greater detail.

Scope and Methodology of SQA Plan

2008-12-03T08:39:00.000-08:00

SCOPE

This SQA process is tailored to fit the current software development effort and is related to the project planning and lifecycle description documents for this project.

METHODOLOGY

The methodology presented here is based on the Software Engineering Institute's (SEI) Capability Maturity Model (CMM) and the Institute for Electrical and Electronics Engineers (IEEE) standards for Information Management. This SQA process:

• Makes use of the principal project participants as defined in the SDLC and SPMP.
• Describes the processes for deliverable reviews and software testing.
• Defines deliverable class standards to be applied during reviews of stage deliverables.
• Identifies the work products produced as a result of the review and testing efforts.

The SDLC defines a series of stages; each stage is defined as a separate operation with specific inputs and outputs. This SQAP implements assessments and reviews at specific points within each of these stages.

FORMAL REVIEWS
For each project deliverable, as many as three types of formal reviews are conducted after the end users and development team have informally agreed that the deliverable content is accurate. The three review types are:

1. End-user review, conducted by at least one Subject Matter Expert (SME) who is familiar with the software product under development.

2. Technical review, conducted by at least one experienced software developer who is familiar with the product under development.

3. Quality Assurance review, conducted by an independent Quality Assurance Reviewer (QAR).
Each review is conducted in alignment with the reviewer's area of expertise and in accordance with the review criteria described in the associated Deliverable Class Standard and review form.

Refer to Chapter 2 for a discussion of Deliverable Class Standards. By tailoring the review focus to the expertise of the reviewer, this SQA plan prevents redundancy and inappropriate reviews.

PERSONNEL ROLES & RESPONSIBILITIES

In a database development effort, three principal roles are defined:
1. Primary End-user Representative (PER)
2. Primary Developer Representative (PDR)
3. Quality Assurance Reviewer (QAR)

The PER acts as the primary point of contact and principal approver for the end-user community. The PER is responsible for ensuring that end-user reviews are conducted on time and by appropriate subject matter experts.

The PDR acts as the primary point of contact and principal approver for the developer community. The PDR is responsible for the conduct of technical reviews in a timely manner and by appropriate development team members.
The QAR acts as the independent quality assurance reviewer for the project. The QAR will work independently from the development team to ensure objective audits and reviews of the work products and processes of this software development project.

STANDARDS
The following standards were used as guides to develop this SQA process. The standards were reviewed and tailored to fit the specific requirements of small database projects using the referenced SDLC:
• ANSI/IEEE 730.1: Standard for Software Quality Assurance Plans
• ANSI/IEEE 1028: Standard for Software Reviews and Audits
• ANSI/IEEE 1012: Standard for Software Verification and Validation
• SEI/CMMI: PPQA key process area

An Introduction to Software Quality Assurance Testing Plan

2008-12-03T08:34:00.000-08:00

Software Quality Assurance Plan

CONTENTS

INTRODUCTION

SCOPE

METHODOLOGY

VERIFICATION AND VALIDATION

VERIFICATION

VALIDATION

STAGE DELIVERABLE REVIEWS

ACTORS

FORMAL ITERATION PROCESS

IN-STAGE ASSESSMENT

STAGE EXIT REVIEW

DELIVERABLE CLASS STANDARDS

REVIEW REQUIREMENTS

REVIEW FORM FORMAT

SOFTWARE PROJECT MANAGEMENT PLAN DCS

REQUIREMENTS DCS

DESIGN DCS

ONLINE HELP DCS

IMPLEMENTATION MAP DCS

TEST PLAN DCS

DEPLOYMENT PLAN DCS

ACCEPTANCE PLAN DCS

INSTALLATION & ACCEPTANCE DCS

SOFTWARE TESTING

ACTORS

PROCESSES

TEST METHODOLOGY

Software Quality Assurance Plan Checklist

2008-12-03T08:25:00.000-08:00

The Software Quality Assurance Plan is a high level plan which can say that each of the subject points may be handled differently by each project, depending upon client requirements.

Topics the plan should include are listed below:

Corporation, Plan Title and version, date effective
Purpose of Plan
Glossary of Terms
How documentation is handled (some examples are):
- Statement of Work
- Planning Documents such as Software Requirements and Functional Specification
- User's Manual / Developer's Manual, etc.
Verification and Validation of software
Project management
Software standards used (generally people use Microsoft standards, but some do not)
User Interface Standards used
Development Tools and Techniques
Configuration Management and how source code is saved
Reviews and Audits
Testing
Bug Reporting and Tracking
Training
Risk Management
Process Improvement
Signoffs needed for plan and plan revisions

Estimation Methods for Defect Estimation

2008-11-29T08:01:00.000-08:00

Download the complete document which includes all the Estimation Approaches for the Defect Estimation-
A Study of Estimation Methods for Defect Estimation (Direct)
A Study of Estimation Methods for Defect Estimation (From Scribd)

Accurate defect prediction is useful for planning and management of software testing. The major goal of a software testing process is to find and fix, during debugging, as many defects as possible and release a product with a reasonable reliability. A trade-off between releasing a product earlier or investing more time on testing is always an issue for the organization. The clear view of the status of the testing process is crucial to compute the pros and cons of possible alternatives. Time to achieve the established goal and percentage of the goal achieved up to the moment are important factors to determine the status of a software testing process. Many defect prediction techniques have addressed this important problem by estimating the total number of defects, which then provide the status of a testing process in terms of remaining number of defects in a software product. The availability of an accurate estimation of the number of defects at early stages of the testing process allows for proper planning of resource usage, estimation of completion time, and current status of the process. Also, an estimate of the number of defects in the product by the time of release, allows the inference of required customer support.

Our goal in this paper is to discuss various estimation methods which are used to develop these defect estimation techniques. We would discuss the assumptions that each method makes about the data model, probability distribution, and mean and variance of data and estimator. We will also discuss the statistical efficiency of the estimators developed from these estimation methods. There are two broad groups of estimation methods called Classical Approach and Bayesian Approach as shown in Figure 1. If there is prior statistical knowledge about the parameter which we are interested to estimate, then an estimator based on Bayesian Approach can be found. Note that the prior knowledge can be in the form of prior mean and variance and probability distribution of the parameter. There are several Bayesian estimators available.

Main advantage of Bayesian estimators is that they provide better early estimates compared to estimators developed from Classical Approach. In the initial phase of software testing not enough
test data is available, consequently in the absence of prior knowledge initial performance of estimator suffers.
Click on the below figure (Estimation Approaches) in order to see the large and clear view.

Download the complete document which includes all the Estimation Approaches for the Defect Estimation-
A Study of Estimation Methods for Defect Estimation (Direct)
A Study of Estimation Methods for Defect Estimation (From Scribd)

The Testing Maturity Model (TMM)

2008-11-28T07:44:00.000-08:00

TMM (Testing Maturity Model) was developed, in 1996, at the Illinois Institute of Technology [2,3]. It reflects the evolutionary pattern of testing process maturity growth documented over the last several
decades. The basis for it was the historical model provided by Gelperin and Hetzel [5]. Their model describes phases and test goals for the periods of the 1950’s through the late 1980’s. Basically four periods can be distinguished: The “debugging oriented” period, where testing was merely seen as an activity to help remove bugs, the “destruction oriented” period focused on testing as an activity to detect implementation faults, the “evaluation oriented” period in which testing became an activity that was integrated into the software life cycle with the purpose to detect requirements, design and implementation faults. Finally, the “preventionoriented” stage where the scope of testing is broadly defined and includes review activities, with the primary goal to prevent requirement, design and implementation faults. The basic idea behind TMM is that every organisation goes through these historical phases, and that by providing the characteristics of these phases the test maturity can be determined. Thus in essence, TMM is an assessment. model rather than an improvement model. But an assessment model can be used as a basis for an improvement programme as well.

TMM has two major components: the Maturity Model, in which five maturity levels are distinguished (like in CMM), and an Assessment Model. Each maturity level, with the exception of the initial level 1, has a structure consisting of: A set of maturity goals, identifying testing improvement goals that must be addressed to achieve maturity at that level (consider these as the Key Process Areas) Supporting subgoals, defining the scope, boundaries and needed accomplishments for a particular level necessary to achieve the goals associated with each level.
The model with its maturity levels and goals is depicted in Figure 1.

Click on the above figure, to view in large.

The Process of Test Process Improvement

2008-11-27T07:41:00.000-08:00

Software testing is still a pain-in-the-neck for many organizations. Because it is only marginally addressed in software process improvement models like CMM, a separate Testing Process Improvement model is needed. The current authors have implemented a structured testing process guided by the “Testing Maturity Model” (TMM). An outline of this model is presented, showing how with growing maturity, testing evolves from detecting defects in software code to testing as essential product quality control instrument. The biggest strengths of TMM are:

It reflects 40 years of industry-wide best test practices and it is designed as a counterpart of the popular CMM model for software development improvement. Weaknesses include the under-representation of test people and organization related issues, and missing maturity goals for the test infrastructure. Based on practical implementation guided by TMM, the process of test process improvement is addressed and experiences are presented.

Read Also:
The Testing Maturity Model (TMM)

Step by Step Guide to Learn QTP

2008-09-04T09:53:00.000-07:00

Click here -> Step by Step Guide to Learn QTP

QTP Tutorial 1: Learning Recording

QTP Tutorials 2 - Using Data Table

QTP Tutorials 3 - Accessing Data Table values thro...

QTP Tutorials 4 - Standard Checkpoint

QTP Tutorials 5 - Page Checkpoint

QTP Tutorials 6 - Bitmap Checkpoint

QTP Tutorials 7 - Image Checkpoint

QTP Tutorials 8 - Text Checkpoint

QTP Tutorials 9 - Table Checkpoint

QTP Tutorials 10 - Checkpoint Return Value

QTP Tutorials 11 - Actions

QTP Tutorials 12 - Script to create file

QTP Tutorials 13 - Regular Expression