Software Process Assessment

The existence of software process does not guarantee the timely delivery of the software and its ability to meet the user’s expectations. The process needs to be assessed in order to ensure that it meets a set of basic process criteria, which is essential for implementing the principles of software engineering in an efficient manner. The process is assessed to evaluate methods, tools, and practices, which are used to develop and test the software. The aim of process assessment is to identify the areas for improvement and suggest a plan for making that improvement. The main focus areas of process assessment are listed below.

1. Obtaining guidance for improving software development and test processes
2. Obtaining an independent and unbiased review of the process
3. Obtaining a baseline (defined as a set of software components and documents that have been formerly reviewed and accepted; that serves as the basis for further development) for improving quality and productivity of processes.

Software process assessment examines whether the software processes are effective and efficient in accomplishing the goals. This is determined by the capability of selected software processes. The capability of a process determines whether a process with some variations is capable of meeting user’s requirements. In addition, it measures the extent to which the software process meets the user’s requirements. Process assessment is useful to the organization as it helps in improving the existing processes. In addition, it determines the strengths, weaknesses and the risks involved in the processes.

The process assessment leads to process capability determination and process improvement. Process capability determination is an organized assessment, which analyzes the software processes in an organization. In addition, process capability determination identifies the capabilities of a process and the risks involved in it. The process improvement identifies the changes to be made in the software processes. The software capability determination motivates the organization to perform software process improvement.

Different approaches are used for assessing software process. These approaches are SPICE (ISO/IEC15504), ISO 9001:2000, standard CMMI assessment method for process improvement, CMM-based appraisal for internal process improvement, and Bootstrap,

SPICE (Software Process Improvement and Capability Determination) is a standard used for both process improvement and process capability determination. SPICE provides a framework for assessing the software process and is used by the organizations involved in planning, monitoring, developing, managing, and improving acquisitions. It is carried out in accordance with the International Organization for Standardization (ISO) and International Electro-technical Committee (IEC), which are used together and known as ISO/IEC 15504. The functions of SPICE (ISO/IEC 15504) are listed below.

1. To develop process-rating profiles instead of pass or fail criteria
2. To consider the environment in which the assessed process operates
3. To facilitate self assessment
4. To ensure suitability for all applications and all sizes of organizations.

SPICE (ISO/IEC 15504) constitutes a set of documents that is used to guide goals and fundamental activities and grade the organization according to its level of capability. In addition, it determines the capability of a process in an organization, based on the results of the assessment of the processes and products. It also develops a maturity model for process improvement. This model is known as SPICE reference model. It is applicable for all processes and comprises following six levels.

1. Not performed: At this level, the processes are unable to accomplish the required outcomes. Thus, no identifiable products are created.
2. Performed informally: At this level, the implemented process accomplishes the defined outcomes. It is not planned and tracked; rather it depends on individual knowledge and identifiable products.
3. Planned and tracked: At this level, the defined process delivers products according to quality requirements within a specified time. The processes and products are verified according to the procedures, standards, and requirements.
4. Well-defined: At this level, the processes based on software engineering principles which are capable of achieving defined outcomes are used.
5. Quantitatively controlled: At this level, the performance measures, prediction capability and objective management are evaluated quantitatively. In addition, existing processes perform consistently within the defined limits to accomplish the desired outputs.
6. Continuously improved: At this level, the existing processes adapt to meet future business goals. For continuous improvement, various kinds of statistical methods are used.

ISO (International Organization for Standardization) established a standard known as ISO 9001:2000 to determine the requirements of quality management systems. A quality management system refers to the activities within an organization, which satisfies the quality related expectations of customers. Organizations ensure that they have a quality management system by demonstrating their conformance to the ISO 9001:2000 standard. The major advantage of this standard is that it achieves a better understanding and consistency of all quality practices throughout the organization. In addition, it strengthens the customer’s confidence in the product. This standard follows a plan-do-check-act (PDCA) cycle, which includes a set of activities that are listed below.

1. Plan: Determines the processes and resources which are required to develop a quality product according to the user’s satisfaction.
2. Do: Performs activities according to the plan to create the desired product.
3. Check: Measures whether the activities for establishing quality management according to the requirements are accomplished. In addition, it monitors the processes and takes corrective actions to improve them.
4. Act: Initiates activities which constantly improve processes in the organization.

An appraisal examines processes to determine their strengths and weaknesses in an organization. The appraisal used in an organization evaluates the internal processes (series of procedures occurring in the organization) for establishing or updating process improvement. In addition, appraisal includes measuring the process improvement progress by conducting audits of the processes. To conduct an appraisal, a scheme known as SCAMPI was developed by the Software Engineering Institute (SEI).

SCAMPI is used for process improvement by gaining insight into the process capability in the organization. The major advantage of SCAMPI is that it supports process improvement and establishes a consensus within the organization regarding areas where the process improvements are needed. The major characteristics of SCAMPI are listed in Table.

                                                Table SCAMPI Characteristics

SCAMPI enables an organization to select and follow an approach for developing a plan for appraisal, which is appropriate for the requirements. These appraisal requirements in CMMI (ARC) comprise criteria for developing, defining, and utilizing the appraisal methods (namely, Class A, Class B, and Class q, which are based on Capability Maturity Model Integration (CMMI) as an improved framework. The objectives of SCAMPI are listed below.

1. To identify strengths and weaknesses of existing processes in the organization
2. To specify an integrated appraisal method for internal process improvement
3. To act as a motivation for initiating and focusing on software process improvement.

SCAMPI is an appropriate tool for benchmarking within the organization. This process emphasizes a rigorous method that is capable of achieving high accuracy and reliability of appraisal results. The major characteristics of classes ‘A’, ‘B’, and ‘C’ are listed in Table. Class ‘A’ assessment method should satisfy all appraisal requirements of CMMI. The major characteristic of Class ‘A’ assessment is a detailed assessment of process (es).With a thorough coverage, the strengths and weaknesses of processes are determined. SCAMPI is a Class ‘A’ assessment method for process improvement.

Class ‘B’ appraisal method should comply with a subset of ARC requirements. As shown in Table, requirements of the Class ‘A’ method are optional for, Class ‘B’. Class ‘B’ assessment helps the organization to gain insight into its process capability. It focuses on areas that need improvement. It does not emphasize detailed coverage and is not efficient for level rating. These types of appraisals are considered efficient for initial assessment in organizations that have just started to use CMMI for process improvement activities. In addition, it is useful in providing a cost effective measure for performing interim assessment and capability evaluations.

                                        Table Characteristics of Appraisal Methods

Class ‘C’ appraisal methods are inexpensive and used for a short duration. In addition, they provide quick feedback to the result of the assessment. In other words, these appraisal methods are useful for periodic assessment of the projects.

Class ‘B’ and Class ‘C’ appraisals are useful for organizations that do not require generation of ratings. The primary reason for all appraisal methods should be to identify the strengths and weaknesses of the processes for their improvements.

CBA-IPI tool is used in an organization to gain insight into the software development capability. For this, the strengths and weaknesses of the existing process are identified in order to prioritize software improvement plans and focus on software improvements, which are beneficial to the organization. The organization’s software process capability is assessed by a group of individuals known as the assessment team, which generates findings and provides ratings according to the CMM (Capability Maturity Model). These findings are collected from questionnaires, document reviews and interviews with the managers of the organization. Thus, the primary goal of CBA IPI is to provide an actual picture of the existing processes in an organization. To achieve this, the assessment team performs the following functions.

1. Provides data as a baseline to the organization in order to check its software capability
2. Identifies issues that have an impact on the process improvement
3. Provides sufficiently complete findings to the organization. These are used to guide the organization in planning .and prioritizing future process improvement activities.

For an assessment to be considered in CBAIPI, it should satisfy the minimum requirements concerning the assessment team, assessment plan, data collection, data validation, and reporting of the assessment results. These requirements are listed in Table.

                                               Table Requirements in CBA IPI

The CBA IPI method is similar to SCAMPI as both are used for process assessment in an organization. However, differences do exist between the two approaches. These differences are listed in Table.

                              Table Differences between CBA IPI and SCAMPI

Bootstrap is an improvement on SEI approaches for process assessment and improvement and covers the requirements laid by ISO 9000. This approach evaluates and improves the quality of software development and management process of an organization. It defines a framework for assessing and promoting process improvement. The basic objectives of this approach are listed below.

1. To support evaluation of the process capability
2. To identify the strengths and weaknesses of the processes in the organization being assessed
3. To support the accomplishment of goals in an organization by planning improved actions
4. To increase the effectiveness of the processes while implementing Standard requirements in the organization.

The main feature of the bootstrap approach is the assessment process, which leads to an improvement in the software development processes. During the assessment, the organizational processes are evaluated to define each process. Note that after the assessment is done, data is collected in a central database. In addition, it provides two questionnaires (forms containing a set of questions, which are distributed to people to gain statistical information): the first to gather data about the organization that develops the software and the second to gather data about the projects.