What SPI is?
Main Frameworks and Appraisal Methods
Mappings
ROI for SPI Initiatives
True and False "Maturity Models": the MM-mania
New SPI topics faced
Quick Guides
Publications & External References

-

Starting from mid '80s, a greater attention has been deserved in Software Engineering to the study of software processes, analysing their structure and the best way to be applied for achieving process improvement. "It is a simple task to make things complex, but a complex task to make things simple”, according to Meyer's law. Software process improvement is definable as “the continual and iterative improvement of both the software process and products throught the use of project experiences” (NASA) or as “a deliberate, planned methodology following standardized documentation practices to capture on paper (and in practice) the activities, methods, practices, and transformations that people use to develop and maintain software and the associated products. As each activity, method, practice and transformation is documented, each is analyzed against the standard value added to the organization” (Szymanski & Neff) or again “a plan derived from the recommendations of a SPA that identifies the specific process capability and performance” (M.Paulk, SEI) and “action taken to change an organization's business need and achieve its business goals more effectively” (ISO, SPICE project, Part 9) where each of these definition “gathers” one of the possible dimension, from iterativity, to the need for documentation, to Software Process Assessment as its starting point, till the achievement of company's goals.
Four basic pillars (Kuntz):

• evolution is possible, and requires time and resources;
  
• a greater process maturity reduces risks and increases performance;
  
• evolution implies there is a pre-established sequence in order to take under control a process with reference to external factors;
  
• maturity will decrease is not constantly maintained.
  

The relationship between Total Quality Management (TQM) and Software Process Improvement (SPI) is summarised in the following image:

Main SPI Frameworks and Appraisal Methods

Maturity is expressed through a nominal scale, usually with a Likert scale (from 1-min to 5-max), retrieving the ideas and concepts from Phil Crosby's studies. Such instantiations are called staged models, where a certain maturity level can be achieved only if all the practices contained in the "n-1" level have been yet successfully performed.

Sw-CMM
The first and most famous SPI staged model is undoubtely the Sw-CMM (Software Capability Maturity Model) by the Software Engineering Institute (SEI), composed in v1.1 by 18 Key Process Areas (KPA) across the 5 Maturity Levels.
For any detailed information, please click here (official webpage).
On the success of this first model, a family of CMMs have been developed at SEI: People-CMM (P-CMM), Software Acquisition CMM (SA-CMM), Systems Engineering (SE-CMM), Integrated Product Development CMM (IPD-CMM).
Twice a year, SEI reports results from Sw-CMM assessments, from 1987 on, freely downloadable from this page

But what about the possibility to run a process included in a ML upper than ours? Should it be evaluated in an appraisal or not? And which value could it have?
For this reason, an evolution of staged architectures was deployed in the mid '90s, called continuous models, where each process defined in the model can be evaluated on its own, no matter to a predefined order of processes, just an evaluation of most relevant processes for that company/SBU under assessment. Most relevant models with this new architecture type were (in order of appearance): SE-CMM, SPICE (ISO/IEC 15504) and CMMI.

SPICE (ISO/IEC TR-2 15504:1998)
The ISO model for SPI was the JTC1/SC7/WG10 standard named SPICE (Software Process Improvement and Capability dEtermination), improving the well-proven structure of Sw-CMM and refining the list of processes of ISO/IEC IS 12207:1995, on a continuous, bi-dimensional architecture (the two dimensions are: capability and process ones). This new standard (the newest version should be released in mid 2005, with a new process category: ACQ, for acquisition processes) subdivides 40 processes into three groups:

• primary processes,
• organization processes
• support processes,

More relevant websites for SPICE are:

• ISOSPICE
• SUGAR: the SPICE Users Group
• Griffith University
• Updated info on new upcoming SPICE version

CMMI (Capability Maturity Models Integration)
SEI started in 1997 to work on the evolution of Sw-CMM, trying to integrate the several maturity model that an ICT company could have been separately implemented at the same time, on the experience also of the FAA i-CMM and the bi-dimensional architecture introduced with SPICE. CMMI would be a transition continuous model between Sw-CMM and the newest SPICE release, according to the notes in several CMMI documents, even if it seems difficult to really think it will be dismissed in few years. Some differences:

• the first one in the process groups, four in CMMI (process, project, engineering, support) and five in SPICE (Customer, Engineering, Supporting, Organizational, Management)
• two architecture versions in CMMI (staged, continuous), while just the continuous one in SPICE
• the four versions for CMMI (sw, se/sw, se/sw/ipd, se/sw/ipd/ss), just one (sw) for SPICE
• different assessment rules for CMMI (ARC, types A-B-C), while just one only in SPICE
• CMMI is a de facto standard by SEI, while SPICE (ISO/IEC TR-2 15504) is an international de jure standard by ISO/IEC

• SEI official webpage
• STSC mapping between Sw-CMM v1.1 and CMMI v1.1 and also the SEI mapping between CMMI v1.1 and v1.2

The following table tries to summarise main SPI models and related appraisal methods with hyperlinks, where available, to access directly models/methods original documents or relevant information on them.

As illustrated, current version of CMMI model(s) is v1.2, released on August 25, 2006.
Main changes in version 1.2 are reported in a list of files provided here (in details: model changes, SCAMPI A Appraisal Method changes and Training changes). Apart from little, fairly minor changes, as described by M.Phillips in a column in a recent issue of "News@SEI":
* to unify the staged and the continuous versions into a unique, single handbook in order to reduce complexity
* to eliminate common features and advanced practices
* to (eventually) integrate SAM and ISM process areas
* to stress more the relevance of achieving customer satisfaction in the informative material
* to (eventually) broaden the coverage of the development environment (currently covered only by the OEI process area
* to consider more the "service management" perspective
* to update also SCAMPI

But the new version 1.3 is coming out: here some anticipations about the points in discussions for an eventual inclusion by 2010 in the different constellations.

Mappings
When is it possible to map models?
A common issue is the request to map objects with a different inner nature. So, it is no so unfrequently to read: how can I map SixSigma with CMM or PMBOK and CMMI (BTW, this presentation by B.Groarke about a joint application of the two at the SSC San Diego)? Or again: in which way can I use MSF (Microsoft Solutions Framework) deliverables with CMMI?
The first question before trying to answer should be: are the "things" intended to compare "comparable"? Are they entities of the same "nature"? The answer is that is not possible to compare a technique with a process model; at least it is possible to qualitatively analyze the way a technique can be used within a certain process or set of processes. In such sense, for instance, it is interesting a 2002 SEI technical note from P.Solomon about the way CMMI can be used to improve Earned Value Management.

Mappings available
Next table presents some mappings among SPI models available on the Web. When a direct mapping does not exists, it would be possible to cross other related existing mappings in order to derive - with care - that relationship.

	Sw-CMM	CMMI	SPICE	ISO 9001:2000	TSP	EIA/IS 731	SE-CMM v1.1	RUP
Sw-CMM		X	X	X	X		X
CMMI	X		X	X	X	X		X
SPICE	X	X		X
ISO 9001:2000	X	X	X
TSP	X	X
EIA/IS 731		X					X
SE-CMM v1.1	X					X
RUP		X

Not referenced in the table, since it's more an integration than a mapping, People CMM v2 supports integrated product development teams in the IPPD extensions of CMMI v1.1.

Other sources about the comparison between CMMI and RUP:
A tutorial by B.Gallagher & L.Brownsword
A Rational whitepaper for achieving CMMI ML2 using IBM Rational's solutions, including RUP, updating a previous paper about how to achieve Sw-CMM ML2/3 with RUP
Enhancing RUP for CMMI compliance: a methodological approach
A CMMI Maturity Level 2 assessment of RUP, by M.Grundmann (Dec2005)

ROI for SPI Initiatives

One of the most relevant questions about implementing an SPI initiative is: how much will it return and how many months for the break-even point? Several papers and presentations tried during last years to describe and report experiences for providing ROI values. Here there is a short list of references about this interesting issue:

the 1997 report by Khaled El Emam & Lionel Briand on "Cost and Benefits of Software Process Improvement" (ISERN-97-12)
a more recent report always by Khaled, dated 2003, on ROI models for Static Analysis Tools (you need to register to the SEIR for free to access it)
the DACS's Software TechNews vol.5 no.4
An STSC2002 Conference presentation by J.Statz & B.Solon
IEEE Software - Vol 21 No.3 (May/June 2004) -- ROI in the Software Industry
A new book by David F.Rico, with a series of related metrics
A recent SEI Special Report on the Demonstration of the impact and benefits of CMMI, that's an evolution ten years ago from the SEI/CMU-94-TR-013 document (see also this 1999 presentation by the Process Group based on those SEI data)
"ROI from SPI" by Sami Zahran
A 1999 TechReport by T.McGibbon (DACS) - A Busines Case for SPI (revised): measuring ROI from SwEngineering & Management
A 1997 paper by Herb Krasner on the payoff for these initiatives
A list of articles and technical reports (most of them with URLs) from the Software Productivity Consortium
An experience in a Sw-CMM ML5 company by K.Dymond, V.Govilkar & A.Kumar

True and False "Maturity Models": the MM-mania

Looking at the different existing SPI models, it seems to be a misuse of the "Maturity Model" labeling: read for instance this paper on the "MM"-mania (in the following, we list th 34 models with related links/websites, for your eventual interest, where available):

1. Capability Maturity Model Integration (CMMI)
2. Capability Maturity Model for Software (SW-CMM)
3. People Capability Maturity Model (P-CMM)
4. Software Acquisition Capability Maturity Model (SA-CMM)
5. Software Engineering Capability Maturity Model (SE-CMM)
6. Integrated Product Development Capability Maturity Model (IPD-CMM)
7. IT Service Capability Maturity Model (IT Service CMM)
8. Organizational Project Management Maturity Model (OPM3)
9. Services Maturity Model
10. Self-Assessment Maturity Model (SAMM)
11. Testing Maturity Model (TMM)
12. Web Services Maturity Model
13. Security Maturity Model (SMM)
14. Operations Maturity Model
15. e-Learning Maturity Model
16. e-Government Maturity Model
17. Earned Value Management Maturity Model (EVM3)
18. Outsourcing Management Maturity Model
19. Change Proficiency Maturity Model
20. Performance Engineering Maturity Model (PEMM)
21. IT Architecture Maturity Model (ACMM)
22. Information Process Maturity Model (IPMM)
23. Project Management Maturity Model (PMMM)
24. Programme Management Maturity Model (PMMM)
25. Learning Management Maturity Model (LM3)
26. Automated Software Testing Maturity Model
27. Website Maturity Model
28. PM2 Maturity Model
29. Internet Maturity Model
30. Usability Maturity Model
31. Software Reliability Engineering Maturity Model
32. System Security Engineering Capability Maturity Model (SSE-CMM)
33. Configuration Management Maturity Model
34. Broccoli Maturity Model

But the list does not end here: there are also:
* Software Maintenance Maturity Model (S^3M)
* Risk Management Maturity Model (RMMM)
* e-Government Maturity Model
* Business Process Maturity Model (BPMM), by B.Curtis & C.Weber (and a presentation by D.Ho on the state-of-the-art on Process Maturity Models, dated 2004)
* Documentation Maturity Model (DMM)
* Metrics Data Base Maturity Model (MDB^MM)
* ISM3 (Information Security Management Maturity Model)
* Agile Maturity Model (AMM) (see Slide 26/31)
* KPI Rule Maturity Model (RMM)
* Requirement Maturity Model (RMM)
* PRTM Customer Requirement Maturity Model (RMM)
* IBM/Rational Requirement Maturity Model (RMM)
* Supply Chain Maturity Model (SCMM)
...
A MM should be intended in the way Phil Crosby produced his Quality Management Maturity Grid and be referred in the Sw-CMM way to the "process" entity).
An example of this trend is an interesting technique (not a model), called OSSM (Open Source Maturity Model), with the aim to help in comparing and choosing OS products. Reading the document, the "maturity" in OSMM is referred to the "OS product maturity to be evaluated" and also "model" is not properly used, since it is described a 7-steps procedure for evaluating OS products for selection according to several indicators (also here the "product indicators" seem to be expressed as a two-tier quality model, as ISO/IEC 9126, with 4 characteristics and 12 metrics).

New SPI topics faced

Starting from the analysis of technical literature, some interesting topics are open for discussion. In particular:

• TQM vs. SPI: what is the right balance?: a paper included into the IRIS20 proceedings by Peter BØttcher discussed this interesting issue. Again, Mark Paulk dedicated a huge part of a SEI "Q&A" document in order to propose the differences and complementarities between the two
• Creativity vs. SPI: how creativity can be properly evaluated in "rigid" implementation schemas? In this paper we proposed some ideas and a metric about how to identify if creativity is properly managed or not in an organization. The question has been discussed since a long time and an interesting paper was the one by James Bach on the 1994 September issue of American Programmer, stressing the problem with a staged approach for some particular KPAs such as the one devoted to creativity & innovation
• How to choose the right SPI framework? From the technical viewpoint, there are at least four criteria to take into account when evaluating which SPI model to adopt for your organization:
  Firstly, the choice of an SPI model must affort firstly in the domain faced by a certain model (referring to the above table, you could see the "Domain" column).
  Secondly, looking at the groups of processes included in that model and level of granularity in their description; do they fit with your exigencies or not? are they sufficiently wide to cover your typical activities? are they sufficiently detailed to reach the desired level of description with the right process notation
  Thirdly, looking at the representation type (staged vs. continuous, as discussed after); which is your objective in assessing your organization?
  Finally, the assessment method applied to the selected SPI model; is it sufficiently granular in order to "make a real photo" of your organization? could it suggest you how to improve for the future?
  Of course, these criteria must be complemented with other ones from the business viewpoint, such as: diffusion of that model, recognizability, (eventual) related certification schemes, etc...
  A more structured approach is the one defined by the European Software Institute (ESI), based on 4 categories (general information, critical characteristics, desired characteristics, and additional characteristics) to be rated against the Measurement criteria listed.
  Finally, it is interesting the evolutionary view proposed by David Rico with 7 criteria for moving from past to future SPI models.
• Continuous or staged version for a certain SPI model?: this is a comprehensive post with some considerations about the opportunities in choosing the proper version to adopt
• Which usage of Process Asset Libraries (PAL) and which software solution can be adopted
• Hybridization between SPI and Performance Management models

while some other practical questions are, for instance, about:

• XP (eXtremeProgramming), SLC (Software Lifecycles) and the Sw-CMM: what are the differences?
• the way Six Sigma and CMMI can be properly used together in the ICT world
• RUP and CMMI: mappings available right now
• "Measurement & Analysis" Process Area in CMMI: some thoughts moving from Sw-CMM to CMMI
• Which could be some parameters for evaluating a SCAMPI assessor?
• Peer Reviews: how much do they cost?

Quick Guides

When dealing with SPI models, you have to read a hundreds of pages. Therefore, in the everyday practice, it could be useful a 'summary' of main elements. Here in the following there are some 'quick guides' on a two-fold A4 sheet. In particular:

• CMMI-DEV v1.2
• CMMI-ACQ v1.2
• CMMI-SVC v1.2

Publications

L.Buglione Misurare il software. Quantità, Qualità, Standards e Miglioramento di processo nell'Information & Communication Technology, 3^rd Edition, Franco Angeli, 2008 (Chapter 4)
L.Buglione, Proposal for a new UI paradigm, Proceedings of the Workshop "Dialogo e modelli di utente nelle interfacce intelligenti", 6° AI*IA Conference, Padova, Italy, September 23-25, 1998
L.Buglione, N.Quintano & D.Reo, Balanced IT Scorecard and EFQM: A Balanced Approach to Performance Measurement for Software Intensive Organisations, Proceedings of the 2nd European Software Measurement Conference - FESMA 1999, October 4-8, 1999, Amsterdam, The Netherlands, ISBN 90-76019-07-X, pp. 415-426
D.Reo, N.Quintano & L.Buglione, Measuring Software Development - There is more than just measuring software!, Proceedings of the 2nd European Software Measurement Conference - FESMA 1999, October 4-8, 1999, Amsterdam, The Netherlands, ISBN 90-76019-07-X, pp. 391-402
L.Buglione & E.Ostolaza, Achieving Business Excellence in SPI: Applying the EFQM/SPICE Integrated Model in Industry, 3rd International Software Quality Week Europe 1999 (QWE'99), 1-5 November 1999, Brussels, Belgium
L.Buglione & E.Ostolaza, Improving business results in Software Intensive Organisations through the EFQM/SPICE Integrated Model, Software Process Improvement 1999 Conference (SPI99) , 30 November - 3 December 1999, Barcelona, Spain
L.Buglione & E.Ostolaza, EFQM/SPICE Integrated Model: a TQM approach to Software Process Improvement, SPICE2000 Conference , June 10–11, 2000, Limerick, Ireland, Proceedings (Editor T.P. Rout), pp. 207-218
L.Buglione & E.Ostolaza, Improving business results in Software Intensive Organisations through the EFQM/SPICE Integrated Model, 44th Congress of the European Organization for Quality , June 12– 16, 2000, Budapest, Hungary, Proceedings Vol.3, HU ISBN 963-00-3180-9, pp. 285-296
L.Buglione & A.Abran, Creativity and Innovation in SPI: an exploratory paper on their measurement, in "Current Trends in Software Measurement", Proceedings of the IWSM 2001 (11th International Workshop on Software Measurement), Montréal, Québec (Canada), August 28-29, 2001, Shaker Verlag, ISBN 3-8265-9681-1, pp. 112-126
J.M. Desharnais, A.Abran, A.Mayers, L.Buglione & V.Bevo, Knowledge Modeling for the Design of a KBS in the Functional Size Measurement Domain, in "Knowledge-Based Intelligent Information Engineering & Allied Technologies", Eds:E.Damiani, R.J.Howlett, L.C.Jain & N.Ichalkaranje, ISBN 1-5860-280-1, pp.26-31, IOS Press, Proceedings of KES2002 (6th International Conference on Knowledge-Based Intelligent Information & Engineering Systems), Podere d'Ombriano - Crema (Italy), September 16-18, 2002
L.Buglione & A.Abran, ICEBERG: a different look at Software Project Management, IWSM2002 in "Software Measurement and Estimation", Proceedings of the 12th International Workshop on Software Measurement (IWSM2002), October 7-9, 2002, Magdeburg (Germany), Shaker Verlag, ISBN 3-8322-0765-1, pp. 153-167
L.Buglione & A.Abran, Assessment of Measurement Indicators in SPI Frameworks, IWSM2003, in "Investigations in Software Measurement", Proceedings of the 13th International Workshop on Software Measurement (IWSM2003), September 23-25, 2003, Montréal (Canada), Shaker Verlag, ISBN 3-8322-1880-7, pp. 287-309
L.Buglione, Sistemi Informativi & modelli di Software Process Improvement: relazioni e impatti, Proceedings of itAIS2005, 2^nd Conference of the Italian Chapter of AIS, 1-2 December 2005, Verona (Italy)
L.Buglione & C.Dekkers, A Murphological View on Software Measurement: a serious joke or a funny serious thing?, Proceedings of SMEF 2006, 3^rd Software Measurement European Forum, 10-12 May 2006, Rome (Italy), pp. 315-329
Buglione L.& Cislaghi M., Valutare i processi nelle organizzazioni ICT: oltre la ISO 19011, Qualità, Rivista dell'AICQ, Augusta Edizioni, Luglio/Agosto 2006, pp.4-8
L.Buglione & A.Abran, ODC and CMMI: Introducing the Root-Cause Analysis at Lower Maturity Levels, Proceedings of MENSURA 2006 (1^st International Conference on Software Process and Product Measurement, Cadiz (Spain), November 6-8 2006
Bégnoche L., Abran A. & Buglione L., A Measurement Approach Integrating ISO 15939, CMMI and ISBSG, Proceedings of the 4th Software Measurement European Forum (SMEF 2007), Rome (Italy), May 9-11 2007, ISBN 9-788870-909425, pp.111-130
Buglione L., Maturity Models: modelli esclusivi o integrabili?, Qualità On-Line, Rivista dell'AICQ, Novembre 2007
Buglione L., Dell'uso contrattuale dei Maturity Models (e non solo), De Qualitate, Nuovo Studio Tecna, Febbraio 2008, Anno XVII, N° 2, pp.36-43
Buglione L., Quanto bisogna essere maturi per usare i modelli di maturità?, Persone & Conoscenze, Aprile 2008, No.38, Edizioni ESTE, pp.49-54
Buglione L., Strengthening CMMI Maturity Levels with a Quantitative Approach to Root-Cause Analysis, Proceedings of the 5th Software Measurement European Forum (SMEF 2008), Milan (Italy), 28-30 May 2008, ISBN 9-788870-909999, pp. 67-82
Buglione L., Rejas-Muslera R., Cuadrado-Gallego J.J., Strengthening Maturity Levels by a Legal Assurance process, EuroSPI² 2008 , European Systems & Software Process Improvement and Innovation, Dublin (Ireland), 3-5 September 2008

External References

Datta S., Effects of Changing Requirements: A Tracking Mechanism for the Analysis Workflow, Technical Report, FSU-SCS-2005-108, Florida State University, School of Computational Science, 2005
Chaur Bernal J., Diseño conceptual de productos asistido por ordenador: Un estudio analítico sobre aplicaciones y definición de la estructura básica de un nuevo programa , Ph.D. Thesis, Universitat Politècnica de Catalunya (Espana), June 2005
Santana Tapia R., Daneva M. & van Eck P., Developing an Inter-Enterprise Alignment Maturity Model: Research Challenges and Solutions, Technical Report, University of Twente (Netherlands), May 2007
Al Qutaish R., SPQ^MM: A Software Product Quality Maturity Model Using ISO/IEEE Standards, Metrology and Sigma Concepts, Ph.D. Thesis, Ecole de Technologie Superieure (ETS), Montréal (Canada), 21 June 2007
Casey C., Do Software Developers Need The Personal Software Process?, Department of Computing, University of Central Lancashire (UK), 2000
Calvo Medrano J.M. & Minguet Mélian J.M., Medida de las subcaracterísticas Capacidad de Análisis y Capacidad de Cambio mediante la norma ISO/IEC 9126, Febrero 2007
Salviano C., Melhoria e Avaliação de Processo de Software com o Modelo ISO/IEC 15504-5:2006, CURSO DE PÓS-GRADUAÇÃO, “LATO SENSU” (ESPECIALIZAÇÃO) A DISTÂNCIA MELHORIA DE PROCESSO DE SOFTWARE, Universidade Federal de Lavras - UFLA, Fundação de Apoio ao Ensino, Pesquisa e Extensão - FAEPE, Lavras - MG

[Bio Sketch] [Misurare il Software] [Publications] [QEST & LIME models] [Presentations] [Links]