Background
Main Methodologies
Agile and SPI Models
Agile and Measurement
Agile and Estimation
Publications

-

It is possible to express the relationship between Agile (or Lightweight) Methodologies and Traditional (or Heavyweight) ones using a musical metaphor (Traditional approaches = Classical Orchestra; Agile Methodologies = Jazz group), as in the following figures.

It is important to stress that AM are not strictly lifecycles but represent istances of the Iterative-Incremental lifecycle family. This means that an AM cannot be compared directly with a Waterfall or a Prototype lifecycle, but has more sense to understand which variations and characteristics each AM expresses and if it has a good fit with the project's exigencies or not.
This webpage would aim to propose a list of thoughts, references, links and information about the "Measurement & Analysis" issue within Agile methodologies, that is something not again discussed in depth, more than discussing those methodologies per se.

Main Methodologies :
Under the "agile" label there are plenty of methodologies. A reference portal can be the Agile Alliance one, providing a series of useful public resources, starting from this roadmap on. Browsing this page, there is the list of the main methods and tools & techniques, from XP and SCRUM (for sure the more known and applied worldwide) to all the other ones.
A good and comprehensive technical report about the description and comparison among those methods has been written at VTT by P.Abrahamsson, O.Salo & J.Ronkainen, stressing for each method its characteristics and eventual commonalities.

Agile and SPI Models :
Which is the relationship between Agile Methods and SPI models? This question started in the late '90s with a paper by Mark Paulk, the Sw-CMM architect, appeared on IEEE Software discussing XP (Extreme Programming) from the CMM perspective. In particular, that paper presented a table showing the compliance level of XP practices with CMM KPAs (see slide 11); this kind of analysis was replied later referred to SCRUM by Ken Schwaber (see slide 12).

Here in the following there is a series of papers, presentation and technical reports found on the net about suggestions for a better joint usage of Agile Methods within SPI models, mostly oriented to CMMI:

Stretching Agile to Fit CMMI Level 3 -- the story of creating MSF for CMMI Process Improvement at Microsoft
An USC presentation dated 2002, based on Boehm's book on "balancing agility"
A presentation by Richard Turner held at 2nd NDIA CMMI Tech Conf (2002)
A 2002 paper on Crosstalk about the pros and cons of using AM with Sw-CMM
The first article by Mark Paulk (2001) on IEEE Software on this issue
The Agile CMMI blog
Using CMMI to balance Agile and Plan-driven methods, a 2003 presentation by Richard Turner
A case study about how to impelment CMMI using Agile approaches in your software developments
The Philips Netherlands experience
An article by Mike Konrad
Implementing CMMI with a combination of agile methods
About skill and cultural issues
3 case studies presented at the SPICE2006 conference
An Approach to Reconcile the Agile and CMMI Contexts in Product Line Development, a paper by Navarrete, Botella & Franch
The Future of CMM and Quality Improvement -- how to match and put together different models and approaches
A presentation by Barry Boehm dated 2002
MS Visual Studio 2005 for Agile & CMMI
SAMI (Sidky Agile Measurement Index) at Tangile Software, with 5 maturity levels, in order to measure the agile potential of an organization, within an Agile Adoption Framework. Here some additional info about the assessment from the Agile2007 conference and here from the Agile Journal (May 2007).

Agile and Measurement :
Measurement in the Agile context focus a lot on the Time perspective. Many measures and indicators are the same than applying other SLC, but some others are particular and have a better fit with AM.

Here a list of papers-presentations recently found on the Net:
http://www.berteigconsulting.com/AppropriateAgileMeasurement.pdf
The RTF metric (Running Tested [Failed]) in a discussion by Ron Jeffries, also referenced in this Nokia experience
An experience at the Israeli Force
Papers from the 1^st Workshop on the Empirical Evaluation of Agile Processes (EEAP2002)
Papers from the 2^nd Workshop on the Empirical Evaluation of Agile Processes (EEAP2003)
The "Agile Journal" June 2006 issue on Agile Metrics
A thread on the introduction to agile metrics
A paper by William Krebs from the XP/Agile Universe 2002 conference
A Metrics Checklist for Agile Measurement by D. Hartmann & R. Dymond, with some thoughts about how to evaluate when they bring (or not) a business value
A post (July 2007) on the perceived missing practice of measuring what is "agile"

Agile and Estimation :
Within an Agile project, typically only its functional side is evaluated, often missing the non-functional part (about this discussion, it could be helpful to take a look to this IFPUG report). When using Agile Methods, four main concepts are widely applied:
Story Points (SP) (a variant are RSP Story Points)
Ideal Time
Velocity
Load Factor

The main problem is that those measures are higly subjective and could be ok if applied in small and stable groups, but cannot fit with a larger organization, as many ICT companies has.

Two main issues to manage applying these measures "as-is" are the lack of historical data and of standard values to share.
About historical data, XP do not consider to formally historicize data (while Scrum is a bit more PM-oriented), that's for instance one of the premises to consolidate a CMMI ML3 evaluation. All these concepts beside on the experience of a stable project team. Story Points are an arbitrary value strongly depending on a group of people and wheter an organization has a matrix organization it is very difficult to collect useful data to share among groups during time, as when applying a Wideband Delphi estimation. Therefore, it could be suggestible to choose more objective sizing measures.
About the standard values, taking a look to several sources, Story Points and Ideal Time have a lot of definitions and value assignments, while - at the aim to build and establish a strong foundation for quantitative-based estimations with historical projects' data - more quantitative-sound measures should be applied for sizing the agile project and tracking it during its lifetime.

A larger and detailed analysis on this issue is ofference in this paper presented at SMEF2007 by A.Abran & L.Buglione (click here for the presentation)

Thus, two fundamental things to track during the project lifetime are effort (estimated vs actual) as well as the project size, where a possible solution in order to refine estimations and have more quantitative evaluation of tasks to perform can be an adaptation to Agile projects of PSU (Project Size Unit), applied to each single User Story, plus a further component for sizing the non-functional requirements and activities. This solution can help in avoiding underestimations, as reported in this experience on the Agile in Action website.
A PSU template for Scrum projects is available at the PSU webpage.

Publications :
Buglione L., Project Size Unit (PSU) - Measurement Manual, v1.01, October 2005
Buglione L., Dimensionare i progetti: che "metro" usare? XPM.it, Giugno 2006
Buglione L., Meglio Agili o Veloci? Alcune riflessioni sulle stime nei progetti XP, XPM.it, Febbraio 2007
Buglione L. & Abran A., Improving Estimations in Agile Projects: issues and avenues, Proceedings of the 4th Software Measurement European Forum (SMEF 2007), Rome (Italy), May 9-11 2007, ISBN 9-788870-909425, pp.265-274
Buglione L., Project Size Unit (PSU) - Measurement Manual, v1.21, November 2007, Available versions: English, Spanish/Castellano, Italiano
Racheva Z., Daneva M., Buglione L., Complementing Measurements and Real Options Concepts to Support Inter-iteration Decision-Making in Agile Projects, 34^th Euromicro/SEAA 2008, Workshop on Software Management, Parma (Italy), 3-5 September 2008

[Bio Sketch] [Misurare il Software] [Publications] [QEST & LIME models] [Presentations] [Links]