Herding Cats

Principles of Program Governance

When there is discussion around development, management of projects, agile, earned value and all the other "bottom up" processes that occupy the blog space, it's sometimes best to step back and consider what is really needed in the business and government world.

GOVERNANCE

Governance is about making the right decisions at the business level that impact the technical aspects of the work needed to deliver a capability. With the capability business "value" can be achieved.

All this agile talk about "value" rarely has any units of measure and turns into just a bunch of words, sometimes even nonsense words. The terms Mission and Vision are the starting point for the governance of projects and programs:

• Vision - Defines the way an organization or enterprise will look in the future. Vision is a long-term view, sometimes describing how the organization would like the world to be in which it operates. For example, "provide affordable manned space flight to low earth orbit using private funding."
• Mission - Defines the fundamental purpose of an organization or an enterprise, succinctly describing why it exists and what it does to achieve its Vision. For example - "build and operate a single stage to orbit vehicle with 4 Pacs that can operate for under $100,000 per flight."

Here's an approach to governance in the Enterprise IT world

Principles of program governance

View more presentations from Glen Alleman

If you print this briefing in the landscape format and bind it on the short side (left side) it will be a book that guides you through the Program Governance process.

Quote of the Day

Years ago I came to the realization that the most poignant of lyrical tensions comes from the awareness that we are living an dying at once. To embrace such knowledge and yet remain compassionate and whole is the consummation of the endeavor of art.

From Listening to Mahler

Success of Complex Projects comes from System Management

The disucssion of agile development seems to be anchored in the development of code, from lists of "features," described in Stories, assembled from iterations into a releases. In the domain of "true" complex systems, this approach fails to address the complexity of the system itself.

Here's a clip from a working session I gave on the issue of "Systems Management," or the lack of.

In the absense of the "systems" view, the stories, features, iterations, and releases are driven by the current needs of the user. While this may be appropriate for some problems, there is no taxonomy of what systems it is appropriate for.

This is of special interest for large procurements encountered in Enterprise IT and Federal Government programs.

Leadership and Management

Lots has been written about management and leadership.

When something goes wrong, ask at the end, “Could our process, properly followed, have detected this situation?

From John Goodpasture and his quote from Alistair Cockburn...

... to “lead” already implies “change”. So a leader leads a change. (Somehow this thought had escaped my notice – had it escaped yours? It hit me pretty hard in the lecture.) If the leader in question is the PM, the person is leading a change in order to stabilize things (we can lead in order to increase stability). (This thought knocked my socks off.) ... and then the PM can “manage” the stable ongoing situation.

Quote of the Day

The very complexity of the natural systems with which the epidemiologist is faced often makes it difficult to tell which of the correlations that he observers are biologically significant. This difficulty is not lessened by the fact that it is quite easy to invent hypothesis that would, if they were true, fit attractively into our puzzle. It is much less easy to determine whether they are true or not; and this step has been omitted with a surprising frequency.

- Topley, 1942, from the Epigraph of Evolution of Virulence in an Experimental Bacteriophage System, PhD Thesis, Sharon Lee Messenger, 1997, University of Texas, Austin. (reference provide by biology student son)

So when there is a conjecture that agile software development is like a biological system or like a physical system, I wonder if the true understanding of what that means is known to the speaker? This whole notion of adopting scientific analogies for use in simple and many times simple minded applications to software development is very sporty business, if there is not a clear statement that the connection is purely "notional" and made of illustrative purposes only, having no connection what so ever to reality.

21st Century Motivation

When think about "agile" and the motivation of participants in projects based on agile, take 18 minutes to see how to think about WHY we want agile.

The Naive Notion of Simplicity

Many speak about simplicity, the search for simplicity, the quest for reducing complexity But or the term complicated there is no clear and concise unit of measure. Neither is there a unit of measure for simplicity. It's like anything of that class - "I know it when I see it."

But Dan Ward has a nice little book titled The Simplicity Cycle.The book has great examples of how we can fool ourselves into thinking that buzz words, and simple (and some times simple minded) approaches to complex problems can be used. When Dan uses the term complex, he uses a clear and concise definition.

This definition...

Lots of interconnected parts equal a high degree of complexity.

Few interconnected parts equal a low degree of complexity.

But there is more to it than than, there always is and that's why simple anologies don't actually work in practice. Dan reminds us...

So, while the consisting of interconnected parts definition is simple, applying that definition and establishing a qualitative assessment of whether or not a system has a lot or a little is—well—not so easy.

Context is everything. When someone says, this is complex and doesn't mention the context, well it's hard to know if that is true of not. A colleague complained "this SharePoint user interface for setting up a Team Site is really complex." Really? Compared to what?

Or, "the flight navigation computer software system architecture is complex." Really, compared to what? 

There are time when comparisons between compelx and not-complex can be made. But in all cases units of measure are needed for the discussion to proceed.

No domain, no context, hard to tell if the statement is useful to anyone.

Quote of the Day - Risk

Without metrics, you’re just another guy with an opinion. — Stephan Leschka, Hewlett Packard

When I hear words from almost anyone about how their approach is better than some other approach, I think of this quote.

And as Daniel Patrick Moynihan said:

Every man is entitled to his own opinion, but not his own facts

Alert - Canadian Forces Software Course

The Canadian Forces Intensive Course in Software Engineering contains many useful ideas applicable to current problems in enterprise class software projects.

The Use, Misuse, and Danger of Metaphor

Metaphors are common in the discussion of project management, software development, and agile methods. Metaphors are simple, intuitive and appeal to the reader by making a connection between the author and the reader through a common language mechanism. Metaphors appear to be an essential to understanding new topics. Topics many times conjectured to be new. Topics sometimes made more complex than needed.

The problem with metaphor is it's interpretation many times depends on the cultural context that generates the metaphor in the first place. More critically, the metaphor - while understandable to the reader - may not actually be the correct metaphor for the problem at hand, may not actually be correct at its basis, or simply is wrong.

Does This Metaphor Have Anything to Do With Managing Agile Projects?

Good question. On the surface there are words like "non-linear" - yep people are many times non-linear. The metaphor of "she went non-linear on me when I told here I didn't want to be her boyfriend any more." The term "couple system" - yep people systems are certainly coupled - in some many ways.

But then the issue of what metaphor to use. Can physics and biological metaphors be used? They are, but they suffer from fundamental errors in connecting the dots. Linear and non-linear systems have been studied since the invention of calculus. Every physics students is told in their freshman years of high scholl that the world is composed of second non-linear partial differential equations, but we'll be studying the linear ordinary differential questions first to understand the principles. Same for biology and chemistry. Start with the linear, move to the non-linear. If you leave the physics, biology, chemistry, mathematics track at the linear stage, you may have missed to history, principles, and practices of the actual mathematics used in these sciences. When they appear again in metaphors around social sciences = complex adaptive systems principles (not the theory, just the metaphorical principles) it seems to be "new all over again."

It's this rediscovery of the established that provides the basis in the social science connection to agile.

Back to the Metaphor Problem

Metaphors introduce a fundamental trade off between the generation of insights and the possibility of misappropriation. The extension of genetics to eugenics owed much of its popularity in the United States and in Germany to its use of culturally resonant metaphors. The use of metaphorical language in molecular biology—with its references to information, signaling, translation, editing, etc.—has contributed substantially to its breathtaking success during the past 50 years.

The trouble is that metaphorical abstractions all too easily become concrete objects and substitute for specific, describable processes.Many of the metaphorical references in the current agile literature mask both the complexity of the metaphor example (double pendulum for example) and the simplifying approaching to what is conjectured to be complex.

For example the notion that software development using "water fall," (as a metaphor, since it's not allowed in the DoD) can be replaced with software development using agile result in dramatic (my emphasis) improvements in success (all unsubstantiated in the scientific sense of double blind experiments in a controlled environment)(I know that is too strict a condition, but there is work needed if the process improvement is to be made outside self contained projects - like inside the government).

Rather agile software development method contribute little to the solution - if proper processes are applied - to the REAL problems of software development:

• Over optimistic assessments of the unknowns
• Use of deterministic estimating process rather than calibrated probabilistic approaches
• Inadequate data elicitation of the underlying risks
• Improper cost distributions for future work
• Overly aggressive management commitments
• Cost estimates based on poor project definition
• Inadequate allocation of budget
• Inadequate management of cost contingencies
• Ineffective procurement process that encourage unrealistically low bids

While agile software development provides verifiable improvements in the production of working products (Technical Solution in the CMMI paradigm) - by the very nature of short cycle 100% working outcomes - the large issues  (shown above) of software intensive projects go unaddressed.

Agile software development is one of many tools needed to Increasing the Probability of Success for software (or for that matter any) project. Our failure to recognize this and by into the metaphorical examples that conjecture chaos can only be controlled with agile, will not solve the problem. It will only delay the solution. The solution requires the integration of discipline around the conditions listed above PLUS software development based on iterative and incremental production of working products.

Alert - How Not to Use Earned Value

In a recent post on PM Hut, it was suggested that the Earned Value Management terms should focus on the "value" of the outcomes of a project, rather than the measures of performance for the work efforts of a project.

This is a common mistake. Many in the agile IT community make this mistake. One example is Questioning the Value of Earned Value on IT Projects. This is the PM Hut post conjecture that the "value" is earned value is not needed, what is needed is business value.

Since they are not the same, it turns out BOTH are needed. But ANSI-748B is about project is about PROJECT performance not BUSINESS performance. There is no reason to complain about ANSI EV for not addressing business performance.

There are several ways to define the business value, the best in my experience is Balanced Scorecard, with KPP's and KPI's for each strategy, connected to the programmatic performance of a project. Here's how to connect the dots between project perfomance measures and business strategy measures.

Back to the Issue At Hand

For those entering the Earned Value domain for the first time, EV’s units of measure are monetary But EV does not measure “value” to the buyer in the way suggested here.

The BCWS of earned value is the budget for the planned work. The BCWP - the earned value - is “earned” as the progress toward completion is measured. This measure is usually in “percent complete,” giving BCWP = BCWS x “percent complete.”

While some have suggested migration to the notation suggested here of PV, EV, AC, the primary users of Earned Value - the US Government and the tools used to report EV are still formatted in the standard notation (BCWS, BCWP, ACWP). The DID 81468 Contract Performance Report contains those named fields.

Project “value” is certainly an important aspect of measuring project performance. The measures of project value are best done through a business management process like Balance Scorecard with its Key Performance Parameters (KPP) and Key Performance Indicators (KPI).

As well Measures of Effectiveness (MoE) and Measures of Performance (MoP) are a means of measuring a projects value to the consumer of the project’s benefits - be they a building or space craft.

EV + Agile = Success

Just as an aside, EV is a PERFECT tool for Scrum projects where IT is being developed in a rolling wave and planning package architecture in a larger context. The "physical percent complete," needed by EV for calculating BCWP = BCWS x "physical percent complete" is EXACTLY the outcome of 100% working products at the end of each iteration. While agilest like to equate "velocity" with EV, it's not technically true, but the story points are very close to "dollars" when you exchange the points work effort in hours, then monetize those hours with the labor rate.

So don't let anyone, especially those

The Point of This Conversation

So please don't confuse the application of EV to the performance of work in support projects, with the "business value," of a project. In the Balanced Scorecard paradigm, most of the value of anything, especially IT is intangible and contains all kinds of externalities.

Be careful when reading or listening about "renaming," "reassigning," or "redefining" the established principles of ANSI-748-B Earned Value Management.

Externalities

LinkedIn Network

Why Wikipedia is a poor reference source for complex issues

Wikipedia is a pretty good source of information for topics that can be conveyed across a web pages and a small set of references. But you should be always be careful in referencing Wikipedia when entering into the scientific domain. One example is around the discussion of self-organizing systems in the domain of agile software development.

The Wikipedia definition of "self organization" is:

Self-Organization is the process where a structure of pattern appears in a system without a central authority or external element imposing it through planning.

This is usually the basis of reducing planning and letting the system emerge. But Wikipedia in this case is missing a few parts. Self-organizing systems have a multitude of characteristics beyond the simple definition in Wikipedia. It is important to understand these attributes to improve the chances that any metaphors, anologies, or other attempts to make connections are actually based of credible facts.

• System-ness - self-organization takes place in a system, in a coherent whole that has parts, interactions, structural relationships, behaviors, state, and a border that limits if from its environment.
• Complexity - self-organizing systems are complex systems. The term complexity has three levels of meaning: (1) There is self organization and emergence in complex systems. (2) Complex systems are not organized centrally, but they are organized in a distributed manner. (3) Is it difficult to model complex systems and to predict their behaviors, even if it is known to a large extent the parts of the system and the connections of the parts.
• Control parameters - a set of parameters influencing the state and behavior of the system.
• Critical values - if certain critical values of the control parameters are reached, structural change takes places and the system enters a phase of instability and criticality.
• Fluctuation and intensification - small disturbances from inside the systen intensify themselves and initiate the formation of order.
• Feedback loops, circular causality - feedback loops occur within the self organizing systems. Circular causality involves a number of processes and results.
• Nonlinearity - in a critical phase of a self organizing systems, causes and effects cannot be mapped linearly.
• Bifurcation points - once a fluctuation intensifies, the system enters a critical phase where its development is relatively open, certain (not all) possible paths of development emerge, and the system has to "make a choice." This means a dialectic of necessity and chance. Bifurcation means a phase transition from stability to instability.
• Selection - in a critical phase that can also be called the point of bifurcation, a selection is made between one of the several alternatives paths of development.
• Emergence of order - in the critical phase (but not before), new quantities of a self-organizing system emerge. This principle is called order from chaos. A self-organizing system is more of a self-organizing system emerge: this principle is also called order from chaos or order through fluctuation. A self organizing system is more than the sum of its parts. The emergence of order includes both (1) bottom-up emergence (a perturbation that causes the system's pans to interact synergetically in such a way that at least one new quality on a higher level emerges) and (2) downward causation (once new qualities of a system have emerged, they, along with the other structural macro-aspects of the system, influence - that is enable and constrain – the behavior of the system's parts). This process can be described as a top-down emergence if new qualities of certain parts (seen as wholes or systems themselves) show up.

This is a critical missing piece for the simple and many times simplistic use of emergence metaphors and anologies. Without defining the phase transition for the system, the bounaries of the system itself, and the choices that can be made by the system - the anologies will just be handwaving.

• Fault tolerance - Outside a critical phase, the structure of the system is relatively stable concerning local disturbances and a change of boundary conditions.
• Openness - can only take place if the system imports entropy that is transformed as a result, energy is exported or dissipated.
• Symmetry breaking -The emerging structures have less symmetry than the foundational laws of the system.
• Inner conditionality - self-organizing systems are influenced by their inner conditions and the  boundary conditions from their environment.
• Relative chance -  is a dialectic of chance and necessity in self-organizing systems: cer1ain aspects are determined, whereas others arc relatively open and subject to chance.
• Hierarchy - the self organization of complex systems produces a hierarchy in two distinctive senses: (1) The level of emergence is a hierarchically higher level-Ihm is, il has additional. new emergent qualities that cannot be found on the lower level that contains the components. The upper level is sublation (Aufhebung in Ihe Hegelian sense of the term) of Ihe lower level. (2) Self organization results in an evolutionary hierarchy of different system types: these types are  hierarchically ordered in the sense that upper levels are more complex and have additional emergent qualities.
• Globalization and Localization - bottom-up emergence means the globalizing sublation of local entities; downward causation means the localization of more global quantities.

With these characteristics, metaphors can be built that relate to actual processes be observed. But the simple - and possibly simple minded - definition from Wikipedia just creates more fog.

The critical outcome of this approach to to move away from the physics, biology, and other hard sciences annologies, and move to to social systems. The references for these social sciences approach start here.

• Fuchs. C. (2003). Dialectical  Philosophy and Self-Organization. In V. Arshinov & C. Fuchs (Eds.). Causality, Emergence, Self-Organization (pp. 195- 244). Moscow: NIA- Priroda.
• Fuchs. C. (2003). The self-organization of matter. Nature, Society, and Thought 16(3). 281-313.
• Fuchs. C. (2003). Some implications of Pierre Bourdieu's work for a theory of social self-organization. European Journal of Social Theory, 6(4). 387--108.
• Fuchs. C. (2003). Structuration theory and social organization, Systemic Practice and Action Research, 16(2), 133- 167.

The First Agile Organization

In 1943 Kelly Johnson built the XP-80, with these principles:

• The manager must be delegated practically complete control of his program in all aspects. He should report to a division president or higher.
• Strong but small project offices must be provided both by the military and industry.
• The number of people having any connection with the project must be restricted in an almost vicious manner. Use a small number of good people (10% to 25% compared to the so-called normal systems).
• A very simple drawing and drawing release system with great flexibility for making changes must be provided.
• There must be a minimum number of reports required, but important work must be recorded thoroughly.
• There must be a monthly cost review covering not only what has been spent and committed but also projected costs to the conclusion of the program.
• The contractor must be delegated and must assume more than normal responsibility to get good vendor bids for subcontract on the project. Commercial bid procedures are very often better than military ones.
• The inspection system as currently used by the Skunk Works, which has been approved by both the Air Force and Navy, meets the intent of existing military requirements and should be used on new projects. Push more basic inspection responsibility back to subcontractors and vendors. Don't duplicate so much inspection.
• The contractor must be delegated the authority to test his final product in flight. He can and must test it in the initial stages. If he doesn't, he rapidly loses his competency to design other vehicles.
• The specifications applying to the hardware must be agreed to well in advance of contracting. The Skunk Works practice of having a specification section stating clearly which important military specification items will not knowingly be complied with and reasons therefore is highly recommended.
• Funding a program must be timely so that the contractor doesn't have to keep running to the bank to support government projects.
• There must be mutual trust between the military project organization and the contractor, the very close cooperation and liaison on a day-to-day basis. This cuts down misunderstanding and correspondence to an absolute minimum.
• Access by outsiders to the project and its personnel must be strictly controlled by appropriate security measures.
• Because only a few people will be used in engineering and most other areas, ways must be provided to reward good performance by pay not based on the number of personnel supervised.

Sound like "agile?" Yes and at the same time, engineering focused, specifications were needed, documents where needed, management where needed.

Ride the Divide

With the season just starting, motivation movies keep us focused, since we're out of shape, over weight, and pretty much whipped from skiing all winter - that is letting gravity do a lot of the work, except in the bumps.

Here's a good start for motivation.

Ride The Divide Movie Trailer from Ride The Divide on Vimeo.

Coming next to the orginal course of the Coors Classic and Red Zinger, the Quizno's Pro Challenge.  

The local team will be there as well.

Alert - Oh My Gawd

We have found the enemy and he is us.

Status of FY 2010 Federal Improper Payments Report. In paragraph two of the summary the number is $125,400,000,000. Yep $125.4 BILLION.

The Myth of Standish Reports - Update

The Standish Report is raising its head again in the Agile presentations I've seen recently at conferences. Forgetting for the moment all the past "criticisms" of Standish...

• Standish Report
• Standish Report and Naive Statistics
• Finally a Challenge to the Standish Report

The numbers in the report are still being used by "agile thought leaders" to establish the basis for their proffered solutions. Forgetting even more that "agile" is a bottom up software development solution. And that most of the issues with large IT programs are leadership at the senior executive level, externalities from the market place at large, and general inability to connect the dots between business strategy and business execution.

Further is the complete lack of transparency in the statistical numbers. But here an alternative on how to report IT project success.

When you go here you'll see 61% are green, 33% "need attention," and 4.84% have "significant concerns."

The definitions of "needs attention" and "significant concerns" is provided in terms of mission suitability rather than simple - and simple minded cost and schedule.

So it's time once more to move beyond the worn out red herrings and start to address how to Increase the Probability of Project Success with processes and tools beyond just applying Scrum to the development team.

Willian poinnted out the bands for measuring success.  http://it.usaspending.gov/faq-agencies

Quote of the Day - Propensity to Believe

We are magical beings in a scientific age. Notwithstanding all the remarkable achievements of our species in terms of understanding and harnessing nature, we are born to magical thought and not to reason.

From the introduction paragraph to "Propensity to Believe," James E. Alcock, in The Flight from Science and Reason, New Your Academy of Sciences,1996.

This sentence caught my eye after reading a similarly placed introduction sentence in a "next generation" agile management book...

For centuries, mathematicians have preferred to work with linear (ordered) systems, and they considered nonlinear (complex) systems to be a special group.

My least favorite reference source is Wikipedia, but that source says simply:

A linear equation is an algebraic equation in which each term is either a constant or the product of a constant and (the first power of) a single variable.

Going upstairs to the library on a quest for a mathematics history book - which I couldn't find - I spotted instead the Principia and several "guides" to the Principia. The Principia (the replica of the original I have is from 1932) is a geometric text, since calculus was banned by the church. The "guides" - Florian Cajori's revised translation of Motte's translation from Latin to English (my version), Cohen, Whitman, and Cohen's translation and "a Guide to Newton's Principia," and my favorite, Chandrasekhar's Newton's Principia for the Common Reader (that's me) all start with Newton "inventing" his Calculus using non-linear relationships between the independent variables of is version of gravity.

Of course Wikipedia's definition of nonlinear is more accessible:

A nonlinear system is any problem where the variable(s) to be solved for cannot be written as a linear combination of independent components. A nonhomogeneous system, which is linear apart from the presence of a function of the independent variables, is nonlinear according to a strict definition, but such systems are usually studied alongside linear systems, because they can be transformed to a linear system of multiple variables.

Lo and behold those orbital mechanics diagrams, with the second order differential equations (all geometrically drawn from the calculus) in Principia and the equations that supported them - but were not published for fear of punishment - are of course Non-Linear integral and differential equations, since they involve variables with exponents.

So back in those past centuries I guess those mathematicians were actually using non-linear math to work out the orbits of planets, making attempts to explain gravity of the orbits, solve those elliptical (not elliptical in the number theory definition) equations of motion to predict where the moon and planets would be some time in the future.

But again care is needed with terminology. The non-linear aspacts of say large amplitude pendulums, forget the double pendulum for the moment, can be approximated with a series expansion. And the indeterminate aspects of these system does not mean the are not solvable or simulated. Just that the behavior cannot be determined in closed form and a progressive solution is needed to determine what happens at a future time.

The propensity to believe that extensions can be established to baseline principles, where established language, and definitions are agreed to, can be used as the basis one's message, while ignoring those principles is sporty business at best and flawed at worse. Such an approach, no matter how well intentioned, lays the resulting message on a foundation of sand. The difficulty of course is that approach - redefining established principles for ones own needs - is actually not necessary. Either use the right analogies or find better ways to explain the message without drawing attention to the fact you're making this stuff up as you go.

Quote of the Day

The trouble with Americans is that they haven't read the minutes from the previous meeting

- Adlai Stevenson

This might be said of others outside America as well. Much of the discussion (of sorts) is missing the connection with the reality of the past:

• Iterative and incremental development is mandated by DoD 5000.02
• Deliverables of working product are part of Integrated Master Plan / Integrated Master Schedule procurement
• Test driven software is the basis of DO-178 flight controls design and development.
• Many defense contracts limit the duration of a Work Package to crossing only one accounting period
• DoD 5000.02 describes "emergent" requirements over the procurement lifecycle. Within a Milestone, Program Events assess the increasing maturity of the product.

For real success in the enterprise IT world, we must end the use of red herrings, like "waterfall," "command and control," high ceremony references and start reading about how major enterprise systems have been developed for decades. And most importantly move on past those ancedotal experiences and look for examples of how improvements have been made in domains where "mission critical" is the starting point.

Alert - Australian Ministry of Defence

The Australian Ministry of Defence has a wonderful site for all kinds of guidance for acquisition activities, including risk management, systems engineering, technology management, project and programme (sic) management.

Thanks to Shim Marom and Craig Brown for their inriduction to the Australian view of the world.