The popular notion that Cynefin can be applied in the software development domain as a way of discussing the problems involved in writing software for money is missing the profession of Systems Engineering. From Wikipedia Cynefin is...
The framework provides a typology of contexts that guide what sort of explanations or solutions might apply. It draws on research into complex adaptive systems theory, cognitive science, anthropology, and narrative patterns, as well as evolutionary psychology, to describe problems, situations, and systems.
While Cynefin uses the term Complexity and Complex Adaptive System, it is applied from the observational point of view. That is the system exists outside of our influence on the system to control its behavior - we are observers of the systems, not engineers of the solutions in the form of a system that provides needed capabilities to solve a problem.
Read carefully the original paper on Cynefin The New Dynamics of Strategy: Sense Making in a Complex and Complicated World This post is NOT about those types of systems, but about the conjecture that the development of software is by its nature Chaotic. This argument is used by many in the agile world for avoid the engineering disciplines of INCOSE style Systems Engineering.
There are certainly engineered systems that transform into complex adaptive systems with emergent behaviors that cause the system to fail. Example below. This is not likely to be the case when engineering principles are applied in the domains of Complex and Complicated.
A good starting point for the complex, complicated, and chaotic view of engineered systems is Complexity and Chaos - State of the Art: List of Works, Experts, Organizations, Projects, Journals, Conferences, and Tools There is a reference to Cynefin as organization modeling. While organizational modeling is important - I suspect Cynefin advocates would suggest the only important item - the engineered aspects of applying Systems Engineering to complex, complicated, and emergent systems is mandatory for any organization to get the product out the door on time, on budget, and on specification.
For another view of the complex systems problem, Principles of Complex Systems for Systems Engineering is a good place to start along with the resources from INCOSE and AIAA like Complexity Primer for Systems Engineers, Engineering Complex Systems, Complex System Classification, and many others.
So Let's Look At the Agile Point of View
In the agile community it is popular to use the terms complex, complexity, complicated, complex adaptive system many times interchangeably and many times wrongly - to assert we can't possibly plan ahead, know what we're going to need, and establish a cost and schedule because the systems complex, and emergent.
These terms are many times overloaded with an agenda used to push a process or even a method. As well, in the agile community it is popular to claim we have no control over the system, so we must adapt to its emerging behavior. This is likely the case in one condition - the chaotic behaviors of Complex Adaptive Systems. But this is only the case when we fail to establish the basis for how the CAS was formed and what sub-systems are driving that behaviors, and most importantly what are the dynamics of the interfaces between those subsystems - the System of Systems architecture - that create the chaotic behaviors .
It is highly unlikely those working in the agile community actually work on complex systems that evolve AND at the same time are engineered at the lower levels to meet specific capabilities and resulting requirements of the system owner. They've simply let the work and the resulting outcomes emerge and become Complex, Complicated, and create Complexity. They are observers of the outcomes, not engineers of the outcomes.
Here's one example of an engineered system that actually did become a CAS because of poor efforts of the Systems Engineers. I worked on Class I and II sensor platforms. Eventually FCS was canceled for all the right reasons. But for small teams of agile developers the outcomes become complex when the Systems Engineering processes are missing. Cynefin partitions beyond obvious emerge for the most part when Systems Engineering is missing.
First some definitions
- Complex - consisting of many different and connected parts. Not easy to analyze or understand. Complicated or intricate. When a system or problem is considered complex, analytical approaches, like dividing it into parts to make the problem tractable is not sufficient, because it is the interactions of the parts that make the system complex and without these interconnections, the system no longer functions.
- Complex System - is a functional whole, consisting of interdependent and variable parts. Unlike conventional systems, the parts need not have fixed relationships, fixed behaviors or fixed quantities, and their individual functions may be undefined in traditional terms.
- Complicated - containing a number of hidden parts, which must be revealed separately because they do not interact. The mutual interaction of the components creates nonlinear behaviors of the system. In principle all systems are complex. The number of parts or components is irrelevant n the definition of complexity. There can be complexity - nonlinear behavior - in small systems of large systems.
- Complexity - there is no standard definition of complexity is a view of systems that suggests simple causes result in complex effects. Complexity as a term is generally used to characterize a system with many parts whose interactions with each other occur in multiple ways. Complexity can occur in a variety of forms
- Complex behavior
- Complex mechanisms
- Complex Situations
- Complex systems
- Complex data
- Complexity Theory - states that critically interacting components self-organize to form potentially evolving structures exhibiting a hierarchy of emergent system properties. This theory takes the view that systems are best regarded as wholes, and studied as such, rejecting the traditional emphasis on simplification and reduction as inadequate techniques on which to base this sort of scientific work.
One more item we need is the types of Complexity
- Type 1 - fixed systems, where the structure doesn't change as a function of time.
- Type 2 - systems where time causes changes. This can be repetitive cycles or change with time.
- Type 3 - moves beyond repetitive systems into organic where change is extensive and non-cyclic in nature.
- Type 4 - are self-organizing where we can combine internal constraints of closed systems, like machines, with the creative evolution of open systems, like people.
And Now To The Point
When we hear complex, complexity, complex systems, complex adaptive system, pause to ask what kind of complex are you talking about. What Type of complex system. In what system are you applying the term complex. Have you classified that system in a way that actually matches a real system. Don't take anyone saying well the system is emerging and becoming too complex for us to manage Unless in fact that is the case after all the Systems Engineering activities have been exhausted. It's a cheap excuse for simply not doing the hard work of engineering the outcomes.
It is common use the terms complex, complicated, and complexity interchanged. And software development is classified or misclassified as one or the both or all three. It is also common to toss around these terms with no actual understanding of their meaning or application.
We need to move beyond buzz words. Words like Systems Thinking. Building software is part of a system. There are interacting parts that when assembled, produce an outcome. Hopefully the desired outcome. In the case of software, the interacting parts are more than just the parts. The software has emergent properties. A Type 4 system, built from Type 1, 2, and 3 systems. With changes in time and uncertainty, modeling these systems requires stochastic processes. These processes depend on estimating behaviors as a starting point.
The understanding that software development is an uncertain process (stochastic) is well known, starting in the 1980's [1] with COCOMO. Later models, like Cone of Uncertainty, made it clear that these uncertainties, themselves, evolve with time. The current predictive models based on stochastic processes include Monte Carlo Simulation of networks of activities, Real Options, and Bayesian Networks. Each is directly applicable to modeling software development projects.
[1] Software Engineering Economics, Barry Boehm, Prentice-Hall, 1981.
