In several management and agile software development books there is mention of "complex adaptive systems," " chaotic systems," and "evolutionary systems". "Most books and articles in the field of complexity are either highly technical and readable only by specialist, or rather shallow popularizations" of the same concepts.
Modeling of dynamic systems is a profession, guided by the mathematical principles. There is a vast literature on this topic. At the same time there is a large amount of "overly simplified" notions about dynamic systems, there use, their outcomes, and their domain and context in a domain.
I'd like to establish some background for such systems, before applying these concepts to actuial projects (in a later post):
- There is conjecture that the populist approach to dynamic systems is suffienct for making decisions involving other peoples money.
- That non-mathematical descriptions of such systems, while providing introductions to the concepts in a notional manner, can be used to guide methods, processes, and benefits for complex projects.
It's not my intent to convince those who have landed on the populist view of Complex Adaptive Systems as their paradigm for software development, that they need to go deeper into the principles. Rather to show those interested in the principles to see how those populist views are limited in their applicability to actual problems of managing complex systems. And possibly starting with the populsit view, explore further the actual mathematics and its application to project management problems.
The study of dynamic system dates to ancient times, focused on planetary orbits in the Renaissance. Ptolemy's Almagest provided an elaborate geometric description of the solar system in 140 A.D.
In the mid-20th century, this topic was limited to scientist. The first investigator to observe "chaotic" behavior in a computer program was Edward Lorenz at MIT. James Gleick's popularization Chaos is a good starting point for the history of chaos modeling.
Four Types of Dymanic Systems
- Type 0 - Static Systems - systems that change slowly or intermittently after a long period of time or change at a scale too small to care about. The definitions of slow and small need to be provided. As well static system can be simple to complex and still the static.
- Type 1 - Solvable Systems - systems that have sets of differential equations that have a "solution." By "solution" it means a mathematical function of time. This could be a "closed form" solution. But solvability comes in two forms - with errors and without errors. And then two other forms - linear and nonlinear.
By the way the popularly misinformed notion that mathematics came to nonlinear equations recently is just that - misinformed. 1690 Jakob Bernoulli reduced the problem of determining the isochrone (the curve in a vertical plane down which a particle, starting at any initial point on the curve, will descend to the lowest point in the same amount of time) to a first-order nonlinear differential equation.
Newton put these to work in 1665 for kinematics, dynamics, and celestial mechanics. It took nine years after his death (1727) before is was published, but it was in his notebook.
Before proceeding with the linear / nonlinear system discussion, we need to define what we mean by "linear." Linear systems are rich with generalizations from which conclusions can be drawn. Solvable nonlinear as well, but working on the linear solution first provides insight to the behavior and possible solutions in the nonlinear domain.
- Type 2 - Perturbation Systems - many systems are approximated by a solvable system. But when these system operate for a long time the "solvable" attribute stops working. In some cases it is not possible to generalize the solution. This is where perturbation comes in. Making small changes in the parameters to "keep the system linear" is one approach.
Perturbation theory is simple. Suppose we have a system S and there are two disturbances one at time t1 and one at a later time t2. We'd like to know what the second disturbance should be to restore the system to its original state S0. Let's call the first disturbance U1 at time t1, and the second disturbance U2 at time t2. We need an expression that shows the operations that move the system from its first state S1 to the second state S2. This is called perturbation theory.
- Type 3 - Chaotic Systems - must populist books on chaos theory have very weak definitions of chaos. Here's one with more rigor.
Chaotic systems comprise the transition between solvable and near solvable systems and those that are completely stochastic or random. Random systems are analyzed using "averages." While this "averaging" sounds simple enough it has many difficulties. This is the pitch around probabilistic risk analysis and the avoidance of averages, which usually leads to poor decision making.
Chaotic systems are simplier to the non-mathematical user since there are tools that can perform the analysis. Monte Carlo Simulation is one used in the project management world.
Chaotic systems are shown in popular books as those nice Lorentz spiral plots of systems behaving chaotically - jumping from one unstable equilibrium point to another, converging to a steady state of the two convergence points.
- Type 4 - Stochastic Systems - The big question is what is random? Random means unpredictable. A random sequence might be defined as xi, i ϵ Z+ where all the x's have equal probability of being any number between + ∞ and -∞.
These Type 4 processes are good representations of many natural systems, including human systems writing software for money. There is a logical sequence of events, that are at their core "random variables." Cost, schedule, and technical performance are many places to work. But the human interactions ae stochastic processes as well.
These systems can be characterized as "being driven by noise." Any modeling parameter can be replaced with a random variable. This replacement is allowed because random variables are usually considered to be exogenous - they are independent of the state variables. This is a common observation of human systems. Again this is the natural home for the Monte Carlo Simulations.
So Why Do We Care About All This?
When we hear or read about Complex Adaptive Systems, Chaos, or Complexity in populist texts or forums, we need to take care for several reasons:
- Is there mathematical rigor around those terms? Or is this a popularized description of the process. Not that there isn't interest in the popularized attributes, but be careful making business decisions with someone elses money if you don't have a handle on the underlying mathematics.
- When there is mention that some theory of chaos is being used to guide the suggested method, take care to confirm both sides of the equation. Is the conjecture about the source of the random behavior actually connected to the reality of the situation. Then is the conjecture that the proposed solution to address this behavior confirmed to actually work for the actual reason it is claimed to be working.
In later posts, I'll show how these concepts are applied the real world projects operating in the presence dynamic environments.
