Patterns amid Complexity

By Greg Fisher

Among the people I talk to about complex systems, a common point made is that the future is inherently uncertain or “unknowable”.  I have emphasised this myself in a number of blogs and papers – it results from the concept of emergence “breaking” determinism (as well as from quantum uncertainty).  But in this blog I want to flesh out what I mean when I say “the future is inherently unknowable” because the issue is much more nuanced than this statement implies, and because I would like to provoke some discussion.  I should mention that it was a conversation with my colleague Rhett Gayle that catalysed this blog, after he rightly questioned my use (as well as that of others) of this phrase.

At the heart of my argument are two points: (i) it’s important to distinguish between randomness and complex systems because the latter includes patterns whereas the former does not; and (ii) it is patterns that determine our ability to anticipate and / or influence the future.  In this blog, let me define “patterns” as systematic relationships between the component parts of some complex system that are expected to persist for a period of time.  My colleague Hank Sohota offered an alternative definition: “patterning” is a population wide tendency to think and / or behave in a particular way.

Visual Pattern by Reginald Leung

It is informative to distinguish between patterns and laws.  The latter involve universality in time and space i.e. they are eternal and they are expected to operate in all circumstances and contexts (consider, for example, natural laws).  By contrast, patterns are ultimately ephemeral and context always matters.  In fact, we could think of laws as a special type of pattern: eternal and context independent.  My reference to patterns has sometimes been misinterpreted as being about laws – that is not what I mean.

There is also the important question of identifiability.  Many patterns in human systems are tacit or implicit to varying degrees, rendering them difficult or impossible to identify and – even if identification were possible – to quantify.  As an aside, many traditional economists try to identify the eternal laws of economic behaviour because they think of a whole economic system as both unchanging and, implicitly, quantifiable.

The point I want to emphasise in this blog is that it is these two issues – the extent to which patterns persist and whether they are identifiable – that largely determine the degree to which we can be precise (or be confident) in predicting or influencing the future.  A good way to think about this is to consider that patterns exist alongside emergent effects, which are unpredictable by definition and might appear to be random.  So it is patterns that provide us with some basis of prediction but they’re not everything – the world is constantly evolving – so we find ourselves in a weird situation of “semi-predictability”.  We live in a world in which “black swan events” live along side relatively stable patterns (such as language).

Let’s look at randomness for a moment.  If everything around us in our universe were random, we wouldn’t exist and this blog would not be possible.  There would be no natural laws (or, if there were, they would be random – in which case they wouldn’t be very good laws), and there would be no forms of self-organisation (such as energy existing as mass and sub-atomic particles combining to make up atoms).  But consider a thought experiment: suppose we take a less extreme version of randomness, say one in which we do exist but nothing around us were predictable, what then?  Well, to be frank, there wouldn’t be much point in getting up in the morning.  We wouldn’t know (for example) if people still spoke English, obeyed laws such as prohibition concerning murder, used the currency in their pocket, drove on the left side of the road (for those in the UK), or could be trusted (etc. etc. etc.).  It would be a weird world in which to live and we probably wouldn’t survive for very long because it’s more than likely we’d soon be killed by a random act.

We don’t live in a random world because patterns have emerged.  They have created all of the order around us and by and large we expect most of this order – these patterns – to continue.  By “order” I do not mean that the societies in which we live are ordered in the same way as Laplace’s clockwork universe.  Nor do I mean that all patterns are only socially useful.  I just mean that there are forms of systematic relationships between people (and technology) that have facilitated our survival and what we understand as “order”.  Given my definition of patterns, this means that language is a type of pattern, as are laws, money, which side of the road we drive on, and trust.  In fact I would posit that we could call everything non-random in the universe a pattern.

There is a relationship between patterns and prediction.  In fact, I would note that not only do patterns exist and persist, we must rely on them in every day life.  We make decisions in the present assuming the persistence of some patterns e.g. I will withdraw £50 from a cash machine today for spending over the next few days.  I do not expect everyone else in the UK to switch to the Thai Baht during that period.  Furthermore, it is particular patterns – many of which we might call institutions – that are responsible for our civilised society and a relatively high standard of living.

But – and this is to assert the point further – it is important to emphasise that the world will change and so too will the patterns around us.  By “expect to persist” in my definition of patterns I was referring to making reasonable judgments that some patterns will remain broadly the same over a particular period.

To conclude, it is easy to conflate the original point of “the future is unknowable” and randomness.  The two are not the same.  If we live in a world that includes some predictable patterns and apparent randomness due to emergent effects, this statement is true but not in extremis.  Let’s not be too coarse-grained about this issue.

There are a number of implications of this mixture of patterns and emergence (apparent randomness) for public policy, which I will turn to in a future blog.

8 Responses to “Patterns amid Complexity”

  1. An excellent and informative post, Greg.

    You make a very important point about the patterning process. In an organizational or societal context, this is itself an emergent phenomenon of people’s everyday interactions, being ‘shaped’ by – and, at the same time, ‘shaping’ those interactions.

    As you imply, it is this process which enables us to ‘go on together’ in the world by creating a degree of expectancy which we take for granted in our everyday affairs. As your colleague suggests, this ongoing process (hence the emphasis on “patterning” rather than “pattern”) becomes embodied in a generalized tendency for people to think and act in particular ways – within those ‘populations’ to which they see themselves ‘belonging’.

    At the same time, the possibility (though not the likelihood) always exists that novelty will emerge from these same conversational interactions, and that people will ‘switch patterns’ in the moment – whether by accident, insight or intention.

    The possibility also exists, of course, that they will become ‘stuck’ – individually and/or collectively – in unhelpful patterns of thinking and acting. Established management ‘wisdom’, based on expectations of certainty, predictability and control, is one such pattern.

    Cheers, Chris

    • gregfisher says:

      Thanks Chris. I agree. I would also add (implied in the blog I think) that orthodox economics is also (mostly) an unhelpful pattern, being also based on certainty, predictability and control. They both seem to include the same ergodic (or “formative teleology)” mindset.

      Best wishes,

      Greg

  2. Greg, thanks for this very useful post – helps to clarify some important points which , as you say, are sometimes misunderstood.

    About patterns persisting but not being eternal, I rather like Ralph Stacey’s saying that the ‘the future is unknowable but not unrecognisable’. Along the same lines, Nicholas Nassim Taleb points out that the unknowability of a roulette wheel (highly constrained) is very different from the unknowability of a complex system (much more flexible in terms of evolving boundaries and interactions) – there is no chance at all of the roulette ball falling into all 37 slots at once, or suddenly turning into a piece of cheese!

  3. […] as part of a pattern. Greg Fisher wrote an outstanding post (read it!) discussing the importance of pattern recognition in complex systems. Here is part of his prescription: There is a relationship between patterns and prediction. In […]

  4. John Tropea says:

    Greg says : “it is patterns that determine our ability to anticipate and / or influence the future…”

    Chris says: “a tendency for patterns to be self-reinforcing and self-replicating) are essential if we are to function in our day-to-day lives.”

    I think this crosses over into intuition and prediction…which Monica Anderson has a lot to say about…

    http://artificial-intuition.com/intuition.html
    “Intuition is a process that uses this kind of correlation data to make short-term predictions that are correct often enough to improve our survival.

    Note that Intuition makes no attempt to model causality, or create any kind of high level models or theories. That would be using Logic. Intuition simply tracks events.”

    http://blogs.hbr.org/hbr/mcafee/2010/01/the-future-of-decision-making.html#comment-49809633
    “You cannot avoid using intuition in everyday situations. The alternative would be to use logic and other methods of Reductionist Science. But we don’t use science in everyday life, not even if we are scientists. As you come to a stoplight, you do not compute a differential equation in order to determine how hard to push the brake pedal – you use your intuition, which is a simple algorithm allowing you to jump to decent conclusions that are correct most of the time, based on your database of experience.

    We have yet to build a robot that can outrun a human on a rocky beach, where the human uses intuition to predict where to place each foot and how to keep their balance on *every step*. Almost every decision we make in everyday life, from taking a step to formulating a sentence with the correct semantics (which computers can’t do either) is 100% intuition based.”

    http://artificial-intuition.com/prediction.html
    “The purpose of Intelligence is Prediction. Evolution of the ability to predict agents and phenomena in the environment improved survival rates and created a strong evolutionary pressure to develop better and longer term predictions. This is the reason Intelligence evolved.”

    • Chris Davies says:

      John – intuition is a really interesting point to raise in the context of complexity because – you are right – it plays a crucial role in helping us make predictions in a complex and uncertain environment. It saves us from devoting disproportionate effort to thinking about things that probably aren’t worth worrying about, a point that Herbert Simon encapsulated in his concept of bounded rationality.

      There’s a parallel, I think, in the way societies develop institutions (in the North-ian sense – http://bit.ly/wzTBoJ). Without rules of thumb and semi-automatic patterns of behaviour, the transactional costs of most interactions would be prohibitively high.

      The problem comes when those institutions create patterns of behaviour that are out of kilter with the prevailing environment. Unpicking institutions to adapt to that change is an expensive process, and can feel extremely uncomfortable if that unpicking has to happen quickly.

      You could argue that we are experiencing this kind of rapid reconfiguration of the institutional framework in the current set of financial/political/social upheavals, and this would explain why many things feel rather painful at the moment.

      regards

      Chris

  5. Kenan Simpson says:

    Hi, first I want to say that finding this blog was a black swan for me because I think very similarly.

    I would add that patterns are the emergence of relationships among randomness that is bounded by laws, and that the laws are themselves patterns.

    Also, I was surprised that the term fractal was not used at all because this is what we are talking about, it seems to me.

  6. A significant challenge for complexity thinkers is the nature of subjective experiences and its contribution to pattern detection or formation. Jan Smuts, the author of ‘holism’ said: “Mind is admittedly an active, conative [desiring to act], organising principle. It is forever busy constructing new patterns of things, thoughts or principles out of the material of its experience. Mind, even more than life, is a principle of whole making. It differentiates, discriminates and selects from its vague experience, and fashions and correlates the resulting features into more or less stable, enduring wholes. Beginning as mere blind tropisms [attending to stimuli], reflexes and conditioned reflexes, mind in its organic nature has advanced …”
    He continued: “The free creativeness of mind is possible because … the world ultimately consists, not of material stuff, but of patterns, of organisations, the evolution of which involves no absolute creation of an alien world of material from nothing…”

  7. Herman Vande Putte says:

    Knowledge, by definition, is about the past. A (scientific) law is based on observations from the past. Whether the law will stand in the future or not, can never be known, it can only be assumed. A law is valid, because its invalidity is not proven yet. This is the essence of the falsification theory of Popper.

Leave a Reply