Emergence and Generative Art

blog
Written By Amy Goodchild

Sometimes, a system is more than the sum of its parts. Simple rules and behaviours at an individual level can lead to complex and surprising phenomena at a macro level. This is emergence - common in the natural world and in generative art.

 

PART ONE: EMERGENCE

Let’s take a look at some definitions:

 

"You have this very simple set of rules that doesn't look like it's going to do anything interesting, but (...) these large super structures start to present themselves that you wouldn't be able to predict just from looking at the rules."

Tyler Hobbs

"You put a few simple rules together and something comes out of that which is entirely unexpected, and moves beyond what you could imagine those simple rules producing."

Casey Reas

"Emergence occurs when an entity is observed to have properties its parts do not have on their own, properties or behaviors which emerge only when the parts interact in a wider whole."

Wikipedia

 

These definitions mention the two key features of emergence:

  • Simple rules lead to unexpected and complex results.

  • The properties of the parts of a system are different to the properties of the system as a whole.

In summary - simple rules are followed by parts of a system and unexpected results emerge from the whole system.

 

Okay… can you give me some examples?

There are loads of examples of emergence in nature, here are some of my favourites.

 

Murmurations

Birds fly in flocks without colliding, while humans can barely make it across a train station without awkwardly bumping in to each other or dragging a suitcase over someone’s feet.

A flock of birds sometimes even produces beautiful swirling patterns, known as a murmuration.

 

There’s no choreographer directing the birds’ movements. Each bird is following its own rules (for example, parakeets always veer right and change altitude to avoid potential collisions) and the combination of every bird’s movements leads to the dramatic patterns we see.

 

Fireflies

Fireflies flash their little beetle butts mainly as a mating ritual, although it’s thought they originally evolved the ability to deter predators. Generally it’s the males putting on a show for the females, and some species of fireflies have even evolved to flash in sync for maximum flirtation potential. Some of these species, found in Southeast Asia, blink in time, while others in North America flash over a period of a few seconds, creating waves of light.

Photo by Flash Dantz

 

Again, there’s no conductor with a baton leading them in time, and no drummer laying down the beat. Instead, each firefly watches for flashes around it, but responds as an individual.

It doesn’t just flash as soon as it sees another flash though, that would create a feedback loop of flashes immediately triggering each other. Instead, each firefly has it's own internal “clock” which it nudges forward a little every time it sees a flash nearby. When its clock strikes 12, that’s when it flashes.

With each firefly following this rule, waves of flashes set off across the cloud, or the cloud completely synchronises itself.

Here’s one I simulated earlier:

 

Snowflakes

It’s not just animal behaviour that leads to emergence, it also happens thanks to physical properties of materials.

Snowflakes begin with an ice crystal formed when a water droplet attaches to a dust particle in the sky. While falling to the ground, the snowflake’s shape builds as more water vapor freezes on to it.

The molecular properties of water affect the shape it creates as it freezes and this, combined with the way the budding snowflake comes into contact with new water vapor particles in the air mean that snowflakes form hexagons and then begin to branch out into 6 arms.

Every water molecule has one oxygen atom and two hydrogen atoms.

When water becomes ice, it forms this hexagonal structure. The oxygen atoms are slightly negatively charged while the hydrogen atoms are slightly positively charged, meaning they will only bond to each other (because opposite charges attract).

 

Variables such as temperature and humidity affect the shape of the branches. These factors produce repeatable results and, in a lab, it is possible to create multiple identical snowflakes by tightly controlling these conditions.

This super interesting video goes deeper into the way snowflakes form in nature and how they can be designed in a lab. For the purposes of this article, the most interesting thing to note is that, again, there is no artist deciding the snowflake’s shape. It is the result of the simple molecular properties of water and the environmental conditions.

Photo by Aaron Burden on Unsplash

 

Paramecia movement

It’s not one of the classic examples of emergence but, as a single celled organism, the paramecium builds an interesting bridge between emergence in alive and non-alive systems. A paramecium has no brain or nervous system but its behaviours make it appear as if it were making decisions.

You can skip to ~10 seconds in this video to see them moving.

 

Paramecia, which are often found in ponds, move through water using their cilia (like little hairs found all over their outer membrane) as oars. When a paramecium bumps into an obstacle, it reverses the movement of its cilia - but how does it “know” to do this? (Spoiler alert: it doesn’t)

The outer membrane of the paramecium is physically deformed by the impact of the obstacle, which allows calcium to enter the cell. The calcium flooding into the cell is positively charged, which depolarises the cell’s membrane (flips its charge around), which in turn causes the cilia to move in the opposite direction, propelling the paramecium backwards away from the obstacle.

It’s a mechanical and physical process. At no point is the paramecium doing anything we would describe as knowing something, thinking, or making a decision. There is no brain, just a series of local interactions between parts of the cell and the environment.

 

Great expectations

In the first section, words like “unexpected” and “unpredictable” were casually thrown around to describe the results of an emergent system. In the snowflake example though, I said that snowflakes can be created or even designed in a repeatable (and therefore predictable) way in a lab.

It turns out that there are different levels of unexpectedness:

1: Completely expected.
I push a ball and the ball moves. This is not an emergent system.¹

2: Somewhat unexpected.
If you had never heard of a snowflake, and I said, “guess what, water vapour falling through the sky at freezing temperatures forms rotationally symmetrical hexagonal structures that turn out to look cool” you might say, “wow, I didn’t expect that.”

However, once we know everything relevant about the properties of water vapour, its motion through the sky and so on, we can piece together an explanation of why it happens.

We can call something an emergent system when there is a bit of thinking needed and a few steps to go through to tie together the link between the input and the output. We didn’t immediately expect it and it’s unpredictable at first glance but the output is actually a deducible consequence of the inputs.

 

Strong and weak

Most emergent systems in nature and in generative art fall into the “somewhat unexpected” category. This is known as “weak emergence”, which I think is kind of a shame because it’s still really cool and we shouldn’t be mean to it.

There is another type of emergence, which is worth discussing even though it takes us on a detour from generative art. It’s known as “strong emergence”, and it can be put into a third category:

 

3: Completely unexpected.
I push a ball and the ball turns into a squirrel.

I spent an hour photoshopping this and I am sorry.
Squirrel photo by Włodzimierz Jaworski on Unsplash.
Beach ball photo by Rodion Kutsaiev on Pexels

 

That’s a silly example but it sort of has to be. For something to be strongly emergent there has to be no deducible mechanism by which the input produces the output. In David Chalmers’ paper on the topic, he refers to strongly emergent phenomena as those “whose existence is not deducible from the facts about the exact distribution of particles and fields throughout space and time (along with the laws of physics)”. In other words, even if we knew absolutely everything about the input and the rules, we could never figure out how it produced the output.

Squirrels spontaneously emerging from balls being pushed around would fit this definition but it isn’t a thing that actually happens. There is a lot of stuff that does happen that we don’t have explanations for though.

One unexplained phenomena I learnt about recently is that birds use the earth’s magnetic field to navigate, but we don’t know for sure how they sense it. However that doesn’t mean the mechanisms for magnetoreception in animals are not deducible, simply that we haven’t deduced them yet.

So are there any examples of strong emergence in the real world? Things that, even in principle, cannot be explained from their constituent parts?

 

The universe experiencing itself

David Chalmers argues that there is one case of strong emergence and that’s consciousness.

Consciousness means I am having an experience of being myself. You are having an experience of being yourself. There is something it is like to be me, or you, (or a bat). But a rock is not having an experience of being itself. There is nothing it is like to be a rock.²

 

Somehow, if you put together a bunch of oxygen, carbon, hydrogen, nitrogen, calcium, phosphorus and a handful of other elements, in just the right way, it grows a brain, becomes conscious and starts asking questions like ‘Who am I?’, ‘What is consciousness?’ and ‘Will Snoop Dogg ever buy my NFTs?’

Chalmers’ position is that the mechanism by which this happens is absolutely not deducible, even in principle. I’m not sure whether I agree but my take on these kinds of questions is that picking a side is not necessary and it’s more worthwhile to think through the consequences of each option while letting your mind boggle. So, enjoy that.

That’s all I’ll say about Strong Emergence. Let’s go back to the, again, unfairly named, Weak Emergence.

 

Part TWO: Emergence in Generative Art

Generally speaking, generative art is made by defining a set of rules, running them and producing results. That sounds familiar. Emergent systems also have a set of rules and a result. Much generative art can be called emergent, and some is specifically positioned to explore or use emergence.

There is quite a collection of “classic” and renowned emergent systems. Let’s look at a few.

 

Wolfram’s Rules

On a 2D grid of cells, each cell can either be “on” or “off”. The first row is set up randomly or with a pattern and the following rows are populated one at a time.

Each cell sets its own state depending on the pattern of the three cells above it.

For example, in rule 30 specifically, the rules are as follows -

Parent Cells 1, 1, 1 1, 1, 0 1, 0, 1 1, 0, 0 0, 1, 1 0, 1, 0 0, 0, 1 0, 0, 0
Next cell state 0 0 0 1 1 1 1 0
 

There are 256 sets of rules, covering all the possible combinations. Here are some examples of the results:

Rule 30

Rule 22

Rule 22 from randomised top row

 

Some rulesets create repeating patterns, such as Rule 22 when seeded with a single “on” cell, while Rule 30 creates seemingly infinite and random patterns.

I have a p5js implementation of this here which you are welcome to play around with.

 

Game of Life

This is similar to Wolfram’s rules, but here each cell looks at its 8 neighbour cells and, instead of iterating from the top to the bottom of the grid, it iterates over time, creating animations.

3 phase pulsar

Finite pattern

Gliders

 

There is really only one set of rules in Conway’s Game of Life and the results vary according to the start state.

 

Cyclic cellular automata

Here we build on the idea of Wolfram’s rules and the Game of Life by introducing more states. In a CCA system, there can be any number of states for each cell, and they loop around in a cycle like 0, 1, 2, 3, 0, 1, 2… etc.

Each cell increments its own state if there are more than a threshold of neighbouring cells that are at a higher state (or exactly one state up). In a cyclic cellular automata the size and shape of the “neighbourhood” can also be varied.

Here’s an example of how that can look:

 

Reaction Diffusion

This system is a little more complicated as each cell holds two pieces of information - an amount of A and an amount of B. You can imagine them as two chemicals, or as something like carrots (A) and rabbits (B).

Every step, four things happen:

  • Some A is added to every cell (feed)

  • Some B is removed from every cell (kill)

  • A and B diffuse from high concentration cells to neighbouring lower concentration cells (A and B have different rates of diffusion)

  • A and B react with each other - two B’s and one A become three B’s.

A more detailed explanation is here and Dan Shiffman implements the algorithm in p5js here.

By changing variables controlling each of those four things (e.g. the amount of A added to each cell, the amount of B removed, etc) we can produce varied results. Many combinations of settings will result in patterns dying out quickly, but we can hunt around for combinations of settings that produce interesting results.

In my fxhash project Consequence Broadcast, I implemented Reaction Diffusion in a shader so it would run efficiently in real-time. Each output selects from a library of effective settings I had found. In many of the pieces, it uses one set of settings in one part of the canvas, and another elsewhere.

 

This produces some very organic looking patterns but, if you compare the results here to those from the Game of Life and a CCA, they do feel connected, and you can still glimpse the underlying grid and rule based nature of the Reaction Diffusion algorithm.

 

Agent Based Systems

The systems we’ve looked at so far are all grid based and, as the complexity of the systems increased, some incredibly cool patterns emerged.

We can also do away with the grid completely and work with ‘agents’ instead.

You can think of these agents as being like creatures, plants or other entities, placed anywhere on a canvas and able to move around if their ruleset allows it. Agent based systems are also often referred to as an ecosystem.

Let’s look at some classic examples in this category.

 

Flocking

This is a system originally developed by Craig Reynolds to simulate the flocking of birds. He called the agents in this system boids, a shortened version of ‘bird-oid object’.

In every frame of the animation, each boid looks around at others nearby and adjusts its own heading based on the behaviour it sees, using three rules:

  • Alignment: The boid goes in the same direction as others are going.

  • Cohesion: The boid goes closer to others.

  • Separation: The boid avoids getting too close, to avoid collisions.

The influence of each of these three rules can be weighted differently, to produce different results in the flock. For example, you could have a flock where alignment is very influential and the boids all turn as one, or a flock where separation is not at all influential, and the boids fly through each other.

In addition we can alter:

  • Perception distance: When the boid looks at other boids , this is how far away it looks (a radius)

  • Separation distance: The minimum distance a boid tries to maintain from others.

Here are some examples of different flock behaviours created by adjusting these settings.

(This simulation also has some central gravity, pulling the boids gently towards the centre of the frame and a bit of Perlin noise added to their movement to add to the natural feel. )

 

Diffusion Limited Aggregation

This one is a real-world phenomenon that is often simulated in code. We start with one particle in the centre of the frame. New particles are introduced one at a time and move about randomly (simulating Brownian Motion) until they collide with another particle, at which point they become still and the next particle is introduced.

At first thought, I would have expected this to form a blobby shape but actually it makes branching tree structures. It makes complete sense when you think about it - the newly introduced particles are more likely to collide with a still particle that is protruding out into the space than they are to collide with a particle at the centre of the aggregate.

 

Endless other ideas

We’ve done a tour of the Greatest Hits here - the algorithms that generally come to mind when we think about emergence and generative art. But it doesn’t stop there. Lots of generative art uses emergence in varied and often innovative ways.

In ‘Arteria’, Camille Roux created a system of “autonomous agents which are drawing paths following a set of rules”, resulting in beautiful and varied patterns.

Nadieh Bremer has recently been exploring double pendulums, where the swinging motion of two conjoined lengths combines to create swirls. We also see how small differences in start state can quickly diverge to create chaotically different outputs.

 

Jos Vromans created a system of subdividing triangles, from which leads to a shaded, almost 3D looking effect.

While Frank Force (KilledByAPixel) demonstrates how complexity can arise from a very small amount of code.

 

Piter Pasma created this ‘simple’ set of steps to slice through and pull apart a spiral shape.

After splicing and pulling 300 times, we see these results -

 

In Neel Shivdasani’s ArtBlocks project, Tropism, he constructs stable graphs and uses a series of steps which have a cascading effect throughout the graph to creates patterns.

Meanwhile Rituals, another ArtBlocks project, this time by Aaron Penne and Boreta, is a “celebration of emergence”, featuring endlessly evolving patterns.

 

Pushing the boundaries

Here are a few things I want to mention even though I haven’t yet explored or understood it myself.

Multi Scale Turing Patterns, inspired by spots and stripes patterns found in nature -

 

Lenia, a cellular automaton that uses “continuous states and continuous space-time”. I haven’t figured out what the latter means but the results look mind blowingly lifelike -

 

Some artists really pushing boundaries with lifelike bio simulations are Ciphrd (I particularly recommend the blog about his project Ethereal Microcosm), Arsiliath and mxsage.

 
 

Extrapolations & Implications

Some of the coded systems we’ve looked at are designed to simulate biology, and some implement invented rules. In both cases, we see complex and often lifelike results. Even the comparatively simple Game of Life can produce entities which seem to have agency. Some of this is surely the human tendency to anthropomorphise but, it’s undeniable that outputs from a system like Lenia could be confused for footage of a petri dish.

A snowflake is shaped in accordance with physical properties. A paramecium moves due to physical properties and chemical triggers. A firefly ‘decides’ when to flash based on an algorithm. Where does it stop? As Artnome puts it, “After playing with the Game of Life, I and many others get a feeling that everything, no matter how complicated and mysterious it appears to be on the surface, may also be reducible to a basic set of algorithms.”

It feels as if the world could be a wind up toy, ticking through a giant set of interdependent rules and algorithms.³ Is there any real difference between a firefly deciding to flash and me deciding to make a joke to a friend? Perhaps the latter is just the result of a more complicated ruleset and a wider range of inputs.

But it certainly seems like there is a difference. Generally humans don’t feel like puppets on strings - we feel as if our decisions and actions are our own. Perhaps consciousness itself is the key? Human behaviour is influenced by some rulesets, but it cannot be simulated with the accuracy or fidelity of a snowflake. Is that due to a lack of data and understanding of the algorithms at play, or is consciousness an unpredictable secret sauce that gives us the ability to operate outside of any algorithm?

It seems to me like the comparison between us and the firefly is more of a gradient than a binary difference.

Some kind of theory

 

Let’s also not forget the idea that consciousness itself emerges, but in a way that is not reducible to its constituent parts. Does consciousness enable us to break from the algorithms, or does breaking from the algorithms provide us consciousness?

This seems like a good place to refer to my earlier point about it not always being necessary to find an answer to a question. As artists, it’s our prerogative to ruminate and speculate on the implications of the possibilities.

 

  1. It occurs to me that you could make an argument that the movement of the ball is an emergent property of the electrostatic forces between me and the ball and probably form a conclusion that every consequence is emergent somehow but let’s just… not. (go back)

  2. Panpsychism - the idea that actually everything is conscious (a rock, a hammer, the sun, my teddy bear… etc) is some of the most legitimised hippy-sounding probably-nonsense I know of. Fun to think about, probably not real. Although in the case of my teddy bear it absolutely is. (go back)

  3. Randomness and chaos also play a part in disputing the wind up toy concept, but that’s a topic for another article. (go back)

Amy Goodchild
Previous

Playing dice: Randomness, determinism and the quantum world
Next

Packing & Expanding Polygons : An ongoing exploration