‘I don’t know if I actually said Eureka, but that Eureka moment really did happen……I suddenly realised that there was a door in front of me that I didn’t even know existed that had suddenly swung open’

Sir Alec Jeffries on DNA-based identification

The unconscious intuitive, ‘Visionary’, mind is at its most magical when it produces new ideas and insights, leaping across subject domains, making new and unexpected links, and constructing richer, more illuminating models of the world. These aerobatics are its unique talent; the conscious analytical ‘Engineer’ mind works on earthbound tramlines which it cannot leave.

There are hundreds of books seeking to teach us how to be more creative. They draw on the advice of highly creative people and the practical tips they contain are useful, but often presented without much underlying argument to connect them. In these articles I want to start from an analysis of the psychology of creativity - what happens our minds and brains when we have a new idea – and draw out from this the ways in which we can facilitate that process in ourselves and our colleagues. This is of course particularly challenging amidst the many competing demands of running any organisation which relies on the ability of its people to produce new ideas.

What are new ideas and where do they come from?

‘[The detective Strike] mentally backed away from his new theory to examine it in its entirety, and from every angle he surveyed it, saw it to be smooth, balanced and complete. The extraneous and the irrelevant were now lying discarded to one side.’

Robert Galbraith, ‘The Running Grave’, 2023

Let’s go back to your unconscious simulation of reality, stocked with simplified models of everything you’ve encountered, which enables you to navigate the world without engaging your conscious mind, as I explained in my earlier articles. All new ideas share the same psychological structure. They’re new models of the world, and if they are good ideas, they increase your understanding of it because they take in more information and link it in a more coherent way. If you like, they are ‘meta’-models which reorganise the components available to you in a more coherent and economical way. This applies regardless of the importance or scale of the idea: whether it’s a scientific discovery or innovation, a work of imagination, an occasion when you ‘think outside the box’ at work, or one of those personal insights which can set us off in a new direction. Whichever it is, you have made new links and disabled old ones, and reorganised your unconscious architecture, so that it illuminates things in a new way.

Your unconscious mind ‘thinks’ associatively. Your brain is what chaos theorists call a ‘self-organising system’, a system in which a series of local interactions produce greater order in the whole. When it’s stimulated by something new, activation spreads outwards through its networks of neurons, like ripples on a pond, as it seeks to relate the new stimulus to its existing architecture, to find links, matching patterns and resonances. Sometimes it can only incorporate the new stimulus if it reorganises, and on occasions that reorganisation may be large scale and widespread, severing many existing links and making new ones – making ‘creative leaps’, and triggering a cascade of changes in different areas. It may need to try out many different options before it has a configuration which encompasses all the relevant components.

There has been a lot of interest in the last decade in a relatively newly discovered network in the brain, the Default Mode Network (DMN), which neuroscientists believe may have a role in forging these new connections. The DMN, sometimes called the ‘daydreaming’ or ‘imagination’ network - we might call it the Visionary network - clicks in when you’re at rest. It’s responsible for simulating experience in a variety of contexts, including times when we remember events in the past, imagine the future or any other hypothetical state, take on another person’s perspective or ruminate about ourselves. It’s more active in general in creative people – a point to which we will come in the next article.

The resting DMN flip flops with the Executive Network (EN), which is engaged when you’re consciously focusing on a task, and responsible for organising information in your working memory, and conscious problem-solving and decision-making. We might think of it as the Engineer network. The DMN is more active when the EN is less so, and vice versa. It’s busiest when you are in a sustained state of what neuroscientists call ‘transient hypofrontality’ (the EN involves the frontal parts of your brain): when you’re drifting off to sleep, out for a walk, in the shower, doing a mundane task like cleaning the house.
When your brain is idling like this the DMN can look for matching patterns and try out multiple different ways of tidying and reorganising – making new links, resolving contradictions and accommodating new information - and send any promising new configurations to your conscious mind. It’s thought that it recognises such patterns by their speed and ease of processing: the reorganisation is more coherent - both comprehensive and economical like an elegant mathematical proof, which provides a complete, self-explanatory answer in the minimum number of steps. And when a new model like this reaches your consciousness, you experience an ‘aha moment’ – a sense certainty that the perspective you now have is the true one, that you can’t go back to your previous way of thinking about things - and a feeling of release, even elation. Your brain has released the reward chemical, dopamine, and this ensures that the new model is saved with a positive emotional tag. It also provides an incentive for you to seek out further new insights and their accompanying rewards.

The dual process (again)

Graham Wallas, whom we met in the articles on decision-making (Articles 3-5), describes the process of idea generation thus. You focus – consciously - on something which needs attention (Wallas calls this ‘priming the mind’) ; you disengage from it and do something else to allow your unconscious mind to work ( ‘incubation’); an idea arrives in your conscious mind (‘illumination’); you evaluate that idea to see whether it works by analysing it consciously (‘verification’); if it does you proceed to develop it; if it doesn’t you remit the matter back to your unconscious mind for further incubation
If you have engaged in a sustained piece of creative work – developing a theory or technology, designing a government intervention, writing a book, producing a film, conceiving a work of art - you will know that this will involve countless iterations of the process as you negotiate the many creative challenges along the way: false starts, blind alleys which don’t survive the verification stage and involve the jettisoning of a lot of previous work, uncertain periods where you feel you’re not making any progress. It took Einstein 10 years to formulate the special theory of relativity, during which he felt confused and frustrated.

Incubation and illumination: coaxing ideas into the light

‘I can’t sit down with a script. I need to be distracted and busy myself – play cards, go for a swim, anything, in order to do the work. I need to leave it to my subconscious and trust that the internal engine is making adjustments and processing everything.’

Dame Judi Dench on immersing herself in a new part.

The neuroscientist Arne Dietrich puts forward a convincing explanation of what happens in the brain when we think hard about a problem and then disengage from it. When we’re working on a problem, he suggests, our brains organise themselves to focus on that task, activating the networks usually associated with that task (he calls this a ‘task set configuration’). But this won’t work if the problem requires a reorganisation, a new configuration. Indeed the current configuration acts as a constraint on new thinking because it inhibits access to new networks which aren’t part of it.

When you stop focusing and enter a state of ‘transient hypofrontality’ the problem remains on the fringe of your consciousness in the form of a much-weakened task set configuration, prompting your unconscious mind to keep working on it. But, because the configuration is weaker, it’s no longer a constraint so the Default Mode Network can make new associations and try out new links which were previously blocked off. The brain can also notice new stimuli which would previously have been classified as irrelevant – it’s sensitised to these because the weakened task set configuration is still there. The quote from the Robert Galbraith thriller, which I was reading when I should have been working, is a minor example of just this. I had been thinking about creativity and insight, and it struck me as a perfect example of the nature of a new idea that works.

As I said earlier, transient hypofrontality occurs when we relax, stop thinking about anything challenging, and are absorbed in the moment. R.E.M sleep, with its vivid dreams, is the gold standard for incubation: memories are consolidated during sleep, bringing out their implications and associations and connecting them to the existing networks in your brain. There is a myriad of stories of new ideas appearing in their creators’ minds when they are sleeping, dozing, or in an otherwise altered state of consciousness, and creative people become adept at inducing these states and capturing what they produce. In one famous instance, Paul McCartney, woke one morning in 1964 with a soulful melody playing in his head. ‘The tune itself came just complete, came just out of a dream’, he explained. He immediately got up and went to the piano - next to his bed for just this reason - and played through it. He added words later to create the song, ‘Yesterday’, one of the greatest pop songs of the twentieth century.

Baths and showers involve a period of sensory deprivation – nothing much to see or hear– providing the perfect conditions for transient hypofrontality, and there are many accounts of their beneficial effects on the creative process. The late Douglas Adams, author of ‘The Hitchhiker’s Guide to the Galaxy’, was reported to have as many as six baths a day when he was suffering from writer’s block (which was often; he is also famous for his complete inability to meet deadlines). The screenwriter, Mark Gatiss (‘Dr Who’, ‘Sherlock’) uses baths in a similar way: he calls them ‘think baths’. Bathing is of course also relaxing and pleasurable. A number of lab studies have shown that we have more ideas when we’re in a good mood. And a study in real life settings produced a similar finding: a strong correlation between the subjects’ reports of increased happiness on certain days and important breakthroughs at work in the following day or two. Broad associative thinking requires relaxation. In contrast, a sense of threat and anxiety narrow your focus and trigger the involvement of your conscious mind to deal with the problem.

Sometimes you can sense that you’re on the verge of a breakthrough. Illumination may be preceded by what Wallas calls intimation, a sense that your unconscious mind is about to produce an insight. It’s a common trope of detective fiction – the detective knows that his unconscious mind has found a crucial link but can’t quite grasp it. The function of this message is to tell us to focus inwards, to block out external stimulation and provide the right conditions for the insight to reach the conscious mind.

Verification

‘And as imagination bodies forth
The forms of things unknown, the poet’s pen
Turns them to shapes, and gives to airy nothing
A local habitation and a name’

Shakespeare, A Midsummer Night’s Dream Act V Scene 1

As we know, the unconscious mind looks for patterns and correspondences - it doesn’t ‘think’ analytically like the conscious mind. Einstein said,
‘These thoughts did not come in any verbal formulation. I rarely think in words at all. A thought comes, and I may try to express it in words afterward.’
The conscious task of verification is essentially one of translation from the unconscious vision into a series of propositions expressed in words or numbers a drawing or design or a thought-through course of action, to see whether the idea can survive in the concrete world of the conscious mind, whether it can be implemented or delivered.

This encounter with reality is a vital stage of the process. As Wallas says,

‘There are thousands of idle ‘geniuses’ who require to learn that, without a degree of industry in Preparation and Verification, of which many of them have no conception, no great intellectual work can be done….’

But there is a risk. If you attempt to do this too soon, before the unconscious process is complete, the half-formed idea will evaporate and may be impossible to recapture. And, of course, there is always pressure to keep checking progress in an organisational setting where tasks have to be completed to deadlines.

So, what lessons can we draw from this analysis?

In brief:

New ideas are formed from the components already available to the unconscious mind of the creator, so it’s important to spend time stocking your unconscious library and learning the language of your craft by absorbing the work of other people, and broadening your experience in any way you can

New stimuli will spark new thoughts: this is why we are often at our most productive after a holiday

Incubation and illumination require time and quiet: they take place in an individual mind, not in meetings. They won’t happen in presenteeist cultures, whether in person or digital, and can’t take place to order to tight deadlines

Solving a complex problem may take weeks, months, or years and involve many iterations of the creative process. If you keep the problem in the back of your mind and return to it every so often you may find you’ve made progress in the interim. Try not to get frustrated. Record your thoughts and return to the issue in due course

People will be more creative if they are relaxed and not anxious

You risk strangling new ideas at birth if you try to evaluate them too soon

And finally, creative people tend to have certain personality characteristics. You need to understand these if you are to survive as a creative, especially in a commercial context, and/or get the best from the creative people you work with. This is the subject of my next article.

Sources
Timothy A Allen and Colin G DeYoung, Personality Neuroscience and the Five-Factor Model, The Oxford Handbook of the Five Factor Model of Personality, Oxford Handbooks Online, 2016
Marcus E Raichle, Creativity and the Brain’s Default Mode Network, Secrets of Creativity (eds Suzanne Nalbantian, and Paul M Matthews, OUP 2019
Roger E Beaty, Mathias Benedek et al, Creativity and the Default Mode Network, Neuropsychologia, 2014; Nov 64: 92-98
Arne Dietrich, How Creativity Happens in the Brain, Palgrave Macmillan 2015
Graham Wallas, The Art of Thought, Solis Press 2014 (first published 1926)
Matthew Walker, Why We Sleep, Penguin 2017
John Kounios and Mark Beeman, The Eureka Factor, Penguin 2015
Steven Johnson, Where Good Ideas Come From, Penguin 2011