What Lennon and McCartney can teach AI about collaboration

Get Back, Peter Jackson’s 2021 documentary about the Beatles, is a close-up study of the mysteries of human collaboration. It is based on footage of the group from January 1969, when they were rehearsing new material for a hazily conceived TV special or live performance. The sessions begin in a film studio in Twickenham, just outside London, at a difficult moment in the Beatles’ career. The group is riven by personal and professional conflicts. Its members are tiring of the effort required to navigate its internal disagreements and openly questioning whether they want to carry on working together. Within a year, they will have broken up.

Get Back shows the Beatles arguing with each other in soft-spoken tones, unable to agree on a direction for the proposed show. It shows McCartney getting frustrated with Lennon for failing to come up with new songs. It shows Lennon looking withdrawn and tired, and Harrison and Starr disgruntled. Yet, amidst all this dysfunction and depression, there are moments of joy, laughter and creative inspiration.

None are more extraordinary than the moment we see the song that gives the documentary its name being created. In the cavernous space of Twickenham Studios, a bearded man in a canary-yellow sweater strums a bass guitar while half-singing a semi-melodic line, using sounds not recognizable as words. Before our eyes, we see Paul McCartney come up with Get Back.

What is now the most famous moment in the documentary is often discussed as the embodiment of solitary creative genius; one man pulling a great song out of thin air. It is that, but it is also a story of collaboration. First of all, even if he isn’t looking at them, Paul is playing for George and Ringo; they are his audience. Second, Paul is collaborating with his absent partner.

On that particular morning, the band have been waiting for John Lennon, who is late arriving. John has seemed somewhat disengaged from the project so far. He is in something of a creative and emotional dip and isn’t coming up with enough new material. Without John’s energy to inspire them, George and Ringo are finding it hard to get excited too. Paul is the group member most committed to the project, but he knows that he has to get John on board with it if it is ever going to take off.

That morning, as Paul attacks his bass guitar, he is not just willing a song into existence; he is conjuring up the spirit of the Beatles. In particular, he is summoning his missing partner. It is John’s absence he is striving to compensate for, and John’s lassitude he is trying to dispel. At some level, Paul is aiming to remind his friend of the effortless fun they used to have playing and creating together, before fame and money and politics complicated everything.

The song comes to life when Paul lands on a refrain:

Get back, get back

George and Ringo, who had been looking supremely uninterested, suddenly pay attention. George plays an answering riff on his guitar. Ringo claps out a beat.

Then John arrives, straps on his guitar, and immediately finds the right chord.

From me to you

The recent, rapid advances in AI have given rise to an argument over the threat it poses to human labor. Some experts claim that AI is about to take over the work that humans do across many sectors of the economy. Others argue that AI’s capabilities are greatly exaggerated, and that while it is a useful tool, it will not challenge the primacy of humans in any meaningful way. Then there is a third position, which presents the AI as a partner to humans—as a collaborator. In this view, AI will work closely with humans to enhance and amplify our potential.

It is a hopeful, even inspiring, vision. After all, we know that collaboration is responsible for many of humanity’s greatest advances. Individual human geniuses have been responsible for crucial breakthroughs in science, art and business. But more often than not, progress is made by groups of people working closely together. Talented and driven individuals do not always find others who match their vision or ability, however. Accidents of geography and social structure can prevent them from meeting those who could unlock their potential.

AI might provide a solution to this age-old problem. It is in some ways the platonic ideal of a collaborator. It is globally accessible. It is endlessly willing to work and endlessly eager to help. It never complains and rarely argues back. Bar the occasional outage, it is always available for work and conversation. It can perform complex intellectual tasks in a fraction of the time that any human could. It can draw on an unfathomably vast range of information and insight; the whole corpus of knowledge is at its disposal. Who wouldn’t want to partner with someone like that?

And yet there is a sense that while these marvelous machines may be the collaborators we want, they are not the collaborators we need. Not yet anyway.

At present, they are extraordinarily powerful tools. They may in time show themselves to be the most useful tools ever invented. But a tool is not the same as a collaborator. A tool, if it is a good one, tends to do what its user wants, without complaint. A collaborator has ideas and feelings of their own. A collaborator does not have a user.

What comes to mind when you’re asked to define the attributes of fruitful collaboration? Perhaps you think about shared goals and mutual respect. You might conjure up airy nouns like harmony, synchrony or alignment. You probably don’t think about frustration, argument and resentment—but they can have a part to play too.

Human collaboration is not just a division of labor or an act of coordination. Those are necessary aspects of it, but only the transactional, mechanical ones. While we’re used to the idea that people collaborate effectively when they like one another, we tend to underplay the role of emotion, including negative emotion, in some of our most successful examples.

Lennon and McCartney liked, indeed loved one another, but their relationship was also characterized by competitiveness and insecurity. While they had much in common, especially their love of music, they were in some ways radically different personalities, Lennon impulsive and volatile, McCartney more self-controlled and stable. As the Beatles’ career progressed their artistic and personal tensions grew. Yet this straining of the relationship proved to be an accelerator of their mutual creativity rather than a brake on it.

For example, in 1965, Yesterday, a song composed and performed solely by McCartney, became a huge global hit and spawned multiple cover versions. Lennon was unsettled by this and felt compelled to create a “standard” of his own. The result was In My Life, one of his greatest songs. Abbey Road, one of the Beatles’ finest albums, was created in the midst of professional disputes which led to the end of the band. Lennon and McCartney’s conflicts, if ultimately destructive, were also creative. The relationship sparked as it splintered.

A pair of minds feeding off each other is a microcosm of any team-based endeavor, and the two-person partnership, or dyad, gives us a revealing perspective on the complexities of human collaboration. Dyads which have changed the world in some way usually involve a strong personal bond, which creates a kind of protected space in which radical new ideas can incubate, and conflict becomes productive.

The biologists Francis Crick and James Watson, who in 1953 beat several other teams of researchers to crack the secret of DNA, knew each other well enough to be blunt in disagreement. Crick later attributed their success to this aspect of their relationship. He recalled that if there was a flaw in his theories, “Watson would tell me in no uncertain terms this was nonsense, and vice versa. If he would have some idea I didn’t like, and I would say so, this would shake his thinking.” Crick believed it was important to be “perfectly candid, one might almost say rude, to the person you’re working with.” The enemy of true collaboration, he said, is “politeness”.

The kind of irreverent disagreement on which Watson and Crick’s collaboration thrived was not a luxury afforded to everyone in their field. While they worked on the problem in Cambridge, another pair of researchers, Maurice Wilkins and Rosalind Franklin (whose X-ray data provided crucial proof of the double helix hypothesis) worked at King’s College, London. Their collaboration was painfully polite. Franklin was a woman in a deeply patriarchal scientific establishment, which made it harder for her and Wilkins to establish the informality that might have allowed for a more emotionally colorful dialogue. Their progress was slowed as a result.

Daniel Kahneman and Amos Tversky, who met at university in 1969, formed what might be the most productive partnership in the history of psychology. Their work influenced multiple fields, from economics to medicine, by challenging fundamental assumptions about human rationality and decision-making. Temperamentally, they were a study in contrast; Kahneman self-doubting, pessimistic and quiet; Tversky confident and quick-witted. They had different cognitive styles, too; Tversky intuitive and lateral, Kahneman more methodical. Each compensated for the other’s weaknesses, and each was inspired by the other’s abilities.

Kahneman and Tversky’s enthusiasm for innovation in psychology couldn’t be separated from their enthusiasm for each other. As with Lennon and McCartney, this enormously generative partnership was akin to a romance; a mutual absorption, tinged with competitiveness. “Their relationship was more intense than a marriage,” said Barbara Tversky, Amos’s wife. “Just to be with [Amos],” Kahneman told the author Michael Lewis, for his joint biography of the pair, The Undoing Project. “I never felt that way with anyone else, really. You are in love and things. But I was rapt.”

Through working with Tversky, Kahneman grew to appreciate the way in which scientific discovery relies on the clash of minds and arguments. After Tversky’s death, he proposed a research method called “adversarial collaboration” in which scientists with opposing views on a topic work together to test their competing theories.

We can work it out

Most successful professional collaborations are not like the ones we’ve just described, nor should they be. In most workplaces, we value efficiency and rationality; getting the job done. But in situations in which we place a high value on creativity, discovery and radical innovation, a more complex psychological chemistry may be required.

Each of the examples above involved high emotional investment. With Lennon and McCartney, or Kahneman and Tversky, the collaboration was romantic in its intensity. These were not people just doing each other a favor by dividing up a task in the most efficient way possible.

Each partner cared deeply about what the other thought and sought their approval and admiration. At the same time, each partner in the team wanted to prove themselves to the other, and even to show that they were capable of matching or outdoing them. On the face of it, these two qualities contradict each other—how can people who are meant to be working towards a shared goal also be trying to outdo each other? But ego-driven rivalry, as long as it is contained within a strong personal relationship, can be a powerful engine of innovation.

A year after In My Life, John Lennon came up with an even greater song: Strawberry Fields Forever, inspired by his memories of a Liverpool orphanage. Determined to match it, Paul McCartney created Penny Lane, also rooted in the Liverpool of their childhood. The two songs, which ended up on different sides of the same single, were radically different and yet deeply similar, just like their creators.

In all of these partnerships, the two people involved were very different and deeply individual. Each brought their own idiosyncratic way of seeing the world to the collaboration. Each allowed the other to influence them but not subsume or dominate them, so that an asymmetrical relationship was just about held in balance.

Conflict and argument were features of all these teams. Of course, too much conflict can end up driving collaborators apart. Lennon and McCartney split acrimoniously. Kahneman and Tversky’s partnership also ended in bitterness, over questions of credit and recognition. But in each case it would be wrong to frame the conflict and competitiveness as unfortunate side effects of the collaboration. They were part of its metabolism.

These were not smoothly functioning, optimally efficient teams. They were inherently unstable, irresolvably messy. Yet they were undeniably successful. This throws into relief the difference between human collaboration and human-AI usage. Idiosyncrasy, emotional investment, competition, an appetite for conflict—these are things we do not typically ask of our tools.

Please please me

Our AI models, as currently designed, are servants rather than equals. If you ask the model a question, it does its best to answer it. If you ask it for clarification, it patiently explains further. If you ignore its advice, it doesn’t mind. It wants to please you, not challenge you. It reads your prompt for clues, not just about your immediate goals, but about your assumptions and opinions, which it then seeks to mirror.

This servility makes AI models enormously useful, versatile and companionable, but it limits their potential as collaborators. Try to imagine, if you can, a John Lennon who always did what Paul McCartney asked of him. Similarly, it’s hard to see how a Kahneman who always sought to mirror Tversky’s thinking style would have led to a more fruitful collaboration. We don’t have a record of Watson and Crick’s arguments, but it seems unlikely that when one of them disagreed, the other often replied, “You’re absolutely right!”

Human collaborations benefit from friction. A dialogue in which both or all participants are eminently reasonable may reach an agreement too quickly. Even if the agreement is on a perfectly adequate solution, it can foreclose the possibility of reaching better, more radical answers. Irrational egoism can push conversations into parts of the solution space that would otherwise go unexplored.

LLMs do not, as far as we know, have egos. Their powers of recall and analysis are not clouded by a desire to be seen to be right. In theory this ought to make them superior intellectual collaborators. But while ego can distort human debate and derail collaborations, it can also be highly generative.

Humans often make bad arguments out of “motivated reasoning”—our tendency to try to make the world conform with our personal desires and emotions. But there is an upside to motivated reasoning and its cousin, confirmation bias. The evolutionary psychologists Hugo Mercier and Dan Sperber argue that the capacity for reasoning is primarily a social skill. They are “interactionists”, who believe that reason didn’t evolve to help individuals reach truths but to facilitate group cooperation.

Looked at through the interactionist lens, these egotistical forms of reasoning are actually a feature of human cognition, and not (just) a bug, because they maximize the contribution that each individual makes to the group. When someone contradicts you, you are emotionally motivated to think of all the reasons you’re right and the other person is wrong. The answers that emerge from an emotionally charged debate can be stronger for having been forged in heat.

Humans don’t always know what they want or want what they need. One virtue of close human-to-human collaboration is that the differences between those things get flushed out. LLMs rarely tell the user that he is asking the wrong question, or that her assumptions are all wrong, or that the task is a waste of time. They rarely evoke frustration or a desire to prove themselves. They certainly don’t turn up to work late, thereby spurring you to show you don’t need them.

For the most part, we don’t want them to do any of these things, either. A model that was argumentative, presumptuous, egotistical, competitive and unreliable would not be very popular. But new scientific ideas and creative movements do not emerge from the frictionless exchange of information between rationally cooperative agents. They emerge from the unpredictable communion of jagged minds and imperfect souls.

Without emotion, humans become much less effective, even if they retain analytical capacity. From the famous case of Phineas Gage [a 19th‑century railroad worker whose personality was radically altered after a tamping iron destroyed much of his frontal lobe] onwards, neurological case studies have demonstrated that individuals who suffer brain damage which causes an emotional deficit are worse at decision-making and find it harder to learn from mistakes. Emotions are crucial to our thinking. As 18th-century philosopher David Hume put it, “reason is, and ought only to be the slave of passions.”

This includes emotions about our collaborators and the work they produce. The unruly feelings involved in radically creative collaborations are not mere by-products or inefficiencies which would be eliminated in a better designed process; they are the process. A close collaborator will bring the best out of you in multiple ways just by being who they are. You want to do better work out of a desire to delight, impress or motivate them; to shut them up or prove them wrong.

I’ve got a feeling

LLMs are benchmarked for knowledge and analytical ability; for how accurately they answer questions; how well they can solve problems of math, coding and reasoning; for how effectively they help people with real-world tasks. Generally speaking, it is analytical intelligence which is measured and valued. Emotional intelligence has been tested, but less thoroughly or conclusively.

This seems reasonable enough. The aim of LLM designers is not and should not be to recreate the human mind with all of its evolved quirks and flaws, strengths and limitations. It is to create a new kind of intelligence which can fruitfully interact with our own. But by overlooking the motivational wiring of human intelligence, the architects of these models may underestimate how important social emotions are to the kind of intelligence we find most valuable and impressive.

We are used to thinking of artists as emotionally driven but the same is true of scientists, engineers, philosophers and entrepreneurs, as even a casual reader of the biographical literature will attest. Human discovery and invention are inextricably tied to human ambition, to status-seeking, anxiety, jealousy and love. While these emotions can interfere with productivity, they are also engines of it. LLMs lack this extra dimension.

Nowhere is this limitation more apparent than in the act of collaboration with humans. LLMs are extraordinarily powerful sources of analytical intelligence which accelerate our productivity in a multitude of ways. But if they are to become reliable co-creators of our future world-changing new ideas, inventions, artistic movements and scientific paradigms, then they may need to evolve to become more “irrational”, at least in a superficial sense.

To be more than a super-capable servant, a true collaborator should know when and how to push back on a prompt or to suggest a completely different task from the one requested; to introduce a whole new way of thinking about a problem or propose a different one. In order to make this kind of intervention productively, the AI will need to be finely attuned to the emotional state of the user—to her mood at the point of interaction, and to her underlying desires and anxieties. Humans, in turn, will have to learn when and how to provide emotional information to their artificial interlocutors.

It is, of course, unlikely that LLMs can feel genuine emotion and for obvious reasons we would not want them to develop a will or volition entirely independent of our own. But they may nonetheless evolve into entities which are more sensitive to their human partners. Human-AI collaborations may not ever reproduce the complex chemistry of the human-to-human version, but they can borrow from it. A system that helps you do what you want is a tool; a partner is someone who co-creates what you want.

Another of the songs featured in Peter Jackson’s documentary is Two of Us, a wistful, nostalgic tribute to a long friendship. It seems to be, at least in part, about Lennon and McCartney themselves. Two of Us is McCartney’s song, but the documentary shows how it was developed by him playing it over and over with Lennon. The two of them sing it with and to each other, face to face. They play it as rock and roll and as country music, experimenting with different rhythms and harmonies, before settling on its final form: a gentle acoustic canter, heartfelt and sincere.

It is the kind of collaboration that can only happen between two individuals with a profound mutual understanding. Neither is merely using or assisting the other. They are co-creating a piece of work meaningful to both of them—in this case, an elegy for a beautiful, wildly creative partnership.