Fish Food 654: What is AI still not good at?

Human vs AI, AI Mode, Meta's personal superintelligence vision, how students use ChatGPT, and strangers on a bench

This week’s provocation: On the limitations of AI

As much as everyone is getting excited about what AI can do, it’s also important to understand the limitations of AI and where there remains a gap between machine and human capabilities. I wrote a LinkedIn post a while back on this but I wanted to dive a little deeper on this topic as I think this is key to our understanding of how to work better with the machines towards better outcomes for ourselves, and also because these are differences which are actually worth celebrating. So this is my take on the most significant limitations of AI, and those that will be around for a good while. For ease I’ve divided them into three main areas.

1. Embodiment and messy reality

• Lived, embodied common sense. The way in which humans and AI learn is fundamentally different and this results in some important differences in how they understand the world. Whilst LLMs mostly learn from patterns, probability and ‘what usually follows what’, humans learn through a combination of lived experience, what we sense, what we observe, and what we study. We build models of the world from years of trying things out, experiencing how things feel and the things we choose to pay attention to.

  Another way of thinking about this is that humans anchor concepts in sensation, action, understanding and consequence. AI anchors them in statistical co-occurrence. If a human ‘knows’ what a wet glass feels like and what happens if you drop it, an LLM will ‘know’ the phrases that usually appear with wet glass. This means that humans can generalise safely in messy, novel situations, whereas AI can generalise quickly across patterns but it can misfire when the surface form looks familiar yet the physical or social stakes differ.

• Improvisation in truly novel physical spaces. Lived experience for humans means that we can improvise better, particularly in the physical or artistic world. We can MacGyver stuff. But more than that, it means that we know when to stop, when it’s better to wait, and when to say ‘I don’t know’. AI may be good at staying consistent once a narrow objective is fixed, but it can become brittle in unstructured environments with odd constraints. It’s incentivised to respond and so refusal and ambiguity are hard for models to calibrate (hence hallucinations).

  Lived experience also means that even as a child we can see something novel and make a good guess as to its purpose or application. Whilst AI is improving in this area, it still largely needs lots of examples or systematic self-supervised learning to figure it out.

• Ethnographic noticing: the small, telling detail. Humans are inherently curious, AIs are not. As a human you can pick up on tiny details that infer everything, but an AI often misses the one off-pattern signal that could redefine the problem (unless specifically prompted to do so). A person can see one odd example and adapt by analogy but an AI often needs lots of similar examples or clever augmentation to internalise the pattern. Innovation often happens at the edges but edge cases are ‘expensive’ for AI in that data is scarce or nonexistent, it may require human experts to decide what ‘correct’ is, and fine-tuning for small slices of data is inefficient. Humans run on predictive processing, meaning that when reality violates expectation, it feels salient. We’re wired to seek stories, so anomalies trigger curiosity, analogies, and reframing. Lived experience gives us the context to judge if the odd thing is noise or a clue. And let’s not forget that many of us are motivated by the idea of finding ‘the thing that everyone missed.’

• Repair, maintenance, and care. A final point about messy reality. Humans understand the value of keeping things going, whether that’s patching things up, or nurturing a relationship, or what living things need to live well. AI is largely built to produce and predict, not to tend and maintain. Societies run on maintenance work that is often undervalued and under-automated.

2. Values, goals, and moral agency

• Choosing goals (not just optimising given ones). One of the hardest parts of strategy is deciding which game to play, so that (not that I want to mix metaphors) we are not climbing the wrong hill. Humans can originate objectives, reinterpret briefs, and understand when a goal is wrong. AI is able to optimise well against objectives it is given but it rarely challenges the brief, renegotiates the mission, or questions whether success criteria are ethical or meaningful.

• Negotiating values and norms. As humans we juggle conflicting values (fairness against efficiency, risk against speed) and can reach ‘good enough’ compromises that others accept as legitimate. AI lacks skin in the game or moral standing. It can simulate ethics talk but cannot own the consequences. Another way of putting this is that legitimacy is earned socially, not computed.

• Trust-building over time. Humans build reputations, take responsibility, apologise in credible ways when things go wrong to repair relationships. AI struggles to be truly accountable, feel remorse, or bear the costs for mistakes. An example of the implications for business of this - users might forgive fallible humans that they trust, but not opaque systems that can offer no recourse.

• Moral courage and doing the right (but costly) thing. Humans sometimes sacrifice comfort for principle when making decisions. Sometimes we stand up for what we think is right, even if it’s not the easiest option open to us (whistleblowing, for example). AI has no skin in the game. It doesn’t feel fear. It has no understanding of what it means to be courageous and principled. So it cannot take ‘risks’ in any meaningful sense.

3. Strategy, time horizons, and exploration

• Causal reasoning in open systems. Humans are uniquely good at navigating messy, multi-factor situations and generating new options to try. Whilst LLMs are excellent at pattern matching, they are not naturally good at counterfactual thinking outside of narrow benchmarks (unless prompted to do so by a human of course).

• Long-horizon projects and exploration. Humans keep on pursuing hazy, multi-year visions, adjusting along the way despite uncertainty. AI struggles when rewards are delayed, feedback is ambiguous, and the target keeps moving, which means that it can struggle with long-horizon projects. Equally people are uniquely good at wandering, daydreaming, following weird hunches, noticing anomalies, and stumbling into breakthroughs. AI, less so.

• Holding multiple incompatible frames at once. The ability to empathise is a great human strength. We can jump from thinking like an economist, to viewing the world like a poet, to following how a scientist would solve a problem. LLMs can output any frame (particularly when prompted to do so), but they struggle to truly juggle and reconcile them dynamically. Breakthrough ideas often live in the seams between frames.

• Institutional navigation and coalition politics. As we all know from bitter experience making an idea a reality often involves bargaining, trade offs, influencing without authority, convincing, persuading, building coalitions, getting over setbacks (as Dr Helen Bevan once said ‘all of my good ideas are battles’). AI lacks embeddedness in the messy power structures and informal networks that comprise most organisations.

4. Culture, meaning, and aesthetics

• Thick context, subtext, and shifting cultural nuance. Humans can (usually) read the room. They can sense power dynamics, catch irony, and know when silence speaks louder than words. Models miss intent which is hidden in tone, timing, status relations, or local history. An AI model could never make a good standup comedian because it lacks the subtlety of observation, storytelling, timing, and wit.

• Taste, judgement and humour. Speaking of humour, AI may be able to deliver puns and templated jokes, but real comedic edge involves social courage and consequence. Good humour plays with norms, timing, risk, and shared pain or embarrassment. Humans use humour to develop shared understanding and bonding. Just as AI could never be a standup comedian I’m going to say that AI could never come up with iconic concepts like ‘Happiness is a cigar called Hamlet’, ‘You’ve been Tango’d’, John Smith’s ‘no nonsense’, Pot Noodle’s ‘The Slag of All Snacks’, Reebok’s ‘Belly’s Gonna Get Ya’, or even Marmite’s ‘gene project’.

  Similarly, taste is a uniquely human thing. As Nitin Nohria in The Atlantic put it, taste is ‘judgement with style’. The English philosopher John Locke (in his essay Concerning Human Understanding) distinguished between wit and judgment, viewing them as distinct mental capabilities. Wit, according to Locke, involves the quick and creative combination of ideas to form ‘pleasant pictures and agreeable visions’. Judgment, on the other hand, is the ability to carefully distinguish between ideas. Essentially, wit is about finding connections and creating novel associations, while judgment is about discerning differences and establishing clarity. In a world awash with slop, taste and judgement is a defining human skill that AI cannot replicate.

• Identity continuity and personal narrative. An LLM can write a story, based on an approximation of thousands of other stories, but it has no enduring self to stitch genuine memories, roles, and aspirations together into a compelling narrative. It lacks a coherent self that people can truly relate to.

• Making meaning out of suffering and finitude. Many creatives and artists would argue that the creative process is an act of suffering from which pearls form (from ‘this idea is terrible!’ to ‘this is great!’). As consumers of creative outputs we can intuitively sense the struggle when we see a work of art that an artist has wrestled to create. It breathes authenticity in a way that AI generated content never will. AI can echo creative outputs but it does not share the condition of being mortal and vulnerable, nor the validity of lived experience.

Some final thoughts on some broad, overarching themes that have emerging for me when I’ve been writing this:

• Human vision + machine iteration: Humans hold the long, fuzzy horizon whilst AI stress-tests scenarios, explores parameter space (systematically looking at the values of parameters within a defined range to identify the optimal configuration) and keeps the set of options wide.

• Outcomes + optimisation: Humans define the goal, describe aims, trade‑offs and point AI at optimising within those human-set boundaries, or proposing candidate metrics for humans to sign off on. This is not to say that AI can’t play a role in goal-setting, but defining outcomes should absolutely be within the human domain.

• Curiosity+ support: Create the space for human wandering, anomaly hunting, and exploration, and then use AI to catalogue, cluster and connect what they find.

• Taste-makers + generators: Put humans in the role of curator and editor and use AI to generate divergent options, variations, modifications. Use feedback from curators to retrain the generator.

• Social nuance + simulation: Let humans steer in situations where feelings, social norms, and context matter. Use AI to pre‑sort, simulate, and surface blind spots.

Moravec’s paradox is the concept that tasks that humans find cognitively challenging (like playing chess or solving mathematical problems) are relatively easy for AI to replicate, while tasks that humans do effortlessly (walking, recognising faces, or navigating through space) are extremely difficult for AI to master. Moravec posited that the ‘easy’ capabilities which humans do without conscious thought are ingrained from millions of years of evolution and natural selection. The point is that recognising the inherent strengths and weaknesses of both humans and AI enables us to design systems which get the most from both.

I’m not an AI evangelist, and I’m not an AI sceptic either. Treading this middle ground is not always the easiest (or the most fashionable) thing to do but it does mean that we are forced to think more carefully about why and how artificial and human intelligence are different. And that’s essential if we are going to put the two together in a meaningful way.

Rewind and catch up:

Separating fads from trends, and second and third order effects of AI

Using synthetic personas and research to explore ideas

Using AI as a thought partner

Have we got personalisation all wrong?

---

If you do one thing this week…

Google’s AI Mode launched in the UK this week. With this, and AI Overviews (AIOs), it’s clear that search behaviour is already changing a lot. Google have said that AIOs are now shown to 2Bn monthly users but also that AI boosts search volume (perhaps because of different types of questions asked and more conversational search). Numbers vary but Sparktoro suggest up to 60% of searches are now zero click and recent Pew Research found that users are less likely to click on links when AIOs are shown on the results page. It appears as though informational searches (rather than more specific queries) are much more likely to have AI Overviews.

It feels as though we are undergoing a quite fundamental rewiring of how people access information on the web. According to SimilarWeb for example, organic visits to news sites fell from 2.3 billion to 1.7 billion between mid 2024 and May 2025, a 26% drop.

---

Links of the week

• ‘Meta's vision is to bring personal superintelligence to everyone.’ Yikes. Sonja Grimmer applied some critical thinking and annotated the Zuckerberg announcement for us all (HT John Willshire)

• I’m still playing around with ChatGPT agent but so far it appears to be magical at some things but less good at others. More on that later, but in the meantime Will Francis did a helpful short video on the Claude integrations feature which enables you to integrate it with a tonne of other apps and then ask it to do all kinds of things.

• YouTube is now the number one destination for viewing TV for anyone Gen Alpha, but perhaps more telling is that viewers aged 55 and over watched almost twice as much YouTube content last year as they did in 2023. Lots of interesting (UK based) stats in Ofcom’s latest Media Habits report

• Jeremy Ettinghausen (friend of ODF) was given unfettered access to three students’ ChatGPT accounts and he analysed 18 month’s worth of prompts. What emerged was just how embedded its use was in their lives, and the breadth of things they used it for, from deep academic work, to being a therapist, to completely mundane enquiries (‘how long does dry cleaning take?’)

• WARC’s annual Future of Strategy survey (always interesting) is now open, so if you are a planner, strategist, consultant data analyst or even a designer or copywriter it’s worth completing.

---

Quote of the week

Arvind Narayan, and the opening paragraph to his book on AI Snake Oil.

---

And finally…

This week I came across the Strangers on a Bench podcast which is a lovely idea - host Tom Rosenthal talks to random strangers on park benches which often leads to some pretty deep and fascinating conversations.

Weeknotes

This week I was mostly staying in London running a three day leadership workshop. Whilst there I popped into The Wallace Collection which was right opposite my hotel and has a fascinating collection of artefacts including lots of Canaletto’s (is the plural of Canaletto Canaletti?), and also the Photographer’s Gallery to see the Dennis Morris exhibition (recommended). I’ll be spending two weeks on the road overseas from this weekend (lots of travel at the moment but hey, you gotta go where the work is, right?).

Thanks for subscribing to and reading Only Dead Fish. It means a lot. This newsletter is 100% free to read so if you liked this episode please do like, share and pass it on.

If you’d like more from me my blog is over here and my personal site is here, and do get in touch if you’d like me to give a talk to your team or talk about working together.

My favourite quote captures what I try to do every day, and it’s from renowned Creative Director Paul Arden: ‘Do not covet your ideas. Give away all you know, and more will come back to you’.

And remember - only dead fish go with the flow.