“Cognitive Debt is where you forgo the thinking in order just to get the answers, but have no real idea of why the answers are what they are.”
Artefacts Newsletter #247, April 25th, 2025
I first wrote about the idea of Cognitive Debt in April 2025. I was drawing together my thoughts on the implications of unleashing Large Language Models (LLMs) at scale. Shortly afterwards I wrote a blog post that seemingly spoke to something about the modern workplace for a lot of folk.
People would get in touch to tell me that Cognitive Debt was, for the first time, giving them a language to pithily describe the feelings, experiences and discussions that were playing out in organisational settings.
Therefore I have created this page as an ongoing reference point.
Firstly, it is an updated articulation (written in early October 2025) of what I mean when thinking about Cognitive Debt.
Secondly, it is a collection of examples found over the summer which I have tagged along the way as Cognitive Debt. Once you have a name for a thing, you recognise it more readily when it rears its head. These are not conclusive proofs about the nature of using LLMs at scale, but instead jumping-off points to think about their application in different circumstances.
Finally, at the bottom of the page, there is the backstory to this concept of Cognitive Debt and how I got to it by playing off Ward Cunningham’s idea of Technical Debt; something a company accrues wittingly or unwittingly, and if left unchecked can add up to serious issues in the longer term.
I shall keep adding more examples for the rest of this year at least. Whilst we may only have a quarter of 2025 left, it feels that there is a lot that might change before the year is out. As Gramsci put it nearly a century ago, “The old world is dying, and the new world struggles to be born: now is the time of monsters.”
Please do get in touch if you have examples of Cognitive Debt to share, want to talk about the implications, or even discuss how this approach can help you.
“I as yet know nothing.
The truth does not come without a tax of effort.”
– Hercule Poirot
What is Cognitive Debt?
I think of Cognitive Debt as the gap underneath a seemingly plausible answer where the thinking should have been. It could be an email without empathy, a presentation that’s missing the point, customer service without the care, or a strategy without the sensemaking.
Cognitive Debt is where you forgo the thinking in order just to get the answers, but have no real idea of why the answers are what they are.
What does the automatically-scripted email do to your relationship with the recipients? Which critical point omitted in a presentation could send your team in the wrong direction? Which subtle detail in a customer’s reaction does your chatbot miss, leading them to leave the next time their renewal comes around? And which mix of research inputs, analysed and synthesised by people rather than processors, could really create a long-standing strategic advantage for your firm?
When we use an LLM for convenience and speed, and forgo the associated thinking around it, what is it that we exchange? How and when might we need to pay that back?
This is Cognitive Debt.
It is all of the holes under your business where the humans would have been.
Of course, businesses are already full of technology and machines that are doing what people once did, both manually and intellectually. What we need to think about carefully is how these technologies are different, and what effects might they have as a result?
When you start thinking in terms of Cognitive Debt, and consider exchanging the speed and convenience of an LLM for the critical thinking power of humans, you can start to make reasonable value judgements around both the immediate and longer term implications of the exchange.
LLMs don’t think like humans (to be clear, they don’t *think* at all). Dropping in an LLM to do the thinking work that people do is not replacing ‘like-for-like’. They are not ‘extra colleagues’, or ‘very-bright interns’, no matter what people claim. As Gergely Orosz succinctly puts it:
“99% of people using LLMs forget how these things work: they are advanced probability machines. They generate the next most likely token (word) based in the input and their training. Under the hood, it’s a giant matrix multiplication that has eerily good output.”
The exchange we make when accruing Cognitive Debt is fairly straightforward. We get answers quicker from an LLM, but we have not employed our own thinking capabilities – questioning, analysing, interpreting, evaluating, judging. It makes it hard to see how these qualities may have differently informed the output we have in front of us.
But the temptation to use the readily available output can often prove too much. Andrew Taylor made an observation earlier this year that has lived in my head rent-free ever since:
“People have a kind of Gell-Mann amnesia towards LLMs. They can think “AI is bad at things I am good at and know about, but good at things I can’t judge because I’m bad at them”…”
It really helps to have a certain level of expertise in the topic you use an LLM for, because not only can you spot the things that feel off, you can also start to notice the omissions and the things it has ignored. A couple of years ago, when the emergence of ‘generative AI’ was in its infancy, I wrote about how there was ‘no such thing as six-fingered text‘. Where image creation GenAI systems would regularly create images which were clearly false (hands with six-fingers), it is much harder for people to spot similar errors in text, especially if they have no expertise around the output.
And we can easily carry out tests and conduct thought experiments to help us see what we might be missing if we use an LLM instead of doing the thinking ourselves. On our own, we can create ways to make sure we avoid the common traps that come with using LLMs as part of our process.
From micro to macro
To get to the heart of the Cognitive Debt issue, we need to think bigger.
It is one thing to think about all of this at an individual level. Proponents of LLM technologies will often default to describing the benefits to a hypothetical user. The inference is that an organisation is therefore a great big collection of individual users; imagine everyone on your organisation could have these superpowers!
I believe it is more crucial to think about the concept of Cognitive Debt within the reality of how larger social settings work. The complex and dynamic structures of any kind of group will be greatly impacted by the introduction of these new technologies. The impacts will not all be positive, and not evenly distributed either.
The promise of the LLM industry is that they can replace thinking activities at scale (and do it well) within large group settings. They might make your existing people more productive by speeding up their processes, or replacing sections of them. They could stretch the existing capabilities of your teams by giving them skills they lacked before. There are even still arguments being made that these technologies could replace roles in companies altogether (though nowadays these voices seem noticably quieter).
Yet knowledge work is never an individual endeavour; the outputs are for colleagues and customers, friends and fans. Emails go to other people, presentations are vehicles to communicate ideas to a group, customer service is there to support customers, and strategy sets directions that affect hundreds, thousands, maybe millions of others. In short, we create for others.
Bringing together a view of your constantly moving people and processes, products and services, instances and incidents, and playing through the scenarios of what happens when you introduce different LLM capabilities seems to be (and should be) the major opportunity for modern management.
Because if we understand Cognitive Debt at scale within organisations, we can start to articulate what kind of thinking, people, ideas and endeavours really matter to us, and need supporting at all costs.
Examples of Cognitive Debt
After I had first written about Cognitive Debt back in the spring, I started to see examples of it everywhere. I have taken a few of my favourites just to flesh out some broader points on the implications of Cognitive Debt for some of the familiar suggested LLM use cases.
A First Draft?
People will often suggest using an LLM to create a ‘first draft from which I can do the editing’. In June 2025, a paper published from MIT by Nataliya Kosmyna et al was titled “Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task”.
It is an early publication of some initial research results (and I look forward to reading more of the follow-up research). The thing that grabbed my attention most was that the group who wrote an essay by starting with an LLM (as opposed to just writing themselves, or gathering information from search engines) were mostly unable to quote directly from their essays, despite having edited and finished them.
Implications: How do people engage with the outputs of an LLM if they haven’t done any of the thinking prior to use? Does using an LLM to get a ‘first draft’ of something runs the risk of turning someone into an unthinking copy-edit drone? The creation of a piece of work needs an intellectual home in a human brain, who can extend some ownership over it. The Cognitive Debt accrued here applies right at the start of the process – a great big nothing-burger where the initial thinking should be.
Content Generator?
It is a tough time in publishing, especially for newspapers. So with continual staffing cuts and outsourcing initiatives, the Chicago Sun-Times unfortunately published a list of summer reading recommendations that (thanks to the LLM the freelancer used) featured books that didn’t exist.
From NBC News – “To our great disappointment, that list was created through the use of an AI tool and recommended books that do not exist,” Melissa Bell, chief executive of Chicago Public Media, which runs the newspaper, said in a statement. “We are actively investigating the accuracy of other content in the special section.”
On the surface, this feels like a very plausible list of books. I’ve heard of some of the authors. I recognise some of the book names. Because some of the list is valid, it creates an impression that it all should be.
Implications: The immediate reaction of course is that ‘well, somebody should have checked’. Which is true, but also demonstrates how an LLM can just push Cognitive Debt around in a system. In isolation, it feels like it is saving time and effort. But often, it is just displacing it to somewhere (or someone) else. If your organisation needs to make sure they get things right, every time, then the consequences of dropping LLMs into workflows without due care become obvious.
Fact Checker?
In an appeal case in the UK’s Upper Tribunal in September 2025, a Barrister was referred to the regulator for having not just included a false citation from ChatGPT, but then trying to resubmit it because he had checked it again… using ChatGPT.
Amazingly, he stood up after the recess and told the court “he had undertaken ChatGPT research during the lunch break and the citation… was correct” . Rather than reading the case study proper, he assumed that the LLM could do it for him. The court found that the Barrister “did not know that AI large language models, and ChatGPT in particular, were capable of producing false authorities”.
Implications: There is a clear need for users to understand what kind of outputs an LLM will create – not statements of fact, or verified opinion from elsewhere, but the statistical relationships between words in response to the prompt. Whose responsibility is it to develop that understanding? Is there guidance offered for all employees? A quick-guide for considering the dos and don’ts? Some examples of situations in which LLM use is encouraged or discouraged? How do you help people become aware of the Cognitive Debt they might accrue as they go about their daily work?
Time Saver?
Over the summer, the US Food and Drug Administration (FDA) unveiled artificial intelligence tool Elsa, that was intended to speed up processes such as drug and medical device approval. However, like most LLMs, it would occasionally make up nonexistent studies, or misrepresented the findings from research reports. It was also unable to access a lot of key information in complex processes. This report from CNN talked anonymously to employees:
“Anything that you don’t have time to double-check is unreliable. It hallucinates confidently,” said one employee — a far cry from what has been publicly promised. “AI is supposed to save our time, but I guarantee you that I waste a lot of extra time just due to the heightened vigilance that I have to have” to check for fake or misrepresented studies, a second FDA employee said.
Implications: Beyond the extra Cognitive Load in double-checking all of the output in a situation like this, there is an interesting extra consideration; what is this LLM not able to access? Often processes in an organisation will depend on multiple types of digital sources, and if you don’t have a clear understanding about what a system is missing in its analysis, it is hard to properly consider its outputs.
Second Opinion?
One of the core use cases for AI general (beyond just LLM) has been using it for assisting medical professionals in checking scans for early-stage signs of diseases like cancer. Stories last year highlighted how AI processes were pick up things that human specialists may miss – though to be clear, we have moved on from the days where people predicted these tools would replace fields like radiology entirely.
Recently though, a study has been conducted to understand the effects of bringing AI into this process. In the study, when the assistance was removed, clinicians ability to find tumours dropped by roughly 20% compared to their performance before the tool was introduced.
As this report highlights, “The AI in the study probably prompted doctors to become over-reliant on its recommendations, “leading to clinicians becoming less motivated, less focused, and less responsible when making cognitive decisions without AI assistance”…“.
Implications: It is interesting to think about our current levels of critical thinking not just as set resource, but as something we need to practice. As shown by the clinicians in this trial, if we stop practising something, we become less able. Introducing more tools that we become wholly reliant upon can start creating a type of Cognitive Debt that we are unaware of until we need it.
Inspired by Technical Debt
Alongside a general lack of understanding as to how LLMs are created, and what they are actually delivering, I believe there is a substantial lack of forethought around what the implications are for organisations who choose to ‘forgo the thinking’ in order to find shortcuts to the answers.
I was looking for a way to express this potential systemic loss to at least give decision makers pause for thought in order to consider the consequences of their LLM use.
What is an LLM really doing, and what might actually happen if we use it here?
I came to Cognitive Debt as a term as an intentional and direct reflection of Ward Cunningham’s Technical Debt metaphor. Cunningham explains more about the specific metaphor here:
“With borrowed money, you can do something sooner than you might otherwise, but then until you pay back that money you’ll be paying interest. I thought borrowing money was a good idea, I thought that rushing software out the door to get some experience with it was a good idea, but that of course, you would eventually go back and as you learned things about that software you would repay that loan by refactoring the program to reflect your experience as you acquired it.”
My experience of this has been an ongoing feature of working on innovation projects over the last 15 years or so. Digital development teams would make technical decisions that allow them to achieve something in the short term – “launch this new feature immediately” – but with the understanding that they needed to repay this Technical Debt hanging over them.
Now, I believe that everyone intends to do everything to the best of their abilities, right up until the point that they don’t. They may be suddenly constrained by time, budget, the actions of others, changes in resources, being redeployed to work on something else, etc etc. It is hard to define exactly why Technical Debt doesn’t get repaid, and usually it’s not really constructive to play the blame game either.
But as it sits there in an organisation, as yet to be repaid, Technical Debt has a tendency to grow and become more expensive. Then people leave, and new folk come in and discover not just the debt exists, but now there are the additional cost of trying to determine how this was put together, why, and what happens if you turn this off…
For me, accumulating Technical Debt comes down to this; we were meant to do this properly, but we did it the quick way. The quick way might work for a while. But at very least, we should consider what the implications of not repaying that debt are.
You can probably see why I felt the Technical Debt comparison was useful for thinking about the widespread use of LLMs in an organisational context.
You must be logged in to post a comment.