I have been thinking a lot recently about Cognitive Debt from one particular perspective; often the need for ‘thinking’ is not being replaced, but instead just reshaped and pushed elsewhere in a system.
Recap: I think of Cognitive Debt as ‘where we have the answers, but not the thinking that went into producing those answers”. It is a phenomenal largely (but not exclusively) fuelled by the deployment of LLMs at scale. Answers are now much, much cheaper to come by.
Additionally, I am most interested in exploring Cognitive Debt not from an individual perspective, but from a group one. It is critical to thinking through the implications of using these technologies inside an organisation, or between an organisation and its employees, a government and its citizens, and so on and so forth.
Lately, I have started noticing examples of not just where the debt is being accrued, but who then has the responsibility to pick it up and repay it.
For instance, where a person or company uses LLMs to (notionally) become much more productive, the intent is clear; “this can do the thinking for me”.
Who has to fact check the sources the employee has put into the report? Work out where the story is in a thousand automated press releases? Or establish which candidates are really worth interviewing in a market where AI is used to both write covering letters and read them?
Too often, an LLM doesn’t replace the need for thinking in a group setting, but simply creates more work for others.
As I started writing this post, my distant economics self shuffled in its fireside armchair, mumbled something about laissez-faire cognitive debt, before returning to its afternoon nap.
Laissez-faire translates as ‘let them do it’, and is (I presume) most often used to describe a hands-off approach to government intervention or regulation in a sector; i.e. laissez-faire economics.
Perhaps I’m wondering if the laissez-faire approach to LLMs has been “yes there are issues with this, but you can solve them”.
And it’s not just a feature at the individual or even organisational level, but a culture that problematically pervades the whole system. Two recent experiences have prompted me to take Google as an example.
Last week, I was lucky enough to get a ticket for IF’s excellent Crossing The AI Divide event. To pull from CEO Sarah Gold‘s introduction:
But here’s the real issue: the gap isn’t between the companies that build with AI and those that don’t. It’s between those who use it safely and confidently and those who don’t.
There was a really interesting panel mix to help consider the question of trust.
Update: Sarah’s talk is now up here:
In addition to Sarah, there were two people from heavily regulated sectors (Christine Foster of Experian & Lily Dart from Monzo), and one person from a sector which is, let’s be honest, much more lightly regulated (Dawn Bloxwich of Google DeepMind). The discussion was expertly chaired by Richard Allan from the House of Lords.
As the conversation unfolded, it brought me back to this idea of pushing cognitive debt around a system, but this time on a more macro scale, between organisations and LLM suppliers. Largely because that was what was unfolding on stage.
Dawn laid out what I suppose is the approved Google DeepMind public position; other companies employing GenAI technologies in their work should not trust them unquestioningly.
It was obviously the responsibility for an organisation operating in a certain market to imagine all of the scenarios and impacts of using LLMs for customer-facing technologies, and mitigate accordingly.
And at no stage could the buck ever stop with Google; their clients (companies like Monzo and Experian, perhaps) must take on the risk and liability for designing these tools in ways that the public would have access to.
“Let them do it”, indeed.
Laissez-faire Cognitive Debt is where you know the answers being generated can’t really be trusted, but it doesn’t really matter because if someone has a problem with it they should do the work to rectify the system’s mistakes.
Now you might very well think that it is a bit rich for the fourth most valuable company in the world to be so seemingly disinterested in taking responsibility for a technology that it produces, and clearly perceives and understands the failings of. But I couldn’t possibly comment.
Then, yesterday morning in an interview with the BBC, Alphabet CEO Sundar Pichai suggested that even when Google are letting LLMs loose in their main consumer facing channel, the search page, then all of the hard work in ascertaining whether something is true or not should lie directly with consumers themselves.
“Let them do it”, is what Sundar implies.
You could argue that there is nothing that has polluted the information environment faster than Google’s slapdash deployment of substandard LLMs at the top of search results.
And, because they operate in what is (for all intents and purposes) an unregulated environment, it is infuriating that Google so blithely unleashes this technology in a way that it would land Experian or a Monzo in the hottest of water in a week at best.
To claim with a straight face that people will understand that search results returned on a search engine aren’t trustworthy or useable because there’s a tiny piece of text at the bottom that says ‘AI responses may include mistakes’ is… well, it’s quite something.
Across both of these examples, we get a clear sensethat Google’s position is to push the hard work of wrangling with the Cognitive Debt of LLMs elsewhere; onto the businesses who want to implement these technologies, or the consumers who use them.
Yet given we know, clearly, the implications that this work will be created wherever these technologies are deployed, surely we can start to shape a regulatory policy environment that can offer more certainty and trust in both the B2B and B2C worlds?
At Crossing The AI Divide, we also heard that age old standard reply from technology companies, as Dawn from DeepMind stated that ‘regulation stifles innovation‘.
MRDA; “Well, she would say that, wouldn’t she?”
My immediate reaction to the point was that ‘sure, but lack of regulation stifles innovation too’. There is no real incentive for Google, OpenAI, Anthropic et al to solve the problems inherent in LLMs (e.g. they will always ‘hallucinate‘ – it’s a feature, not a bug) if they can do what they like in a regulation-free environment.
Yet good regulation can help shape markets well, because it:
- Builds trust and confidence
- Creates new markets
- Promotes competition
- Encourages investment and innovation
- Ensures quality and safety
How do I know all this?
Well, Google told me so…
You must be logged in to post a comment.