Everyone's busy making use of the new toys ICT has gifted to us. And so we should.
There's great promise. It puts wonderful prose in our mouths. It delivers leaps in apparent quality. It achieves it with resource and time efficiency.
But those features bring with them some big traps.
GenAI is totally empirical. It has no model of the world, and no understanding of the meaning of a single word, let alone a sentence, let alone a line of argument.
It has no conception of values, intentions, motivations or emotions.
From the viewpoint of language theory and semiotics, it's all about the most simplistic form of syntactics (the relations among small segments of words, words and phrases), with no semantics (the relations between data and real-world things), and no pragmatics (the relations between data and people who interpret it).
As a result of that, GenAI has no understanding of the question it's given and no idea what the response it provides might mean to people who read it.
It has no appreciation of the person asking the questions, of the contexts in which the questions are being asked, the person's intentions, the purposes to which the responses might be put, or the consequences that an error or a misunderstanding might have.
Crucially, it cannot provide a humanly understandable explanation for its responses. That's because, being purely empirical, it's a-rational.
You can ask it to provide an explanation. But all that it can generate is another concoction based on a micro-grammar. It can't tell you the basis on which evidence was gathered, evaluated, and drawn on, to reach a conclusion.
In all, there are 20 attributes of GenAI that cause all the problems. We tend to think of 'hallucinations' – plausible-sounding but incorrect, nonsensical or made-up pseudo-facts – but that's merely problem number 20 in that list.
We were warned about what these features add up to. GenAI is 'a stochastic parrot', sewing glowing sentences together from wisps of probabilities of what the next likely word in the sentence ought to be.
But, thanks to the mature natural language generation component at the back end of each GenAI, the prose it produces is very smoothly expressed, sounds authentic, and – more dangerously still – sounds authoritative. It satisfies every criterion for high-quality fraud.
So how can we extract the value, and avoid the pitfalls?
After GenAI arrived, many organisations rushed to offer guidance. Tertiary education institutions faced immediate challenges, so they were first on the scene. Conservative government agencies published po-faced instructions, emphasising compliance with their governing statutes.
Many documents from corporations and industry associations are also driven by regulatory compliance concerns. Most aren't very helpful to people at the workface.
Distilling the more useful advice down, what I came up with is a 5-phase approach:
Play / Prepare / Perform / Prove / Exploit
Playful experimentation is a great place to start. Apart from being fun, it gives you insights into how your requests shape the responses. That gives a glimpse into how other people use GenAI to drive their preferred output, and manipulate attitudes and outcomes.
And, of course, it leads to an understanding of how you and I do the same, both intentionally and accidentally.
After playing games and building understanding and confidence, the convenience and ease of use make it tempting to get stuck into active use of the tool.
There are two reasons to hold back and prepare first. One is that what we ask drives the results. So, we're at risk if getting what we asked for, not what we meant to ask for. (That's a variant of a dictum uttered by Ada Lovelace, a contender for the title of 'world's first programmer', in 1842).
The second reason is that, in order to evaluate the responses you receive, you need to have done some pre-thinking about what shape the answer is likely to have.
Rather than making a request or giving an instruction, it's best to think of your input as a prompt, intended to draw out information, and to enable you to narrow down to more specific aspects later, and to consider exceptions, ambiguities and complexities.
We're now ready to submit our prompt and think about the response. To get quality results for anything beyond simple tasks, it's generally necessary to enter into a dialogue, by extending or qualifying the initial prompts, or starting again with a more carefully composed variant of the original prompt.
And now that you have a response, you can use it, right?
Well, if what you're doing won't have any consequences for anyone, you can go right ahead.
But if the outcomes of what you're doing will be impactful, you need to do what GenAI has not done, and cannot do. The stochastic parrot has no knowledge of the world, no common sense, and no idea what your conversation with it means. And it takes no responsibility for any damage you might do.
We have an obligation to identify key facts and assumptions that appear to underlie the responses GenAI gives us, and to fact-check or validate against reliable sources of information about the real world.
And we have the responsibility to think about what the next steps look like. Who will be impacted? What will their experiences be? How diverse will those experiences be? What exceptions are there, will they be recognised, and will they be appropriately handled?
For the protection of yourself and others -- sanity-check, document, and apply the pub test.
GenAI can be valuable when you use it intelligently, and as a tool to assist in thinking work.
But think of GenAI as an explosive. It can be harmful to others, and it can be harmful to you.
Roger Clarke serves as ACS’s vice president of Membership Boards. The interpretations and arguments expressed in this article are made in his personal capacity.
