This is the third post in a short series I am writing about AI literacy for technical writers. So far I have covered what AI is good at, where it fails, and how to structure content so that AI can use it effectively. This one is about something that follows directly from all of that: how to check AI output properly.
Before I looked into this area more closely, I assumed that reviewing AI output would be pretty much the same as peer-reviewing other writers’ work. Read it through, tidy it up, fix the obvious problems. But it turns out with AI content, that’s nowhere near enough.
You can’t fully trust AI output
By now, I’d guess most of us have experienced AI content that’s well-written, yet completely inaccurate. In the last couple of weeks, I’ve seen:
Instructions in the wrong order
Explanations that actually explain the opposite of how a feature works in reality
Paragraphs that contradict earlier statements.
That’s the problem with AI. It creates confident and often plausible content, but it has no way of knowing whether it is right or wrong. It’s only as good as its training data, and even then, it can misinterpret details or try to repeat patterns that don’t apply in every scenario.
With AI-generated content, it all needs to be checked for accuracy as well as style and flow. This is especially important for safety-critical information. If AI leaves out a warning, presents a step in the wrong order, or misrepresents how something works, the consequences for the user can go well beyond mild frustration.
What if you ask AI to fact-check its own output?
Well, it has the same problem. It uses the same pattern-matching to review the output as it did to create it. It’s unlikely to find problems with content that it thought was correct in the first place.
A common place we can see this is with AI bots that are getting answers from training data that is out-of-date. They will keep repeating the same mistakes based on the same incorrect source data, until they learn otherwise. As far as the AI knows, the information IS correct.
Validating AI output with humans back in the loop
So how should we go about validating the information AI produces? Well, there are several steps we can take:
Verify claims against a trusted source
Every fact the AI produces needs to be tracked back to the source of that information. That could be a help centre, a specifications document, a Slack thread, or even your own hands-on experience with the product. If you can’t trace where the information came from, treat it as dubious until you can confirm it.Test step-by-step instructions yourself
A bit of a no-brainer really and something I’d suggest you do no matter who or what created the instructions. Well-written instructions are of no use if they are incorrect. In fact, they can be worse than no instructions at all.Pay attention to the details
AI sometimes creates information that’s plausible, but wrong. This can be hard to catch, but keep an eye out for setting names that are slightly off, steps missing, and explanations that are mostly right but miss important details. Catching these requires you to know the product well enough to spot a near-miss.Be on red alert for missing information
AI generates content based on the content it already knows. This can result in important information being missed, such as critical warnings, edge cases, and details about pre-requisites.Check terminology
Check that AI is using the correct terminology from your style guide. It will use the terms it sees as the most common in its training data, and these may stray away from your style guide. This is especially true if, internally, your SMEs use different terms for features your customers actually see.
With all of these, the better you know your product, the easier it will be to catch the AI errors. That’s why checking AI output can be a bigger challenge for people who are new or less familiar with the product.
ACTCA - A quick checklist
If you find acronyms helpful, ACTCA is what you need for validating AI outputs:
Accurate
Can you verify every factual claim?Complete
Is anything missing that a real user would need?Tested
Have you actually followed the steps in the product?Consistent
Does the terminology and tone match the rest of the documentation?Appropriate
Is the level of detail right for this audience?
These are all things a good technical writer asks of any content. It’s just more important to double-check them with AI.
Is validation the future for technical writers?
In the short-term, it could well be.
Validation is where technical writing expertise becomes genuinely difficult to replace. AI can produce volume and fluency. It cannot produce accuracy without someone who knows the product, knows the users, and knows what good looks like. Validation is not optional. In a lot of ways, it is the whole job.
