Is AI Output Actually any Good?

This story starts in my kitchen. Or rather, it starts with my wife in our kitchen.

When I think about when I started incorporating this technology into my work processes, I noted my wife adapted to this technology much quicker than I ever did. She uses it daily for practical, real-world tasks, asking questions like, "What can I cook based on these specific ingredients I have in the pantry?" Here I am, leveraging AI in my professional life but leaving it out of my personal one, continuously fascinated by the fact that she has become a more avid and experienced user than me.

The new useful tool

I’m a problem solver at my core, I’m also a natural sceptic. Despite that scepticism, I now rely heavily on artificial intelligence to get my work done, whether that’s accelerating how I solve complex challenges or documenting old software features. Those are just a few examples of how I leverage technology on a day-to-day basis in my role as a Programme Lead at Smudge.

Yet, as much as I use it, I often find myself wondering: Is what this technology producing actually "good?" And how can I confidently assess the output?

For a long time, I viewed this wave of technology as something built strictly for developers. We’ve seen the rise of dedicated coding agents like Codex and Claude Code designed specifically for engineering workflows. The capabilities of Generative AI have come a long way. It has evolved into something incredibly useful for everyone, providing us all with a powerful tool to increase our productivity, and make our lives easier.

But it’s not without its traps.

What does good look like?

The core issue is simple: is what it’s telling us actually good? These large language models are famously adept at hallucinating - making things up confidently just to give us an answer. Even though many industry experts have contributed to training these models to get them to where they are today (a journey Reuben outlined well in his history of AI article), the underlying question remains. We know AI is a very useful and impressive tool, the question remains is it actually any good?

This is exactly where our own knowledge, expertise, and value as a practitioner needs to plays a significant role. We need to apply judgement and retain our responsibility as an expert when evaluating whether or not what a LLM delivers as an output is sound and worthy of being good, or not.

Take the example of how my wife and I approach cooking. If we both input the prompt, "I have x ingredients that I want to make dinner with, please make suggestions," the AI will spit out a recipe with clearly defined steps. But what I cook from that output and what my wife makes will be two very different meals. Why is this?

My skill as a cook is sorely lacking. My wife, on the other hand, is a fantastic cook (and I look forward to what she makes every single day, which is why she tends to be in the kitchen more than me). Because of her expertise, she can look at the AI's recipe, spot the flaws, and make creative adjustments. I can’t do that, as I don't have the skill to be able to judge what the LLM suggested. Left to my own devices, I wouldn't know if the advice was fundamentally sound, or not. She does.

Your understanding of what "good" looks like, earned through your own real-world experience, is what allows you to reach a proper conclusion on if the output is useful, and good.

Generative AI is not a silver bullet

Given the incredible tools we now have for developing code using AI it is far too easy to take an AI output at face value without questioning it, simply because the technology "just works." It is an incredibly easy trap to fall into - accepting the first generated response as the answer. We have to learn to question it. We need to treat this new digital companion like an entry-level peer who is eager to please but fundamentally less experienced, even when it gives answers that sound like they came from a subject matter expert.

Generative AI is not a silver bullet. In reality it's just another tool to help increase our productivity, and it requires active governance. We are already seeing the rise of "Vibe Coders," people who generate code using AI without fully understanding what they are looking at. This lack of scrutiny often leads to major downstream problems, resulting in software that is neither production-ready nor secure enough for corporate environments. It's potentially neither useful nor good.

Through these tools, we have all gained the power to create and retrieve answers instantly. But if we don't understand the underlying subject matter, we cannot judge if what we are doing is actually good. The foundational skill isn't there. This disconnect is exactly what makes the AI tools we have today incredibly powerful, but also remarkably dangerous.

As we move forward into this technology-driven future, we need to practice active curiosity. Every time we look at a generated screen, a line of code, or a recipe, we have to ask ourselves the ultimate question: Is what the AI telling me actually any good?

Key takeaways for navigating AI output:

• AI makes output faster, but it doesn't make deciding any easier. True quality control still requires human expertise.
• Beware the confidence trap. Generative models are designed to give plausible answers, not always accurate ones.
• The practitioner is the anchor. Your real-world experience is the only tool capable of separating a "good vibe" from a truly production-ready solution.