Hold onto humanity: Critical thinking in the age of AI

AI adoption is no longer the hard part. The real challenge is adopting it without quietly outsourcing the capability that makes engineering organisations valuable: the ability to think.

One question ran through the entire roundtable. As AI becomes the default way of working, what happens to human judgement? More importantly, how do organisations build teams, engineering practices and software delivery processes that preserve it?

The roundtable surfaced three priorities for technology leaders.

Move thinking upstream

Teams should define the intended outcome, supporting evidence, constraints, and acceptance criteria before asking AI to produce an answer. This makes the output easier to assess and reduces the risk of accepting polished work that solves the wrong problem.

Match the technology and controls to the risk

A large language model is one tool among many. Depending on the problem, teams may need deterministic rules, classical machine learning, simulation, conventional software, another type of AI model, or a combination of approaches. The level of human oversight should reflect the consequences of error, not the novelty of the technology.

Make AI-assisted work measurable and accountable

Organisations need to record more than what happened and when. They should also capture what generated an action, which evidence informed it, which controls applied, who approved it, and why the final decision was made.

Ideas, frameworks and practical approaches from the discussion have been brought together in this white paper for engineering leaders, CTOs, and technology decision-makers building AI into software delivery while keeping human judgement at the centre of every important decision.

A simple question sat at the centre of the discussion, even if the answer proved much harder.

AI is sold on convenience. Save hours. Automate the task. Let an agent handle it. Every one of those promises asks people to hand over a small piece of their thinking. Each individual decision feels rational. Taken together, they become a gradual transfer of value.

One of Ime’s most thought-provoking ideas was that AI does not think. It predicts, and it rationalises.

A clear distinction emerged between reasoning and rationalising. Reasoning works forward, from evidence to a conclusion. Rationalising works backwards, from an answer to a justification. AI models, he argued, are expert rationalisers. People who rely on them without enough scrutiny gradually begin to think the same way.

A different view also had its say. One participant building AI for healthcare argued that children growing up alongside AI companions could learn faster than any previous generation, as long as expectations rise with the technology and people keep tackling harder problems rather than easier ones.

Nobody argued for using less AI. The real debate centred on where human judgement remains essential, and how organisations can hold onto it.

When asked how many people in the room had completed prompt-engineering training, and then how many still applied those practices consistently, the answer was a knowing silence.

Deadline pressure changes behaviour. People type a quick prompt, skim the output, and iterate by instinct. If the answer looks convincing, it gets used.

No one was criticising that behaviour. The discussion focused on why it happens so easily. Critical thinking rarely disappears through one conscious decision. It fades through hundreds of small compromises made in the name of speed.

Engineering organisations cannot rely on individual discipline alone. Better habits need to be built into the way work is planned, reviewed, and delivered.

A case study by Vention: How specification-first delivery reduced defects

Vention was approached by a client with a non-functional fintech platform, limited documentation, and real money already in motion. Vention decided to move the thinking upstream. Before development began, QA engineers and business analysts wrote the specification for each piece of work and validated it from a quality perspective. Every ticket followed a structured format, with a user story and acceptance criteria written to remove ambiguity before any AI-generated code.

The MVP launched in three months. Thanks to the specification-first approach, the defect rate fell by 60-70% from the start of the project. The starting point was a broken inherited codebase, so the figure reflects the full turnaround. The lesson is clear: when intent, reasoning, and the expected outcome are documented before AI touches the work, quality follows.

Inherited, non-functional codebases with money already in motion are the classic starting point of Vention's software project rescue engagements, and the discipline there is the same: specification first, code second

One of the most practical conversations of the morning started with a simple observation. Machines are often held to a standard that people themselves never meet. Ask the same person the same question on different days, and the answer may well be different.

The analogy prompted an immediate debate. A marketing technology leader argued that marketing can tolerate a reasonable range of outcomes. Healthcare decisions often require complete accuracy. An engineering leader from financial services challenged the assumption that marketing is always low-risk, noting that poorly targeted financial products can still harm vulnerable customers.

One conclusion united the room. Acceptable levels of variation depend on the decision, not the technology. A model that is perfectly appropriate in one workflow may introduce unacceptable risk in another. Defining those boundaries for every use case is where effective AI governance begins.

The technology industry increasingly uses “AI” and “large language model” as though they mean the same thing. They are not. LLMs are inherently probabilistic, meaning the same prompt can yield different answers because they are designed to predict plausible language rather than deterministic outcomes.

World models offer an interesting contrast. Instead of learning from text, they learn how the world behaves, making them a better fit for problems where predictable, deterministic behaviour matters.

That defined the key idea here: good engineers don’t rely on a single tool. LLMs, other AI models, classical machine learning and conventional software all have their place. Choosing the right technology for the problem is often more important than choosing the latest one.

A case study by Vention: The right tool was not the fashionable one

A trade intelligence company engaged Vention to build a platform that generates on-demand reports from complex global trade data. Source records were extensive but often delayed, inconsistent, or incomplete, so analysts needed missing values filled without sacrificing transparency.

Rather than defaulting to an LLM, Vention chose gradient boosted trees. The data was tabular, and every prediction needed to be explainable and auditable. Trained on decades of cleaned historical data, the model achieved more than 80% accuracy on held-out records.

Conventional software then verified totals, calculations, and chart-to-table consistency before publication. The result was a system that produces a complete report in around five minutes.

The lesson is simple: start with the required outcome and level of risk, then choose the right technology, not simply the most fashionable one.

A question about governance and provenance led to one of the most practical conversations of the roundtable.

Most engineering organisations already log what happened and when. AI systems raise the bar. Teams also need to capture who or what generated an action, why it happened, and what reasoning led to it.

One participant questioned whether audit records themselves could be manipulated. Rather than debating products, the discussion quickly shifted to first principles. Records should be immutable and write-once, with corrections added as new entries rather than edits. That approach preserves the full history and allows every decision to be replayed when needed.

Another idea surfaced repeatedly throughout the discussion. Governance requirements should be defined before technology choices are made, not afterwards. Careful thinking often leads to surprisingly straightforward solutions, where AI supports the system rather than becoming the system.

One message came through consistently from engineering leaders around the table. AI doesn’t lower expectations for engineers. It raises them.

Writing code is no longer enough as engineers are increasingly expected to explain their decisions, justify trade-offs and communicate directly with stakeholders.

Several participants described how that shift is already changing the way teams work. Instead of assigning individual tickets, they increasingly give small teams ownership of complete features. Engineers arrive with recommendations rather than waiting for instructions, and judgement becomes just as valuable as implementation.

The conversation closed with a broader question: how much of our work should ever be delegated?

One participant admitted preferring an AI agent to mediate communication with colleagues. Others pushed back. Human relationships reveal context that no dataset can fully capture. A staff engineer considering resignation may stay because of one conversation, even when every available metric suggests nothing is wrong.

Someone half-jokingly predicted that genuine human interaction may eventually become something people actively seek out and pay a premium for.

Perhaps the strongest takeaway from that morning came from the discussion itself. Two hours of debate, disagreement, curiosity and laughter produced ideas that no AI-generated summary could fully capture.

AI can make engineering teams faster and more productive. Protecting human judgement, critical thinking, and meaningful collaboration remains our responsibility.

The discussion surfaced several practical frameworks that engineering leaders can apply immediately. None require new tools or complex governance programmes. Each is designed to strengthen judgement, improve consistency, and make AI adoption more intentional.

Ime, the keynote speaker, returned to the same discipline throughout the discussion. Before handing any task to an AI tool, ask three questions in order.

1. Intent. What outcome am I trying to achieve, and why? Focus on what actually needs to happen, not simply what would be nice to have.
2. Reasoning. What steps and thought process should lead to the answer? What evidence should support it? Reasoning works forwards from evidence. Rationalising accepts an answer and justifies it afterwards.
3. Action. What should be done, and how will I check the result against the original intent before anything is acted on?

The message was deliberately modest. Businesses don’t need a full critical thinking curriculum. Applying these three questions consistently can significantly improve both prompt quality and the way AI-generated outputs are reviewed.

Not every AI decision carries the same level of risk. Acceptable variation depends on the context.

1. Classify the decision. Is the output customer-facing or internal? Is it reversible? Who is affected if it is wrong? Marketing content naturally allows more flexibility than patient advice or financial decisions.
2. Decide the human's role. Should people remain in the loop, supervise the outcome, or simply receive notifications afterwards? Match the level of oversight to the level of risk.
3. Set tolerance thresholds before deployment. The keynote speaker compared it to agreeing on acceptance criteria with a hardware supplier before the first shipment, not after defects appear.
4. Revisit those thresholds over time. A use case that begins as an internal drafting tool may gradually move into customer-facing decisions. Governance should evolve alongside it.

A case study by Vention

To keep AI-enabled delivery transparent at scale, Vention relies on Vention Forge, its proprietary AI-powered delivery intelligence platform. Forge tracks how teams execute the BMAD Method (an agile AI-driven development framework) across the SDLC, giving project and portfolio leaders visibility into AI-assisted delivery adoption, execution flows, productivity patterns, and implementation outcomes.

Governance should be designed before systems are built, rather than added afterwards. Starting with the right questions often leads to simpler, more effective solutions.

1. Record what happened, when it happened, and what generated the action.
2. Capture why the action was requested and who, or what, requested it.
3. Log the reasoning behind the output, not just the outcome itself.
4. Link every decision back to supporting evidence, whether that is an email, meeting, document, or message.
5. Preserve records using immutable, write-once principles. Corrections should be added as new entries rather than edits so the full history remains intact.
6. Audit failures as carefully as successes. Records explaining why something went wrong often become the most valuable governance data an organisation holds.

A case study by Vention: Making hiring decisions traceable

A regulated staffing company engaged Vention to replace a fragmented hiring process built around emails, spreadsheets, and manual document checks. Candidates had to provide licences, identity documents, and other compliance-sensitive records, making every approval explainable and defensible.

Vention built a structured digital workflow with append-only audit logs, immutable consent records, and versioned policy tracking. Every action that changed a candidate's status was recorded with supporting metadata, allowing the company to reconstruct each step of the process and verify which checks had been completed.

The outcome was exactly what the company set out to achieve: a faster hiring process without sacrificing accountability.

One of the participants described how highly regulated engineering teams create room for experimentation without compromising governance.

1. Form small cross-functional pods of two or three engineers focused on one clearly defined business problem. Treat the work as a proof of concept rather than a long-term programme.
2. Spend the first phase agreeing on engineering fundamentals, including design principles, coding standards and architectural patterns. Those guardrails become the foundation for everything that follows.
3. Give the team freedom to innovate within those boundaries. Engineering excellence becomes the primary control mechanism, rather than additional compliance processes.
4. Separate experimentation from governance. Several participants argued that innovation and control compete when forced into the same phase. Explore first, evaluate afterwards, then repeat the cycle.
5. Whatever the outcome of the proof of concept, the organisation keeps stronger engineering practices. Even unsuccessful experiments leave the team better equipped for future work.

Choose one high-impact AI use case and write a one-sentence tolerance statement. Clarify what level of variation is acceptable, where the human sits in the process, and whether the use case is running on evidence or hope.

Update your definition of ‘done’ for AI-assisted work. Engineers should be able to explain the intent, the reasoning, the evidence behind the output, and why they trust it, instead of just presenting the final result.

Assess one AI initiative against the audit trail framework described above. If you're only recording what happened and not why, you have observability without accountability

A case study by Vention: From AI experimentation to shared engineering practice

One of Vention's large fintech clients had already equipped its engineers with AI tools, training, and opportunities to experiment, but adoption remained individual and inconsistent. Shared workflows, reusable practices, and reliable measures of success were all missing.

Vention helped turn that experimentation into a structured transformation across three pilot teams. The programme introduced shared AI practices, stronger documentation, specification-driven development, reusable instructions, and systematic knowledge sharing. Instead of treating AI as a tooling rollout, the organisation began embedding it into the way engineering teams plan, collaborate, and deliver.

The initiative is still in its early stages, and productivity remains within historical ranges. Tangible business outcomes are expected after 9-12 months of sustained adoption. Part of the lesson is that meaningful AI transformation begins with disciplined changes to engineering practice, while business results follow over time.

The roundtable reached a clear conclusion: successful AI adoption depends as much on engineering discipline as it does on technology.

Choosing the latest model is only part of the equation. Lasting results come from clear ownership, strong engineering practices, governance built into delivery, and human judgement at every stage where decisions matter.

Vention applies those principles through its AI-enabled software development approach. AI is embedded throughout the software development lifecycle to support planning, development, testing and code review, while engineers remain accountable for architecture, trade-offs, validation and business outcomes.

Every engagement starts with the business problem before technology is considered. AI discovery workshops help organisations identify practical opportunities, define acceptable levels of risk, and establish the guardrails needed for responsible adoption. The same principles guide our dedicated engineering teams and staff augmentation engagements, embedding AI where it creates measurable value while keeping people accountable for the decisions that matter.

The objective is simple: help organisations build software teams that deliver faster, maintain quality and preserve the human judgement that great engineering depends on.