How NOT to use AI

Generative AI is genuinely powerful. It is also genuinely misused, often by well-intentioned teams who reach for it because it feels like the modern thing to do. This article is not a critique of AI. It is a critique of how we decide when to use it.

First, what generative AI actually is

Generative AI is the family of models behind tools like ChatGPT, Claude, and Gemini. It works by predicting the most plausible continuation of a given input. It has absorbed vast amounts of human language, reasoning patterns, code, and domain knowledge. As a result, it can write, summarise, classify, translate, explain, and generate at a level that often looks indistinguishable from human work.

But here is the key distinction that most AI adoption decisions miss: generative AI is a probabilistic reasoner, not a deterministic calculator. It does not compute the right answer. It generates a highly likely answer.

For many tasks, that is exactly what you need. For others, it is precisely what you cannot afford.

The core question to ask before any AI project: “Does this task require a reliable, auditable, rule-based output or does it require flexible interpretation of complex, variable input?”

The answer tells you almost everything.

Where AI genuinely excels

Before we talk about misuse, it is worth being specific about where AI delivers real value.

The pattern is consistent: AI shines when the logic is too complex or variable to encode as rules, but the tolerance for imprecision is meaningful.

The sweet spot: Tasks where a human expert would say “it depends”, and the dependencies are too numerous or contextual to capture in a flowchart.

Complex reasoning over unstructured input

When a customer writes a long, rambling complaint email and you need to identify the product, the sentiment, the urgency, and the recommended team to route it to, that is a genuinely hard classification problem. A rules-based system would need thousands of conditions. AI handles it in one pass.

Generating first drafts with contextual nuance

Writing a proposal, a job description, a project brief, or a summary of a meeting: these tasks require judgment about tone, audience, emphasis, and structure that cannot be reduced to templates. AI is a capable first-draft engine for all of them.

Synthesising across many documents

• A lawyer reviewing 200 contracts for non-standard clauses.
• A product manager reading 500 support tickets to identify theme patterns.
• An analyst summarising quarterly earnings calls across 30 companies.

These are tasks where human review is slow, expensive, and inconsistent. AI handles the synthesis and the human validates the output.

Use AI when the input is messy and the output can tolerate review. Avoid it when the input is structured and the output must be exact.

The traps: tasks where AI is the wrong tool

The following are common categories of AI projects where ambition meets reality, often revealing that a simpler, more deterministic approach would have served the goal better.

1. Usecase: Contract fee extraction

Risk: High risk misuse

Task: Extracting exact figures from legal contracts

A common pitch: “We have 10,000 contracts. Let AI pull out the fees, penalties, and payment terms.”

The problem is that contracts are written in natural language with enormous variations like: “an amount equal to 1.5% per annum of the outstanding principal, compounded monthly, exclusive of VAT” is not the same as “1.5% monthly fee.” AI will extract a number. It may not extract the right interpretation. In financial or legal contexts, that ambiguity has real consequences. A deterministic parser built by a lawyer and an engineer is auditable, testable, and correct by design.

AI approach

Model reads contract, outputs “fee: 1.5%”.

• Confidence: high.
• Correctness: uncertain.
• Auditability: none.

Better approach

Structured extraction with defined fields, validation rules, and a human review step for edge cases. Boring, reliable, auditable.

2. Usecase: Finance and invoicing calculations

Risk: High risk misuse

Task: Using AI to compute totals, apply discounts, or reconcile invoices

If a finance team asks AI to process invoices, to check line items, apply VAT, apply early-payment discounts and flag discrepancies, they are asking a language model to do arithmetic. Language models can do arithmetic. They can also get it wrong in ways that are nearly invisible.

The correct tool for invoice processing is a deterministic function: input goes in, calculation happens, output comes out, same way every time. A spreadsheet formula is more trustworthy here than the most advanced AI model available today. Using AI for financial calculation introduces variance where finance demands precision.

Suggestion: Leave the calculations to deterministic (rule-based) systems! I’ve been advised to use dedicated software.

3. Usecase: Data validation and ETL pipelines

Risk: Architectural misuse

Task: Replacing validation logic with AI inference

ETL (Extract, Transform, Load) pipelines are the plumbing of data infrastructure. They exist to move data reliably from one place to another, applying defined transformations and quality checks along the way.

When teams try to use AI to “intelligently” validate data or infer transformations, they are substituting a probabilistic model for a deterministic contract.

If the rule is “a valid UK postcode matches this regex and the district must exist in this lookup table,” write that rule.

An AI model that infers whether a postcode looks valid introduces latency, cost, and unpredictability into a pipeline that needs to be fast, cheap, and always correct.

Suggestion: Use AI upstream to clean and classify data before it enters the pipeline, but not in the pipeline.

4. Usecase: Form processing with defined schemas

Risk: Overcomplicated solution

Task: AI-based form parsing when structured input already exists

If users submit a form with defined fields, like: name, date, amount, category, and you then use AI to “understand” what they entered, you have added complexity without adding value. A form with structured fields is already a data contract. Parse it as one.

AI adds genuine value when the input is a free-text field or an uploaded document with no schema, where humans express the same intent in dozens of different ways. If the input is already structured, honour that structure.

Suggestion: Build deterministic functions. They are faster, cheaper, testable, and they do not hallucinate.

The decision framework

Before commissioning an AI feature or project, run it through these four questions:

• Could a junior developer write an explicit rule for every case? If yes, write the rules. AI is unnecessary.
• Is the output used in a financial, legal, or compliance context? If yes, every error has a cost. AI needs a verification layer, and if that layer is thorough enough, ask whether AI is adding anything.
• Does the input already have a defined schema? Structured data should be processed by deterministic code. AI is for unstructured input.
• Is accuracy 100% required, with no tolerance for review? AI is not the tool. If accuracy is high but human review is part of the workflow, AI as a first-pass is viable.

The real cost of AI misuse

The failure mode of a badly-applied AI project is rarely a dramatic crash. It is a slow erosion of trust. The model extracts 97% of fees correctly, and the 3% errors slip through into contracts or payment runs for months before anyone notices. The invoice processor seems to work, until a discount is misapplied on a high-value client. The ETL pipeline “mostly” validates correctly, until a batch of corrupted records reaches production.

In deterministic systems, errors are reproducible and fixable. In probabilistic systems, errors are statistical: they happen at a rate, not at a moment. That distinction matters enormously when the output has business consequences.

The hardest question to ask: “If this AI feature produces a wrong answer 2% of the time, what is the cost of that 2%?” If the answer is “we would not know until it was too late,” reconsider the architecture.

The takeaway

Generative AI is not a universal upgrade to every workflow it touches. It is a powerful tool for a specific class of problem: messy, variable, human-language input that needs to be interpreted, summarised, or generated with contextual nuance. For everything else, including calculation, validation, structured parsing and rule-based routing, the deterministic function you would have written five years ago is still the right answer.
The most useful skill a product manager can develop in the current moment is not knowing how to use AI. It is knowing when not to.

Are you interested in this topic? Do you have any questions about the article? Book a free consultation and let’s see how the Code Factory team can help you, or take a look at our services!