CompCode Academy

Lesson 3 of 6 · Free to read

Spot the four ways AI code goes wrong

Spot the four ways AI code goes wrong

Audience: Final-year students and engineers in their first three years
Prerequisites: Lessons 1 and 2
Time: about 25 minutes
What you will keep: a checklist you run before accepting AI code
This lesson does not cover: security review in depth (see the security module).

Why this matters

AI coding tools fail in a small number of repeatable ways. Once you know them, you can catch most mistakes in under a minute, before they reach a pull request. Each one below comes with how to spot it and how to recover without starting over.

1. Fabrication

What it looks like: a confident call to a function, option or package that does not exist. It often looks plausible because it is named like things that do exist.

How to spot it: you do not recognise the name, or it is suspiciously convenient. Search the installed package (pip show, your editor's "go to definition") or the official docs. If you cannot find it, assume it is invented.

Recovery: "Which functions in your answer are you not certain exist in version X? Use only what is in the installed package." Then verify what it names.

2. Stale context

What it looks like: the AI edits a file as it was earlier in the conversation, uses a name you already changed, or "fixes" code that no longer exists.

How to spot it: the diff does not apply cleanly, or it refers to names you do not see in the file.

Recovery: give it the current file again, or start a fresh conversation with a short brief. Long conversations drift; a clean restart is often faster than arguing.

3. Overconfidence

What it looks like: "Done! All tests pass." But the tests were never run, or the change only works for the example you gave.

How to spot it: there is no test output, or the claim is broader than the evidence.

Recovery: "Run the tests and paste the full output." Accept nothing as done without output you can read. If the tool cannot run commands, run them yourself.

4. Scope creep

What it looks like: you asked for one fix and got a refactor of three files, renamed variables and a new dependency.

How to spot it: the diff is much larger than the change you described.

Recovery: reject the diff. Re-brief with a constraint: "Change only list_tasks. No renames, no new packages." Large unrelated diffs hide real bugs from reviewers.

A worked example

You ask for a function that returns tasks due this week. The AI answers with code that calls date.week_range().

  • Fabrication? Python's datetime.date has no week_range method, and you can check that in seconds with dir(date) or the docs.
  • You ask it to use only the standard library. It returns working code and says "tested and working".
  • Overconfidence? No test output was shown. You ask it to write two tests, one for a Monday and one for a Sunday, and run them. The Sunday case fails because the week boundary was off by one.

Two checks, two minutes, and the bug never reaches your teammates.

Your accept checklist

Before you accept AI-written code, check:

  • Every function and package it uses exists in the version you have installed.
  • It edited the current version of each file.
  • You saw the tests run and pass.
  • The diff contains only the change you asked for.
  • You can explain every changed line to a teammate.

Exercise: catch one on purpose

In any small project, ask your AI tool to use a function you suspect does not exist in a library you know. Record the prompt, what it answered, how you proved it right or wrong, and the prompt that fixed it. Keep this; it goes into your prompt log in lesson 4.

Key points

  • Fabrication, stale context, overconfidence and scope creep cover most failures.
  • Each has a quick check and a quick recovery.
  • "Done" means test output you read, not a claim.

New free lessons and course updates. Unsubscribe from any email.