Self-Debug Prompting
Self-debug prompting is a pattern in which the model generates code, an interpreter executes it, and the model receives the execution result — error messages, failed test output, or unexpected values — as additional context for a revised attempt. The loop continues until the code runs correctly or a retry budget is exhausted.
It mirrors a human developer's edit-run-read-error-revise cycle and is particularly effective for code-generation tasks where first-draft code often has small bugs (off-by-one errors, wrong imports, type mismatches) that are obvious once the interpreter surfaces them. Quality depends on the interpreter's error messages being informative; opaque runtimes reduce the benefit.
Origin: "Teaching Large Language Models to Self-Debug" (Chen et al., Google, 2023) formalized the pattern and showed gains on code-generation benchmarks; similar execution-feedback ideas appear in concurrent work like Self-Edit and CodeT.
How it works
- 1
The model generates an initial code attempt for the task, typically with a brief plan or docstring up front.
- 2
An interpreter or sandbox executes the code against test cases; stdout, stderr, exception traces, and return values are captured.
- 3
The full execution trace is appended to the conversation as a new turn, framed as "the code raised X" or "the test expected Y but got Z".
- 4
The model produces a revised version. The loop continues until tests pass or a retry budget (commonly 3–5 attempts) is exhausted, at which point the best attempt is returned.
Example
A coding agent is asked to write a function that parses ISO 8601 durations. Its first attempt raises a ValueError on the test case "PT1H30M". The interpreter's error — "could not convert string to float: '1H30'" — is fed back to the model, which realizes it needs to split on letter boundaries, not just digits, and produces a corrected version that passes on the second try.
Not to be confused with
- Self-refine
- General output-refinement pattern where the model critiques and rewrites any kind of output. Self-debug is the code-specific specialization where the "critic" is a real interpreter, not the model itself.
- Reflexion
- Adds a verbal-reinforcement memory across episodes — the agent writes a lesson after each failure and carries it forward. Self-debug is single-task and stateless across runs.
- Test-driven prompting
- Hands the model the test cases up front so it writes code to satisfy them. Self-debug uses execution feedback reactively after each attempt; the two are often combined.
Put this into practice
Build polished, copy-ready prompts in under 60 seconds with SurePrompts.
Try SurePrompts