The question
When an AI coding agent produces unreliable backend code, is the model the problem, or the brief it was given? The industry answer is usually to reach for a bigger model. This study tests the other variable.
Method
The same coding agent built the same backend module six times under identical conditions except one: three runs worked from a raw software requirements document, and three worked from a structured build kit, a specification prepared to be complete before any code is written. Every run was scored for reliability and all six outputs are published.
All six runs are published with their inputs, outputs and scores.
Finding
The completeness of the specification, not the capability of the model, determined whether the generated backend was reliable. The study title is the result.
This finding is also the engineering principle behind CodeNuk, where complete specifications feed a deterministic generation pipeline.