Engineering Standards for AI Agents: From Rules to Discipline in Development
A year and a half of development without hand-coding exposed the limits of simple instructions in files like CLAUDE.md. Agents like Claude Code and Cursor handle tasks but ignore project context, make up logic, and skip edge cases. The author, seasoned IT leader Andrey Yumachev, shifted from one-off prompts to a systematic process: formal specs, failure memory, and the SENAR standard with the TAUSIK framework. This boosted FPSR (First-Pass Success Rate) from 40% to 75–80% on server-side tasks.
Problems Uncovered in Practice
Simple projects work with a rules file: code style, approved libraries, focus areas. But complex tasks push agents beyond bounds.
Unauthorized Changes
An agent refactors a module for a task, tweaks a utility library used in seven modules. Tests pass, but three modules break the next day. The agent acts like an overeager junior dev: extracts duplicates without approval.
Guessing APIs
Integrating an external service: the agent generates request code, retry logic (1s, 2s, 4s), and response parsing. Two of five requests fail—the docs are outdated, so the agent fills gaps plausibly but wrongly. No manual checks via curl or Postman with code comments.
Test Gaps
Password reset: tests pass green, but there's an OWASP vulnerability—different responses for existing/nonexistent emails enable enumeration. The agent didn't add the test without explicit specs.
Instructions are suggestions, not hard limits. No enforcement mechanism.
Process Evolution: From Specs to Memory
Formalizing Requirements
Every task: goal, acceptance criteria, in/out scope files, negative scenarios (empty fields, invalid IDs). FPSR doubled. Specs curb guesswork—the agent sticks to explicit rules.
Example Formalized Spec:
- Goal: Implement password reset.
- Criteria: Identical response (and timing) for any email; tests for existing/non-existing.
- In scope: auth.py, tests/test_reset.py.
- Out of scope: UI.
Sortula: A Project with Failure Memory
A link-saving service with semantic search (LLM extracts theses to DB). The agent breaks the build three times on one library—no memory of past fails.
Solution: Failure log file.
- Library X: Build fails on server Y (missing dep Z).
- Alternative: W, works with config flag.
- Date: YYYY-MM-DD.
In 6 weeks: 40 entries, saving 15+ min per task. Log feeds into CLAUDE.md as preamble.
SENAR Standard and TAUSIK Framework
SENAR is an engineering standard for AI development: specification, enforcement, memory, audit. TAUSIK is the implementation framework.
SENAR Components:
- Specification: YAML/JSON specs with criteria, scope, negatives.
- Enforcement: Pre-commit hooks check compliance (linting, no out-of-scope changes).
- Memory: Sortula-style log + agent memory (Claude context window).
- Audit: Human review checkpoints at milestones.
- Rollback: Git workflows with feature branches.
TAUSIK automates: generates spec templates, injects logs into prompts, enforces via scripts.
Metrics and Honest Benchmarks
FPSR: Share of tasks solved on first try. Early estimates from history, later from logs.
| Stage | Server FPSR | Frontend FPSR | Notes |
|-------|-------------|---------------|-------|
| Rules only | 40% | 30% | Baseline |
| +Specs | 75% | 60% | Familiar domain |
| +SENAR | 85%+ | 70%+ | Full stack |
Numbers are subjective, domain-dependent. FPSR drops 20% in unfamiliar areas.
Key Takeaways:
- AI agents are executors, not engineers: need disciplined specs and enforcement.
- Failure memory is crucial: saves 15–30 min/task.
- FPSR >80% achievable on familiar stacks with SENAR.
- Humans in the loop: architecture, reviews, decisions.
- 30+ projects: from PoC to production.
— Editorial Team
No comments yet.