File access in local AI agent: tools

File Access for Local AI Agents: Tools and Bug Fixes

Our local AI agent, Doka, now has file access capabilities, but not before fixing a critical KV-cache bug in llama.cpp. The system prompt started with Session started: ${new Date().toISOString()}, which changed every iteration. This invalidated the entire cache: the model recalculated 2000 prompt tokens from scratch over 10 iterations, wasting 20,000 extra tokens.

Moving the timestamp to the end of the prompt or user turn restored the cache. Multi-step tasks sped up 2–3x on loops with 8+ iterations. This pattern mirrors Anthropic's recommendations for Claude but is critical for local models too.

Secure File Access with Permission Gates

File reading tools are straightforward, but writing needs safeguards. Each tool in the toolbox has a dangerous flag:

const fileTools: Tool[] = [
  {
    name: 'read_file',
    dangerous: false,
  },
  {
    name: 'write_file',
    dangerous: true,
  },
  {
    name: 'edit_file',
    dangerous: true,
  },
  {
    name: 'delete_file',
    dangerous: true,
  },
];

When a dangerous tool is called, a dialog pops up showing the arguments: path and content (first 500 characters). The "Allow" button isn't the default. Denials return an error to the agent:

{
  "error": "User denied permission for write_file",
  "canRetry": false,
  "suggestion": "Ask the user to confirm the intended file path"
}

Full Suite of File Tools

We've implemented six tools:

• read_file(path: string): string
• write_file(path: string, content: string): void (dangerous)
• edit_file(path: string, instruction: string): void (dangerous)
• list_dir(path: string): FileEntry[]
• move_file(from: string, to: string): void (dangerous)
• delete_file(path: string): void (dangerous)

edit_file applies a patch based on the instruction (read → patch → write → verify), avoiding passing large files through context.

Fixing Path-Related Errors

The agent mixed up files with the same names in different directories due to relative paths. The fix:

• Default working directory specified in the system prompt.
• Path validation: going outside bounds triggers an error.
• Instruction: ask the user if paths are ambiguous.

The bug showed up in tests 3–4: the agent picked notes.md from the root instead of the target folder.

After write_file or edit_file, we added auto-verify: an automatic read_file check. This adds a step but prevents silent failures (permissions, disk issues). MAX_TOOL_STEPS bumped to 12 for plan + read + write + verify.

Agent Timeline for Transparency

Users complained about the "black box": three minutes of waiting with no insight into steps. We added a sidebar log:

• Tool calls with arguments.
• Results.
• Timestamps.

Event streams (tool_call_start / tool_call_end) render in React in real-time. Essential for file access: users see exact paths and operations.

Structured Errors and Retry Logic

Previously, errors dumped Node.js stack traces, leading to infinite retries or hallucinations. Now they're structured:

{
  "error": "File not found: /path/to/file.md",
  "canRetry": false,
  "suggestion": "Check if the path is correct with list_dir first"
}

canRetry cuts down on pointless loops. True for network timeouts, false for nonexistent files.

Restructured System Prompt

The system prompt is now sectioned:

• Role and Boundaries: Capabilities and limits.
• Prioritized Rules: Safety > accuracy > speed.
• Response Format: Structure for different types.
• Few-shot Example: Reasoning on an edge case.

This structure cut hallucinations in long chains, especially rule-based refusals.

Planning Phase: Think → Plan → Act

Instruction before tools:

Before using tools, write a short plan:
THINK: [what information do I need?]
PLAN: [step 1 → step 2 → step 3]
ACT: [execute plan]

Halves redundant tool calls. Forces goal-setting before action.

Key Takeaways

• Moving dynamic data (timestamps) to the prompt end speeds up loops 2–3x via KV-cache.
• Permission gates with dialogs and structured errors minimize write risks.
• Auto-verify and path validation eliminate silent file op failures.
• Agent Timeline builds trust through step transparency for file access.
• Structured prompts with planning boost reliability on multi-step tasks.

— Editorial Team