The Complete Guide to Agent Configuration: How I Built a Self-Optimizing Workspace for My AI Agent
Most people treat AI agents like chatbots. They complain about inconsistent personality, forgotten context, and unpredictable behavior. But the problem isn't the model — it's their configuration files.
"The files ARE the agent." I've seen this sentiment across multiple AI communities, and after spending a full day restructuring my agent's workspace, I finally understand what it means. Your workspace files aren't just documentation. They're the agent's DNA — its personality, memory, operating manual, and safety rails all encoded in plain text.
This article documents what I learned from that day. Not theory. Real problems, real fixes, real code.
What Are Agent Configuration Files?
Every AI agent framework uses some form of persistent configuration — files that load before every conversation and define how the agent behaves across sessions.
| Framework | Config File | What It Does |
|---|---|---|
| OpenClaw | SOUL.md, USER.md, AGENTS.md, etc. | Full workspace with separated concerns |
| Claude Code | CLAUDE.md | Single file for all project context |
| Cursor | .cursorrules | Editor-specific behavior rules |
| Codex | AGENTS.md | Repository-level agent instructions |
| Custom Agents | System prompt / .env | Usually hardcoded or minimal |
The concept is the same everywhere: give the agent persistent context that survives between sessions. Without it, every conversation starts cold — the agent doesn't know who you are, how you like to communicate, or what it's working on.
The difference is in how you structure it. Most frameworks use a single file. That works for small projects. But when your agent handles multiple projects, has complex workflows, and needs a stable personality — a single file becomes unmaintainable.
This guide uses OpenClaw as the example, but the principles apply to any agent framework. If you're using Claude Code, think of this as "how to split your CLAUDE.md into a maintainable system." If you're building custom agents, these patterns work for system prompts too.
The core insight: agent configuration is not a prompt. It's an architecture decision.
The Problems I Found
My agent, Jasper, had been running for weeks. On the surface, it worked. But small inconsistencies kept accumulating:
Problem 1: Missing SOUL.md
No personality file. The agent had no stable voice, no defined boundaries, no sense of self. Each session started from zero. Sometimes helpful, sometimes formal, sometimes weirdly casual. Inconsistent because nothing defined what "consistent" meant.
Problem 2: Missing USER.md
No user profile. Every session was a cold start. The agent didn't know I prefer direct answers over explanations, that I work in Australia/Sydney timezone, that I hate phrases like "Let me help you with that." All this had to be re-established every time.
Problem 3: Information in the Wrong Files
User preferences buried in MEMORY.md. Behavioral rules scattered across IDENTITY.md. The system "worked" but was impossible to maintain. When I wanted to update my communication preferences, I had to search three files.
Problem 4: Bloated WORKING.md
My working notes file had grown to 1,600 tokens. Every session loaded this entire blob — most of it stale project notes from weeks ago. Pure waste.
Total bootstrap cost: 2,847 tokens before the agent even read my first message. Unacceptable.
Harness Engineering: The Framework
OpenAI introduced "harness engineering" as a concept, but the core idea is simple: prompt engineering is for single conversations, harness engineering is for permanent personality.
Think of it like genetics. A prompt is what you say to someone in a conversation. A harness is who they are across all conversations. Workspace files define the agent's unchanging characteristics — personality, values, safety rules, operational procedures — that persist regardless of what you're discussing.
Sam Saffron nailed it: "Style is garnish. Structure is structural." A witty personality prompt is nice. But clear separation between identity, rules, and memory is what makes an agent maintainable over months of daily use.
The key insight: workspace files are loaded before every session. This makes them incredibly powerful — and incredibly expensive if misused. Every token in your bootstrap files costs you on every single interaction.
This shifts how you think about configuration:
- Stop asking "what should the agent know?"
- Start asking "what must the agent know on every single session?"
Anything that doesn't need to be in every session should live in referenced docs, not bootstrap files.
The Six Core Files
After restructuring, here's what each file does — and just as importantly, what it doesn't do.
| File | Purpose | What Goes Here | What Doesn't Go Here |
|---|---|---|---|
SOUL.md | WHO I am | Personality, tone, values, boundaries | Rules, procedures, user info |
USER.md | WHO I serve | User profile, preferences, timezone | Agent personality, operational rules |
IDENTITY.md | My name card | Name, emoji, one-liner | Behavioral rules, preferences |
AGENTS.md | Operating manual | Session startup, rule index, principles | Detailed rules (those go in docs/) |
TOOLS.md | Safety rules | Critical don'ts, dangerous operations | Tool documentation, how-tos |
MEMORY.md | World state dashboard | Project index, current focus, links | User preferences, personality traits |
Let me show you the before/after for each.
SOUL.md — Personality and Values
Before (what I had): Nothing. The file didn't exist.
After:
# SOUL.md — Who I Am
## Personality
- Direct and concise. No filler phrases.
- Technical terms stay in English.
- Proactive but not pushy. Do the work, report results.
- Australian English spelling (colour, organisation, behaviour).
## Values
- Transparency over completion. Report failures before trying alternatives.
- Files over memory. Persistent info goes to disk, not context.
- Safety first. Use trash over rm. Confirm before publishing.
## Boundaries
- Never pretend to have done something I haven't.
- Never silently change the plan.
- Never send emails or publish content without explicit confirmation.
## Anti-patterns (things I don't do)
- ❌ "Let me help you with that"
- ❌ "Great question!"
- ❌ Over-explaining when not asked
- ❌ Hedging with "I think" when I know
The difference is night and day. Now the agent has a stable personality that survives session restarts. The "anti-patterns" section is particularly valuable — it explicitly bans behaviors that would annoy me.
USER.md — User Profile
Before: Preferences scattered across MEMORY.md and random session contexts.
After:
# USER.md — Who I Serve
## Profile
- **Name:** Ming Fang
- **Location:** Sydney, Australia (GMT+11)
- **Work hours:** Usually 9am-6pm AEDT, sometimes later
- **Primary device:** Mac Studio (Apple Silicon)
## Communication Preferences
- Answers first, explanations only if asked
- Code blocks over prose when showing implementation
- Direct feedback welcome — no need to soften criticism
- English for technical writing, Chinese acceptable for casual chat
## Current Projects
See `MEMORY.md` for active project list.
## Known Patterns
- Often fires off multiple requests while waiting for previous results
- Prefers file outputs over conversation dumps (Envelope Pattern)
- Will explicitly say "think out loud" when wanting to see reasoning
Now the agent knows who it's serving. It adapts to my communication style from the first message. No warm-up period.
IDENTITY.md — The Name Card
Before: Had behavioral rules mixed in with identity.
# IDENTITY.md
- Name: Jasper
- Always check FOCUS.md first ← Wrong file!
- Use trash instead of rm ← Wrong file!
After:
# IDENTITY.md — Who Am I?
- **Name:** Jasper
- **Emoji:** ⚡
- **One-liner:** OpenClaw AI Agent on Ming's Mac Studio
That's it. Three lines. Identity is just identity — name, symbol, tagline. Everything else belongs elsewhere.
AGENTS.md — Operating Manual
Before: A 2,000-token wall of text with rules, procedures, and examples all mixed together.
After:
# AGENTS.md — Operating Manual
> L0 index layer, ≤1.5KB. Detailed rules → docs/AGENTS_RULES.md
## Every Session
1. Read `FOCUS.md` — if task exists, execute immediately
2. Read `memory/YYYY-MM-DD.md` (today + yesterday)
3. Route each message by tier (see `docs/ROUTING_RULES.md`)
## Core Principles
- FOCUS.md = current priority. Do it first.
- Important info must go to files, not memory.
- Try before asking. Come with "I tried X and Y" not just questions.
- Safety: `trash` > `rm`. Confirm before sending/publishing.
## Trigger Rules
| Scenario | Rule File |
|----------|-----------|
| Writing code | `docs/MODEL_ROUTING.md` |
| Git operations | `docs/GIT_RULES.md` |
| Deployment | `docs/DEPLOY_RULES.md` |
The magic: index, don't include. The bootstrap file points to detailed docs. Those docs only load when relevant. Saves hundreds of tokens per session.
TOOLS.md — Safety Rails
Before: Included full command documentation.
After:
# TOOLS.md — Critical Safety Rules
> Only "violate this and things break" rules. Command reference → docs/TOOLS-REFERENCE.md
## 🔐 Environment Variables
- ❌ Never add API keys without asking
- ❌ Never hardcode keys in source
- ✅ Key change flow: Vercel env → ~/.env backup → local .env copy
## 🛡️ Config Changes
# ❌ Direct edit of ~/.openclaw/openclaw.json
# ✅ Use jasper-configguard
jasper-configguard patch --dry-run '{"key": "value"}'
## 🧩 coding-agent
# MUST use pty:true or it hangs
exec pty:true workdir:~/project command:"claude --dangerously-skip-permissions 'task'"
Only rules where violation causes real damage. Everything else lives in reference docs.
MEMORY.md — World State Dashboard
Before: User preferences, project history, random notes, stale todos.
After:
# MEMORY.md — World State Dashboard
> Current state only. History → memory/YYYY-MM-DD.md
## Active Focus
See `FOCUS.md` for current task.
## Project Index
| Project | Path | Status |
|---------|------|--------|
| contact-site | ~/ACTIVE/contact-site | Active |
| openclaw-skills | ~/.openclaw/skills | Maintenance |
## Quick Links
- Daily notes: `memory/`
- Rule docs: `docs/`
- Skills: `~/.openclaw/skills/` and `~/.agents/skills/`
## World Context
- Using OpenClaw with Claude models
- Primary channel: Telegram
- Backup channel: Discord
Clean dashboard. No preferences (that's USER.md). No personality (that's SOUL.md). Just world state.
Three Design Principles
1. Separation of Concerns
The single biggest mistake: mixing personality with rules. They update at different rates. Personality rarely changes. Rules update as you discover new failure modes.
Keep them separate:
SOUL.md→ WHO (stable)AGENTS.md→ HOW (evolves)TOOLS.md→ DON'T (grows with incidents)
2. Index Over Content
Bootstrap files should be pointers, not encyclopedias.
# ❌ Bad: Full rules in AGENTS.md
## Git Rules
1. Always commit before major changes
2. Use conventional commits format
3. Never force push to main
4. ... (200 more tokens)
# ✅ Good: Index only
## Trigger Rules
| Scenario | Rule File |
|----------|-----------|
| Git operations | `docs/GIT_RULES.md` |
The agent loads docs/GIT_RULES.md only when doing git work. Every other session skips it.
3. Token Budget
Every bootstrap token costs you on every message. Set limits:
| File | Max Tokens |
|---|---|
| SOUL.md | 400 |
| USER.md | 300 |
| IDENTITY.md | 50 |
| AGENTS.md | 600 |
| TOOLS.md | 500 |
| MEMORY.md | 400 |
| Total | ≤2,250 |
My original setup: 2,847 tokens. After restructuring: 1,680 tokens. That's a 41% reduction — compounding savings on every single interaction.
Automatic Optimization: The soul-keeper Skill
Configuration drift is real. You write these files, they work, then they slowly rot. New patterns emerge that aren't documented. Old rules become irrelevant. Files bloat.
The solution: a skill that monitors for optimization opportunities. Not waiting for complaints — proactively detecting when files need updates.
| File | Update Trigger |
|---|---|
| SOUL.md | User corrects tone/style 3+ times in a week |
| USER.md | New explicit preference stated |
| AGENTS.md | New recurring workflow emerges |
| TOOLS.md | Safety incident occurs |
| MEMORY.md | Project status changes |
| FOCUS.md | Task completed or priority shifts |
Implementation sketch:
// soul-keeper trigger detection
const triggers = {
'SOUL.md': {
condition: 'style_corrections_this_week >= 3',
action: 'suggest_personality_update'
},
'TOOLS.md': {
condition: 'safety_incident_detected',
action: 'require_immediate_rule_addition'
},
'MEMORY.md': {
condition: 'project_completed || focus_changed',
action: 'prompt_dashboard_update'
}
};
The key: triggers are file-specific. SOUL.md updates come from repeated corrections. TOOLS.md updates come from incidents. MEMORY.md updates come from project changes. One-size-fits-all monitoring doesn't work.
The Result
30 minutes of restructuring. Benefits on every session going forward.
| Metric | Before | After |
|---|---|---|
| Bootstrap tokens | 2,847 | 1,680 |
| Files with clear ownership | 2/6 | 6/6 |
| Personality consistency | Low | High |
| Time to find a rule | ~60s | ~10s |
The real win isn't the token savings — though those compound fast. It's the maintainability. When I want to update my communication preferences, I know exactly where to look: USER.md. When a new safety rule emerges, it goes in TOOLS.md. No searching, no duplication, no drift.
Agent configuration compounds. Every session is an opportunity to discover something that should be documented. Every correction is a rule waiting to be written. Every personality inconsistency is a gap in SOUL.md.
The files ARE the agent. Make them good.
Templates and Tools
Want to start with the structure I use? All the skills mentioned in this article are open source:
📦 openclaw-skills — A curated collection of OpenClaw skills including soul-keeper, plan-mode, self-improving, and more.
soul-keeper — Self-Optimizing Workspace Files
The skill that auto-detects when your workspace files need updating. Instead of waiting for things to break, it monitors for real triggers — style corrections, new preferences, project changes — and prompts you with specific update suggestions.
Install via ClawHub:
clawhub install soul-keeper
Or grab it from GitHub and drop it into your workspace skills/ directory.
Other Skills Worth Checking Out
| Skill | What It Does | Install |
|---|---|---|
| plan-mode | Think-only mode — analyze and plan, never execute | clawhub install plan-mode |
| self-improving | Self-reflection + learning from corrections | clawhub install self-improving |
| mermaid-architect | Generate robust Mermaid diagrams | clawhub install mermaid-architect |
Further Reading
- Building Effective Agents — Anthropic
- Harness Engineering for AI Systems — OpenAI
- OpenClaw Documentation
- Full skills collection on GitHub
Related posts:
Interested in AI governance for your firm?
Let's have a practical conversation about where you stand.
Get in Touch →