Agent Implementation
Technical deep dive into agent system - Understand the multi-process architecture, MCP tool integration, and execution engine that powers AI agents.
Detailed technical documentation of the agent implementation system.
Overview
Synapse Desktop implements AI agents through a multi-process architecture with MCP (Model Context Protocol) tool integration. The system enables agents to interact with local resources, web services, and external APIs through a secure IPC layer.
Agent Execution Model
Step Types
Agents execute tasks through a series of steps:
- thought: Agent reasoning and planning
- tool_call: Tool invocation with arguments
- tool_result: Tool execution output
- final: Final answer or summary
Step Status
Each step has a status:
- thinking: Agent is processing
- acting: Tool is being executed
- done: Step completed successfully
- error: Step failed
MCP Tool Catalog
Built-in tool definitions with schema:
Web Category
- web_search: Search web with query
- browser: Open URL and extract content
- browser_automation: Playwright automation (navigate, click, type, etc.)
System Category
- terminal: Execute shell commands
- file_system: File operations (read, write, list, delete)
- screenshot: Capture screenshots
Code Category
- code_writer: Write/edit code files
- code_reader: Read and analyze code
- linter: Static analysis
- test_runner: Execute test suites
Database Category
- sql_executor: Run SQL queries
Integrations Category
- api_caller: HTTP REST/GraphQL requests
- github_api: GitHub REST API interaction
- email: Send emails via mailto:
Tool Configuration
Tools are loaded from localStorage (configured in Settings > Skills & Tools):
function getMCPToolsFromSettings(): MCPTool[] {
const saved = localStorage.getItem('mcp_tools')
if (!saved) return []
const enabledIds = JSON.parse(saved) as string[]
return MCP_CATALOG_BASE.filter(t => enabledIds.includes(t.id))
}
OpenClaw Component
Tool Management Interface
Features:
- Toggle tool enable/disable
- View tool descriptions and parameters
- Category-based organization
- Visual feedback with icons
Tool Structure:
interface MCPTool {
id: string
name: string
description: string
category: string
enabled: boolean
icon: React.ReactNode
inputs: string[]
tags: string[]
}
Categories:
- Web (Search, Browser, Automation)
- Code (Writer, Reader, Linter, Test Runner)
- System (Terminal, File System, Screenshot)
- Database (SQL Executor)
- Integrations (API Caller, GitHub, Email)
Chat Component
Interactive chat interface for:
- Direct AI model interaction
- Agent conversation history
- Real-time streaming responses
- Message persistence
Main Process IPC Handlers
Shell Operations
shell:run-command: Execute shell commands with timeout (15s)shell:open-external: Open URLs in system browser
File System Operations
fs:home-dir: Get user home directoryfs:read: Read file contentsfs:write: Write file contentsfs:list: List directory entriesfs:mkdir: Create directoriesfs:ensure: Create file with default content if missingfs:delete: Delete files
Playwright Browser Automation
pw:navigate: Navigate to URLpw:click: Click element by selectorpw:type: Type text into inputpw:get-text: Get element or page textpw:screenshot: Capture screenshotpw:evaluate: Execute JavaScript in pagepw:close: Close browser instance
Ollama Integration
ollama:list: List available local modelsollama:pull: Pull new models (with progress events)
Daemon Server
daemon:start: Start HTTP daemon on port 3999daemon:stop: Stop daemon serverdaemon:status: Get daemon status
MCP Server Management
mcp:start: Start MCP server with stdio JSON-RPCmcp:stop: Stop MCP servermcp:list-tools: List available tools from servermcp:call-tool: Execute tool with argumentsmcp:status: Get server status
MCP Server Implementation
Protocol
Protocol: stdio JSON-RPC (Model Context Protocol)
Initialization Flow
- Spawn child process with command/args
- Send
initializehandshake with protocol version - Send
notifications/initializedconfirmation - Server becomes ready for tool calls
Request/Response Pattern
// Request
{
jsonrpc: "2.0",
id: number,
method: string,
params: object
}
// Response
{
jsonrpc: "2.0",
id: number,
result?: object,
error?: { message: string }
}
Timeout
20 seconds per request
Process Management
- Automatic cleanup on app quit
- Error handling and restart capability
- Status tracking (ready/not ready)
Preload Script API
Exposed API (window.api)
interface ElectronAPI {
// Process info
versions: NodeJS.ProcessVersions
// Event handling
send: (channel: string, data?: unknown) => void
on: (channel: string, callback: (...args: unknown[]) => void) => void
// Shell
openExternal: (url: string) => Promise<void>
runCommand: (cmd: string) => Promise<{stdout, stderr, exit_code}>
// File system
fsRead: (path: string) => Promise<string>
fsWrite: (path: string, content: string) => Promise<{success}>
fsList: (path: string) => Promise<{name, type}[]>
fsDelete: (path: string) => Promise<{success}>
fsHomeDir: () => Promise<string>
fsMkdir: (path: string) => Promise<{success}>
fsEnsure: (path: string, defaultContent: string) => Promise<{created}>
// Playwright
pwNavigate: (url: string) => Promise<{url, title}>
pwClick: (selector: string) => Promise<{success}>
pwType: (selector: string, text: string) => Promise<{success}>
pwGetText: (selector?: string) => Promise<{text}>
pwScreenshot: (outputPath?: string) => Promise<{path}>
pwEvaluate: (script: string) => Promise<{result}>
pwClose: () => Promise<{success}>
// Daemon
daemonStart: () => Promise<{running, port}>
daemonStop: () => Promise<{running}>
daemonStatus: () => Promise<{running, port}>
onDaemonTask: (cb: (payload) => void) => void
// Ollama
ollamaList: () => Promise<string[]>
ollamaPull: (model: string) => void
onPullProgress: (cb: (data) => void) => () => void
// MCP
mcpStart: (name: string, cfg: MCPServerConfig) => Promise<{success, error?}>
mcpStop: (name: string) => Promise<{success}>
mcpListTools: (name: string) => Promise<{tools, error?}>
mcpCallTool: (name: string, toolName: string, args) => Promise<{result?, error?}>
mcpStatus: () => Promise<{servers: Array<{name, ready}>}>
}
Extending the System
Adding New Tools
- Define tool in catalog (AgentPage.tsx):
{
id: 'my_tool',
name: 'my_tool',
description: 'Description',
category: 'Category',
parameters: {
type: 'object',
properties: {
param1: { type: 'string', description: 'Param description' }
},
required: ['param1']
}
}
- Add to OpenClaw (OpenClaw.tsx):
{
id: 'my_tool',
name: 'my_tool',
description: 'Description',
category: 'Category',
enabled: false,
icon: <MyIcon className="w-4 h-4" />,
inputs: ['param1'],
tags: ['tag1', 'tag2']
}
- Implement IPC handler (main/index.ts):
ipcMain.handle('my:tool', async (_event, param1: string) => {
// Implementation
return { result }
})
- Expose in preload (preload/index.ts):
myTool: (param1: string) => ipcRenderer.invoke('my:tool', param1)
Adding New MCP Servers
Configure server in Settings or programmatically:
await window.api.mcpStart('my-server', {
command: '/path/to/server',
args: ['--arg1', '--arg2'],
env: { CUSTOM_VAR: 'value' }
})
Debugging
Main Process Logs
View in Electron DevTools > Console or terminal.
Renderer Logs
View in Electron DevTools > Console (Cmd+Option+I).
MCP Server Logs
Main process logs MCP stdout/stderr with prefix:
[MCP Main] server-name stdout: ...
[MCP Main] server-name stderr: ...
IPC Debugging
Add logging in preload script:
console.log('[IPC]', channel, data)
Related Documentation
- Architecture - System architecture overview
- MCP Integration - MCP protocol details
- IPC API - IPC API reference