Arg0/Docs

Agent Implementation

Technical deep dive into agent system - Understand the multi-process architecture, MCP tool integration, and execution engine that powers AI agents.

Detailed technical documentation of the agent implementation system.

Overview

Synapse Desktop implements AI agents through a multi-process architecture with MCP (Model Context Protocol) tool integration. The system enables agents to interact with local resources, web services, and external APIs through a secure IPC layer.

Agent Execution Model

Step Types

Agents execute tasks through a series of steps:

  • thought: Agent reasoning and planning
  • tool_call: Tool invocation with arguments
  • tool_result: Tool execution output
  • final: Final answer or summary

Step Status

Each step has a status:

  • thinking: Agent is processing
  • acting: Tool is being executed
  • done: Step completed successfully
  • error: Step failed

MCP Tool Catalog

Built-in tool definitions with schema:

Web Category

  • web_search: Search web with query
  • browser: Open URL and extract content
  • browser_automation: Playwright automation (navigate, click, type, etc.)

System Category

  • terminal: Execute shell commands
  • file_system: File operations (read, write, list, delete)
  • screenshot: Capture screenshots

Code Category

  • code_writer: Write/edit code files
  • code_reader: Read and analyze code
  • linter: Static analysis
  • test_runner: Execute test suites

Database Category

  • sql_executor: Run SQL queries

Integrations Category

  • api_caller: HTTP REST/GraphQL requests
  • github_api: GitHub REST API interaction
  • email: Send emails via mailto:

Tool Configuration

Tools are loaded from localStorage (configured in Settings > Skills & Tools):

typescript
function getMCPToolsFromSettings(): MCPTool[] {
  const saved = localStorage.getItem('mcp_tools')
  if (!saved) return []
  const enabledIds = JSON.parse(saved) as string[]
  return MCP_CATALOG_BASE.filter(t => enabledIds.includes(t.id))
}

OpenClaw Component

Tool Management Interface

Features:

  • Toggle tool enable/disable
  • View tool descriptions and parameters
  • Category-based organization
  • Visual feedback with icons

Tool Structure:

typescript
interface MCPTool {
  id: string
  name: string
  description: string
  category: string
  enabled: boolean
  icon: React.ReactNode
  inputs: string[]
  tags: string[]
}

Categories:

  • Web (Search, Browser, Automation)
  • Code (Writer, Reader, Linter, Test Runner)
  • System (Terminal, File System, Screenshot)
  • Database (SQL Executor)
  • Integrations (API Caller, GitHub, Email)

Chat Component

Interactive chat interface for:

  • Direct AI model interaction
  • Agent conversation history
  • Real-time streaming responses
  • Message persistence

Main Process IPC Handlers

Shell Operations

  • shell:run-command: Execute shell commands with timeout (15s)
  • shell:open-external: Open URLs in system browser

File System Operations

  • fs:home-dir: Get user home directory
  • fs:read: Read file contents
  • fs:write: Write file contents
  • fs:list: List directory entries
  • fs:mkdir: Create directories
  • fs:ensure: Create file with default content if missing
  • fs:delete: Delete files

Playwright Browser Automation

  • pw:navigate: Navigate to URL
  • pw:click: Click element by selector
  • pw:type: Type text into input
  • pw:get-text: Get element or page text
  • pw:screenshot: Capture screenshot
  • pw:evaluate: Execute JavaScript in page
  • pw:close: Close browser instance

Ollama Integration

  • ollama:list: List available local models
  • ollama:pull: Pull new models (with progress events)

Daemon Server

  • daemon:start: Start HTTP daemon on port 3999
  • daemon:stop: Stop daemon server
  • daemon:status: Get daemon status

MCP Server Management

  • mcp:start: Start MCP server with stdio JSON-RPC
  • mcp:stop: Stop MCP server
  • mcp:list-tools: List available tools from server
  • mcp:call-tool: Execute tool with arguments
  • mcp:status: Get server status

MCP Server Implementation

Protocol

Protocol: stdio JSON-RPC (Model Context Protocol)

Initialization Flow

  1. Spawn child process with command/args
  2. Send initialize handshake with protocol version
  3. Send notifications/initialized confirmation
  4. Server becomes ready for tool calls

Request/Response Pattern

typescript
// Request
{
  jsonrpc: "2.0",
  id: number,
  method: string,
  params: object
}

// Response
{
  jsonrpc: "2.0",
  id: number,
  result?: object,
  error?: { message: string }
}

Timeout

20 seconds per request

Process Management

  • Automatic cleanup on app quit
  • Error handling and restart capability
  • Status tracking (ready/not ready)

Preload Script API

Exposed API (window.api)

typescript
interface ElectronAPI {
  // Process info
  versions: NodeJS.ProcessVersions
  
  // Event handling
  send: (channel: string, data?: unknown) => void
  on: (channel: string, callback: (...args: unknown[]) => void) => void
  
  // Shell
  openExternal: (url: string) => Promise<void>
  runCommand: (cmd: string) => Promise<{stdout, stderr, exit_code}>
  
  // File system
  fsRead: (path: string) => Promise<string>
  fsWrite: (path: string, content: string) => Promise<{success}>
  fsList: (path: string) => Promise<{name, type}[]>
  fsDelete: (path: string) => Promise<{success}>
  fsHomeDir: () => Promise<string>
  fsMkdir: (path: string) => Promise<{success}>
  fsEnsure: (path: string, defaultContent: string) => Promise<{created}>
  
  // Playwright
  pwNavigate: (url: string) => Promise<{url, title}>
  pwClick: (selector: string) => Promise<{success}>
  pwType: (selector: string, text: string) => Promise<{success}>
  pwGetText: (selector?: string) => Promise<{text}>
  pwScreenshot: (outputPath?: string) => Promise<{path}>
  pwEvaluate: (script: string) => Promise<{result}>
  pwClose: () => Promise<{success}>
  
  // Daemon
  daemonStart: () => Promise<{running, port}>
  daemonStop: () => Promise<{running}>
  daemonStatus: () => Promise<{running, port}>
  onDaemonTask: (cb: (payload) => void) => void
  
  // Ollama
  ollamaList: () => Promise<string[]>
  ollamaPull: (model: string) => void
  onPullProgress: (cb: (data) => void) => () => void
  
  // MCP
  mcpStart: (name: string, cfg: MCPServerConfig) => Promise<{success, error?}>
  mcpStop: (name: string) => Promise<{success}>
  mcpListTools: (name: string) => Promise<{tools, error?}>
  mcpCallTool: (name: string, toolName: string, args) => Promise<{result?, error?}>
  mcpStatus: () => Promise<{servers: Array<{name, ready}>}>
}

Extending the System

Adding New Tools

  1. Define tool in catalog (AgentPage.tsx):
typescript
{
  id: 'my_tool',
  name: 'my_tool',
  description: 'Description',
  category: 'Category',
  parameters: {
    type: 'object',
    properties: {
      param1: { type: 'string', description: 'Param description' }
    },
    required: ['param1']
  }
}
  1. Add to OpenClaw (OpenClaw.tsx):
typescript
{
  id: 'my_tool',
  name: 'my_tool',
  description: 'Description',
  category: 'Category',
  enabled: false,
  icon: <MyIcon className="w-4 h-4" />,
  inputs: ['param1'],
  tags: ['tag1', 'tag2']
}
  1. Implement IPC handler (main/index.ts):
typescript
ipcMain.handle('my:tool', async (_event, param1: string) => {
  // Implementation
  return { result }
})
  1. Expose in preload (preload/index.ts):
typescript
myTool: (param1: string) => ipcRenderer.invoke('my:tool', param1)

Adding New MCP Servers

Configure server in Settings or programmatically:

typescript
await window.api.mcpStart('my-server', {
  command: '/path/to/server',
  args: ['--arg1', '--arg2'],
  env: { CUSTOM_VAR: 'value' }
})

Debugging

Main Process Logs

View in Electron DevTools > Console or terminal.

Renderer Logs

View in Electron DevTools > Console (Cmd+Option+I).

MCP Server Logs

Main process logs MCP stdout/stderr with prefix:

text
[MCP Main] server-name stdout: ...
[MCP Main] server-name stderr: ...

IPC Debugging

Add logging in preload script:

typescript
console.log('[IPC]', channel, data)