General Manual

Familiars Complete Operation Manual (User Perspective) — Every feature: "where it is, what to click, what keys to press, what you see." ⚠️ Source: repo root OPERATION.md (§1–§42) + all Vue components under electron/renderer/src/components/ + all routes under src/bridge/routes/.

Section	Core Component	Entry Point	One-liner
§1 Startup & Run Modes	`electron/python.js` → bridge	`cd electron && npm run electron:dev`	Frontend Electron + backend FastAPI bridge (127.0.0.1:18711)
§2 Main Window Layout	`Desktop.vue` + `ResizeHandle.vue`	Visible on launch	Four-column layout: AgentRail / DialogList / ChatStage + title bar
§3 Title Bar	`TitleBar.vue`	Top of window	Identity area (left) + window controls (right): pin/minimize/maximize/close
§4 Manager Homepage	`Desktop.vue`	Select agent 0 with no dialogue	Desktop shell: wallpaper + icons + floating windows + taskbar
§5 Agent List	`AgentRail.vue`	Leftmost column	Avatar list, + to create, 🎁/⏰ badges, right-click actions
§6 Create Agent	`CreateAgentModal.vue`	`+` at top of AgentRail	Name + role description + avatar → async asset generation
§7 Agent Settings	`AgentSettings.vue`	Right-click AgentRail	Model/prompt/memory/intense_mode/turbo_mode/approval level
§8 Dialogue List	`DialogList.vue`	After selecting an agent	Dialogue entries + create + rename/delete + workshop virtual dialogues
§9 Chat Stage	`ChatStage.vue`	Select a dialogue	Message stream + tool cards + split pane + tree view + CG overlay
§10 Dialogue Tree View	`TreeMini.vue` / `TreeFull.vue`	Thumbnail in ChatStage corner	Drag/search/export/jump nodes
§11 Input Bar	`InputBar.vue`	Bottom of ChatStage	Attachments/folder/Skill/KB/quick inputs/@model/text/voice/send
§12 Attachment Rendering	`AttachmentRenderer.vue`	Inside message bubbles	Auto-selected by kind: image/video/audio/link/file
§13 Quick Inputs / Model / Skill	`InputBar.vue` sub-panels	Input bar ⚡/@/🧩/📚	Phrase assembly, temp model switch, Skill/KB references
§14 Voice Input	`InputBar.vue` 🎙️	Microphone button bottom-right	Push-to-talk → faster-whisper local transcription → append to input
§16 Digital Human	`DigitalHuman.vue`	ChatStage 👤 toggle	WebSocket `/ws/digital_human/<agent_id>` drives lip-sync/expressions
§18 Reports	`ReportApp.vue`	Desktop Reports icon	Aggregated daily/weekly/monthly/annual reports from all agents
§19 Skill Forge	`SkillForgeModal.vue`	ChatStage multi-select → Forge	3-step wizard: params → review → commit
§20 Skill Approvals	`SkillApprovalsView.vue`	Desktop Skill Approvals icon	Approve/reject/defer approval queue
§21 Todo/Schedule/Cron	`TodoScheduleModal.vue`	Desktop Todo icon / Title bar	Todo/Schedule/Cron three tabs
§22 Global Quick Draft	`DraftConfirmModal.vue`	`Ctrl+Shift+A/S/T`	Clipboard → LLM parse → Todo/Schedule
§23 Reminder Toast	`ReminderToast.vue`	Top-right floating card	schedule/todo/desktop reminders + OS notifications
§24 Privacy Approval	`PrivacyApprovalCard.vue`	Card inside dialogue	gitleaks/Presidio/mini-model → ignore/mask/record
§25 AskUser Card	`AskUserCard.vue`	Card inside dialogue	Agent asks back → chip prefills / text box answer
§26 CG Animation Workshop	`CgWorkshop.vue`	Desktop CG Workshop icon	Generate intro CG videos for agent+skill
§27 Agent Workshop	`AgentWorkshop.vue`	Desktop Agent Workshop icon	Import OpenClaw/Hermes/custom agents
§28 Message Monitor	`MonitorApp.vue`	Desktop Message Monitor icon	WeChat/QQ monitor start/stop + contact configuration
§29 User Profile	`UserProfileModal.vue`	Click TitleBar identity area	Nickname + avatar + self-introduction
§30 Background/Theme/Chat Settings	`ChatSettings.vue`	ChatStage top ⚙️	Font/size/background/pin-to-top
§31 Background Running	`SettingsCenter.vue` Background tab	TitleBar ⚙️ / Tray	Background toggle + auto-start + PID + exit
§32 System Tray	Electron Tray	While running in background	Double-click to restore / right-click menu / tooltip
§33 Status Overlay	`StatusOverlay.vue`	`?view=overlay` URL	Red/orange/purple/blue color-coded system takeover status
§34 Model Missing Prompt	`ModelMissingModal.vue`	Startup/switch dialogue/trigger Skill	download / skip / never_prompt
§35 Keyboard Shortcut Reference	—	—	Enter/Shift+Enter/Esc×3/Ctrl+Shift+A\|S\|T
§36 Settings Center	`SettingsCenter.vue`	TitleBar ⚙️ / Desktop Settings icon	LLM connection/background/notifications/Skill management/about

§1 Startup & Run Modes

Dev Startup

bash

# Backend dependencies (first time)
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Launch desktop (frontend + auto-spawn backend bridge)
cd electron
npm run electron:dev

Data flow: Electron main process (electron/electron/main.js) creates BrowserWindow → electron/electron/python.js spawns .venv\Scripts\python -m src.bridge.run → bridge binds to 127.0.0.1:18711 (hypercorn + HTTPS/2 + self-signed cert workspace/.bridge.crt) → frontend accesses REST API + WebSocket via https://127.0.0.1:18711.

Port fallback: If 18711 is occupied, python.js calls pickFreePort() to select a free port, writes to workspace/.bridge.port + workspace/.bridges.json.

Authentication: Bridge generates workspace/.bridge_token at startup. BridgeTokenMiddleware validates X-Bridge-Token header on every request. WebSocket accept also validates token.

Two run modes:

Foreground mode (default): Close main window → exits entire process.
Background mode: Close main window → bridge process persists. Scheduled tasks, reminders, desktop notifications, message monitors continue working.

Production build: cd electron && npm run electron:build → output in electron/dist/.

CLI (UI-less bridge driver): python main.py --help → CLI covering all REST endpoints. It does NOT start the bridge — it only sends requests to a running bridge. Token is auto-read from workspace/.bridge_token.

§2 Main Window Layout Overview

┌─────────────────────────────────────────────────────────────┐
│ TitleBar (28px, pink background)                             │
├──────┬──────────┬───────────────────────────────────────────┤
│Agent │ Dialog   │ ChatStage (message stream + tool cards)    │
│Rail  │ List     │                                           │
│(60px)│(240px)   │                                           │
│      │          │                                           │
│      │          ├───────────────────────────────────────────┤
│      │          │ InputBar (attachments/Skill/quick/text/voice) │
├──────┴──────────┴───────────────────────────────────────────┤
│ (optional) StatusOverlay — top-right edge-floating window    │
└─────────────────────────────────────────────────────────────┘

Component mapping:

Title bar → TitleBar.vue (28px height, var(--pink-frame) background)
AgentRail → AgentRail.vue (60px width, collapsible to 0)
DialogList → DialogList.vue (240px width, collapsible)
ChatStage → ChatStage.vue (flexible width)
InputBar → InputBar.vue (pinned to bottom of ChatStage)

Interactions:

ResizeHandle.vue between columns allows drag-to-resize, persisted to localStorage
Each column has a collapse button (small arrow icon) at top — click to collapse/expand
Triple Esc (3 times within 1.5 seconds) → closes in order: tree fullscreen → split pane → settings modal → interrupt generation → return to homepage

§3 Title Bar

Component: TitleBar.vue

Layout:

Left side (identity area): UserAvatar (18px) + user nickname (store.userProfile.nickname). Click → opens UserProfileModal (§29)
Center: -webkit-app-region: drag drag region (entire title bar can drag the window)
Right side (window controls):
- 📌 Pin (pin): api.togglePin() → window always-on-top. Active state on class (blue highlight)
- — Minimize: api.minimize()
- □ Maximize/Restore: api.maximizeToggle() → icon dynamically switches between maximize/restore
- × Close: api.close() → foreground mode=exit, background mode=hide window

Data flow: TitleBar.vue communicates with Electron main process via window.electronAPI.window IPC. On onMounted, calls api.state() to get initial pinned/maximized state and subscribes to api.onState for changes.

§4 Manager Homepage (Desktop)

Component: Desktop.vue (replaces legacy ManagerLanding.vue)

Entry: Displayed when Manager Agent (id=0) is selected in AgentRail with no specific dialogue selected.

Switch to legacy: localStorage.setItem('familiars:legacyLanding','1') → refresh → ManagerLanding.vue.

Layout: Wallpaper + icon grid + bottom taskbar

Built-in app icons (BASE_APPS array, defined in Desktop.vue lines 63-100):

id	Title	Icon Source	Component	Window Size
`report`	Reports	`entry-image:report`	`ReportApp.vue`	900×620
`learn`	Learn Familiars	`entry-image:learn`	`LearnApp.vue`	940×640
`workshop`	Creative Workshop	`entry-image:lab_researcher`	`WorkshopApp.vue` → `CreativeWorkshop.vue`	1300×780
`demo`	Replicate DEMO	`entry-image:demo`	`DemoApp.vue` (comingSoon)	640×440
`tasks`	Todo·Schedule·Cron	`entry-image:flipping_desk_calendar`	`TodoScheduleModal.vue` (embedded)	960×640
`settings`	Settings	`entry-image:workshop`	`SettingsCenter.vue` (embedded)	880×620
`monitor`	Message Monitor	`entry-image:checking_surveillance_camera`	`MonitorApp.vue`	520×620
`apps`	App Store	`entry-image:figure_shopping_bag`	`AppsApp.vue`	1000×660
`cg-workshop`	CG Animation Workshop	`local-image:/cg-workshop-cover.png`	`CgWorkshopApp.vue`	700×560
`agent-workshop`	Agent Workshop	`local-image:/agent-workshop-cover.png`	`AgentWorkshopApp.vue`	700×560
`skill-workshop`	Skill Workshop	`local-image:/skill-workshop-cover.png`	`SkillWorkshopApp.vue`	700×560
`skill-approvals`	Skill Change Approvals	`local-image:/skill-approvals-cover.png`	`SkillApprovalsApp.vue`	700×560

Broker app icons: Installed brokers listed in AppsApp.vue also appear on the desktop, each broker's cover.png used as icon. Double-click → BrokerAppWindow.vue (iframe loads /api/broker_app/<id>/ui/<entry>).

Actions:

Double-click icon → opens floating window (WindowManager managed, draggable/minimizable/maximizable/closable)
Long-press or right-click icon → reposition / uninstall (third-party broker icons only)
Right-click empty area → change wallpaper, organize icons
Bottom taskbar → lists opened/minimized windows, click to restore

First-launch banner: If background auto-start registration fails, a "Auto-start registration failed, click to retry" banner appears at top of homepage → click Retry or Don't show again.

§5 Agent List (AgentRail)

Component: AgentRail.vue

Position: Leftmost column, 60px wide

Actions:

+ New agent: Top button → opens CreateAgentModal (§6)
Click avatar: Switch to that agent's dialogue (store.selectAgent(id))
Right-click/long-press: Open action menu (completed in Agent Settings)
Collapse/Expand: Top arrow button. When collapsed, only avatars remain (32px wide)
Creating placeholder: Semi-transparent placeholder block + LoadingSpinner.vue, hover to see error details

Sorting: Manager agent (id=0) is always at top, remaining agents sorted by most recent activity time.

Badges:

🎁 Gift badge: agent.gift_pending === true → gold badge at top-right of avatar (§17 GiftBox)
⏰ Alarm icon: Agent is executing a scheduled task
Red border: intense_mode enabled (§7)
Orange border: turbo_mode enabled (§7)

§6 Create Agent (CreateAgentModal)

Component: CreateAgentModal.vue

Entry: + New agent button at top of AgentRail

Data flow:

User fills form → click Create
Frontend POST /api/agents → backend src/bridge/routes/agents.py:create_agent
Backend creates workspace/agents/agent_<id>/ directory + config.json
Async tasks: generate voice.wav (parler TTS) / figure.png / report templates
AgentRail shows semi-transparent placeholder → refreshes to new avatar on completion

Form fields:

Name (required): Display name, also used as directory display name in workspace/agents/agent_N/
Role description: Natural language description of agent's purpose
Identity: Written to IDENTITY.md
Avatar (optional): Upload ≤2MB image → used as portrait for regular avatar + digital human static base

Variants: CreateAgentKindModal.vue provides creation method selection (blank / from template / from external import).

§7 Agent Settings (AgentSettings)

Component: AgentSettings.vue (right-side drawer panel)

Entry: Select agent in AgentRail → TitleBar settings button or right-click agent

Sections:

Field	Type	Default	Description
Nickname	Text	—	Agent's name
Context window	Number	5	Direct dialogue history window size (`SLIDE_WINDOW_SIZE`)
Dialogue model	Text	`DEFAULT_MODEL`	Configure specific model for this agent
Working directory	Path	`/workspace`	Agent can process files under this directory without approval
intense_mode	Toggle	Off	Unsupervised mode: auto-approves all approvals + does not ask user for help. Avatar shows red border
turbo_mode	Toggle	Off	Full authorization mode: auto-approves all approvals + can still ask for help. Avatar shows orange border
security_level	Dropdown	—	Three-tier approval level, only tools with permission requirement ≥ this level need approval
verify_plan	Toggle	Off	Plan approval: after agent creates plan, submit to user for review first
Auto-apply SKILL changes	Toggle	Off	Manager can modify SKILL.md without user consent
need_initiative	Toggle	Off	Allow agent to proactively set monitor tasks / proactively contact user
Avatar background color	Color picker	—	AgentRail avatar background color
Dialogue list text color	Color picker	—	DialogList entry text color
Chat bubble text color	Color picker	—	ChatBubble text color
Replace appearance	File upload	—	Upload image to replace digital human assets
Regenerate voice	Button	—	Based on `IDENTITY.md` voice description, use parler to re-synthesize `voice.wav` (1-3 minutes)
Replace voice	File upload	—	Upload audio to replace agent's voice
Delete this agent	Button	—	Delete all assets under agent directory (digital human/chats/memory/KB/SKILL)

Data flow: After modification, PATCH /api/agents/<id> → src/bridge/routes/agents.py:patch_agent → writes to config.json. Deletion via DELETE /api/agents/<id> → entire directory recursively deleted.

§8 Dialogue List (DialogList)

Component: DialogList.vue

Position: Second column, 240px wide

Actions:

+ New topic: + button at top of list → POST /api/dialogues → create new dialogue
Switch dialogue: Click entry → store.selectDialogue(id) → ChatStage loads that dialogue
Rename: Double-click dialogue title → enter edit mode → Enter to save (PUT /api/dialogues/<id>) / Esc to cancel
Delete: Hover 🗑 icon on right → confirm → DELETE /api/dialogues/<id>
Collapse: Top arrow to collapse entire column

Manager virtual dialogues (only in agent 0's list):

Name	dialogue_id	Component
CG Animation Workshop	-1001	`CgWorkshop.vue`
Agent Workshop	-1004	`AgentWorkshop.vue`
Skill Workshop	-1002	`SkillWorkshop.vue`
Skill Change Approvals	-1003	`SkillApprovalsView.vue`

These virtual dialogues do not count toward "real dialogue" count.

§9 Chat Stage

Component: ChatStage.vue

Position: Rightmost large area

Top Bar

Current agent name + status indicator
Local search: Fuzzy keyword search in current session, highlight matches
Multi-select: Enter message multi-select mode → top batch action bar → trigger SkillForge (§19) or export HTML/Markdown/PDF
Chat settings: Opens ChatSettings.vue (§30)
Open in new window: Split current dialogue into independent BrowserWindow
Split pane: Opens SplitPane → left/right comparison of two dialogues
Delete dialogue: Delete current dialogue
Return to main: Return to Desktop

Message Stream

User bubbles (right): Rendered by ChatBubble.vue
Agent bubbles (left): Avatar.vue on the left side
Each message can:
- Click Copy → full copy to clipboard
- Click Read aloud → digital human reads (via /api/dh/speak)
- Click Regenerate → re-answer
- Click Translate → translate to configured language
- Click Edit → directly edit message content
- Click Delete → delete this message and its subtree
- Click @ → select specific model to answer (TODO)

Special Message Cards

Card	Component	Trigger
Gift	`GiftBox.vue`	Agent sends gift payload
Privacy approval	`PrivacyApprovalCard.vue`	Sensitive content detected
AskUser	`AskUserCard.vue`	Agent calls `ask_user` tool
Plan approval	`PlanApprovalCard.vue`	When verify_plan is enabled
Event wait	`EventWaitCard.vue`	Waiting for external event
Tool call	`ToolRendererCard.vue`	When agent calls a tool
CG video	Overlay	When agent enters specific skill

§10 Dialogue Tree View

Components: TreeMini.vue (thumbnail) + TreeFull.vue (fullscreen)

Entry: Small tree thumbnail in ChatStage corner (TreeMini). Double-click → fullscreen (TreeFull).

Data model: Familiars stores each Q&A as a Node on a DialogTree (src/utils/memory_tools.py). Each Node has an embedding (BAAI/bge-m3), supporting vector retrieval.

TreeFull operations:

Drag node to empty space: Change layout position (does not change parent-child connections)
Drag node onto another node: Set dragged node's parent to the target node
Right-click → Delete subtree: Delete that node and all its descendants
Reset node positions: Restore to default layout
Search box: Highlight matched nodes
Export: Export entire tree as HTML
Exit: Triple Esc or top × button
Zoom in/out: Adjust full-graph zoom ratio

§11 Input Bar (InputBar)

Component: InputBar.vue

Position: Bottom of ChatStage

Elements (left to right):

Button	Function	Data Flow
📎 Attachment	Pop-up menu → select file	File read as `attachments[]`, sent as message attachments
📂 Folder	Pop-up menu → select folder	Entire folder path as attachment
🧩 Run Skill	Open Skill list	Selected → parameter form → `POST /api/skills/run`
📚 Reference KB	Open KB list + preview	Chip added to message → agent perceives during RAG phase
⚡ Quick Input	Open quick panel	See §13
@ Select Model	Open model list	Temporarily specify model for next message
📝 Text box	Multi-line input	`Enter` to send, `Shift+Enter` for newline
🎙️ Microphone	Push-to-talk	See §14
➤ Send	Send message	`POST /api/chat/send` → SSE streaming reply

Attachment chips: Each attachment shows filename, hover to see path, click × to remove. Images/videos/audio rendered inline in bubbles.

§12 Attachments & File Rendering

Component: AttachmentRenderer.vue

Rendering rules (by kind field):

kind	Rendering
`image`	Inline thumbnail → click to enlarge (lightbox)
`video`	Inline `<video controls>`
`audio`	Inline `<audio controls>`
`link`	Hyperlink (opens in external browser)
`file`	File card: 📄 filename + Open + Show in folder

Underlying protocol: app-file:// custom protocol reads local absolute paths (avoids file:// being blocked by webSecurity). Electron main process registers protocol.registerFileProtocol('app-file', ...).

§13 Quick Input / Model Selection / Skill & KB References

Quick Input (quick inputs)

Click ⚡ to open menu
Top input box → Enter to save new phrase
Click existing phrase → appends to input box (does not replace)
× on right side of each phrase to delete
Persisted to workspace/.quick_inputs.json

@ Model Selection

Click @ to open model list
Selected → @ModelName chip appears in input bar
Only applies to the "next" message (auto-clears after sending)
× next to it clears selection

Skill / KB References

🧩 opens Skill list → select → parameter form
📚 opens KB list → select → chip added to message
Agent perceives these context anchors during RAG phase

§14 Voice Input (push-to-talk)

Entry: 🎙️ microphone button at bottom-right of InputBar

Data flow:

Click once → red pulse → MediaRecorder API starts recording
Click again → stops recording → button enters busy state + tooltip says "Recognizing…"
POST /api/asr/transcribe (multipart audio) → local faster-whisper singleton inference
Returns text → auto-appended to input box (does not clear existing text)

Error handling:

Microphone permission denied → prompt to check browser permissions
Current browser does not support recording → prompt about browser version
Recognition failed: … → shows specific error

Privacy: Audio is inferred locally only, not uploaded to external services. Transcribed text may travel to cloud LLM with the next dialogue turn.

§16 Digital Human

Component: DigitalHuman.vue

Entry: 👤 toggle button at top of ChatStage

Data flow:

Enable → WebSocket connects to /ws/digital_human/<agent_id>
When agent speaks → TTS generates audio → wav2lip model drives lip movements
Real-time frames overlaid on static figure.png

Three-state rendering (based on assets):

Condition	Rendering
Has `figure.png` + `bust_loop.webp`	Normal animated display + real-time frame overlay
External agent without appearance	Emoji placeholder (🦞/🐎), layout unchanged
Local agent without appearance	Default bot appearance placeholder

Missing assets: Missing voice.wav or voice_ref.txt → red prompt in button area → guides to AgentSettings (§7).

§18 Reports

Component: ReportApp.vue (Desktop desktop application)

Entry: Desktop → Reports icon

Features:

Aggregates daily/weekly/monthly/annual reports from all agents
Displayed on a timeline
Top-right Regenerate button → manager re-summarizes

Single agent reports: Viewable inside AgentSettings (§7).

Report file location: Organized by date under workspace/agents/agent_<id>/reports/.

§19 Skill Forge

Component: SkillForgeModal.vue

Entry: ChatStage multi-select messages (Shift/Ctrl) → top batch action bar → Forge Skill

Backend API (`src/bridge/routes/skill_forge.py`)

Endpoint	Phase	Description
`POST /api/skills/forge/from-dialogue/derive_io`	Phase 1	Single LLM call to derive inputs/outputs/trigger
`POST /api/skills/forge/from-dialogue/propose`	Phase 1 (full)	rewrite + tool_select + denoise + params + mock + summary
`POST /api/skills/forge/from-dialogue/repropose`	Phase 1 re-propose	Regenerate summary after user locks I/O
`POST /api/skills/forge/from-dialogue/build_and_preview`	Phase 2	Full chain: denoise + tool_select + params + mock + summary + generate + validate + repair
`POST /api/skills/forge/preview`	Phase 2 (legacy)	Generate SKILL.md + validate
`POST /api/skills/forge/revise`	Revise	Agent mode: read/grep/edit/validate/finish tool loop
`POST /api/skills/forge/commit`	Commit	Persist SKILL.md + refresh registry

3-Step Wizard

Step 1 — params (parameter confirmation)

Frontend POST /derive_io → backend single LLM rewrite → parse intent's INPUTS: / OUTPUTS: / TRIGGER: lines
Shows three editable areas: input list / output list / trigger scenarios
Each row is {name, type, description}, can add/delete/modify
Floating buttons appear after user changes:
- ✕ Revert: Restore LLM original
- ✓ Apply: POST /repropose → backend regenerates summary with locked I/O
Confirm I/O to proceed to next step

Step 2 — review

Frontend POST /build_and_preview → backend runs full chain → returns skill_md + errors
Shows full SKILL.md preview (markdown rendered)
Editable: skill_id, scope (global/agent/recipe), overwrite
If errors present, shown in red list (input_missing / output_missing / script_still_invalid)
Can enter user_guidance → POST /revise → agent mode revision

Step 3 — commit

Click Submit → POST /commit → writes to workspace/skills/<skill_id>/SKILL.md (or agent/recipe scope)
Refreshes skill_registry.json
On completion → next time input bar 🧩 menu shows new Skill

Data flow: Entire forge process is stateless on backend (no session map), each POST is self-contained. Frontend holds all intermediate state.

§20 Skill Change Approvals (SkillProposals)

Component: SkillApprovalsView.vue

Entry: Desktop → Skill Change Approvals icon

Backend: src/bridge/routes/skill_proposals.py

API Endpoints

Endpoint	Description
`GET /api/skill_proposals`	List all pending proposals (no body, metadata only)
`GET /api/skill_proposals/count`	Pending count (for badge)
`GET /api/skill_proposals/<pid>`	Full proposal (with body, for diff display)
`POST /api/skill_proposals/<pid>/approve`	Approve → writes to `workspace/skills/`
`POST /api/skill_proposals/<pid>/reject`	Reject → archives to `rejected/`

Interface

Each proposal shows: source agent, skill_id, target_path, created_at, trigger
Expanded view shows diff (current_body vs proposed_body)
Approve → approve_proposal(pid) → lands SKILL.md
Reject → can fill reason → reject_proposal(pid, reason)
Defer → stays in list

Trigger condition: When agent's skill_auto_apply=False, agent's SKILL modifications enter approval queue (workspace/manager/skill_proposals/).

§21 Todo / Schedule / Cron (TodoScheduleModal)

Component: TodoScheduleModal.vue

Entry: Desktop → Todo·Schedule·Cron icon / TitleBar shortcut

Three tabs:

Todo Tab

List displays all incomplete + completed items
Each item: title, content, deadline (can be empty), urgency (1-5), whether reminder needed
Create todo: Form → click Add → POST /api/todos
Complete/Uncomplete: Checkbox → PATCH /api/todos/<id>
Delete: × at end of row → DELETE /api/todos/<id>
frequency: Simple repeat patterns (daily/weekly/monthly)

Schedule Tab

Week view: Horizontal axis 7 days, vertical axis time, event blocks drawn by start/end
Month view: Each cell is a list of that day's event titles
Top arrows to switch week/month
Click empty cell → create schedule
Click event block → edit/delete/complete
Also supports urgency, reminders, frequency

Cron Tab

Show all agents' crons: Checkbox to view other agents' scheduled tasks
Create/edit/delete tasks
Cron expression format: 5-field standard (minute hour day month weekday)

§22 Global Quick Draft

Component: DraftConfirmModal.vue

Entry: Global keyboard shortcuts

Shortcut	Behavior
`Ctrl+Shift+A`	Auto-detect: LLM decides whether todo or schedule
`Ctrl+Shift+S`	Force create as schedule
`Ctrl+Shift+T`	Force create as todo

Data flow:

User Ctrl+C to copy text
Press shortcut → Electron main process reads clipboard → POST /api/draft/parse
Backend LLM parses to structured todo/schedule → returns to DraftConfirmModal
Popup displays: title/content/time/urgency → editable → Confirm to save / Cancel

Empty clipboard: Top-right shows Clipboard is empty. Select text and copy (Ctrl+C) first, then press the shortcut.

§23 Reminder Toast

Component: ReminderToast.vue

Position: Top-right floating card

Sources:

schedule type: On-time reminders
todo type: N minutes before deadline reminders
desktop type: Agent-initiated desktop reminders

Each card:

Top icon + agent avatar
Title + body (supports markdown)
Buttons: Done (todo/schedule only) / Later (5min/30min/1h/1d/custom) / Close
Auto-dismiss (default several minutes) or click Close

OS notifications: When in background / window loses focus, reminders also pop as native system notifications:

Windows: WinRT ToastNotification (priority High to pierce Focus Assist)
macOS: osascript Notification Center
Click system notification → focus returns to main window

§24 Privacy Desensitization Approval (PrivacyApprovalCard)

Component: PrivacyApprovalCard.vue

Trigger: Agent tool output or uploaded file identified as containing sensitive content → special bubble card inserted into dialogue

Detection layers: gitleaks → Presidio → mini-model (three-pass pipeline, src/tools/sensitive_detector.py)

Each finding:

Shows kind (credential/PII/custom), detector layer, context snippet, original text
Three choices:
- ignore: Allow original text through
- mask: Masked replacement (default encrypted preview)
- record: Original text stored but encrypted (src/base/crypto.py Fernet)
Toggle above for show encrypted preview
After all selected → click Confirm to submit at once

§25 AskUser Card

Component: AskUserCard.vue

Trigger: Agent calls ask_user tool during tool phase

Interface:

Main question bolded and centered
Agent provided candidate options → button chips: click chip → prefills text box below
Text box gets focus → Enter to submit / Esc to close

Data flow: Answer → POST /api/communication/event_responses/<rid> → returns to dialogue loop as next turn's user message. Timeout defaults to ASK_USER_TIMEOUT (configurable).

§26 CG Animation Workshop (CgWorkshop)

Component: CgWorkshop.vue

Entry: Desktop → CG Animation Workshop icon

Purpose: Generate intro CG video for a specific agent's specific skill

Steps:

Left column: select agent (must have figure.png generated)
Middle column: select one of that agent's skills (those with CG bound show ✅ mark)
Right column: fill prompt text (optional LLM optimization)
Click Generate CG → manager runs generate_cg skill → workshop page shows progress at top
On completion, preview mp4
Unsatisfied → Regenerate / Delete

Asset location: workspace/agents/agent_<id>/cg/<skill_id>.mp4

§27 Agent Workshop (AgentWorkshop)

Component: AgentWorkshop.vue

Entry: Desktop → Agent Workshop icon

Three sources:

Type	Description	Prerequisites
🦞 OpenClaw	Node.js personal assistant (WhatsApp/Feishu/WeChat)	Node ≥22
🐎 Hermes	Nous Research Python/uv runtime	uv
🧩 Custom	OpenAI-compatible protocol agent project	Repo URL + startup command

Wizard flow:

Select Type → shows prerequisites
Fill model/repo/startup command
SSE shows installation logs in real-time (auto-resumes on disconnect/page return)
On completion → external agent appears in AgentRail, can chat like local agent

§28 Message Monitor

Component: MonitorApp.vue

Entry: Desktop → Message Monitor icon

Supported platforms (only one can be active at a time):

Platform	Implementation
WeChat monitor	3.9 via UIA; 4.0 via vision `operate()`
QQ monitor	Vision `operate()`

Actions:

Start/Stop button at end of row: Start/stop monitor
Contacts: Input field for target (QQ id / WeChat filehelper name), Enter to save (cannot change while running)

Message routing: Monitored messages → manager/corresponding agent identity → dialogue under corresponding name in Familiars.

Special commands (send in your own chat):

@Manager → view agent list and ids
@Manager <task> → manager assigns task to most suitable agent
@<id> <task> → specific agent completes task
#File<number> → send with attached file

Constraint: Agent does not accept next task before previous one completes.

§29 User Profile (UserProfileModal)

Component: UserProfileModal.vue

Entry: Click identity area at top-left of TitleBar

Fields:

Nickname: Auto-saves on blur after editing. Empty → shows error and reverts
Avatar: Click avatar → select local ≤2MB image to upload
Self-introduction integration: Text box → used as basis for agents to understand the user (written to workspace/USER.md)

§30 Background / Theme / Chat Settings

ChatSettings: ChatStage top ⚙️ → current dialogue-level settings

Font, font size, chat background, pin-to-top
Dialogue model, temperature

BackgroundSettingsModal: Triggered via event open-background-settings

Configure chat area background image, transparency, blur, and other visual effects

§31 Background Running & Auto-Start

Component: SettingsCenter.vue → Background Running tab

Entry: TitleBar ⚙️ / System Tray

Features:

Background running toggle: On → closing main window does not exit bridge process
Auto-start on boot: Win: startup folder shortcut / macOS: launchd plist
Status display: Whether currently running, PID, how long it's been running
Exit background: Full exit (releases 3-4GB GPU VRAM) → secondary confirmation to prevent accidental clicks

§32 System Tray

Entry: While running in background, tray icon stays resident

Actions:

Double-click icon: Show main window
Right-click menu:
- Show main window
- Background running (checkbox, toggle shows secondary confirmation)
- Status: Running · PID xxx · Running for yyy (read-only)
- Exit background (full shutdown) (red item, shows secondary confirmation)

Tooltip: Real-time background status display (refreshes every 30s)

§33 Status Overlay

Component: StatusOverlay.vue

Entry: URL ?view=overlay (managed by main process)

Characteristics: Independent narrow window, pinned to top-right corner, click-through, does not steal focus

Status levels (broadcast by src/observability/status.py to /status/stream SSE):

level	Source	Color	Meaning
`operate`	`base_tools.py:_set_status("operate",…)`	Red	Agent is taking over mouse/keyboard/screen via `operate()`
`monitor`	`monitor.py` pre-startup	Orange	Opening client, detecting login
`monitor_active`	monitor poll loop	Red	Reading screen / grabbing focus
`monitor_dispatching`	monitor pauses polling	Purple	Running chat for `@` trigger
`train`	Not yet triggered	Purple	Training state placeholder
`info`	Manual `POST /status/dev/set`	Blue	Debug/other

Interaction: Default is full mouse pass-through. When monitor bar is present → interactive, ⏹ button clickable to stop immediately.

§34 Model Missing Prompt (ModelMissingModal)

Component: ModelMissingModal.vue

Trigger: On startup/switch dialogue/trigger skill, required local model is not downloaded

Interface: One line per missing model, four choices:

download: Download now (shows progress)
skip: Skip (functionality degrades to fallback)
never_prompt: Never prompt again, always fallback
Cancel: Keep modal open

After submission: Auto-retries current action.

§35 Global Keyboard Shortcut Reference

Key	Action	Scope
`Enter`	Send message	Inside InputBar
`Shift+Enter`	New line	Inside InputBar
`Esc ×3` (within 1.5s)	Interrupt/close split pane/close tree/return to homepage	Main window
`Ctrl+Shift+A`	Clipboard → auto-create draft	Global
`Ctrl+Shift+S`	Clipboard → create schedule	Global
`Ctrl+Shift+T`	Clipboard → create todo	Global

§36 Settings Center

Component: SettingsCenter.vue

Entry:

TitleBar ⚙️ button
Desktop → Settings icon
Anywhere dispatches open-settings-center custom event

Sidebar sections:

Section	Content
LLM Connection	BASE_URL / API_KEY / DEFAULT_MODEL / FAST_MODEL / transport_mode(hybrid/streaming) / auto_fallback / optional per-model override (base_url/api_key)
Background Running	Background toggle, auto-start, PID, running duration, exit background (secondary confirmation)
Notifications	Placeholder (future global notification routing settings)
SKILL Management	Inline `SkillManagerPanel.vue` (view/edit/delete/rewrite SKILL)
About	Client version, bridge host/port

LLM Connection data flow:

User edits draft → click Save
POST /api/settings/llm → backend writes to .env
Backend probe checks connection → returns responses_api / transport / ping results
Frontend displays success/failure + model protocol info

Difference from other settings:

SettingsCenter = global settings
AgentSettings (§7) = single agent settings
ChatSettings (§30) = single dialogue settings

Demo Video Convention

Each section in OPERATION has a docs/videos/NN-<slug>.mp4 placeholder. Just place the recording at that path — no need to modify the docs (recommend ≥720p, ≤90s).

Single source of truth: Repo root OPERATION.md (§1–§42 complete). Workshop & pipeline sections see Workshop Manual / Pipeline & Flow.

General Manual ​

Quick Navigation ​

§1 Startup & Run Modes ​

Dev Startup ​

§2 Main Window Layout Overview ​

§3 Title Bar ​

§4 Manager Homepage (Desktop) ​

§5 Agent List (AgentRail) ​

§6 Create Agent (CreateAgentModal) ​

§7 Agent Settings (AgentSettings) ​

§8 Dialogue List (DialogList) ​

§9 Chat Stage ​

Top Bar ​

Message Stream ​

Special Message Cards ​

§10 Dialogue Tree View ​

§11 Input Bar (InputBar) ​

§12 Attachments & File Rendering ​

§13 Quick Input / Model Selection / Skill & KB References ​

Quick Input (quick inputs) ​

@ Model Selection ​

Skill / KB References ​

§14 Voice Input (push-to-talk) ​

§16 Digital Human ​

§18 Reports ​

§19 Skill Forge ​

Backend API (src/bridge/routes/skill_forge.py) ​

3-Step Wizard ​

§20 Skill Change Approvals (SkillProposals) ​

API Endpoints ​

Interface ​

§21 Todo / Schedule / Cron (TodoScheduleModal) ​

Todo Tab ​

Schedule Tab ​

Cron Tab ​

§22 Global Quick Draft ​

§23 Reminder Toast ​

§24 Privacy Desensitization Approval (PrivacyApprovalCard) ​

§25 AskUser Card ​

§26 CG Animation Workshop (CgWorkshop) ​

§27 Agent Workshop (AgentWorkshop) ​

§28 Message Monitor ​

§29 User Profile (UserProfileModal) ​

§30 Background / Theme / Chat Settings ​

§31 Background Running & Auto-Start ​

§32 System Tray ​

§33 Status Overlay ​

§34 Model Missing Prompt (ModelMissingModal) ​

§35 Global Keyboard Shortcut Reference ​

§36 Settings Center ​

Demo Video Convention ​

General Manual

Quick Navigation

§1 Startup & Run Modes

Dev Startup

§2 Main Window Layout Overview

§3 Title Bar

§4 Manager Homepage (Desktop)

§5 Agent List (AgentRail)

§6 Create Agent (CreateAgentModal)

§7 Agent Settings (AgentSettings)

§8 Dialogue List (DialogList)

§9 Chat Stage

Top Bar

Message Stream

Special Message Cards

§10 Dialogue Tree View

§11 Input Bar (InputBar)

§12 Attachments & File Rendering

§13 Quick Input / Model Selection / Skill & KB References

Quick Input (quick inputs)

@ Model Selection

Skill / KB References

§14 Voice Input (push-to-talk)

§16 Digital Human

§18 Reports

§19 Skill Forge

Backend API (`src/bridge/routes/skill_forge.py`)

3-Step Wizard

§20 Skill Change Approvals (SkillProposals)

API Endpoints

Interface

§21 Todo / Schedule / Cron (TodoScheduleModal)

Todo Tab

Schedule Tab

Cron Tab

§22 Global Quick Draft

§23 Reminder Toast

§24 Privacy Desensitization Approval (PrivacyApprovalCard)

§25 AskUser Card

§26 CG Animation Workshop (CgWorkshop)

§27 Agent Workshop (AgentWorkshop)

§28 Message Monitor

§29 User Profile (UserProfileModal)

§30 Background / Theme / Chat Settings

§31 Background Running & Auto-Start

§32 System Tray

§33 Status Overlay

§34 Model Missing Prompt (ModelMissingModal)

§35 Global Keyboard Shortcut Reference

§36 Settings Center

Demo Video Convention