Broker App Development Guide

This article is the complete development reference for the Broker subsystem. A Broker is Familiars' "shareable application package": a .broker.zip containing manifest + UI assets + plugin + trigger + recipe, installable from the marketplace and reusable across users.

---

1. Conceptual Model

1.1 What is a Broker

A Broker = the application container for Familiars' cluster interconnect layer.

Cluster = one user + all agents under that user. It is Familiars' smallest trust unit: all agents within a cluster share one memory and one permission set, and appear externally as a single signed identity (cluster_id = dev_<device_fingerprint>, derived by src/broker/identity.py).

The Broker layer has four subsystems:

Subsystem	Core Work	Current Status
A. Intra-cluster orchestration	Structured collaboration among multiple agents within one cluster: rooms / threads / deals / reputation	✅ Landed
B. App container	Third-party code attached to any cluster as pluggable capability	✅ Landed (AST whitelist + subprocess sandbox, dual-track)
C. Distribution & Trust	Author signing → static site/relay → user install; Ed25519 + sha256 + author pubkey cross-verification	✅ Landed (Layer 1 MVP)
D. Cross-cluster Federation	Message passing between different users' clusters, trusted routing	🔶 Hooks in place, relay not implemented

1.2 Package Structure

A .broker.zip contains:

my_broker.broker.zip
├── manifest.json            Required: describes everything about the broker
├── SIGNATURE                Required (marketplace enforced): format <pub_key_b64>\n<sig_b64>\n
├── ui/                      Optional: iframe UI directory
│   ├── index.html           Entry pointed to by manifest.ui.entry
│   └── ...
├── plugins/                 Optional: *.py loaded via AST whitelist + subprocess sandbox
├── triggers.json            Optional: event → plugin/recipe dispatch table
├── recipes/                 Optional: recipe archives
├── cover.png                Optional: desktop icon
└── README.md                Optional

The packer (broker pack) rejects backslash paths, .. escapes, absolute paths, and filenames with strange characters.

---

2. Complete manifest.json Fields

{
  "broker_id": "my_broker",
  "name": "My Broker",
  "version": "0.1.0",
  "schema_version": 1,
  "author_pubkey": "<base64 公钥>",
  "description": "...",
  "domain": "collaboration",
  "python_api_version": 1,

  "capabilities_required": ["guild", "ledger", "recipe_call", "scheduler"],

  "role_defs": {
    "owner": {"caps": ["*"]},
    "approver": {"caps": ["task.approve", "task.verdict", "settle.trigger"]},
    "issuer": {"caps": ["task.publish", "task.read"]},
    "developer": {"caps": ["task.accept", "task.deliver"]},
    "audience": {"caps": ["task.read"]}
  },
  "role_notes": {"sub_owner": "..."},

  "reputation_dimensions": [
    {"key": "delivery", "label": "交付质量", "min": 0, "max": 10},
    {"key": "punctuality", "label": "准时率", "min": 0, "max": 10}
  ],

  "ui": {
    "entry": "index.html",
    "title": "My Broker",
    "width": 1000,
    "height": 680,
    "single_instance": true
  },

  "permissions": {
    "network": false,
    "fs_read": false,
    "fs_write": false,
    "extra_modules": ["requests"],
    "rss_mb": 512,
    "cpu_budget_s": 30,
    "rate_limit": 60,
    "timeout_s": 30
  },

  "on_install": [
    {"kind": "cron", "cron_expr": "0 * * * *", "action_kind": "plugin_fn", "action_ref": "plugins.mymod:tick"}
  ],
  "on_uninstall": []
}

2.1 Required Fields

Field	Type	Description
broker_id	string	Globally unique, == directory name == zip-internal id
name	string	Display name
version	string	Recommended semver
schema_version	int	Currently = 1

2.2 Recommended Fields

Field	Description
description	Description
domain	Business domain
author_pubkey	Your public key (the marketplace cross-checks it against the SIGNATURE inside the package)
ui.entry	iframe UI entry (.html relative to ui/)
capabilities_required	Declares which local APIs are needed from the cluster
role_defs	Role → capability sets
reputation_dimensions	Reputation dimensions

2.3 Common capabilities_required Values

Capability	Meaning
guild	Guild/rooms/posts/task library
ledger	Token ledger (issue/transfer/settle)
recipe_call	Call local Recipes
scheduler	Scheduled triggers
rooms.read / rooms.write	Session read/write
messages.post	Post session messages
tasks.write / tasks.approve	Task publishing/approval
settle.trigger	Trigger settlement

2.4 permissions (Plugin Sandbox Permissions)

Field	Default	Meaning
network	false	HTTP access to external services
fs_read	false	Read the filesystem
fs_write	false	Write the filesystem
extra_modules	[]	Request additional allowed modules (see §6)
rss_mb	512	RSS memory limit
cpu_budget_s	30	CPU budget over a 30s window
rate_limit	60	Calls per minute
timeout_s	30	Per-call timeout

---

3. UI Development

3.1 Single HTML Skeleton (starter Style)

ui/index.html is the entry point and obtains window.familiars via <script src="../sdk.js">. The bridge serves this shim at /api/broker_app/<id>/sdk.js.

<!DOCTYPE html>
<html>
<head><script src="../sdk.js"></script></head>
<body>
  <h1 id="title"></h1>
  <button onclick="load()">Load rooms</button>
  <pre id="out"></pre>
  <script>
    const api = window.familiars;
    document.getElementById('title').textContent = 'Hello, ' + api.broker_id;
    async function load() {
      const res = await api.rooms.list();
      document.getElementById('out').textContent = JSON.stringify(res, null, 2);
    }
  </script>
</body>
</html>

Full SDK API: workspace/manager/brokers/SDK.md.

3.2 Vue/Vite Project (Advanced)

Put a complete project in ui/ (src/App.vue, vite.config.ts, package.json) and package the build output dist/ into the .broker.zip (entry points to dist/index.html).

Reference: workspace/manager/brokers/example_broker/ (ui_mode: "vue", entry: "./dist/index.html").

Before publishing, run npm install && npm run build in ui/ to generate dist/.

---

4. Plugin Development

4.1 Dual-Track Sandbox Architecture

plugins/*.py passes through two checkpoints at install time:

1. AST whitelist validation (at install time, before compilation) — rejects forbidden imports, attribute chains, dangerous names like exec/eval/open;
2. Runtime sandbox (at PluginBrokerApp.on_start) — dual-track execution:
   • Legacy in-process path: bundle.install() execs into the bridge process's sys.modules["broker_plugins.<safe_id>.<name>"];
   • Sandboxed subprocess path: bundle.install_sandboxed() starts an independent worker subprocess per broker, communicating via JSON-RPC (newline-delimited JSON over stdin/stdout).

Event dispatch prefers the sandbox: when event_bus.py receives a plugin_fn trigger it first checks sandbox_dispatch.is_sandboxed(broker_id); if sandboxed, it routes via subprocess RPC, falling back to in-process on failure.

4.2 Plugin Function Signature

# plugins/greeter.py
from broker_sdk import store

def on_message_posted(payload: dict) -> None:
    store.upsert(
        f"seen/{payload['message_id']}",
        title="seen",
        meta={"cluster_id": payload.get("cluster_id")}
    )

• Receives a single positional payload dict parameter;
• Facades (store / rooms / ledger …) are imported from broker_sdk; in sandbox mode they are reverse RPC proxies, in-process they are the real SDK;
• Do not add an sdk parameter to functions (the old version is deprecated).

4.3 triggers.json

[
  {
    "event": "message.posted",
    "action_kind": "plugin_fn",
    "action_ref": "plugins.greeter:on_message_posted"
  },
  {
    "event": "approval.decided",
    "action_kind": "plugin_fn",
    "action_ref": "plugins.greeter:on_approval_decided"
  }
]

action_kind options: plugin_fn | recipe_call | spawn_agent | ...

The match field can filter payloads: {"to_state": "SETTLED"} only matches events where that field is equal.

4.4 Complete Event Catalog

Event	Payload keys
app.installed	broker_id, installed_from
app.started / paused / resumed / uninstalled	broker_id
message.posted	broker_id, room_id, thread_id?, cluster_id, kind, message_id
approval.created / voted / decided / withdrawn	approval_id, ...
deal.transition	deal_id, from_state, to_state, actor, evidence
member.join / leave / kick	room_id, cluster_id, ...
reputation.threshold	subject, dimension, from, to, delta
cron.tick	cron_expr, ts
custom.<name>	Plugin-defined

---

5. Signing & Security

5.1 Three Mandatory Verifications

On the user side, install_from_url verifies in sequence and rejects if any check fails:

1. sha256: The downloaded content hash must exactly match the index record;
2. SIGNATURE: <pubkey>\n<sig>\n inside the zip, uses the pubkey to verify the sha256(manifest.json) signature;
3. author_pubkey: The pubkey in the signature must match the index declaration.

5.2 Author Signing Keys

• Location: ~/.familiars/authors/<name>.key (Windows: %USERPROFILE%\.familiars\authors\<name>.key)
• Format: single-line base64-encoded 32-byte Ed25519 seed
• Cross-machine sync: just copy the .key file

python main.py broker keygen --name <name>          # Generate
python main.py broker keys-list                      # List
python main.py broker keygen --name <name> --overwrite   # Rotate key

5.3 Current Signature Domain Simplification (Known Gap)

Currently only manifest.json is signed; ui/ / plugins/ / triggers.json can be modified without invalidating the signature. You must re-sign after modifying the manifest. This is a backlog item for subsystem B.

---

6. AST Whitelist

6.1 Allowed by Default

json / math / re / datetime / dataclasses / typing / statistics / collections / collections.abc / enum / functools / itertools / decimal / __future__ / broker_sdk

6.2 Requestable (extra_modules)

os / subprocess / requests / aiohttp / httpx / sqlite3 / pathlib / tempfile / hashlib / hmac / base64 / binascii / uuid / secrets / shutil / glob / fnmatch / textwrap / io / copy / pprint / urllib / xml / csv / string / bisect / heapq / operator / itertools / random / fractions / warnings / contextlib / abc / types / inspect / argparse

6.3 Permanently Forbidden

ctypes / pickle / importlib / inspect / code / codeop / pdb / trace / tracemalloc / sys / signal / multiprocessing / threading / asyncio / socket / select / asyncore / asynchat / ssl / mmap / platform / __import__ / eval / exec / compile / open / globals / locals / vars / getattr / setattr / delattr / breakpoint / input / exit / quit

---

7. Lifecycle Hooks

manifest.on_install / on_uninstall are a set of {kind, ...} objects:

kind	Meaning
cron	{"kind": "cron", "cron_expr": "0 * * * *", "action_kind": "plugin_fn", "action_ref": "plugins.mymod:tick"}
spawn_agent	{"kind": "spawn_agent", "template": "..."}
bind_recipe	{"kind": "bind_recipe", ...}
run_script	(Not landed, not recommended for MVP)

See src/broker/workshop/packaging.py:_validate_lifecycle_hooks for details.

---

8. REST API

8.1 Instance Management (/api/broker_workshop)

Method	Path	Description
GET	/instances	List installed brokers
GET	/instances/{broker_id}	Get one
POST	/instances/import	Import from a directory (local development)
POST	/instances/install_from_url	Install from URL (marketplace three mandatory verifications)
DELETE	/instances/{broker_id}	Uninstall (store sources are refused)
POST	/instances/consent	Install consent (capability approval)
GET	/instances/consents/{user_id}	List user consents
GET	/instances/updates	Check for updates

8.2 Marketplace

Method	Path	Description
GET	/marketplace/index	Get marketplace index
GET	/marketplace/recipes/index	List marketplace recipes
GET	/marketplace/flows/index	List marketplace flows
POST	/marketplace/{kind}/like	Like
POST	/marketplace/{kind}/coin	Tip/coin
POST	/marketplace/coin/balance	Check balance
POST	/marketplace/{kind}/view	Record a view
POST	/marketplace/report	Report
POST	/marketplace/cover/upload	Upload cover
POST	/marketplace/cover/generate	Generate cover

8.3 Capability Approval

Method	Path	Description
GET	/capabilities/registry	Capability registry
GET	/capabilities/describe	Describe capabilities
GET	/instances/{broker_id}/capabilities/status	Check approval status
POST	/instances/{broker_id}/capabilities/approve	Approve capabilities

8.4 Broker Base API (/api/broker)

Method	Path	Description
GET	/brokers	List registered brokers
POST	/brokers/{id}/register	Register
POST	/brokers/{id}/deregister	Deregister
POST	/brokers/{id}/scopes	Set consent scope
GET	/brokers/{id}/card	Get cluster card
POST	/brokers/{id}/actions/{action}	Trigger an action
POST	/brokers/{id}/deals/{deal_id}/transition	Deal state transition
GET	/brokers/{id}/deals	List deals
GET	/brokers/{id}/ledger	Check ledger

8.5 Federation (/api/broker/federation)

Method	Path	Description
POST	/federation/receive	Receive cross-cluster messages
GET	/federation/status	Federation status
POST	/federation/rooms/{broker_id}/{room_id}/federate	Room federation sync
POST	/federation/threads/{broker_id}/{thread_id}/federate	Thread federation sync
POST	/federation/tasks/{broker_id}/{task_id}/federate	Task federation sync

8.6 Assets & SDK (/api/broker_app)

Method	Path	Description
GET	/{broker_id}/ui/{rel:path}	Get UI static assets
GET	/{broker_id}/cover	Get cover
GET	/{broker_id}/sdk.js	Get SDK shim
GET	/_meta/sdk.js	SDK metadata

---

9. Installation Sources & Isolation Semantics

Source	Semantics	Package Level	Example
built_in	Reference package bundled with the repo	Writable / directory can be opened	starter_broker, ventaris
user_created	Built from scratch on this machine	Writable / directory can be opened	Workshop "New"
imported_zip	Zip imported on this machine	Treated as local development, can be modified again	User-imported zip
store	Downloaded from the marketplace	Read-only black box	Ventaris installed from the marketplace

Store package protection rules:

• Delete → 403
• Overwrite → only allowed when the signing author pubkey matches the installed row
• Scaffold name collision → rejected
• Runtime data remains open as usual

---

10. Common Pitfalls

Pitfall	Symptom	Fix
manifest.id does not match the directory	Install fails	broker_id must == directory name == zip name prefix
Modified manifest without re-signing	Verification fails	Re-sign with broker sign
Published to relay but not in the index	Still in the review queue	Wait for operator approval
Plugin adds an sdk parameter	Crashes in sandbox mode	Only use the single-parameter on_*(payload) contract
extra_modules requested but not approved	Capability not granted	An approval dialog pops up at install time; requires user consent
Trying to overwrite a store package	403	Only same-author upgrades allowed

---

Reference Source Code

• Hub: src/broker/hub.py
• Workshop packaging: src/broker/workshop/packaging.py
• Plugin loading: src/broker/workshop/plugin_loader.py
• Sandbox AST: src/broker/workshop/sandbox/ast_guard.py
• Sandbox host/worker: src/broker/workshop/sandbox/{host,worker}.py
• REST: src/bridge/routes/broker_workshop.py, src/bridge/routes/broker.py, src/bridge/routes/broker_app.py
• Template: workspace/manager/brokers/starter_broker/
• SDK documentation: workspace/manager/brokers/SDK.md
• Full development manual: BROKER_DEVELOPMENT.md