Yonexus/PROTOCOL.md

# Yonexus Protocol Specification

Version: draft v0.1
Status: planning

---

## 1. Purpose

This document defines the **Yonexus built-in communication protocol** used between OpenClaw instances.

Yonexus has two roles:
- `main`: the central WebSocket server/hub
- `follower`: an outbound WebSocket client connected to `main`

This protocol covers only the **built-in/system channel** required for:
- connection setup
- pairing
- authentication
- heartbeat
- status/lifecycle events
- protocol-level errors

Application-level business messages are transported separately through the Yonexus rule dispatch layer, but they still use the same WebSocket connection.

---

## 2. Transport

## 2.1 Protocol Transport

Transport is WebSocket.

- `main` listens as WebSocket server
- `follower` connects as WebSocket client
- all protocol frames are UTF-8 text messages in the first version

Binary frames are not required in v1.

## 2.2 Endpoint

The `follower` connects to `mainHost`, which may be configured as:
- full URL: `ws://host:port/path` or `wss://host:port/path`
- or raw `host:port` if implementation normalizes it

Recommended canonical configuration for docs and production:
- prefer full WebSocket URL

---

## 3. Message Categories

Yonexus messages over WebSocket are split into two categories:

### 3.1 Builtin Protocol Messages

Builtin/system messages always use the reserved rule identifier:

```text
builtin::${json_payload}
```

User code must never register a handler for `builtin`.

### 3.2 Application Rule Messages

Application messages use the normal format:

```text
${rule_identifier}::${message_content}
```

When received by `main` from a follower, before dispatch they are rewritten to:

```text
${rule_identifier}::${sender_identifier}::${message_content}
```

---

## 4. Builtin Message Envelope

Builtin messages use this wire format:

```text
builtin::{JSON}
```

Where the JSON payload has this envelope shape:

```ts
interface BuiltinEnvelope {
  type: string;
  requestId?: string;
  timestamp?: number;
  payload?: Record<string, unknown>;
}
```

Field semantics:
- `type`: builtin message type
- `requestId`: optional correlation id for request/response pairs
- `timestamp`: sender-side UTC unix timestamp in seconds or milliseconds (must be standardized in implementation)
- `payload`: type-specific object

## 4.1 Timestamp Unit

To avoid ambiguity, protocol implementation should standardize on:

```text
UTC Unix timestamp in seconds
```

If milliseconds are ever used internally, they must not leak into protocol payloads without explicit versioning.

---

## 5. Builtin Message Types

The first protocol version should define these builtin message types.

## 5.1 Session / Connection

### `hello`
Sent by follower immediately after WebSocket connection opens.

Purpose:
- identify follower intent
- declare identifier
- advertise available auth material/state

Example:

```text
builtin::{
  "type":"hello",
  "requestId":"req_001",
  "timestamp":1711886400,
  "payload":{
    "identifier":"follower-a",
    "hasSecret":true,
    "hasKeyPair":true,
    "publicKey":"<optional-public-key>",
    "protocolVersion":"1"
  }
}
```

Notes:
- `publicKey` may be included during first connection and may also be re-sent during re-pair
- `hasSecret` indicates whether follower believes it already holds a valid secret
- `hasKeyPair` indicates whether follower has generated a key pair locally

### `hello_ack`
Sent by main in response to `hello`.

Purpose:
- acknowledge identifier
- indicate required next step

Possible next actions:
- `pair_required`
- `auth_required`
- `rejected`

Example:

```text
builtin::{
  "type":"hello_ack",
  "requestId":"req_001",
  "timestamp":1711886401,
  "payload":{
    "identifier":"follower-a",
    "nextAction":"pair_required"
  }
}
```

---

## 5.2 Pairing Flow

### `pair_request`
Sent by main when follower needs pairing.

Purpose:
- start pairing challenge
- deliver pairing code and expiry

Example:

```text
builtin::{
  "type":"pair_request",
  "requestId":"req_002",
  "timestamp":1711886402,
  "payload":{
    "identifier":"follower-a",
    "pairingCode":"ABCD-1234-XYZ",
    "expiresAt":1711886702,
    "ttlSeconds":300,
    "publicKeyAccepted":true
  }
}
```

### `pair_confirm`
Sent by follower to confirm pairing.

Purpose:
- prove receipt of pairing code before expiry

Example:

```text
builtin::{
  "type":"pair_confirm",
  "requestId":"req_002",
  "timestamp":1711886410,
  "payload":{
    "identifier":"follower-a",
    "pairingCode":"ABCD-1234-XYZ"
  }
}
```

### `pair_success`
Sent by main after successful pairing.

Purpose:
- return generated secret
- confirm trusted pairing state

Example:

```text
builtin::{
  "type":"pair_success",
  "requestId":"req_002",
  "timestamp":1711886411,
  "payload":{
    "identifier":"follower-a",
    "secret":"<random-secret>",
    "pairedAt":1711886411
  }
}
```

### `pair_failed`
Sent by main if pairing fails.

Example reasons:
- expired
- invalid_code
- identifier_not_allowed
- internal_error

Example:

```text
builtin::{
  "type":"pair_failed",
  "requestId":"req_002",
  "timestamp":1711886710,
  "payload":{
    "identifier":"follower-a",
    "reason":"expired"
  }
}
```

---

## 5.3 Authentication Flow

After pairing, reconnect authentication uses the stored `secret`, nonce, timestamp, and follower private key.

## 5.3.1 Authentication Payload Construction

Follower constructs plaintext proof data as:

```text
secret + nonce + timestamp
```

Where:
- `secret`: the current shared secret issued by main
- `nonce`: 24 random characters
- `timestamp`: current UTC Unix timestamp in seconds

Recommended future improvement:
- use a canonical delimiter or JSON encoding before signing to avoid ambiguity

For v1 planning, the required logical content is unchanged, but implementation should prefer a canonical serialized object like:

```json
{
  "secret":"...",
  "nonce":"...",
  "timestamp":1711886500
}
```

and then sign the serialized bytes.

## 5.3.2 Signature Primitive

The requirement says the follower uses the private key to encrypt/sign the proof.

For implementation, the recommended primitive is:
- **digital signature using the private key**
- verified by main using the stored public key

This should replace any literal “private-key encryption” wording in implementation docs.

### `auth_request`
Sent by follower to authenticate after pairing.

Example:

```text
builtin::{
  "type":"auth_request",
  "requestId":"req_003",
  "timestamp":1711886500,
  "payload":{
    "identifier":"follower-a",
    "nonce":"RANDOM24CHARACTERSTRINGX",
    "proofTimestamp":1711886500,
    "signature":"<base64-signature>",
    "publicKey":"<optional-public-key-if-rotating>"
  }
}
```

Validation performed by main:
1. identifier is allowlisted
2. identifier exists in registry
3. follower is in paired state
4. public key matches expected key if provided
5. signature verifies successfully against canonical proof payload
6. proof contains correct secret
7. `abs(now - proofTimestamp) < 10`
8. nonce has not appeared in recent nonce window
9. handshake attempts in last 10 seconds do not exceed 10

### `auth_success`
Sent by main on successful authentication.

Example:

```text
builtin::{
  "type":"auth_success",
  "requestId":"req_003",
  "timestamp":1711886501,
  "payload":{
    "identifier":"follower-a",
    "authenticatedAt":1711886501,
    "status":"online"
  }
}
```

### `auth_failed`
Sent by main if authentication fails.

Allowed failure reasons include:
- unknown_identifier
- not_paired
- invalid_signature
- invalid_secret
- stale_timestamp
- future_timestamp
- nonce_collision
- rate_limited
- re_pair_required

Example:

```text
builtin::{
  "type":"auth_failed",
  "requestId":"req_003",
  "timestamp":1711886501,
  "payload":{
    "identifier":"follower-a",
    "reason":"stale_timestamp",
    "rePairRequired":false
  }
}
```

### `re_pair_required`
Sent by main when unsafe conditions or trust reset require full pairing again.

Example:

```text
builtin::{
  "type":"re_pair_required",
  "requestId":"req_004",
  "timestamp":1711886510,
  "payload":{
    "identifier":"follower-a",
    "reason":"nonce_collision"
  }
}
```

---

## 5.4 Heartbeat

### `heartbeat`
Sent by follower every 5 minutes after authentication.

Example:

```text
builtin::{
  "type":"heartbeat",
  "timestamp":1711886800,
  "payload":{
    "identifier":"follower-a",
    "status":"alive"
  }
}
```

### `heartbeat_ack`
Optional response by main.

Purpose:
- confirm receipt
- provide server time or status hints if desired

Example:

```text
builtin::{
  "type":"heartbeat_ack",
  "timestamp":1711886801,
  "payload":{
    "identifier":"follower-a",
    "status":"online"
  }
}
```

---

## 5.5 Status / Lifecycle Notifications

### `status_update`
Sent by main when follower state changes.

Example:

```text
builtin::{
  "type":"status_update",
  "timestamp":1711887220,
  "payload":{
    "identifier":"follower-a",
    "status":"unstable",
    "reason":"heartbeat_timeout_7m"
  }
}
```

### `disconnect_notice`
Sent before main deliberately closes a follower connection.

Example:

```text
builtin::{
  "type":"disconnect_notice",
  "timestamp":1711887460,
  "payload":{
    "identifier":"follower-a",
    "reason":"heartbeat_timeout_11m"
  }
}
```

---

## 5.6 Errors

### `error`
Generic protocol-level error envelope.

Example:

```text
builtin::{
  "type":"error",
  "requestId":"req_999",
  "timestamp":1711887000,
  "payload":{
    "code":"MALFORMED_MESSAGE",
    "message":"missing type field"
  }
}
```

Recommended builtin error codes:
- `MALFORMED_MESSAGE`
- `UNSUPPORTED_PROTOCOL_VERSION`
- `IDENTIFIER_NOT_ALLOWED`
- `PAIRING_REQUIRED`
- `PAIRING_EXPIRED`
- `AUTH_FAILED`
- `NONCE_COLLISION`
- `RATE_LIMITED`
- `RE_PAIR_REQUIRED`
- `FOLLOWER_OFFLINE`
- `INTERNAL_ERROR`

---

## 6. Builtin State Machines

## 6.1 Follower State Machine

Suggested follower states:
- `idle`
- `connecting`
- `connected`
- `pairing_required`
- `pairing_pending`
- `paired`
- `authenticating`
- `authenticated`
- `reconnecting`
- `error`

Typical transitions:

```text
idle
 -> connecting
 -> connected
 -> (pairing_required | authenticating)

pairing_required
 -> pairing_pending
 -> paired
 -> authenticating
 -> authenticated

authenticated
 -> reconnecting (on socket close)
 -> connecting
```

On `re_pair_required`:

```text
authenticated | authenticating -> pairing_required
```

## 6.2 Main-Side Follower Trust State

Per follower trust state:
- `unpaired`
- `pending`
- `paired`
- `revoked`

Per follower liveness state:
- `online`
- `unstable`
- `offline`

These are related but not identical.

Example:
- follower may be `paired` + `offline`
- follower may be `pending` + `offline`

---

## 7. Security Windows and Replay Protection

## 7.1 Nonce Requirements

Nonce rules:
- exactly 24 random characters
- generated fresh for each auth attempt
- must not repeat within the recent security window

## 7.2 Recent Nonce Window

Main stores for each follower:
- the last 10 nonces seen within the recent validity window

If a nonce collides with the recent stored set:
- authentication must fail
- main must mark the situation unsafe
- follower must re-pair

## 7.3 Handshake Attempt Window

Main stores recent handshake attempt timestamps for each follower.

If more than 10 handshake attempts occur within 10 seconds:
- authentication must fail
- main must mark situation unsafe
- follower must re-pair

## 7.4 Time Drift Validation

Main validates:

```text
abs(current_utc_unix_time - proofTimestamp) < 10
```

If validation fails:
- auth fails
- no successful session is established

Implementation note:
- use server time only on main as source of truth

---

## 8. Rule Message Dispatch Semantics

## 8.1 Message Format

All non-builtin messages use:

```text
${rule_identifier}::${message_content}
```

### Examples

Follower to main:

```text
chat_sync::{"conversationId":"abc","body":"hello"}
```

Main rewrites before matching:

```text
chat_sync::follower-a::{"conversationId":"abc","body":"hello"}
```

## 8.2 Rule Matching

Initial rule matching should be exact string match against `rule_identifier`.

Dispatch algorithm:
1. parse first delimiter section as `rule_identifier`
2. if `rule_identifier === builtin`, route to builtin protocol handler
3. otherwise iterate registered rules in registration order
4. invoke the first exact match
5. ignore/log if no match is found

## 8.3 Processor Input

Processor receives the final message string exactly as seen after protocol-level rewrite.

This means:
- on follower, processor input remains `${rule_identifier}::${message_content}`
- on main for follower-originated messages, processor input becomes `${rule_identifier}::${sender_identifier}::${message_content}`

---

## 9. Connection Rules

## 9.1 Allowed Connection Policy

Main should reject connection attempts when:
- identifier is absent
- identifier is not in configured `followerIdentifiers`
- protocol version is unsupported
- builtin hello/auth payload is malformed

## 9.2 One Active Connection Per Identifier

Recommended v1 policy:
- only one active authenticated connection per follower identifier

If a second connection for the same identifier appears:
- main may reject the new one, or
- terminate the previous one and accept the new one

This behavior must be chosen explicitly in implementation.

Recommended default:
- terminate old, accept new after successful auth

---

## 10. Persistence Semantics

## 10.1 Main Persists

At minimum:
- identifier
- public key
- secret
- trust state
- pairing code + expiry if pairing is pending
- last known liveness status
- metadata timestamps

May persist or reset on restart:
- recent nonces
- recent handshake attempts

Recommended v1:
- clear rolling security windows on restart
- keep long-lived trust records

## 10.2 Follower Persists

At minimum:
- identifier
- private key
- secret
- protocol version if useful
- last successful pair/auth metadata if useful

---

## 11. Versioning

Protocol payloads should include a protocol version during `hello`.

Initial protocol version:

```text
1
```

If main does not support the provided version, it should respond with:
- `error`
- code: `UNSUPPORTED_PROTOCOL_VERSION`

---

## 12. Recommended Canonical JSON Shapes

To reduce ambiguity during implementation, these payload models are recommended.

```ts
interface HelloPayload {
  identifier: string;
  hasSecret: boolean;
  hasKeyPair: boolean;
  publicKey?: string;
  protocolVersion: string;
}

interface PairRequestPayload {
  identifier: string;
  pairingCode: string;
  expiresAt: number;
  ttlSeconds: number;
  publicKeyAccepted: boolean;
}

interface PairConfirmPayload {
  identifier: string;
  pairingCode: string;
}

interface PairSuccessPayload {
  identifier: string;
  secret: string;
  pairedAt: number;
}

interface AuthRequestPayload {
  identifier: string;
  nonce: string;
  proofTimestamp: number;
  signature: string;
  publicKey?: string;
}

interface HeartbeatPayload {
  identifier: string;
  status: "alive";
}
```

---

## 13. Example End-to-End Flows

## 13.1 First-Time Pairing Flow

```text
Follower connects WS
Follower -> builtin::hello
Main -> builtin::hello_ack(nextAction=pair_required)
Main -> builtin::pair_request(pairingCode, expiresAt)
Follower -> builtin::pair_confirm(pairingCode)
Main -> builtin::pair_success(secret)
Follower stores secret
Follower -> builtin::auth_request(signature over secret+nonce+timestamp)
Main -> builtin::auth_success
Follower enters authenticated state
```

## 13.2 Normal Reconnect Flow

```text
Follower connects WS
Follower -> builtin::hello(hasSecret=true)
Main -> builtin::hello_ack(nextAction=auth_required)
Follower -> builtin::auth_request(...)
Main -> builtin::auth_success
Follower begins heartbeat schedule
```

## 13.3 Unsafe Replay / Collision Flow

```text
Follower -> builtin::auth_request(nonce collision)
Main detects unsafe condition
Main -> builtin::re_pair_required(reason=nonce_collision)
Main invalidates existing secret/session trust
Follower returns to pairing_required state
```

## 13.4 Heartbeat Timeout Flow

```text
Follower authenticated
No heartbeat for 7 min -> main marks unstable
No heartbeat for 11 min -> main marks offline
Main -> builtin::disconnect_notice
Main closes WS connection
```

---

## 14. Implementation Notes

## 14.1 Parsing

Because the top-level wire format is string-based with `::` delimiters:
- only the first delimiter split should determine the `rule_identifier`
- for `builtin`, the remainder should be treated as JSON string and parsed once
- message content itself may contain `::`, so avoid naive full split logic

## 14.2 Payload Encoding

Recommended message content for application rules:
- JSON string payloads where applicable
- but Yonexus itself only requires string content

## 14.3 Logging

Main should log at least:
- connection open/close
- hello received
- pairing created/succeeded/expired/failed
- auth success/failure
- nonce collisions
- handshake rate-limit triggers
- status transitions
- unhandled rule messages

Sensitive values must never be logged in plaintext:
- `secret`
- private key
- raw proof material
- full signature verification internals unless safely redacted

---

## 15. Open Clarifications

Before implementation, these should be finalized:

1. Exact signing algorithm:
   - Ed25519 is a strong default candidate
2. Secret length and generation requirements
3. Pairing code format and length
4. Whether `pair_request` should require operator confirmation or stay automatic for allowlisted identifiers
5. Whether `heartbeat_ack` is mandatory or optional
6. Whether follower should auto-reconnect with backoff strategy after disconnect
7. Whether duplicate active connections should replace old sessions or be rejected

---

## 16. Summary of Reserved Builtin Types

Current reserved builtin `type` values:
- `hello`
- `hello_ack`
- `pair_request`
- `pair_confirm`
- `pair_success`
- `pair_failed`
- `auth_request`
- `auth_success`
- `auth_failed`
- `re_pair_required`
- `heartbeat`
- `heartbeat_ack`
- `status_update`
- `disconnect_notice`
- `error`

These names are reserved by Yonexus and must not be repurposed by user rules.