BSV Multicast NACK Retransmission Protocol

Abstract

This BRC specifies the NACK-based retransmission and endpoint discovery protocol for the BSV multicast transaction distribution pipeline. It defines five UDP datagram formats — ADVERT, NACK, MISS, ACK, and THROTTLED — along with tier/preference-based endpoint selection, an escalation state machine, and configurable retransmit modes. The protocol operates on top of the BRC-124 data-plane and enables reliable gap recovery without requiring connection state at the ingress or listener tiers.

Copyright

This BRC is licensed under the Open BSV License.

Motivation

The BRC-124 data-plane delivers BSV transactions over IPv6 multicast with best-effort semantics. Network congestion, interface buffer overflows, and MLD snooping transitions can cause individual frames to be lost. Because BRC-124 frames carry a stable per-flow HashKey and a monotonic SeqNum stamped by the ingress proxy, listeners can detect exactly which frame is missing without a central sequence authority.

This BRC adds the reliability layer that allows listeners to recover those gaps:

1. Gap detection — listeners identify missing frames when a frame's SeqNum advances by more than 1 from the last-seen value for a given HashKey flow.
2. NACK dispatch — listeners send a 64-byte NACK datagram to a retry endpoint identifying the missing frame by its flow (HashKey) and sequence number.
3. ACK/MISS/THROTTLED responses — every served NACK receives a deterministic 16-byte response; ACK confirms retransmit dispatched, MISS triggers immediate escalation, and the optional THROTTLED signals honest congestion (hold and retry the same endpoint without escalating).
4. Endpoint discovery — retry endpoints periodically multicast a 56-byte ADVERT beacon; listeners maintain a dynamic registry sorted by tier and preference, with no manual configuration required in well-connected deployments.
5. Flood prevention — deduplication and fill-suppression mechanisms prevent retransmit storms at scale.

Specification

Common Message Preamble

All BRC-126 datagrams begin with a 7-byte preamble:

The MsgType byte at offset 6 is in the same position as FrameVersion in BRC-124 data frames. Values 0x10–0x2F are reserved for BRC-126 control messages and are distinct from the data-frame version codes (0x01–0x07).

MsgType Values

ADVERT Wire Format (MsgType 0x20) — 56 bytes

Sent periodically by retry endpoints to the beacon multicast group. Listeners use it to build and maintain their endpoint registry.

ADVERT Flags Bitmask

HasParent: When set, the endpoint maintains a connection to a parent (higher-tier) endpoint and forwards NACKs on local cache miss. Listeners need only know their local tier; inter-tier forwarding is handled transparently.

Beacon group addresses are derived from the shard address scheme using the reserved control-plane index 0xFFFD:

Addresses use the IANA-aligned layout: bytes 0–1 carry the scope prefix, bytes 2–11 are zero (IANA 96-bit boundary), bytes 12–13 carry the IANA Bitcoin group-id (0x000B), and bytes 14–15 carry the group index. Operators MAY override the group-id via -mc-group-id.

NACK Wire Format (MsgType 0x10) — 64 bytes

Sent by a listener to a retry endpoint when a gap is detected. Identifies the missing frame by its flow (HashKey) and sequence number range. For current single-frame retrieval, StartSeq == EndSeq; range requests (StartSeq < EndSeq) are reserved for future use.

The listener opens a per-request ephemeral UDP socket ([::]:0), sends the NACK, and waits up to 300 ms for a single response (MISS or ACK).

HashKey (offset 8) is the HashKey field from the BRC-124 frame, computed as XXH64(senderIPv6 ∥ groupIdx ∥ subtreeID). It uniquely identifies the flow. The retry endpoint uses HashKey as a per-flow rate-limiting key (NACK storm cap). A value of 0 bypasses the per-flow check.

StartSeq / EndSeq (offsets 16/24) specify the range of missing sequence numbers. For current single-frame retrieval, StartSeq == EndSeq. The retry endpoint looks up the frame using the 16-byte cache key HashKey ∥ StartSeq.

SubtreeID (offset 32) is carried for informational purposes; the cache key is HashKey ∥ SeqNum and does not require SubtreeID for disambiguation.

Flags / Proxied (offset 7, bit 0x01) marks a NACK that an endpoint issued on behalf of a downstream multicast domain (cross-domain proxying — see NACK Proxying). An endpoint receiving a NACK with this bit set MUST serve it from its own cache but MUST NOT re-proxy it, bounding any proxy chain to a single hop. The bit was previously reserved and is ignored by legacy endpoints, which simply never re-proxy.

NACK Proxying (cross-domain recovery)

A retry endpoint serving a downstream multicast domain (one fed by a listener's multicast egress rather than directly by the ingress proxy) can only cache what the listener actually emitted. A frame the listener never put on the downstream wire — egress send error, interface flap, or in-fabric loss — is missed identically by the downstream endpoint and every downstream consumer, so a downstream-only cache cannot repair it. NACK proxying recovers such frames from an upstream endpoint that received them directly from the ingress proxy:

1. A downstream consumer NACKs the downstream endpoint, which suffers a local cache miss and returns MISS immediately.
2. The downstream endpoint forwards the NACK to a statically configured upstream endpoint with the Proxied flag set (0x01). Recovery is asynchronous ("cache-warm"); no NACK worker is held waiting.
3. The upstream endpoint serves the proxied NACK from its cache. Because the requester (the downstream endpoint) is not joined to the upstream shard groups, the frame is returned by unicast to the NACK source — a proxied NACK is always served a unicast copy regardless of the upstream's advertised retransmit mode.
4. The downstream endpoint re-caches the recovered frame (keyed HashKey ∥ SeqNum, per-FrameVer TTL) and multicast-retransmits it into the downstream domain; the consumer's gap auto-fills via Tracker.Fill().

One-hop bound. The Proxied flag prevents an upstream endpoint from re-proxying, so any proxy chain is at most one hop. Upstream discovery is by static configuration, because a separated downstream domain generally cannot receive upstream multicast ADVERT beacons. The HasParent ADVERT flag (0x02) signals that an endpoint has an upstream parent configured. When several downstream endpoints run with proxying enabled against a shared cache backend, an in-flight claim deduplicates the upstream NACKs so only one endpoint recovers each gap.

MISS Response (MsgType 0x11) — 16 bytes

Sent unicast to the NACK source when the requested frame is not in the endpoint's cache. On receipt, the listener advances immediately to the next endpoint (no backoff delay).

ACK Response (MsgType 0x12) — 16 bytes

Sent unicast to the NACK source when the frame was found and retransmit dispatched. Whether the listener may cancel the gap on receipt depends on the Flags byte — see Repair confirmation below.

The listener uses the SeqNum echo to confirm the gap entry matches before cancelling it.

Repair confirmation. An ACK asserts that a retransmit was dispatched, not that it was delivered, so the Flags byte determines whether the listener may close the gap. On an ACK with 0x02 (unicast sent) the listener MUST NOT cancel the gap entry: it MUST wait for the retransmitted frame itself and, if the frame does not arrive, MUST resume escalation as though the attempt had timed out. This matters because the retransmit can be lost on the same path that dropped the original. An ACK without 0x02 conveys no such confirmation channel, so a listener MAY cancel on the ACK alone, accepting that a lost retransmit will not be retried; deployments that require repair to be verifiable SHOULD therefore enable unicast retransmit.

THROTTLED Response (MsgType 0x13) — 16 bytes

Sent unicast to the NACK source when the request was rejected by a congestion-control tier that limits per-gap or per-flow request rate (see Flood Prevention). It is a flow-control signal, not a failure: the endpoint is healthy, and for a per-gap throttle a retransmit for this exact gap was likely just served and is propagating over the multicast data plane. On receipt a listener MUST hold the gap for the hinted backoff and retry the same endpoint; it MUST NOT escalate to another endpoint and MUST NOT count the throttle as a recovery failure.

Backoff hint. The suggested hold is ThrottleHintBase << bucket, where ThrottleHintBase = 125 ms and bucket is the Flags low nibble. Endpoints SHOULD use bucket 2 (~500 ms) for a per-gap throttle and bucket 3 (~1 s) for a per-flow throttle. Listeners SHOULD apply jitter (e.g. uniform over [hold/2, hold]) to de-synchronise and MAY clamp the hold to a local maximum. The gap's absolute TTL remains the upper bound; a multicast repair cancels the gap regardless.

Emission rules.

• THROTTLED is OPTIONAL and SHOULD default to disabled; it is a load-shedding refinement for high-fan-out deployments.
• An endpoint MUST NOT send THROTTLED for a flood-tier (per-source) rejection: that tier sheds abusive or spoofed-source load, and answering it would permit reflection. (The 16-byte response is smaller than the 64-byte NACK, so the protocol is never a bandwidth amplifier regardless.)
• An endpoint that does not implement THROTTLED stays silent; the listener falls back to timeout + backoff.
• A listener that does not recognise 0x13 treats it as an unparseable response (timeout-equivalent), so emission is backward-compatible.

Tier / Preference Model

Retry endpoints are organised into tiers representing proximity to the transaction source, and assigned a preference within each tier.

Endpoints discovered via ADVERT carry their operator-assigned Tier (0–254) and Preference (0–255). Endpoints registered via static configuration seed the registry at Tier=0xFF, Preference=0.

Listeners sort the endpoint registry by (Tier ASC, Preference DESC). NACK dispatch always selects the head of the sorted list; on MISS or timeout, the listener advances to the next position.

Escalation State Machine

  ┌──────────┐
  │ PENDING  │  gap registered; hold-off jitter applied
  └────┬─────┘
       │ timer fires
       ▼
  ┌────────────────┐
  │ NACKED(Tier-K) │  NACK sent to current (tier, preference) endpoint
  └───┬────┬───┬───┘
      │    │   │
      │    │   └── Timeout ──► exponential backoff; retry next sweep
      │    │
      │    ├── MISS ──► advance to next endpoint at same tier (Preference DESC);
      │    │            if tier exhausted, escalate to Tier K+1;
      │    │            retry IMMEDIATELY (no backoff on MISS)
      │    │
      │    └── THROTTLED ──► hold for the hinted backoff; retry the SAME endpoint;
      │                      do NOT escalate, do NOT count as a recovery failure
      │
      ├── ACK (no 0x02) ──► gap entry cancelled; done
      └── ACK (0x02 unicast sent) ──► await frame ──► FILLED
                                   └── timeout ──► advance endpoint

  Any state ──► FILLED (multicast repair arrived via data plane)
               gap entry cancelled; in-flight NACK socket times out harmlessly

• ACK received, Flags without 0x02: Cancel gap entry. No further NACKs sent for this gap (the repair is trusted, not confirmed).
• ACK received with 0x02 (unicast sent): Do NOT cancel. Await the retransmitted frame; on arrival the gap is FILLED, and on timeout resume escalation from the next endpoint.
• MISS received: Advance to next endpoint at same tier by Preference; if tier exhausted, move to next tier; retry immediately.
• Timeout: Apply exponential backoff (capped at nack-backoff-max); retry on next sweeper tick.
• THROTTLED received: Hold the gap for the hinted backoff and retry the same endpoint. Do not escalate and do not count it as a recovery failure — it is congestion control (see THROTTLED Response).
• Multicast fill: The data-plane receive goroutine calls Tracker.Fill() independently; gap cancelled regardless of NACK state.

Beacon Scopes

Three scope bytes are defined for the ADVERT wire format. The reference implementation's -beacon-scope flag accepts site, org, global, both, or all (both and all are equivalent: the ADVERT is sent to the site, org, and global beacon groups simultaneously):

Beacon group addresses are shown in ASM (FF0x) form. Under SSM, substitute the FF3x prefix per BRC-129 (FF35::B:FFFD site, FF3E::B:FFFD global; inter-domain scope is SSM-only per RFC 8815). Because beacon groups are control-plane, their source — the emitting retry endpoint — cannot be discovered from within the group; SSM receivers (S,G)-join using the per-group bootstrap source lists defined in BRC-129, not via ADVERT. The ADVERT and NACK wire formats are unchanged across modes.

Scope byte 0xFF is used in ADVERT datagrams when the sender intends to cover site, org, and global simultaneously (sends three ADVERTs). Listeners parse the scope byte from each individual datagram.

To cover multiple scopes, run separate endpoint instances each configured with a different -beacon-scope value. Each scope can use independent Tier and Preference tuning.

Listeners compute TTL as 3 × BeaconInterval. An endpoint not heard for that duration is evicted from the registry. Static seeds (-retry-endpoints) are never evicted.

Configurable Retransmit Modes

Retransmit behavior is controlled by flags on the retry endpoint:

Deployment profiles:

• On-fabric (default): -beacon-flags-multicast=true — all listeners on the fabric receive the retransmit simultaneously.
• Edge / unicast: -beacon-flags-unicast=true -beacon-flags-multicast=false — for listeners not on the multicast fabric, and wherever repair must be confirmed rather than trusted (see Repair confirmation above). A multicast retransmit is also delivered to every listener in the group, including those that observed no loss.
• High-volume: -suppress-ack=true — reduces per-frame ACK traffic; MISS responses are preserved for escalation correctness.

Flood Prevention

Examples

ADVERT Datagram (56 bytes)

A site-scope endpoint at fd20::24, port 9300, Tier 0, Preference 128, 60-second interval, multicast retransmit enabled:

E3E1F3E8                                  // Network Magic
02BF                                      // Protocol Version
20                                        // MsgType = ADVERT
05                                        // Scope = site-local
FD200000000000000000000000000024          // NACKAddr = fd20::24
2454                                      // NACKPort = 9300
00                                        // Tier = 0
80                                        // Preference = 128
003C                                      // BeaconInterval = 60 s
0010                                      // Flags = MulticastRetransmit
A1B2C3D4                                  // InstanceID (CRC32c of hostname)
00000000                                  // Reserved
00000000000000000000000000000000          // Reserved (16 bytes)

NACK Datagram (64 bytes)

Single-frame retrieval: missing frame for flow HashKey=0xA1B2C3D400000001, SeqNum=1235:

E3E1F3E8                                  // Network Magic
02BF                                      // Protocol Version
10                                        // MsgType = NACK
00                                        // Flags (reserved)
A1B2C3D400000001                          // HashKey (XXH64 of sender+group+subtree)
00000000000004D3                          // StartSeq = 1235
00000000000004D3                          // EndSeq = 1235 (single frame)
baadf498a00ca5a44d1c4d9d103b49017f53cd8cb2a70a9c67fc884ecdd622b5  // SubtreeID

MISS Response (16 bytes)

E3E1F3E8                                  // Network Magic
02BF                                      // Protocol Version
11                                        // MsgType = MISS
00                                        // Flags
0000000000000000                          // SeqNum = 0 (cache miss)

ACK Response (16 bytes)

Frame found and retransmitted via multicast:

E3E1F3E8                                  // Network Magic
02BF                                      // Protocol Version
12                                        // MsgType = ACK
01                                        // Flags = multicast_sent
00000000000004D3                          // SeqNum of retransmitted frame (1235)

References

• BRC-12: Raw Transaction Format — Payload format for transaction data within BRC-124 frames
• BRC-124: Multicast Transaction Frame Format — Data-plane frame format; defines HashKey, SeqNum, and Subtree ID stamped by the proxy
• BRC-129: Multicast Group Address Assignments — SSM/ASM addressing, scopes, and (S,G) source discovery

Constants Reference