WebRTC DataChannel: files without buffer errors

WebRTC DataChannel File Transfer: Buffer Management and Reliability in Production

In WebRTC DataChannel, the dc.send() method queues data in the sender's SCTP buffer but doesn't guarantee delivery to the receiver. On slow relay connections, the buffer fills up with megabytes of data, the progress bar hits 100%, and the file ends up truncated. A typical P2P file-sharing setup includes a signaling server for coordination, direct peer-to-peer connections as the primary path, and TURN relays as fallback. The server doesn't store content but manages transfer states: from share creation to recipient queues.

Key challenges: backpressure from the buffer, loss of the File object on page refresh, mismatched readiness between peers, and reconnections. This breakdown draws from a production implementation with relay limits and custom TURN support.

Architecture Without Server-Side File Storage

The server coordinates via REST API and WebSocket: it creates shares, signs connections with ECDSA, and tracks states. A share's lifecycle—active → matched → transferring → active—lets the sender reuse for multiple recipients. Queues automatically distribute to new participants.

Terminal states: expired, cancelled. Failed states recover via reactivate. Content is encrypted end-to-end with DTLS, and relays proxy traffic without accessing data.

Problem 1: TURN Fallback and Optimal Chunk Size

P2P isn't always possible due to symmetric NAT, firewalls, or mobile networks. Connection types are determined from ICE candidates: p2p, server_relay (coturn with quotas), custom_relay (custom TURN without limits).

Chunk size adapts to minimize retransmission delays:

function getOptimalChunkSize(classification: string): number {
  switch (classification) {
    case 'p2p':          return 1024 * 1024;  // 1 MB
    case 'custom_relay': return 512 * 1024;   // 512 KB
    case 'server_relay': return 64 * 1024;    // 64 KB
    default:             return 512 * 1024;
  }
}

Smaller chunks on server_relay speed up recovery: a lost packet retransmits fully, with delays scaling by size. Pre-checking TURN via a dedicated RTCPeerConnection with iceTransportPolicy: 'relay' takes about 5 seconds.

Problem 2: Backpressure and Overlapped I/O in the Send Loop

A naive loop sends chunks sequentially, idling on file.slice().arrayBuffer(). The production implementation uses double buffering and high/low watermarks:

const BUFFER_THRESHOLD = 8 * 1024 * 1024; // 8 MB
const BUFFER_LOW       = 6 * 1024 * 1024; // 6 MB

dc.bufferedAmountLowThreshold = BUFFER_LOW;
let nextBuffer = await file.slice(0, chunkSize).arrayBuffer();

while (offset < file.size) {
  const buffer = nextBuffer;
  const readPromise = offset + buffer.byteLength < file.size
    ? file.slice(offset + buffer.byteLength, nextEnd).arrayBuffer()
    : null;

  if (dc.bufferedAmount >= BUFFER_THRESHOLD) {
    await waitForBufferDrain(dc, computeDrainTimeout(dc.bufferedAmount, speed));
  }

  dc.send(buffer);
  offset += buffer.byteLength;

  const effectiveSent = Math.max(0, offset - dc.bufferedAmount);
  const pct = Math.min((effectiveSent / file.size) * 100, isLast ? 100 : 99);

  nextBuffer = readPromise ? await readPromise : null;
}

• Overlapped I/O: Reads the next chunk in parallel with sending the current one.
• Dynamic timeout: estimatedDrainMs = (bufferedAmount / speed) * 1000, capped at 5–60 seconds.
• Honest progress: offset - dc.bufferedAmount, never exceeding 99% until fully drained.

Completion requires buffer drain + transfer_ack from the receiver before transfer_done.

Problem 3: Receiver-Side and Disk Backpressure

The receiver copies ArrayBuffer to avoid DataChannel buffer reuse:

const chunk = event.data.slice(0);  // Copy
writeChain = writeChain.then(() => writer.write(chunk));

if (bytesReceived === fileInfo.size) {
  dc.send(JSON.stringify({ type: 'transfer_ack' }));
}

writeChain is a sequential promise chain. Slow disks (like USB drives) create backpressure up the stack, throttling SCTP data intake.

Additional Production Challenges

• File Loss: File objects don't serialize; store as ArrayBuffer or Blob in IndexedDB.
• Reconnections: Signaling can drop before DataChannel; treat peer_left as normal.
• Safari Quirks: Handle ordered/unordered modes and buffers separately.

Key Takeaways:

• dc.send() writes to the SCTP buffer, not the network; always monitor bufferedAmount.
• Adapt chunk sizes: 1MB for P2P, 64KB for server_relay to optimize retransmits.
• Wait for transfer_ack after drain before signaling transfer_done.
• Use overlapped I/O and dynamic timeouts for top performance.
• Verify relay quotas bilaterally; run TURN health checks upfront.

— Editorial Team