How to frame a TCP byte stream without losing messages
TCP preserves byte order. It does not preserve the message boundaries your device or partner protocol expects.
- Published
- Updated
- Reading time
- 9 min read
Tested environment Node.js 22 on Windows 11 using synthetic, fragmented byte sequences over a local TCP harness.
Plan before the graph
Start with the protocol contract, not the first packet capture. A single application message can arrive in several TCP reads. Several messages can also arrive in one read. If a channel treats each read as one message, it will eventually split valid work or join unrelated work.
Write down three facts before you configure the listener:
- How does the sender mark the beginning and end of one message?
- Is the length measured in bytes, characters or encoded code points?
- What must happen when the declared boundary is impossible or incomplete?
Choose one framing rule
Use the strongest boundary the sender actually provides. A length prefix is explicit and works with binary data. A delimiter is easy to inspect but only works when the delimiter cannot appear unescaped in the payload. Fixed width is predictable when the protocol truly fixes every record. An idle gap is a last resort because network timing is not a data contract.
| Framing rule | Good fit | Main failure to handle |
|---|---|---|
| Length prefix | Binary or variable records with an explicit byte count | Impossible length or a connection that closes before the full frame |
| Delimiter | Text records with a reserved terminator | Escaped delimiters and partial terminators across reads |
| Fixed width | Records with one contractual byte size | Wrong encoding or a sender that changes record length |
| Idle gap | Legacy equipment with no reliable boundary marker | Slow delivery that looks like the end of a message |
Keep incomplete bytes owned
The buffer is part of the work record. Do not discard it because the current read ended. Append new bytes, extract every complete frame in order and retain the remainder. Put a clear maximum on the buffer so a corrupt length cannot consume memory without limit.
export function readLengthPrefixed(buffer: Uint8Array) {
if (buffer.byteLength < 2) return { messages: [], remaining: buffer };
const view = new DataView(buffer.buffer, buffer.byteOffset, buffer.byteLength);
const length = view.getUint16(0, false);
const frameEnd = 2 + length;
if (buffer.byteLength < frameEnd) {
return { messages: [], remaining: buffer };
}
return {
messages: [buffer.slice(2, frameEnd)],
remaining: buffer.slice(frameEnd),
};
}The example reads one frame so the boundary is visible. A production loop repeats the same operation until the remaining buffer no longer contains a full frame. It also rejects lengths above the protocol limit and records why the connection was closed.
Test through the real read path
Do not prove framing with one complete message passed directly into the parser. Feed the bytes through the same read path the listener uses. At minimum, test one byte at a time, several messages in one read, a boundary split between reads, an invalid length and a disconnect with an incomplete frame.
Read the transcript
The source sends an ordered byte stream. We first record the framing rule from the protocol contract, including the byte order and maximum frame size.
The first read ends halfway through a frame. The listener keeps those bytes and appends the next read. It emits nothing until the declared boundary is complete.
A later frame declares a size above the configured limit. The channel rejects that frame, records the reason and leaves the earlier completed work unchanged.
Approve what failure means
A malformed frame should not disappear and it should not be retried without a rule. Decide whether the listener closes the connection, parks the work for review or emits a typed rejection. Preserve enough safe evidence to explain the choice without logging the payload by default.
What this does not cover
This guide does not define TLS authentication, certificate rotation or a vendor-specific healthcare protocol carried inside the frame. Those layers need their own contracts and tests. It also does not claim that idle-gap framing is reliable for every device. Measure it on the real counterpart before approval.