How We Cut a Client's JSON Bandwidth by 70% Without Breaking a Single Downstream Service

A client came to us with a problem that started as a line item on a cloud bill and ended as an architecture question. They run a large fleet of field devices - industrial sensors, gateways, and edge boxes - all reporting home over JSON. The schema was sensible, the team liked JSON, and a whole ecosystem had grown around it: an ingestion pipeline, a half-dozen microservices, analysts’ notebooks, a couple of legacy dashboards, and a long tail of scripts that all assumed “the data is JSON.”

The brief

A typical message looked like this:

{
  "device_id": "sensor-4471",
  "fw": "2.3.1",
  "ts": 1718834000,
  "readings": [
    { "metric": "temperature", "value": 22.5,  "unit": "celsius", "status": "nominal" },
    { "metric": "humidity",    "value": 61.0,  "unit": "percent", "status": "nominal" },
    { "metric": "pressure",    "value": 1013.2,"unit": "hpa",     "status": "nominal" }
  ]
}

Individually, harmless. At fleet scale - hundreds of thousands of messages a minute - it became three compounding costs:

Egress and ingestion. The data was tiny; the envelope was enormous. Count how many times "metric", "value", "unit", "status", and "nominal" appear in that one message - then multiply by every reading, in every message, forever. They were paying to ship the same field names billions of times a day.
Decode cost downstream. Every consumer re-parsed full JSON, allocating a fresh string for every key and value, just to read two or three fields.
Resilience. On flaky cellular links, bigger payloads meant more partial transfers and more retries. Bytes on the wire were, quite literally, a reliability problem.

The constraints made the obvious answers unattractive:

Just gzip it. Helps on the wire, but it’s CPU on both ends - and on a battery-powered MCU, compression isn’t free. Worse, it does nothing for reading: you must inflate the entire blob before you can touch a single field. It’s a transport trick, not a data format.
Move to Protobuf. Smallest and fastest, but schema-first. Every one of those downstream consumers would need the .proto and a codegen step. Breaking the entire “it’s just JSON” ecosystem was off the table.
MessagePack or CBOR. Genuinely smaller than JSON, but they still write every key in every object. Same redundancy, smaller font.

What the client actually needed was contradictory on its face: the compactness of a schema-based format, the ergonomics of JSON, and zero migration cost for everything already downstream.

So we built TSON - Terse JSON - and deployed it as a drop-in at the edges of their pipeline. It’s open source (MIT) and lives at github.com/siktec-lab/tson. This is how it works, why it’s fast, and - the part that ended up mattering most - why being able to read fields directly off the binary changed the economics for their hot-path services.

The core idea: separate the structure from the values

The whole format hangs on one observation about real-world JSON: the structure repeats far more often than the data changes. The same object shape - the same set of keys - appears thousands of times; only the numbers inside move.

So TSON stops storing structure and data together. A document is three blocks:

┌──────────────────────────────────────────────────────────┐
  HEADER  13 bytes: version + offsets to the 3 blocks
├──────────────────────────────────────────────────────────┤
  DEFINITIONS   each distinct object "shape", once
    #7  Object {metric:String, status:String,
                 unit:String, value:Float}
├──────────────────────────────────────────────────────────┤
  DICT          each repeated string value, once
    [0] "nominal"   [1] "celsius"   [2] "percent" ...
├──────────────────────────────────────────────────────────┤
  DATA          pure typed values — no keys, no quotes
    entry → def #7 → ["temperature", →dict, →dict, 22.5]
    entry → def #7 → ["humidity",    →dict, →dict, 61.0]
└──────────────────────────────────────────────────────────┘

The first time the compiler sees an object with the keys metric, status, unit, and value, it records that shape as a definition - the field names, sorted, with their types. Every later object of the same shape becomes a 2-byte reference to that definition plus its bare values. The field names are written once for the entire document, regardless of how many thousands of readings follow.

Two design choices in there are worth pausing on:

Fields are sorted and positional. Inside a definition, fields are stored in a fixed order, so a value’s position implies its key. The data block carries no keys at all - just values, in order. That’s where the bulk of the savings comes from.
Types live in the definition, not the values. Because the definition says value is a Float, the data block stores a bare 4-byte f32 with no type tag per value. The schema pays for the type information once.

Repeated string values get the same once-only treatment through the dict block, which we’ll come back to.

On the client’s real telemetry, the result was a consistent ~70% reduction:

Payload	JSON	TSON	Savings
telemetry batch (500 readings)	54.4 KB	16.2 KB	70.2%
routing config (200 rules)	27.9 KB	8.4 KB	69.7%
128 KB mixed dataset	249 KB	104 KB	58.1%

And - this is the part that kept the migration sane - it’s still self-describing. The definitions travel inside the document. There’s no external schema file, no codegen, no version-skew between a producer and a consumer that compiled against different .protos. A decoder that has never seen your data can fully reconstruct the original JSON from the bytes alone.

The wire format, concretely

A few details from the binary spec, because they explain the performance later.

Header - 13 fixed bytes. Version plus three little-endian u32 offsets to the definition, dict, and data blocks. A decoder reads 13 bytes and instantly knows where everything lives:

[version:u8][def_off:u32][dict_off:u32][data_off:u32]

Type tags. Eight of them, one byte each, used only inside definitions:

Tag	Type	Wire size
0x00	Null	0 B
0x01	Bool	1 B
0x02	Int	4 B (i32 LE)
0x03	UInt	4 B (u32 LE)
0x04	Float	4 B (f32 LE)
0x05	String	variable
0x10	Array	variable
0x11	Object	variable

Data entries. The data block is a flat sequence of length-prefixed entries:

[def_index:u16][payload_len:u32][ …payload… ]

That payload_len is doing quiet, important work: it makes the data block seekable. You can jump from one entry to the next without decoding the one in between - which is exactly what makes streaming and partial reads cheap.

Detail: a string length prefix that respects short strings

Most binary formats slap a fixed 4-byte length on every string. But real JSON is full of short strings - field values like "nominal", "celsius", status codes, IDs. Spending 4 bytes to say “this is 7 bytes long” is mostly waste.

TSON uses a hybrid, self-describing length prefix where the first byte tells the decoder how wide the length is - and doubles as the dict-reference sentinel:

First byte	Overhead	Max length	Meaning
`0x00..=0x7F`	1 byte	127 B	inline string, 1-byte length
`0x80..=0xBF`	2 bytes	16 KB	inline string, 2-byte length
`0xFE`	4 bytes	16 MB	inline string, 3-byte length
`0xFF`	5 bytes	—	not a string - a dict reference

The strings that dominate real payloads (< 128 bytes) cost a single overhead byte. The decoder reads one byte, learns the width, consumes exactly that many more - no modes, no look-ahead, no cross-value state. Every value is self-contained, which is what makes single-pass streaming possible.

That last row is the hinge between this section and the next: 0xFF means “this value is dict entry N.” That’s how a repeated string becomes a 5-byte pointer instead of paying its full length every time.

Lazy string interning: dedup without the dictionary tax

Naive interning has a trap. If you build a dictionary of every string, you discover that half of them appeared only once - and now a one-off string costs a dict slot plus a reference, which is worse than just inlining it.

TSON’s compiler avoids this with lazy promotion. A string only earns a dict entry once it has proven it repeats:

First sighting → emit it inline, remember “seen once.”
Second sighting → promote it into the dict now, emit a 5-byte StrRef.
Third onward → already in the dict, just reference it.

The dict therefore holds only strings that genuinely appear ≥ 2 times. Unique strings stay inline and cost nothing extra. For a document of all-unique values, the dict block is empty - zero overhead. You get interning’s win without interning’s tax. Here’s the actual compiler path:

fn emit_string(&mut self, s: &str) -> TsonData {
    // Already promoted (seen ≥ 2 times) → just reference it.
    if let Some(&idx) = self.dict_map.get(s) {
        return TsonData::StrRef(idx);
    }
    // Second sighting → promote into the dict now.
    if self.seen_once.contains_key(s) {
        let idx = self.dict.len() as u32;
        self.dict.push(s.to_string());
        self.dict_map.insert(s.to_string(), idx);
        return TsonData::StrRef(idx);
    }
    // First sighting → inline, and remember we've seen it.
    self.seen_once.insert(s.to_string(), ());
    TsonData::String(s.into())
}

For the client’s telemetry - where "nominal", the unit strings, and the metric names recur constantly - this turned kilobytes of repeated text into a handful of dict entries and a sea of 5-byte references.

Making the implementation fast: the encoder that stopped allocating

Designing the format makes it small. Implementing it well makes it fast - and this is where a profiler paid for itself.

The first encoder did the idiomatic, innocent thing: each node in the value tree returned its own Vec<u8>, and the parent copied that into its buffer.

// The natural version — and a quiet performance sink.
fn encode_value(value: &TsonData) -> Vec<u8> {
    match value {
        TsonData::Int(v) => v.to_le_bytes().to_vec(),   // heap alloc per int
        TsonData::Array(_, _, items) => {
            let mut buf = Vec::new();                    // heap alloc per array
            for item in items {
                let child = encode_value(item);          // alloc…
                buf.extend_from_slice(&child);           // …copy…
            }                                            // …then drop. Repeat ×N.
            buf
        }
        // …
    }
}

For a document with N nodes, that’s on the order of N heap allocations and N copies - every leaf builds a throwaway Vec just to be copied into its parent and freed. Death by a thousand mallocs.

The fix is almost dull: stop returning buffers; write into one shared buffer.

// Append straight into the output — no per-node Vec, no intermediate copy.
fn encode_value_into(value: &TsonData, buf: &mut Vec<u8>) {
    match value {
        TsonData::Int(v) => buf.extend_from_slice(&v.to_le_bytes()),
        TsonData::Array(self_def, elem_def, items) => {
            buf.extend_from_slice(&self_def.to_le_bytes());
            buf.extend_from_slice(&elem_def.to_le_bytes());
            buf.extend_from_slice(&(items.len() as u16).to_le_bytes());
            for item in items {
                encode_value_into(item, buf);   // recurse into the same buffer
            }
        }
        // …
    }
}

One wrinkle: each data entry needs its payload_len prefix, but you don’t know the length until the payload is written. The fix is a reserve-and-backpatch: write a placeholder, encode the payload, then patch the length using the recorded offset.

let len_pos = buf.len();
buf.extend_from_slice(&0u32.to_le_bytes());        // placeholder
let start = buf.len();
encode_value_into(&chunk.data, buf);                // write the payload
let payload_len = (buf.len() - start) as u32;
buf[len_pos..len_pos + 4].copy_from_slice(&payload_len.to_le_bytes());  // backpatch

The benchmark delta (Criterion, release build) was not subtle:

Operation	Before	After	Change
encode (telemetry, 54 KB)	160 µs	11 µs	−93% (~14×)
encode (128 KB)	251 µs	28 µs	−88% (~9×)
compile (telemetry)	768 µs	440 µs	−43%
full round-trip	1.23 ms	0.71 ms	−45%

Encode went from the second-most-expensive stage to the cheapest - roughly 0.45 µs for a small document. A few smaller wins compounded it: definition lookups dropped from O(n) scans to O(1) index access (definitions are stored in index order, so the index is the slot); UTF-8 validation collapsed from copy-then-validate into a single validated pass; and the release profile got codegen-units = 1 so the recursive hot paths inline cleanly.

The point for the client: a server compiling millions of readings per minute now spends its CPU discovering schemas and interning strings - the genuinely useful work - not shuffling bytes between throwaway buffers.

The part that changed the architecture: reading fields without going back to JSON

Here’s where TSON stopped being “a smaller JSON” and started being a better runtime data model for the client.

Most binary formats are a black box until you fully deserialize them. You receive bytes, you inflate them into objects, then you can read a field. TSON’s layout - definitions up front, positional values, length-prefixed entries - means a consumer can read fields directly off the decoded structure without ever rebuilding a JSON object.

This mattered enormously for the client’s hot-path services: routers, filters, and alerting rules that only needed one or two fields out of each message. They were paying to reconstruct an entire serde_json::Value (or a Python dict, or a JS object) - allocating every key and value - just to read status.

Field access by name

let doc = tson::from_bytes(&bytes)?;

// Read a field straight off the document — no JSON rebuild.
if let Some(tson::TsonData::String(status)) = doc.get("status") {
    if status == "nominal" { /* fast-path: drop it */ }
}

doc.get("status") resolves the name against the definition table and returns a borrowed reference into the already-decoded values. No intermediate JSON, no new allocations for the fields you didn’t ask about.

O(1) field access in a hot loop

For services that pull the same field out of millions of messages, you resolve the field’s position once and then index straight in - no string comparison at all on the hot path:

// Resolve "status" to its positional index once…
let status_idx = doc.index("status").expect("known field");

// …then read it positionally, with zero string work, per message.
for _ in 0..messages {
    if let Some(value) = doc.get_by_index(status_idx) {
        route(value);
    }
}

Because fields are positional inside a definition, get_by_index is a plain slice index - the kind of thing a branch predictor loves.

Walking nested values directly

The same applies inside arrays and nested objects. You can navigate the decoded tree with .field(), .values(), and .len() and only touch the values you care about:

let reading = entry.data.field("readings", defs)   // the array value
    .and_then(|arr| arr.values().first());          // first reading object

if let Some(r) = reading {
    let metric = r.field("metric", defs);            // borrowed, no alloc
    let value  = r.field("value", defs);
    // act on `metric` / `value` without ever building JSON
}

For the client, rewriting their routing and alerting services to read TSON fields directly - instead of loads() → inspect → discard - removed the dominant allocation cost from those services entirely. The messages were smaller on the wire and cheaper to act on at the destination.

Streaming decode with O(1) memory

The edge devices cared about a different property. Because the definition and dict blocks are small (load them into RAM once) and the data block is a flat sequence of length-prefixed entries, a device can walk a large archive one entry at a time - never holding more than a single reading in memory.

let mut reader = tson::TsonStreamReader::new(&bytes)?;

// Schema is parsed once, up front:
println!("{} definitions, {} dict entries",
    reader.definitions().len(), reader.dict().len());

// Then entries stream by, O(1) memory each:
for entry in reader {
    let chunk = entry?;       // one TsonChunk
    process(chunk);           // handle it, then it's dropped
}

A gateway with a few kilobytes of free RAM can process a 200 KB archive of readings, because at any instant it holds exactly one. Field names are resolved by index, strings by dict reference - no per-entry allocation of keys, ever. That’s the structural payoff of paying for the schema once, up front, and amortizing it across every entry.

The response path: replying in TSON without rebuilding a schema

One more use case the client hit: services that receive a TSON message, inspect a few fields, and reply with a TSON message. Naively, the reply would re-discover its own schema and rebuild a dict from scratch.

Instead, the response path reuses the definitions and dict that already came in with the request:

use tson::{TsonData, emit_with_context};

// `defs` and `dict` come straight from the parsed incoming document.
let response = TsonData::Object(7, vec![
    TsonData::String("ack".to_string()),
    TsonData::Float(35.2),
]);

let bytes = emit_with_context(&response, &incoming_defs, &incoming_dict)?;

No schema re-discovery, no dict rebuild - just encode the values against context that’s already in hand. For a request/response service under load, that’s a meaningful chunk of work that simply doesn’t happen.

Same ergonomics, three languages

The reason the rollout didn’t turn into a migration project: at the boundary, TSON looks exactly like the json module everyone already uses.

# Python  —  pip install tson-bin
import tson
blob = tson.dumps('{"device_id":"sensor-4471", ...}')   # JSON string → TSON bytes
obj  = tson.loads(blob)                                   # TSON bytes → dict

// Node.js  —  npm install @siktec-lab/tson
const tson = require('@siktec-lab/tson')
const blob = tson.dumps(jsonString)   // → Buffer
const obj  = tson.loads(blob)         // → object

// Rust  —  cargo add tson
let doc   = tson::compile_json(json)?;                          // JSON → document
let bytes = tson::to_bytes(&doc)?;                              // → compact binary
let value = tson::decompile_to_value(&tson::from_bytes(&bytes)?)?;  // → serde_json::Value

dumps / loads - the same muscle memory as the standard library. We slotted TSON in at the transport edge (encode just before the wire, decode just after), and everything in between kept speaking JSON. The analysts’ notebooks, the legacy dashboards, the services nobody wanted to touch - all unchanged. They handed JSON in and got JSON out. The hot-path services that opted in to reading TSON fields directly got the extra performance; everyone else got smaller payloads for free.

The Rust core is also no_std-capable (it only needs alloc), which is what let the same format run on the constrained devices at the other end of the link.

Where it landed, vs. the alternatives

	TSON	MessagePack	CBOR	Protobuf	gzip’d JSON
Self-describing	✅	✅	✅	❌ needs `.proto`	✅
Field-name dedup	✅	❌ repeats keys	❌	✅	~ (entropy only)
String interning	✅	❌	❌	❌	~ (entropy only)
Read a field w/o full decode	✅	❌	❌	❌	❌
Streaming, O(1) memory	✅	❌	❌	❌	❌
Seekable on disk	✅	❌	❌	✅	❌
`no_std` / microcontroller	✅	~	~	❌	❌
Zero downstream migration	✅	~	~	❌	✅

The two rows in bold are the ones that decided it. Reading a field without a full decode is what made TSON cheaper at the destination, not just on the wire - gzip can’t do it (you must inflate first), and MessagePack/CBOR/Protobuf all require materializing objects before access. Combined with streaming and no migration, it hit every corner of the client’s original, contradictory brief.

There is an honest trade-off, and it’s the right shape for this workload: TSON moves work to compile time. Discovering schemas and interning strings costs more than a dumb serialize. But the access pattern is exactly write-once-read-many - a server compiles a reading once; a fleet of devices and a swarm of consumers read it many times. You pay once and save on every read.

Results

For the client, concretely:

~70% smaller payloads on their repetitive telemetry - directly off the egress bill, faster ingestion, fewer retries on bad links.
~14× faster encoding and a ~45% faster round-trip after the allocation rewrite, so compiling at fleet scale stopped being a bottleneck.
Allocation-free field reads on the hot-path services that opted in - routing and alerting stopped rebuilding objects they immediately threw away.
O(1)-memory streaming so the edge devices could process large archives.
Zero downstream breakage, because the interface was still “JSON in, JSON out” for everyone who didn’t need more.

If your system moves a lot of structured, repetitive JSON - telemetry, event streams, API fan-out, config snapshots - the field names you ship over and over are pure overhead, and rebuilding full objects just to read two fields is pure waste. TSON’s whole bet is that you can say the structure once, pack the values tight, read them in place, and let everyone who doesn’t care keep pretending it’s still JSON.

Because, honestly, it still is. It’s just terse about it.

TSON is open source (MIT) at github.com/siktec-lab/tson, with native bindings for Rust (cargo add tson), Python (pip install tson-bin), and Node.js (npm install @siktec-lab/tson). The encoder, the streaming reader, the direct-access API, and the full binary spec - plus the benchmarks behind every number above - are all in the repo.