What is Protocol Buffers?
Protocol Buffers (protobuf) is Google's language-neutral binary serialization format. You define messages in a .proto schema, compile code generators, and exchange compact binary payloads over gRPC, Kafka, and many APIs. Unlike JSON, the wire format does not include field names. only field numbers and wire types. so a schema-free decoder can recover structure and values, but not the human-readable names.
How to read a protobuf payload by hand
- Read a tag varint: low 3 bits are the wire type; the rest is the field number
- Wire type 0: read another varint (the value)
- Wire type 1: read the next 8 bytes little-endian
- Wire type 2: read a length varint, then that many bytes
- Wire type 5: read the next 4 bytes little-endian
- Repeat until the buffer is consumed
Example: 08 96 01 → tag 0x08 = field 1, wire type 0; value varint 0x96 0x01 = 150.
Decoding without a .proto (like protoc --decode_raw)
Running protoc --decode_raw < message.bin prints field numbers and values without a schema. This online tool does the same in your browser and adds:
- Multiple type interpretations per field (uint vs sint vs bool, float vs fixed32, string vs nested message)
- Recursive expansion of nested messages
- Packed-repeated detection
- JSON export keyed by field number
What you still cannot recover without the .proto: field names (user_id vs 1), enum symbolic names, and oneof groupings.
Where protobuf shows up
- gRPC: request and response message bodies
- Kubernetes / etcd / Envoy: many control-plane APIs use protobuf
- Android / mobile: compact payloads between apps and backends
- Observability: OpenTelemetry OTLP traces and metrics
- Storage: protobuf blobs in Kafka topics, object stores, and databases
Protobuf vs JSON vs CBOR
- Smaller than JSON: field numbers instead of names; binary ints and floats
- Schema-first: evolving messages safely needs the
.proto and field-number discipline
- Not self-describing: unlike JSON or CBOR, you need the schema (or a wire-format viewer like this) to interpret payloads
- CBOR is closer to self-describing JSON-in-binary. try our CBOR Decoder when that is what you have