Demand for Readable and Efficient Binary Format Implementation
Problem
Multiple users point out that current binary formats lack standard structure (e.g., missing magic header, version, CRC) and desire a binary representation that is both compact and readable; they also seek automatic parser generation for performance and safety, especially in memory‑constrained environments.
Pain points
- Lack of standard binary format structure (missing magic header, version, CRC)
- Desire for compact yet readable format using single‑character field names and position‑dependent layout
- Need for automatic parser generation to improve speed and safety
- Efficient parsing and indexing on memory‑constrained devices
Questions to verify
- Do you use binary formats that require high efficiency in real projects? What specific pain points have you encountered?
- Do you need schema validation or automatic parser generation for binary data? Do current tools meet your needs?
- Does your application run in memory‑constrained environments where parsing speed and memory usage are critical?
Evidence
quote
I'm missing some important parts that I would argue any (binary)format needs.
quote
I also want to be a real developer so i just GZIP the JSON to make it binary.
quote
Schemas save you space, but then you lose the ability to understand the data without the schema. That's where JSON has always been handy despite its inefficiencies.
quote
One of my favourite yak shaving adventures in a previous job was writing a parse in place UBJSON decoder for ~1MB of data on a device with about as much free memory.
quote
Congratulations, your strings dont support Unicode and you just ignored probably more than 3/4 or more of the world's languages.
Disclaimer: this card is distilled by AI from community discussions as “pain point evidence”. It is a starting point for need analysis and validation, not a “validated market need”. AI has analytical limits; please open the original discussion above to manually verify before key decisions.