Packing/unpacking binary data in C - doubles, 64 bits

Ben Scott dragonhawk at gmail.com
Thu Sep 10 17:36:02 EDT 2009


On Thu, Sep 10, 2009 at 5:06 PM, Dana Nowell
<DanaNowell at cornerstonesoftware.com> wrote:
> Which one is a bit more complex as this long winded post
> will hopefully show.

  Thanks for that response; it's very informative!

> A typical 'standard' floating point network format is
> a sign bit followed by N exponent bits followed by M 'mantissa' bits,
> see IEEE 754-2008 interchange format for a more 'in depth' discussion.

  What about byte order (endianess)?  Does IEEE 754-2008 define that?
When Bruce asked I Google'ed but found inconsistent answers and gave
up.  I suspect it may be that the original IEEE 754 spec did not
specify but later revisions did; is that the case?

  Are there standard or common C facilities to convert whatever one's
machine might use for "float" to something sure to be IEEE 754-2008?

> It appears you dislike suggestions of 'use ASCII' as they are space
> inefficient ...

  His application is in the realm of an HPC application -- the lower
end of HPC, but there.  In the past he has mentioned 100+ megabytes of
data.  The CPU overhead of doing ASCII/binary and back again can
become rather significant.  So can the increased network bandwidth
used by a less-compact ASCII representation.

  Just for the sake of example: Bruce said 160 MB of data.  Let's
assume it's all 4-byte integers.  That's roughly 42 million integers.
Calling sprintf() and sscanf() 42 million times is going to slow
things down.  Likewise, if we assume a newline separated format and
all significant digits used, an ASCII representation is going to use
11 bytes per integer, turning 160 MB into 440 MB.

-- Ben


More information about the gnhlug-discuss mailing list