Filesystem overhead

bscott at ntisys.com bscott at ntisys.com
Mon Jul 28 11:21:28 EDT 2003


Hello list,

  Red Hat Linux 7.3
  EXT3 filesystem
  Linux kernel 2.4.20-18.7
    EXT3 filesystem driver 2.4-0.9.19
  glibc 2.2.5-43
  GNU fileutils 4.1-10.1

  I'm aware of the fact that there is overhead in any filesystem, due to
things like metadata, internal fragmentation, and such.  Up until now, I've
pretty much just accepted what tools like "du" tell me without wondering
about the details.  Today, though, I was doing the math, and the actual
amount surprised me.

  For example, I have an image of a data CD in a single file.  The actual
size of the logical file (as reported by "stat", "ls", and other tools) is
526,397,440 bytes.  However, the "du" utility says it uses 526,917,632
bytes.  That is a difference of 520,192 bytes, or almost half a megabyte.

  Now, compared to the size of the file, that is an overhead of roughly 0.1
percent -- tiny.  But the absolute size of 0.5 megabytes for a single file
is large enough that I'm curious as to what uses so much space.

  My understanding of "internal fragmentation" is that it is the unused
space between the logical end-of-file and the end of the last physical block
allocated on disk for file storage.  For example, with 512 byte blocks, a
1025 byte file needs two whole blocks allocated for it, leaving 511 bytes of
unused space in the second block.  Since this filesystem uses a block size
of 4096 bytes (as reported by "dumpe2fs"), that would be a maximum of 4095
bytes of "slack space".  So that can't be it.

  The "stat data" in the "stat64" structure (from /usr/include/bits/stat.h)
is only 96 bytes.  While I imagine the actual filesystem stat data might be
a different size, I wouldn't think it would be *that* much bigger.

  Since this is a single file, any space used by directory entires should
not matter.

  So there must be something going on that I am not aware of.  Does anyone
here know what it might be?

  In once sense, it doesn't really matter, since I'm pretty much stuck with
it.  But I would at least like to know what is going on.  :)

  advTHANKSance

-- 
Ben Scott           | Net Technologies, Inc.  | 978-462-8795
Network Engineer    | Salisbury, MA, USA      | 866-NTI-LINUX (684-5468)
bscott at ntilinux.com | http://www.ntilinux.com | Fax: 978-499-7839




More information about the gnhlug-discuss mailing list