Filesystem overhead
michael.odonnell at comcast.net
michael.odonnell at comcast.net
Mon Aug 4 11:27:35 EDT 2003
|>Very cool, that was revealing. Perhaps this discussion
|>can evolve into how journalling (e.g. ext3, etc.) works
|>and why it is good/bad. Anybody?
|
| I would like to see some real metrics on:
| ext2
| ext3
| JFS
| XFS
| ReiserFS
FWIW (which may not be much in the context of the
current discussion) I include here some very raw
numbers obtained during filesystem benchmarking I
performed last fall. I'm not really free to go into
much background but I am able to say that we were
narrowly focussed on characterizing systems such that
we'd know how to get the best performance when writing
absolutely synchronously ie. the data had to actually
end up on a disk platter and not just get parked in
some cache, like in the buffer cache or in the drive
controller's cache.
I generated the numbers with a bash script that kept
launching an app called DIO (see http://oss.mclx.com/)
with different parameters after first mucking around
with various system characteristics like mount
attributes, raw device options, filesystem size, etc.
We tested local disks and also our customized variant
of the "nb" (Network Block) device.
IIRC we concluded at the time that XFS ignores the
sync/async attribute during mounts and always runs
async which, given our goal, we weren't too happy
about, though this isn't a problem for the average user
and it sure makes it seem zippier. Also, XFS makes
some of its lowlevel I/O requests with oddball block
sizes that local disks don't mind but which confused
the nb stuff, so we couldn't benchmark XFS against nb.
A conclusion one might draw from our numbers is that
the journalled filesystems are all plenty fast enough
that you really aren't giving up much throughput in
return for the enhanced reliability. YMMV, though...
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
20021025 IBMx330 2.4.18-xfs-1.1 SMP - SCSIwriteCaches disabled
FIELDS:
Kb/S
| time
| | rawdev
| | | DIObsize
| | | | fstype
| | | | | fsKblocks
| | | | | | fsKperBlock
| | | | | | | MOUNTsync
| | | | | | | | DIOosync
| | | | | | | | | DIOappend
| | | | | | | | | | DIOsize
| | | | | | | | | | |
66 122.4 nb0 8k ext3 100000 4 sync osync append 8M
68 119.2 nb0 32k ext2 100000 1 sync - append 8M
68 119.9 nb0 8k ext3 100000 4 sync - append 8M
68 120.2 nb0 32k ext2 100000 1 sync osync append 8M
70 115.9 nb0 32k ext3 100000 4 sync - append 8M
70 116.8 nb0 32k ext3 100000 4 sync osync append 8M
75 108.6 nb0 8k ext2 100000 1 sync osync append 8M
75 108.9 nb0 8k ext2 100000 1 sync - append 8M
94 86.4 sda5 8k ext2 100000 1 sync osync append 8M
94 86.5 sda5 8k ext2 100000 1 sync - append 8M
103 79.4 sda5 32k ext2 100000 1 sync - append 8M
103 79.4 sda5 32k ext2 100000 1 sync osync append 8M
162 50.4 nb0 8k reiserfs 100000 4 async osync append 8M
163 50.1 nb0 8k reiserfs 100000 4 sync osync append 8M
176 46.4 nb0 8k ext3 100000 1 sync osync append 8M
177 46.3 nb0 8k ext3 100000 1 sync - append 8M
185 44.1 nb0 32k reiserfs 100000 4 async osync append 8M
185 44.2 nb0 32k reiserfs 100000 4 sync osync append 8M
195 41.9 nb0 32k ext3 100000 1 sync - append 8M
196 41.6 nb0 32k ext3 100000 1 sync osync append 8M
205 39.8 sda5 8k reiserfs 100000 4 async osync append 8M
205 39.9 sda5 8k reiserfs 100000 4 sync osync append 8M
206 39.6 sda5 8k ext3 100000 4 sync osync append 8M
207 39.4 sda5 8k ext3 100000 4 sync - append 8M
212 38.6 sda5 8k ext3 100000 1 sync - append 8M
213 38.5 sda5 8k ext3 100000 1 sync osync append 8M
224 36.5 nb0 8k ext2 100000 4 sync osync append 8M
225 36.4 sda5 32k reiserfs 100000 4 async osync append 8M
226 36.2 nb0 8k ext2 100000 4 sync - append 8M
226 36.2 sda5 32k reiserfs 100000 4 sync osync append 8M
232 35.2 sda5 32k ext3 100000 4 sync osync append 8M
235 34.8 sda5 32k ext3 100000 4 sync - append 8M
239 34.2 sda5 32k ext3 100000 1 sync osync append 8M
243 33.7 sda5 32k ext3 100000 1 sync - append 8M
275 29.7 sda5 8k ext2 100000 4 sync - append 8M
275 29.8 sda5 8k ext2 100000 4 sync osync append 8M
293 27.9 nb0 32k ext2 100000 4 sync osync append 8M
293 27.9 nb0 32k ext3 100000 1 async osync rewrite 8M
294 27.8 nb0 32k ext3 100000 1 sync - rewrite 8M
295 27.7 nb0 32k ext2 100000 4 sync - append 8M
296 27.6 nb0 32k ext3 100000 1 sync osync rewrite 8M
303 27.0 nb0 32k ext2 100000 1 async osync append 8M
361 22.6 sda5 32k ext2 100000 4 sync osync append 8M
362 22.6 sda5 32k ext2 100000 4 sync - append 8M
375 21.8 nb0 8k ext3 100000 1 async osync append 8M
385 21.3 nb0 8k ext2 100000 1 async osync append 8M
387 21.1 nb0 8k ext3 100000 4 async osync append 8M
393 20.8 nb0 8k ext3 100000 1 sync osync rewrite 8M
393 20.8 nb0 8k ext3 100000 1 async osync rewrite 8M
394 20.8 nb0 8k ext3 100000 1 sync - rewrite 8M
422 19.4 nb0 32k ext3 100000 1 async osync append 8M
435 18.8 nb0 8k ext3 100000 4 async osync rewrite 8M
436 18.8 nb0 8k ext3 100000 4 sync - rewrite 8M
436 18.8 nb0 8k ext3 100000 4 sync osync rewrite 8M
455 18.0 nb0 8k ext2 100000 4 async osync append 8M
484 16.9 sda5 8k ext3 100000 4 async osync append 8M
498 16.4 sda5 8k ext3 100000 1 async osync append 8M
504 16.2 sda5 8k ext3 100000 4 sync osync rewrite 8M
506 16.2 sda5 8k ext3 100000 4 sync - rewrite 8M
507 16.2 sda5 8k ext3 100000 4 async osync rewrite 8M
508 16.1 sda5 8k ext3 100000 1 sync osync rewrite 8M
509 16.1 sda5 8k ext3 100000 1 sync - rewrite 8M
516 15.9 sda5 8k ext3 100000 1 async osync rewrite 8M
541 15.1 sda5 8k ext2 100000 4 async osync append 8M
553 14.8 sda5 8k xfs 100000 4 sync osync append 8M
555 14.7 sda5 8k xfs 100000 4 async osync append 8M
585 14.0 nb0 32k ext2 100000 1 sync - rewrite 8M
585 14.0 nb0 32k ext2 100000 1 sync osync rewrite 8M
585 14.0 nb0 32k ext2 100000 1 async osync rewrite 8M
601 13.6 sda5 8k ext2 100000 1 async osync append 8M
1098 7.5 nb0 8k ext2 100000 1 sync osync rewrite 8M
1099 7.5 nb0 8k ext2 100000 1 sync - rewrite 8M
1102 7.4 nb0 8k ext2 100000 1 async osync rewrite 8M
1227 6.7 nb0 8k reiserfs 100000 4 sync osync rewrite 8M
1245 6.6 sda5 8k ext2 100000 1 sync - rewrite 8M
1247 6.6 sda5 8k ext2 100000 1 sync osync rewrite 8M
1249 6.6 sda5 8k ext2 100000 1 async osync rewrite 8M
1251 6.5 nb0 8k reiserfs 100000 4 sync - rewrite 8M
1255 6.5 nb0 8k reiserfs 100000 4 sync - append 8M
1259 6.5 nb0 8k reiserfs 100000 4 async osync rewrite 8M
1263 6.5 sda5 8k reiserfs 100000 4 async osync rewrite 8M
1268 6.5 sda5 8k reiserfs 100000 4 sync osync rewrite 8M
1270 6.4 sda5 8k reiserfs 100000 4 sync - rewrite 8M
1275 6.4 sda5 8k ext2 100000 4 sync osync rewrite 8M
1275 6.4 sda5 8k reiserfs 100000 4 sync - append 8M
1277 6.4 nb0 8k ext2 100000 4 sync osync rewrite 8M
1278 6.4 sda5 8k RAW - - - - - 8M
1278 6.4 sda5 8k xfs 100000 4 sync osync rewrite 8M
1278 6.4 sda5 8k ext2 100000 4 async osync rewrite 8M
1281 6.4 sda5 8k xfs 100000 4 async osync rewrite 8M
1283 6.4 sda5 8k ext2 100000 4 sync - rewrite 8M
1287 6.4 nb0 8k ext2 100000 4 sync - rewrite 8M
1287 6.4 nb0 8k ext2 100000 4 async osync rewrite 8M
1290 6.3 nb0 8k RAW - - - - - 8M
1293 6.3 nb0 32k ext3 100000 4 async osync append 8M
1447 5.7 nb0 32k ext3 100000 4 sync - rewrite 8M
1475 5.6 nb0 32k ext3 100000 4 async osync rewrite 8M
1513 5.4 nb0 32k ext2 100000 4 async osync append 8M
1519 5.4 nb0 32k ext3 100000 4 sync osync rewrite 8M
1777 4.6 sda5 32k ext3 100000 4 async osync append 8M
1807 4.5 sda5 32k ext3 100000 1 async osync append 8M
1853 4.4 sda5 32k ext3 100000 1 sync - rewrite 8M
1858 4.4 sda5 32k ext3 100000 1 sync osync rewrite 8M
1868 4.4 sda5 32k ext3 100000 1 async osync rewrite 8M
1874 4.4 sda5 32k ext3 100000 4 sync - rewrite 8M
1894 4.3 sda5 32k ext3 100000 4 async osync rewrite 8M
1899 4.3 sda5 32k ext3 100000 4 sync osync rewrite 8M
2033 4.0 sda5 32k ext2 100000 4 async osync append 8M
2064 4.0 sda5 32k ext2 100000 1 async osync append 8M
2074 3.9 sda5 32k xfs 100000 4 sync osync append 8M
2084 3.9 sda5 32k xfs 100000 4 async osync append 8M
3948 2.1 nb0 32k reiserfs 100000 4 async osync rewrite 8M
4014 2.0 sda5 32k ext2 100000 1 async osync rewrite 8M
4088 2.0 nb0 32k reiserfs 100000 4 sync - append 8M
4130 2.0 sda5 32k ext2 100000 1 sync - rewrite 8M
4143 2.0 sda5 32k ext2 100000 1 sync osync rewrite 8M
4307 1.9 nb0 32k reiserfs 100000 4 sync osync rewrite 8M
4347 1.9 nb0 32k ext2 100000 4 sync - rewrite 8M
4358 1.9 sda5 32k xfs 100000 4 async osync rewrite 8M
4416 1.9 nb0 32k ext2 100000 4 sync osync rewrite 8M
4417 1.9 nb0 32k ext2 100000 4 async osync rewrite 8M
4457 1.8 sda5 32k reiserfs 100000 4 sync osync rewrite 8M
4465 1.8 sda5 32k reiserfs 100000 4 sync - rewrite 8M
4482 1.8 sda5 32k RAW - - - - - 8M
4499 1.8 sda5 32k ext2 100000 4 sync osync rewrite 8M
4501 1.8 sda5 32k reiserfs 100000 4 async osync rewrite 8M
4508 1.8 sda5 32k reiserfs 100000 4 sync - append 8M
4514 1.8 sda5 32k ext2 100000 4 sync - rewrite 8M
4515 1.8 sda5 32k xfs 100000 4 sync osync rewrite 8M
4515 1.8 sda5 32k ext2 100000 4 async osync rewrite 8M
4547 1.8 nb0 32k RAW - - - - - 8M
4611 1.8 nb0 32k reiserfs 100000 4 sync - rewrite 8M
7449 1.1 nb0 32k ext3 100000 1 async - append 8M
7487 1.1 nb0 8k ext3 100000 4 async - append 8M
7507 1.1 nb0 8k ext3 100000 1 async - append 8M
8200 1.0 nb0 32k ext2 100000 1 async - append 8M
8261 1.0 nb0 32k reiserfs 100000 4 async - append 8M
8262 1.0 nb0 8k ext2 100000 1 async - append 8M
8293 1.0 nb0 32k ext2 100000 4 async - append 8M
8318 1.0 nb0 8k reiserfs 100000 4 async - append 8M
8405 1.0 nb0 8k ext2 100000 4 async - append 8M
8628 0.9 nb0 32k ext3 100000 1 async - rewrite 8M
8691 0.9 nb0 32k ext3 100000 4 async - append 8M
8793 0.9 nb0 32k ext2 100000 4 async - rewrite 8M
8850 0.9 nb0 8k ext3 100000 1 async - rewrite 8M
8907 0.9 nb0 32k ext2 100000 1 async - rewrite 8M
8949 0.9 nb0 32k reiserfs 100000 4 async - rewrite 8M
9081 0.9 nb0 8k ext3 100000 4 async - rewrite 8M
9081 0.9 nb0 32k ext3 100000 4 async - rewrite 8M
9207 0.9 nb0 8k ext2 100000 1 async - rewrite 8M
9225 0.9 nb0 8k reiserfs 100000 4 async - rewrite 8M
9958 0.8 nb0 8k ext2 100000 4 async - rewrite 8M
13489 0.6 sda5 8k ext3 100000 1 async - append 8M
13779 0.6 sda5 32k ext2 100000 1 async - append 8M
13892 0.6 sda5 32k ext3 100000 1 async - append 8M
14644 0.6 sda5 8k ext2 100000 1 async - append 8M
15902 0.5 sda5 8k ext3 100000 1 async - rewrite 8M
16091 0.5 sda5 32k ext3 100000 1 async - rewrite 8M
16681 0.5 sda5 32k ext2 100000 1 async - rewrite 8M
17308 0.5 sda5 8k ext2 100000 1 async - rewrite 8M
20433 0.4 sda5 32k ext3 100000 4 async - append 8M
20544 0.4 sda5 32k reiserfs 100000 4 async - append 8M
20687 0.4 sda5 8k ext3 100000 4 async - append 8M
20851 0.4 sda5 8k reiserfs 100000 4 async - append 8M
21107 0.4 sda5 8k xfs 100000 4 sync - append 8M
21315 0.4 sda5 32k ext3 100000 4 async - rewrite 8M
21775 0.4 sda5 8k ext2 100000 4 async - append 8M
22018 0.4 sda5 8k xfs 100000 4 async - rewrite 8M
22379 0.4 sda5 8k xfs 100000 4 sync - rewrite 8M
22493 0.4 sda5 32k reiserfs 100000 4 async - rewrite 8M
22494 0.4 sda5 32k ext2 100000 4 async - append 8M
22508 0.4 sda5 8k xfs 100000 4 async - append 8M
22867 0.4 sda5 32k xfs 100000 4 async - append 8M
23104 0.4 sda5 8k ext3 100000 4 async - rewrite 8M
23125 0.4 sda5 8k ext2 100000 4 async - rewrite 8M
23522 0.3 sda5 32k xfs 100000 4 async - rewrite 8M
23527 0.3 sda5 32k ext2 100000 4 async - rewrite 8M
23659 0.3 sda5 32k xfs 100000 4 sync - append 8M
24363 0.3 sda5 32k xfs 100000 4 sync - rewrite 8M
24502 0.3 sda5 8k reiserfs 100000 4 async - rewrite 8M
More information about the gnhlug-discuss
mailing list