Linux vs. Solaris file IO performance

[Michael O'Donnell]

>Does anybody know what settings could be tweaked
>on a Solaris box so that it would do file IO
>more efficiently like a linux box?


It happens that I am currently involved in
some fairly intensive Linux filesystem testing.
It's one of those situations where I decided we
needed to get some "quick numbers" describing
filesystem performance and sat down to write a
cheap'n'nasty test script.  The numbers turned out
to be more interesting than we expected, though,
and people kept saying, "Hey, that's cool, but I
wonder what happens when you vary this here other
thingy?"  and before long my script was running
almost 200 different tests.  And there's still
plenty of attributes that I haven't instrumented...

It's been an eye-opening experience.  One tends
to forget how many things can be tweaked, and some
of them can make surprisingly large differences
in I/O throughput from an app's point of view;
I can't believe the same isn't true for Slolaris.

I've got several different kinds of boxes rigged
up to test ext2, ext3, reiserfs and xfs over
several kinds of storage.  I'm varying attributes
like filesystem blocksize, filesystem type,
I/O transfer size, file append versus rewrite,
writes with and without OSYNC, mounting with sync
versus async option, etc, etc...

I was interested to see that all four filesystem
types appear near both the top and bottom of the
results (some of my best numbers are over 300
times better than my worst ones) and I'd have
to say that there's no single "best" filesystem,
at least according to my preliminary results.

I guess if I wanted to make an "average"
Linux box's disk I/O feel really snappy from an
interactive user's perspective (ignoring various
issues that some people [like me] don't have the
luxury of ignoring) I'd make sure it had lots of
RAM, fast disks (enable the on-disk write caches if
SCSI) and mount my filesystems with async option.
And I'd probably use ReiserFS or XFS.

And, of course, it's possible that Solaris (or
Sun's HW) have fundamental design differences that
will keep them from ever going as fast as Linux
on a PeeCee...

 .