DLSLUG Notes 5-April-2007: Todd Underwood on ZFS

[Ted Roche]

Fifteen attendees made it to the April meeting of the Dartmouth Lake
Sunapee Linux User Group, held as usual in Carson Hall at the Dartmouth
College campus on the first Thursday of the month.

Todd Underwood [1], Vice President, Operations and Professional
Services at Renesys presented "ZFS: The Last Word in File Systems."
Renesys is in the business of collecting, analyzing and archiving data
about what's happening on the internet and, not surprisingly, that's a
lot of data and growing geometrically. One of Todd's projects is to
provide fast and reliable storage for the hundreds of gigabytes per day
acquired and the tens of terabytes of data stored. He presented a survey
of what's out there, what his needs were, and how he reluctantly
narrowed the search down to SUN Solaris and ZFS. While he had nothing
but praise for ZFS, he expressed some reservations about SUN. Strong
reservations.

Todd dug into the ZFS architecture. ZFS is truly amazing: disk contents
are always coherent, with writes all checksummed, writes as atomic
transactions, with "fancy FS internals" like IO scheduling, dynamic
block sizes and prefetch queues, huge limits (128-bit data). ZFS
flattens the Linux file system model of multiple layers into a single
monolith, and eliminates entire classes of problems introduced by the
multiple layer architecture. There is no fsck. Devices are accumulated
into pools. File systems are assigned to pools. Storage addition and
maintenance is fairly trivial. ZFS performance is remarkable, at disk
speed (with compression, sometimes in excess of disk speed).

Todd presented the rough outlines of his storage system: $17k worth of
hardware from SUN and Dell yields 7.5 terabytes of storage with nearly a
gigabyte per second throughput. Todd's disk storage challenge is solved,
as the company's demand can't match the throughput capacity of his
system, for now, and the system can be expanded with additional external
storage.

As for the porting of ZFS onto other architectures, Todd expressed the
opinion that running on Solaris or OpenSolaris is likely the best
current solutions. Porting onto BSD is underway but not yet ready. He
had heard that ZFS was ported to OS X but could not confirm (Googling
ZFS OSX yields interesting results).

Todd promises to send along slides from his presentation; I'll try to
post links to them to the website.

After the main presentation, there was a good session of questions and
answers. I asked a question on replication of Postgres and Todd
recommended the Sequoia product.

Next meeting, still tentative, will be on writing a FireFox extension by
 Roger Trussell.

Thanks to Todd for his thorough presentation, to Bill McGonigle for
organizing the meeting, to Bill Sconce for sharing his notes with me for
this message, and to all for attending and participating.

[1] http://www.renesys.com/about/management.shtml#a-todd

-- 
Ted Roche
Ted Roche & Associates, LLC
http://www.tedroche.com