SpinRite (was: FYI)

[Mike Bilow]

On 2013-04-28 11:19 ET, Ben Scott wrote:
> On Sun, Apr 28, 2013 at 12:27 AM, Bill Ricker <bill.n1vux at gmail.com> wrote:
>> If the disk is failing, perhaps what it needs in SpinRight to recover the
>> iffy blocks. Not Free, not Open, but good stuff and not expensive.
>    Oh boy.  This is going to get into religious territory.

Conceded. I should disclaim that I am the author of the BruteForce(TM) 
Hard Drive Utilities from the same era.


>    I am of the opinion that while SpinRite is not a total scam, it's
> highly overrated, mostly obsolete, and all of it's useful
> functionality is now available in free programs elsewhere.  SpinRite
> *may* have had some relevance back in the days of MFM, when hard
> drives were powered by steam and people still thought MS-DOS was a
> good idea, but it's not worth paying for these days.  And some of the
> claims made by the author are bunk and always have been.

SpinRite was originally designed for one primary purpose: it was the 
only software that was capable of turning off ECC when test-reading 
older technology drives. Admittedly this was of value only a very long 
time ago in the MFM/RLL era and was quickly rendered much less useful as 
the amount of abstraction increased with the ATA interface. It was still 
possible to request that ECC be disabled with ATA and SCSI, and 
presumably this could give a decent indication of the health of the 
media in ways that were substantially hidden in ordinary operation, but 
eventually there were standardized methods built into interface 
technology that provided that kind of information obtained from logging 
in normal operation, such as SMART for ATA.

Steve Gibson's theory behind SpinRite was that the principal cause of 
partial (as opposed to catastrophic) data loss was dimensional 
instability, most of which resulted from thermal migration beyond the 
tiny mechanical tolerances of cylinder position, and this was an 
expected consequence of normal aging. Therefore, he reasoned, he could 
test-read every sector on the media with ECC disabled, and if it was bad 
then he could re-enable ECC to recover the data and rewrite it more 
emphatically, thereby compensating for the dimensional changes inside 
the drive. A significant piece of evidence in support of his theory was 
the information-theoretic phenomenon that degradation occurred faster 
with RLL encoding despite its being physically identical to MFM 
encoding, implying a tighter tolerance.

Although I believe Gibson was correct, as a practical matter SpinRite 
ceased to be really useful once similar functionality was built into 
intelligent drive controllers necessary for zone-bit recording -- which 
was sometime around when drive sizes started to exceed 32MB. The last 
drive that I had in operation where SpinRite was truly valuable was a 
5.25-inch full-height 120MB Maxtor than had been formatted out to 180MB 
using RLL; in those days, Maxtor was a very high-end, extremely 
well-engineered independent brand rather than the bargain subsidiary of 
Seagate that it is today. That Maxtor drive had triple-redundant spindle 
sensors so that despite becoming noisy it continued in operation with no 
data loss when one failed, and that was why it was retired from service 
in operating condition.


>    SpinRite will read every block on the disk, to make sure they still
> can be read.  This is useful.  But even CHKDSK/SCANDISK will do that,
> and have since DOS 6, circa 1993.

As explained, SpinRite went directly to the hardware, which was the only 
way to bypass ECC. By the way, CHKDSK does not by default read every 
sector: the "/R" switch is required to enable that behavior, and it 
certainly could never disable ECC.


>    SpinRite will read-and-rewrite blocks.  There are scenarios where
> this may be a plausible benefit, such as allowing the drive's built-in
> relocation mechanism to relocate a marginal sector.  But "badblocks
> -n" will do the same thing, for free.

SpinRite was not looking for bad blocks, which are easy enough to find, 
but for "gray area" blocks that were good enough to be readable with ECC 
enabled but not good enough to be readable with ECC disabled. It was the 
first implementation of "early warning" technology, such as SMART, that 
is now built into every drive. In the SpinRite era, only SCSI drives had 
sector relocation mechanisms and these usually had to be specifically 
enabled by setting the particular Mode Page that controlled Read-Write 
Error Recovery; SpinRite simply copied the file.


>    SpinRite will read blocks over and over again.  If a read fails, it
> will keep trying until it succeeds, which is useful on a failing but
> not dead drive.  But dd_rhelp will do the same thing, for free.
>
>    To read a bad block, SpinRite will try tricks like seeking to
> adjacent cylinders/heads/sectors and back again, in various
> directions.  This was plausible for ancient drives, but everything
> made in the past 20 years or so has abstracted the real disk geometry
> away from the host, even when presenting "CHS".  So these tricks are
> meaningless on anything that isn't old enough to run for congress.

This is largely true, but even so simple an action as explicitly 
flushing the cache can help. SpinRite was, again, originally intended as 
a preventative maintenance tool and took off into feature creep where it 
was marketed as a recovery tool and began to be regarded as a magic 
panacea. Its underlying theory of operation was certainly plausible and 
in my opinion correct, but it continued to be sold on its reputation 
long after it was no longer useful. Even so, in the modern era you are 
probably better off using something like "smartmontools" to initiate a 
long self-test on the device.rather than manually test-reading every 
block on a regular basis.


>    And, of course, SpinRite is from Steve Gibson, who always talks like
> an infomercial host.  Billy Mays could have taken lessons from Gibson.
>
>    Gibson claims SpinRite "interfaces directly to the hard disk
> system's hardware", which somehow gives it magical abilities.
> Everything I've seen suggests this is an outright lie.  SpinRite
> flat-out won't work if the drive isn't presented using BIOS interrupt
> 0x13.

Gibson has a personality, but he walks the walk as well as talks the 
talk. I've had occasion to ask him very detailed technical questions and 
he knew his stuff. Gibson's claim about interfacing directly to hardware 
with register-level awareness was absolutely true in the days of 
proprietary controllers: keep in mind that companies such as Miniscribe 
placed their API under NDA, which was manifestly stupid. Several drive 
controllers just used off-the-shelf chipsets for which data sheets were 
publicly available, but they still put their API under NDA even though 
it was little more than that of the chipset. Well into the ATA era, all 
controllers continued to have proprietary registers and so forth that 
were often subject to NDA. Eventually these functions were subsumed 
under vendor-independent parts of later revisions of ATA, for which 
Linux and open source were major driving forces, and this nonsense is a 
thing of the past.


>    He claims things like "hardware register level awareness of IDE and
> SCSI drives".  SCSI drives *don't have hardware registers*.  The SCSI
> spec is quite abstract and hides all that stuff. Further, you don't
> talk to a SCSI drive, you talk to a host adapter.  You literally
> *cannot* talk directly to the drive.
>
>    You can, however, request additional sense data and mode pages,
> which provide a wealth of useful information about the drive.  In the
> DOS environment under which SpinRite runes, this is done through the
> ASPI interrupt calls.  It's a useful capability, and I expect it's
> what SpinRite does, but it isn't the Amazing Scientific
> Breakthrough!!!1! Gibson claims it is.  He just Read The Fscking
> Manual and learned how to use ASPI.

It's not that simple. First, SpinRite dates to an era when there were 
actually SCSI 1.0 devices before all of this confusion was cleaned up in 
SCSI 2.0. Second, even in SCSI 2.0, it was common for drives to ship in 
a default configuration that was very stripped down on the assumption 
that SCSI drives were being sold to OEMs who intended them to be 
integrated into very expensive systems. While it was possible for an end 
user to just buy a SCSI drive and plug it into a controller, much of the 
supply of drives consisted of particular models that had been given a 
"personality profile" to work with AIX, HP/UX, Ultrix, or whatever else. 
Drive manufacturers simply had no expectation until much later that some 
random administrator would buy SCSI drives and build a custom server to 
run Novell NetWare or something like that, and there was a need for 
software to fill that niche. As late as the 1990s, Linux had very 
primitive tools that would at least allow manual inspection and 
modification of SCSI Mode Pages, but this was literally on the level of 
bit-twiddling and demanded expertise not only in storage technology but 
hexadecimal arithmetic. This was very exacting and tedious work, hardly 
a matter of looking in a book and filling in blanks, especially since 
writing a bad value to a SCSI Mode Page could turn an expensive drive 
into a brick.


>    I do think SpinRite did things other software wasn't doing, at the
> time and place it was introduced in.  Even something as simple as
> pattern testing wasn't common in the dark ages of DOS.  (Other
> platforms had it, but the IBM-PC was the ghetto of the computer
> world.)  I acknowledge that.  It was valuable at the time, and even
> today, I suppose a nicely-presented, integrated package might still
> have value.

SpinRite used pattern testing specifically to trigger worst-case 
scenarios with the (2,7) RLL encoding that became the de facto standard. 
Technically, the physical drive is identical whether the encoding is 
relatively sophisticated such as (2,7) RLL or relatively unsophisticated 
such as (1,3) RLL (which is a special case known as "MFM"), and Gibson 
realized that the bandwidth requirements of such encoding methods would 
translate directly into tolerance constraints for dimensional stability 
and timing. This was by no means obvious in the 1980s, and it must be 
remembered that SpinRite emerged in an era contemporaneous with music CD 
technology that was driving cutting-edge research into (2,10) RLL and 
the first commercial use of Reed-Solomon ECC.


>    But that doesn't mean Gibson's bullshit doesn't stink.

Yes, SpinRite was misunderstood and overhyped, and it stuck around as a 
magic elixir for far longer than it should have, but 25 years ago it was 
a remarkably effective and prescient utilization of stone knives and 
bear skins.


>> (And it makes possible the Security Now! podcast.)
>    Regardless of the efficacy of SpinRite, Steve Gibson is in way over
> his head when it comes to security.  His habit of being uninformed and
> making stuff up has burned him more then once.

I cannot defend Gibson on security.

-- Mike