mismatch_cnt != 0, member content mismatch, but md says the mirror is good

[Benjamin Scott]

On Mon, Feb 22, 2010 at 5:51 PM, Bill McGonigle <bill at bfccomputing.com> wrote:
> Did you ever run a 'repair' on the /boot mirror?

  No.  I've been avoiding that until I have a better understanding of
what's going on.  Heck, I've been avoiding *rebooting* until I have a
better understanding of what's going on.  :)  (Thank goodness this
isn't a Windows box!  ;-)  )

liberty$ uptime
 18:08:11 up 364 days, 18:13,  1 user,  load average: 0.22, 0.08, 0.06
liberty$

  Hmmm.  And now that I see that, I can't reboot before tomorrow.  ;-)

> 1) what's different about the two stage2's?

liberty$ sudo umount /boot
liberty$ sudo mdadm --stop /dev/md0
liberty$ sudo mount -t ext2 -o ro /dev/sda1 /mnt/sda1
liberty$ sudo mount -t ext2 -o ro /dev/sdb1 /mnt/sdb1
liberty$ diff -u <( od -c /mnt/sda1/grub/stage2 ) <( od -c
/mnt/sdb1/grub/stage2 )
--- /dev/fd/63  2010-02-22 18:05:49.290275334 -0500
+++ /dev/fd/62  2010-02-22 18:05:49.292275237 -0500
@@ -22,8 +22,8 @@
 *
 0000760   [ 360  \r  \0 265  \0  \0  \v   B 360  \r  \0 027  \0      \b
 0001000 352   p 202  \0  \0  \0 003 002 377 377  \0  \0  \0  \0  \0  \0
-0001020  \0  \0   0   .   9   7  \0   (   h   d   0   ,   0   )   /   g
-0001040   r   u   b   /   g   r   u   b   .   c   o   n   f  \0  \0  \0
+0001020  \0  \0   0   .   9   7  \0   /   g   r   u   b   /   g   r   u
+0001040   b   .   c   o   n   f  \0   b   .   c   o   n   f  \0  \0  \0
 0001060  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
 *
 0001160 372   1 300 216 330 216 320 216 300   g   f 211   -   $ 220  \0
liberty$ sudo umount /mnt/sda1 /mnt/sdb1
liberty$ sudo mdadm --assemble /dev/md0
mdadm: /dev/md0 has been started with 2 drives.
liberty$ sudo mount /boot
liberty$

> 2) should they be different?

  Well, I think they should be different, based on how GRUB was
installed (individually, on each hard disk, booted from floppy).  One
should try to boot from the first BIOS fixed disk, the other should
try to boot from the second BIOS fixed disk.

  However, looking at the difference above, I think they're different
*in the wrong way*.  It looks like one of the disks specifies
"(hd0,0)/grub/grub.conf" while the other just specifies
"/grub/grub.conf".  That doesn't seem right.

  Although I *did* test booting from either disk before deploying this
system.  Although again... there was a lot going on though at the
time.  Maybe I screwed up my tests, too?  Hmmm.

> 2a) if not, does it matter if they are?
> 2b) if so, do /boot's that are in sync have a problem?

  If GRUB was installed from Linux, against the RAID device (/dev/md0
or whatever), then I would think the two members should be identical,
because the RAID driver in the kernel would have written the same
blocks to both members.

-- Ben