Snap 4000 fsck'ing after power hit

[cestes]

Hi All,

I stumbled across this forum after a problem and I've learned a lot just reading.

We've got a problem and a plan of attack and I'd like some advice and/or suggestions. Anything I learn I'll gladly report back.

The Situation: We're moving to a new building. We moved our Snap Server 4000 (bought on eBay with 4*250GB disks configured RAID 5) to the new building so users could access it. The computer room isn't completed (no UPS yet) and the Snap Server was plugged into wall power. Guess what? We took a power hit! The box didn't come back after the power did. It kept giving errors. I called Adaptec and paid the one-time incident fee. I learned a lot and their techs were really helpful. We tried lots of things but no luck. It looks like the hardware is fine, but the file system is not reparable by the fsck. Finallly they told me today to call some data recovery places and gave me a list of numbers.

I spoke with one place that started me thinking. The sales person got an engineer on the phone to ask some questions. Finally he says, "Well, I'm thinking I can work this remote. No need to ship it anywhere. What you'd have to do is get an IDE RAID controller like a Promise, hook the drives up to that and put it in a spare PC you've got laying around."

The Plan: So, I start thinking to myself. I can certainly hook everything up as requested. I started reading and see that the Snap is running XFS inside. Gee, so does Knoppix. So, I went on eBay and bought an Adaptec 2400A RAID card that does IDE RAID 5. I'll slap the card and the 4 drives into a PC and boot Knoppix. Next I should be able to do 'xfs_repair' and I'm home free, right?

OK, maybe not so simple. How do I hook the drives up? What order? Master vs. Slave? Etc. Do I tell the controller that the 4 drives are RAID5 or will it figure it out? Will Knoppix see it right away? Etc.? Finally, will xfs_repair do the job?

Basically, has anyone pulled their RAID5 disks out, repaired them, and put them back again?

Any advice, suggestions, etc. welcome.

Thanks and I hope I'll have some good news early next week!

-Chris-

[re3dyb0y]

Is possible, but not probable

Knoppix will see the drives, but it wont let you read them - ie the files off it (well, no one has so far)

At best youu will be able to make an image, but doesnt mean you can get the datas

I think i've got it on the wiki, but, there is a 4000 service manual about which reference the drive id's to their positions

[rpmurray]

"I think i've got it on the wiki, but, there is a 4000 service manual about which reference the drive id's to their positions"

If you can find that in the wiki, you've got sharper eyes than I do. I still haven't found anything that shows which drives are which in a 4000.

[blue68f100]

Your best bet in repairing the unit is.????
If data must be recovered. BACKUP ALL DRIVES USING DD. It can make a image of the drives, for backup only. You can not read it. If you do image the drives with dd. You will be looking at 24 hrs +. To make a raw copy of the drives.

Do you have a copy of SpinRite from GRC.com. It works with most any drive and OS, XFS is not listed, though, raids are. It is the best option if you are going to need data recovery, with out sending in to a recovery service, which cost $$$$$.

If you hook the drives up in a attempt to read the drives you will proably loose all data. It will not reconize the File System. If you want to try use FreeBSD it is what the snap OS is build off of.

Only a 4000 will be able to read the data from a raid 5. Install into another 4000. But with the boot damged ????? Don't know.....

I would try to determine if you have a hardware failure or Drive/os failure.

Since the boot drive and Flashram (?) are damage. I would mark all drives as for position. I would then install a new drive in to position 1 (first). See if the unit formats the drive ands loads the OS. If so you will be able to recover. If not it gets more difficult.

If you are able to boot with 1 drive. The unit should have held the raid configuration. If so, see if you can set drive 1 up as a spare. Install the remaining drives. With any luck. It should comeup and start resync /rebuild the raid 5 set.

If Not.

Install a drive with the boot image file allready pre-loaded on the the drive. This will test the hardware. If it fails to boot. You have a hardware problem. Either the MB, Ram, or PS is bad. Get you multimeter out and check the PS. These are limited on power. And 250gig drives seam to be close the the max it can handle.

This should be a starting point.

If you do image the drives with dd. You will be looking at 24 hrs +. To make a raw copy of the drives.

[re3dyb0y]

Originally Posted by rpmurray

"I think i've got it on the wiki, but, there is a 4000 service manual about which reference the drive id's to their positions"

If you can find that in the wiki, you've got sharper eyes than I do. I still haven't found anything that shows which drives are which in a 4000.

Lol, yeah, i thought it was....

Well, i had it my files, so have uploaded it for you http://rapidshare.de/files/19065196/...guide.pdf.html



Cheers


Sam

[cestes]

Thanks to all for the info...

To be clear, it's not a hardware problem... everything appears to power up fine. Its not a raid problem (AFIK). It's not a boot/OS problem, since the thing boots up and I can talk to it. It's a file system (XFS) problem that I'm having.

The machine comes up and starts doing a fsck on device 60000, which is my RAID 5 device and that's where it runs into trouble.

I'll take the advice to make copies of everything before I start messing around with it!

Question: Do I need to know the chunk size, parity scheme, etc. that defines the RAID 5 if I'm going to try and read it, or will my controller be able to figure that out?

Thanks again!

-Chris-

[blue68f100]

You are not going to be able to read it without another snap. Due to the XFS file system. Moving it to another non snap will be detramental. The XFS is unique to the snaps, only.

Does the error log indicates which drive is having a problem?

Do you have a copy of SpinRite? It is the best disk recovery program on the market.

[cestes]

Originally Posted by blue68f100

The XFS is unique to the snaps, only.

I'm not sure about that at all. XFS was developed by SGI and is used on a lot of systems:

http://en.wikipedia.org/wiki/Xfs

One of the engineers I spoke to at one of the big data recovery shops basically spelled out the approach. I've got to figure that he's somewhat competent as he was getting ready to charge me between $7,500 and $15,000 for the recovery job!

I'm going to pick up some identical drives and images each drive and play with those to avoid breaking my real ones.

Thanks again for all your input!

-Chris-

[blue68f100]

Older versions of XFS suffered from out-of-order write hazards, which can result in problems such as files being appended to during a crash gaining a tail of garbage on the next mount.

Snap has been using this/modified since 98. So this may explain the error. When snap went from v1 to 2, it required a hd reinit as part of the process. So they are using a Modified XFS.

While you are at it. SpinRite supports raid drives. It's a $90 program. It may be worth the try. It's written in assembly language, and run from a dos (x86). It is well know for its recovery process. It's a lot cheaper than $15k.

DD on 250gig drive will take about ??????
dd defaults to 512k bytes, for it raw copy. If I recall you can change it to 1024K and it will be faster.

Have you looked closely at the Error Log? Generally it tells you which drive.

[re3dyb0y]

You can change it to 1024K

If they are gonna charge that much, buying a new unit would work probably better, and then allow you to have 2 units!

[re3dyb0y]

I know on Mandrake/Mandrivia that also does support XFS/it is in the list of hard drives you could mount

I just wasn't able to get it to install at the time

XFS has been merged into the mainline Linux 2.4 (as of 2.4.25, when Marcelo Tosatti judged it stable enough) and 2.6 kernels, making it almost universally available on Linux systems. Installation programs for the SuSE, Gentoo, Mandriva, Slackware, Ubuntu and Debian Linux distributions all offer XFS as a choice of filesystem. FreeBSD gained read-only support for XFS in December 2005.

Seems most of the newer OS's also include it.....