2200 Panic / GPF

[duanem]

I have a 2200 that I am getting a General Protection Fault on.

I have tried a number of things to repair, but I cannot determine the cause or how to get around it. I reset to factory defaults, which most posts I read indicate will help to resolve a GPF. I realize the mirror, through diagnostic efforts, is now broken, but the errors I was getting on the panic were the same before the broken mirror.

Essentially the server boots up and runs through its preliminary startup, does a fsck, and as soon as that completes, it goes offline and is unreachable.

The log from the latest boot up until the time the server halts.

The killer is I just need the data and I can see from the bootup that the data is still on the disk based on the number of files and volumes.

Any help would be greatly appreciated. The results from the panic / GPF are also included although it is from a boot earlier than the log, the results from both remain consistent.

04/30/2009 20:08:48 Command: info log temp

Requested: Log=Temporary, Boot=N/A Direction=Fwd Types=All Source=All

04/30/2009 19:48:21 4 S SYS | System Initialization : Server v4.0.860
Build Date: Mar 2 2005 17:50:55
Boot Count: 4
04/30/2009 19:48:21 4 D SYS | Executable built by KEVIN
04/30/2009 19:48:21 4 D SYS | Hardware platform:2.0.4 Model:0 (64 MBytes) S/N:825503
04/30/2009 19:48:21 4 D SYS | ETH: Reset- Eaddr set to 00 C0 B6 0C 98 9F
04/30/2009 19:48:21 4 D SYS | Update IP...
04/30/2009 19:48:21 4 D SYS | Initial file system BIO cache size is 6709632 bytes, 808 buffers
04/30/2009 19:48:21 4 D SYS | DISK: Initial ARBs: 808 Memory: 132512
04/30/2009 19:48:21 4 D SYS | Code Page set to 437
04/30/2009 19:48:21 4 D SYS | QDL System is ENABLED
04/30/2009 19:48:21 4 D SYS | Disk cache flush enabled
04/30/2009 19:49:16 4 D SYS | ide_BaseDriverEntry: Ready Error at start.
04/30/2009 19:49:16 4 D SYS | ide_BaseDriverEntry: Ready Failed
04/30/2009 19:49:16 4 D SYS | SNMP agent enabled for ReadWrite.
04/30/2009 19:49:16 4 D SYS | SNTP disabled.
04/30/2009 19:49:16 4 D SYS | AFP: started
04/30/2009 19:49:16 4 D SYS | 'FTPD' started.
04/30/2009 19:49:16 4 D SYS | Update WorkGroup...
04/30/2009 19:49:16 4 D SYS | Intf: 0, dev: 0: Model: Maxtor 6L160P0
04/30/2009 19:49:16 4 D SYS | Firmware Rev: BAH41G10 Serial #: L30MHSMG
04/30/2009 19:49:16 4 D SYS | Intf: 1, dev: 0: Failed to find drive.
04/30/2009 19:49:17 4 E L01 | File System : Logical set member 0 not found. Original device ID: 50000
04/30/2009 19:49:17 4 W L01 | File System : 1 member(s) missing in logical device (original ID: 50000)
04/30/2009 19:49:17 4 D SYS | MirrorInitSet: device 0 unavailable to initialize set
04/30/2009 19:49:17 4 I L00 | File System Check : Executing fsck /dev/ride0g /fix /fixfatal
04/30/2009 19:49:17 4 I L00 | File System Check : partition is clean.
04/30/2009 19:49:17 4 D SYS | Dev 10006 (/priv) NO QDL validation required.
04/30/2009 19:49:17 4 D SYS | Update FDB 0x10006...
04/30/2009 19:49:17 4 I L00 | File System : Opened FDB for device 0x10006
04/30/2009 19:49:17 4 D SYS | Scheduled ACL Set and Propagate at /priv/os_private for FDB_ID_12
04/30/2009 19:49:17 4 I L01 | File System Check : Executing fsck /dev/rmirr0 /fix
04/30/2009 19:49:17 4 D SYS | Propagate on /priv/os_private: Success - 8 files, 0 dirs; Errors - 0 files, 0 dirs
04/30/2009 19:49:17 4 W L01 | File System Check : partition is NOT clean.
04/30/2009 19:49:17 4 D SYS | Fsck - Using primary superblock
04/30/2009 19:49:17 4 D SYS | 39932176 bytes pre-allocated
04/30/2009 19:49:17 4 D SYS | Memory allocation for i-node cache: 90% of free RAM
04/30/2009 19:49:17 4 D L01 | File System Check : Failed to allocate 29847554 bytes for rcd_lncntp!!!
04/30/2009 19:49:17 4 D SYS | -- Swap-based Fsck --
04/30/2009 19:49:17 4 D SYS | 7615 i-node cache blocks, cache hash table: 3067 entries
04/30/2009 19:49:17 4 D SYS | 256 i-nodes per generic cache block
04/30/2009 19:49:17 4 D SYS | 170 i-nodes per directory cache block
04/30/2009 19:49:17 4 I L01 | File System Check : ** Phase 1 - Check blocks and sizes
04/30/2009 19:49:24 4 D SYS | AFP: Registering "SNAP825503". Net 65280, Node 159, Socket 40
04/30/2009 19:49:28 4 D SYS | AFP: Allocated 63 volumes, 14336 files, 256 users
04/30/2009 19:49:28 4 D SYS | AFP: initialization complete
04/30/2009 19:49:36 4 I NET | DHCP: T1 length was 0 is now 129600
04/30/2009 19:49:36 4 I NET | DHCP: T2 length was 0 is now 194400
04/30/2009 19:49:36 4 I NET | DHCP/BOOTP: Setting IP address to 192.168.0.3
04/30/2009 19:49:36 4 D SYS | Update IP...
04/30/2009 19:49:36 4 D SYS | BOOTP: DNS = 100A8C0
04/30/2009 19:51:29 4 D SMB | SMB : Becoming master browser for WORKGROUP
04/30/2009 19:51:29 4 D SMB | SMB : Becoming master browser for WORKGROUP
04/30/2009 20:03:51 4 I L01 | File System Check : ** Phase 1b - Rescan for more duplicate blocks
04/30/2009 20:03:51 4 I L01 | File System Check : ** Phase 2 - Check pathnames
04/30/2009 20:04:02 4 I L01 | File System Check : ** Phase 3 - Check connectivity
04/30/2009 20:04:02 4 I L01 | File System Check : ** Phase 4 - Check reference counts
04/30/2009 20:04:06 4 I L01 | File System Check : ** Phase 4b - Check backlinks
04/30/2009 20:04:11 4 I L01 | File System Check : ** Phase 5 - Check cylinder groups
04/30/2009 20:08:43 4 W L01 | File System Check : Modified flag set in superblock (Fixed)
04/30/2009 20:08:43 4 W L01 | File System Check : Clean flag not set in superblock (Fixed)
04/30/2009 20:08:43 4 D SYS | 39933340 bytes used during fsck()
04/30/2009 20:08:43 4 I L01 | File System Check : 21437 files, 863182 used, 18762561 free (0 frags, 18762561 blocks, 0.0%% fragmentation)
04/30/2009 20:08:43 4 D SYS | Elapsed time: 1166 s.
04/30/2009 20:08:43 4 D SYS | Fsck cache statistics:
04/30/2009 20:08:43 4 D SYS | total memory used for cache: 33282004 bytes
04/30/2009 20:08:43 4 D SYS | total number of directories: 1311
04/30/2009 20:08:43 4 D SYS | maximum depth of recursion in sorting: 0
04/30/2009 20:08:43 4 D SYS | number of swaps in sorting phase: 98
04/30/2009 20:08:43 4 D SYS | ----- generic i-node cache -----
04/30/2009 20:08:43 4 D SYS | cache entries: 10153
04/30/2009 20:08:43 4 D SYS | cache hits: 34009415 (99%)
04/30/2009 20:08:43 4 D SYS | cache misses: 116592 (0%)
04/30/2009 20:08:43 4 D SYS | reused cache entries: 108977
04/30/2009 20:08:43 4 D SYS | total reads from swap device: 58296
04/30/2009 20:08:43 4 D SYS | total writes to swap device: 59630
04/30/2009 20:08:43 4 D SYS | total writes skipped (clean blocks): 49347
04/30/2009 20:08:43 4 D SYS | average successful cache lookup: 1.00 iterations
04/30/2009 20:08:43 4 D SYS | maximum successful cache lookup: 1 iterations
04/30/2009 20:08:43 4 D SYS | average unsuccessful cache lookup: 1.00 iterations
04/30/2009 20:08:43 4 D SYS | maximum unsuccessful cache lookup: 1 iterations
04/30/2009 20:08:43 4 D SYS | ----- directory i-node cache -----
04/30/2009 20:08:43 4 D SYS | cache entries: 5077
04/30/2009 20:08:43 4 D SYS | cache hits: 19656 (99%)
04/30/2009 20:08:43 4 D SYS | cache misses: 7 (0%)
04/30/2009 20:08:43 4 D SYS | reused cache entries: 15
04/30/2009 20:08:43 4 D SYS | total reads from swap device: 7
04/30/2009 20:08:43 4 D SYS | total writes to swap device: 15
04/30/2009 20:08:43 4 D SYS | total writes skipped (clean blocks): 0
04/30/2009 20:08:43 4 D SYS | average successful cache lookup: 1.00 iterations
04/30/2009 20:08:43 4 D SYS | maximum successful cache lookup: 1 iterations
04/30/2009 20:08:43 4 D SYS | average unsuccessful cache lookup: 1.00 iterations
04/30/2009 20:08:43 4 D SYS | maximum unsuccessful cache lookup: 1 iterations
04/30/2009 20:08:43 4 I L01 | File System Check : Cleanup completed...
04/30/2009 20:08:44 4 D SYS | Dev 50000 (/0) requires QDL validation(level 0)
04/30/2009 20:08:44 4 D SYS | QDLScanPath record count mismatch for dir /0/os_private. Expected 34, found 38.
04/30/2009 20:08:44 4 D SYS |
QDLScanPath /0/os_private: found damaged database. Database will be reconstructed

eventLogGetRecord() printed 95 records at 04/30/2009 20:08:49



F Call Stack :
$0011B912
$002DF0FE
$00227FAD
$00222020
$002117B1
$002024AF
$00202371
$001277C0
$001A4869
$001A6F63
$001A2CAA
$0010CC67
$0010413C
$001041C3
================================
System 4/29/2009 9:20:00 AM
F PANIC : General Protection Fault (#13) at $001FFEF0
EAX=646F6F66 EBX=00FA3430 ECX=004AD58C EDX=646F6F66
ESP=004AD518 EBP=004AD530 ESI=646F6F66 EDI=00000000 System 4/29/2009 9:20:00 AM

[blue68f100]

Drive 1 (dev 0) has failed by the address. If you disconnect drive 1 it should come on up. But it need to rebuild QDLScanpath.

While you have it open confirm that the cooling fan is working.

These Stack calls can some times be fatal. But since yours happen with a HD failure I suspect not.

You may need to disconnect the failed HD, and swap positions for the unit to come up. The SnapOS has problems sometime booting off drive 2.

There is a user called Snap-tech that does recovery work. I do not believe that your that far gone yet.

[duanem]

Unfortunately, I have tried it without the other drive, and tried it with the working drive in both slots. Still doesn't come up. Haven't checked the fan however.

The panics were happening before the other drive started having issues.

The drive problems and broken mirror came about after the server locked up.

Once the server stops responding, with a fast-flashing "system" light, the only way to get it back online is to power it down and start it up again.

I will see if I can get a cleaner, more accurate log. That one was a few days old, but is the only time I had the QDL statements in there before it stops functioning.

From what I can determine from the logs, it appears the server is booting up, doing a fsck on the mirror, completing the fsck successfully, mounting the mirror, starting qdl, then failing. From the mount to the failure, is only about 30 seconds or less and it would be nearly impossible to execute any other commands.

[blue68f100]

Have you tried a full system reset? Doing so will require Assist.

Spinrite by GRC can do some repairs, and had worked for some.


If you must have the data contact snap-tech, he is cheaper than any one else when it comes to this.