Modify SCSI Backplane for SATA Drives

[Harbinger]

I just built a server using an older Intel motherboard and SCSI-based server case. I modified the SCSI backplane so that I could still use the case's drive bays but with SATA drives instead. I blogged about it, here's the link:

http://vvenesect.wordpress.com/2011/...e-sata-drives/

Since it's been a little slow lately I thought this might stir some conversation. Enjoy!

[Phoenix32]

I think some people around here might get offended at your comment about nobody here having "the Chops" to do the conversion on the 4100. Plenty of people here, or at least during that time frame, had/have the chops to do it.

I was a late-comer to that particular subject, but was involved in the late ends of the conversations about it. It was not about "the chops", but rather about being worth the tremendous effort that would have to go into it.

Given the slow ethernet speed of the unit and limitatios of the OS, it just wasn't worth the hard work that would have to go into making it work with larger drives than it could already do given the Snap 4400, 14000, 4200, 4500 etc that were already available.

As a side note, YOU WERE WRONG IN WHAT YOU SAID IN THAT BLOG. It was not about a driver in the SNAP OS, it was in the firmware/BIOS. That is not an easy mod and was just not worth doing.

[Phoenix32]

While my last message was coming from a bit irritated point of view (slamming the great people that are/were on this forum), this message is a genuine question. I do not mean to question you personaly or piss anyone off, but I do have a question I have to ask to make this make sense to me and maybe others.

Why? Beyond just the fun of doing some hardware hacking?

Why not a Guardian OS unit and an expansion like a Snap Disk 10 if you want more than 4 drives, or a Snap Disk 30 or S50 if you want more than 8 (or 12 if you use two Snap Disk 10s)?

If you you have to have SATA, then doing the SATA mod is easy enough, or there is always the 410, 520, and the Snap Disk 30 and S50 are already SATA. These units can be purchased fairly inexpensive these days.

Like I aid, I am NOT trying to offend you, I swear. Maybe I am just dense, but I somewhere missed the whole point of it all. The snaps I just mentioned would do everything you stated you wanted, except maybe to not use GOS. But, since GOS is by far the best file server OS going, it is hard to beat. I am just confused a little bit, please explain.

[Harbinger]

Originally Posted by Phoenix32

I think some people around here might get offended at your comment about nobody here having "the Chops" to do the conversion on the 4100. Plenty of people here, or at least during that time frame, had/have the chops to do it.

I was a late-comer to that particular subject, but was involved in the late ends of the conversations about it. It was not about "the chops", but rather about being worth the tremendous effort that would have to go into it.

No offense meant. It just seemed to me, in the many discussions about what would be necessary to do, that what was needed in a modification was clear. Several people dug into it, as I recall, but gave up because they ran into a brick wall either in their understanding or capabilities. No one stepped up afterward with greater abilities to make the mod(s) with that information and thus my comment.

Originally Posted by Phoenix32

Given the slow ethernet speed of the unit and limitatios of the OS, it just wasn't worth the hard work that would have to go into making it work with larger drives than it could already do given the Snap 4400, 14000, 4200, 4500 etc that were already available.

Perhaps so, in your (and others) opinions. Those of us already with 4100s didn't necessarily want to go and acquire one of those other units; we wanted to upgrade the units we already had even with their other limitations. Sure, it would take a while to transfer files and load them up, but they'd make dandy music servers if they just had more space. Part of the challenge at times is doing something just because you can (or wonder if you can pull it off in the first place).

Originally Posted by Phoenix32

As a side note, YOU WERE WRONG IN WHAT YOU SAID IN THAT BLOG. It was not about a driver in the SNAP OS, it was in the firmware/BIOS. That is not an easy mod and was just not worth doing.

Point taken, thanks for the clarification. I'll go back and make an update/correction to the post.

[Harbinger]

Originally Posted by Phoenix32

While my last message was coming from a bit irritated point of view (slamming the great people that are/were on this forum), this message is a genuine question. I do not mean to question you personaly or piss anyone off, but I do have a question I have to ask to make this make sense to me and maybe others.

Why? Beyond just the fun of doing some hardware hacking?

Why not a Guardian OS unit and an expansion like a Snap Disk 10 if you want more than 4 drives, or a Snap Disk 30 or S50 if you want more than 8 (or 12 if you use two Snap Disk 10s)?

If you you have to have SATA, then doing the SATA mod is easy enough, or there is always the 410, 520, and the Snap Disk 30 and S50 are already SATA. These units can be purchased fairly inexpensive these days.

Like I aid, I am NOT trying to offend you, I swear. Maybe I am just dense, but I somewhere missed the whole point of it all. The snaps I just mentioned would do everything you stated you wanted, except maybe to not use GOS. But, since GOS is by far the best file server OS going, it is hard to beat. I am just confused a little bit, please explain.

Part of the reason was price. I knew I could pick up some parts inexpensively (and re-use others I already had) to build a home server. Part of it was having something I'd built and understood better than an off-the-shelf box that used proprietary assemblies. I also wanted to use an open-source software solution that didn't require costly upgrades. I'd had a chance (I thought) to acquire an 18000 around the beginning of the project but that fell through; now I read that the newest GOS won't work because the hardware is too "old." There's probably no reason the software wouldn't work, they've probably just left device support out of the OS build. With an open-source package those things can be separately added back if they're removed.

THe last part is simply to show people what's possible if they take an old, "outdated" unit and perform some old-fashioned hacking work to extend its life. These things are showing up in surplus stores now, cheap, just because they use an outmoded storage technology. Why scrap them if you can still use them?

No offense taken. For you such an undertaking might not be interesting, given your experience with the Snap platform already. Nothing wrong with that. But others who are starting out might find the project challenging on an intellectual and capabilities-level. I know I did.

[Phoenix32]

Originally Posted by Harbinger

Perhaps so, in your (and others) opinions. Those of us already with 4100s didn't necessarily want to go and acquire one of those other units; we wanted to upgrade the units we already had even with their other limitations. Sure, it would take a while to transfer files and load them up, but they'd make dandy music servers if they just had more space. Part of the challenge at times is doing something just because you can (or wonder if you can pull it off in the first place).

I guess it comes down to the chain of thinking. Not that one is right and the other is wrong, just different. IMO, and probably by most that would have the chops to complete that type of undertaking, it comes from a mindset with backups in mind as well as the data itself. If you are or get deep into these kinds of things, sooner or later, if not already, you will get bitten by the ever dreaded, "I didn't back up my data and now it is all gone" syndrome.

With there being a limitation of 100baseT, much over the 512GB limit woud take forever and a day or two to back up. And then with a further limitation of not being able to go over 1TB ever due to the base OS being used for Snap OS, it just kind of made it a wash for most of the guys I had talked to who could have done it. It would have been damn hard and time consuming and just didn't seem worth it is all. If it had been easy, it probably would have gotten done just for fun, but...

Again, not that one is right and the other wrong, just a different mindset by most of the people who probably could have done it is all.

[Phoenix32]

Originally Posted by Harbinger

Part of the reason was price. I knew I could pick up some parts inexpensively (and re-use others I already had) to build a home server.

That is a part I don't agree with, but to each their own. I think the GOS units can be had pretty cheap if you look.

Originally Posted by Harbinger

- Part of it was having something I'd built and understood better than an off-the-shelf box that used proprietary assemblies.

- I also wanted to use an open-source software solution that didn't require costly upgrades.

- THe last part is simply to show people what's possible if they take an old, "outdated" unit and perform some old-fashioned hacking work to extend its life.

Now those are all points I can see and agree with (I do it for fun etc. too sometimes) and it makes more sense for the post and blog now, thank you.

Originally Posted by Harbinger

No offense taken. For you such an undertaking might not be interesting, given your experience with the Snap platform already. Nothing wrong with that. But others who are starting out might find the project challenging on an intellectual and capabilities-level. I know I did.

Thank you, at least someone around here can take things the way they are meant and not get all B-hurt if someone does not agree with them. THANK YOU!

[Terry Kennedy]

Originally Posted by Phoenix32

I think some people around here might get offended at your comment about nobody here having "the Chops" to do the conversion on the 4100. Plenty of people here, or at least during that time frame, had/have the chops to do it.

Indeed. I had been independently upgrading 4100's to 120GB drives for over a year before I discovered this site. I also had/have a few other tricks up my sleeve related to Snap Extensions which have never been documented on this site.

As a side note, YOU WERE WRONG IN WHAT YOU SAID IN THAT BLOG. It was not about a driver in the SNAP OS, it was in the firmware/BIOS. That is not an easy mod and was just not worth doing.

Actually, as far as I know, I have the most detail on what was going on, since I had gotten as far as convincing Snap engineering that it was possible and was working on getting pricing to have the work done...

The reason that the 4100's didn't get LBA48 support at the same time as the rest of the legacy Snap product line was because the Snap engineering staff was under the impression that it was not possible to do it on the particular Promise controller which was (only) used in the 4100. I sent them a set of LBA48 diffs against the FreeBSD driver for the chip in question, and pointed them to the Linux driver, which also did LBA48.

The only BIOS issue is that the operating system (on disk-based OS units like the 1100) needs to be within the first 127GB of the drive as the BIOS doesn't know about LBA48. On flash-based OS units like the 4100, this is irrelevant as the OS is in flash.

What it eventually boiled down to was that there was no amount of money that could convince Snap to provide a feature whose sole purpose was to enable customers to avoid the need to purchase new Snap products.

[Terry Kennedy]

Originally Posted by Phoenix32

With there being a limitation of 100baseT, much over the 512GB limit woud take forever and a day or two to back up. And then with a further limitation of not being able to go over 1TB ever due to the base OS being used for Snap OS, it just kind of made it a wash for most of the guys I had talked to who could have done it. It would have been damn hard and time consuming and just didn't seem worth it is all. If it had been easy, it probably would have gotten done just for fun, but...

There's also the issue of what is going to happen when the server attempts to recover from an unclean shutdown. Even with 512MB (memory upgrade), a 4100 with 4 120GB drives falls back to swap-based fsck, which takes approximately forever. If you disable Java and the Snap Extensions, the fsck just barely fits in RAM. If larger drives were possible, it could take days to check the filesystem.

[Phoenix32]

Well, we must have talked to different Snap Engineers then, LOL (typical of them back then).

What I was able to get out of them was that the controller did in fact support LBA48 (after I proved it them, the asshats), but the BIOS would need upgraded (the actual firmware portion) and then a driver (which you mentioned) inserted into the Snap OS for the 4100 and LBA48. The driver would be an easy fix, but the hard firmware would not be. In fact, they told me some used ROMs instead of EPROMS, which could then accordingly not be updated without a chip change. I stopped pursuing it myself at that point, but do not know what if anything came after that.

[Terry Kennedy]

Originally Posted by Phoenix32

Well, we must have talked to different Snap Engineers then, LOL (typical of them back then).

What I was able to get out of them was that the controller did in fact support LBA48 (after I proved it them, the asshats)

We must have been talking to different people, since I went through the same thing you did, to convince them the controller could do LBA48.

but the BIOS would need upgraded (the actual firmware portion) and then a driver (which you mentioned) inserted into the Snap OS for the 4100 and LBA48. The driver would be an easy fix, but the hard firmware would not be. In fact, they told me some used ROMs instead of EPROMS, which could then accordingly not be updated without a chip change. I stopped pursuing it myself at that point, but do not know what if anything came after that.

I think they were confused. There are callbacks available in the BIOS, but they only work in real mode, not protected mode. Thus, they aren't usable from inside SnapOS. Once SnapOS loads, the only thing the BIOS gets used for is normal reboots and the tail end of panic processing (which, as far as I could tell, never worked right anyway).

I actually spent a couple days with their engineering staff in their office going over all of this, getting an estimate on the amount of time it would take, and so on, before management (theirs, not mine) decided they didn't want to pursue it for business reasons.

[Harbinger]

Thanks for the good information, Phoenix32 and Terry. That information was scatted through a number of posts and a couple of threads, and I thought I'd remembered things correctly. I've made a correction to the blog with appropriate credits; please check it and verify that I got it right.

At any rate, now I've got an otherwise good 4100 that I'm not sure what I'll do with...I could put one 1TB disk on my Linksys NSLU2, outstrip the 4100's capacity, and still end up with that music server I'd originally wanted.

[Terry Kennedy]

Originally Posted by Harbinger

Thanks for the good information, Phoenix32 and Terry. That information was scatted through a number of posts and a couple of threads, and I thought I'd remembered things correctly. I've made a correction to the blog with appropriate credits; please check it and verify that I got it right.

Above the "UPDATE" section, in your original text, you've got MB instead of GB. In the "UPDATE" section, you might want to clarify that the "They" in "They did not believe" refers to the Snap engineers, not Phoenix32 or myself.

I don't know that the piece in your update about the flash memory being near maximum capacity is correct. Most of the LBA48 stuff is already in the OS image the 4100 runs - the missing piece is the device driver changes, which are quite small. I doubt the image would brow by more than 256 bytes.

At any rate, now I've got an otherwise good 4100 that I'm not sure what I'll do with...I could put one 1TB disk on my Linksys NSLU2, outstrip the 4100's capacity, and still end up with that music server I'd originally wanted.

Yup. 4100's were (mostly*) great for their time, but that was a long time ago. These days, you can beat a 4100's performance with a portable USB HDD plugged into the USB port of the average home router.

* Mostly - there is a glaring oversight in the 4100, as well as the other Snap models from that era - there is no parity protection in memory, so there is a chance of silent data corruption.

- 6 RU, 2.8TB raw, 2.1TB usable, 50Mbyte/sec

- 6 RU, 64TB raw, 48TB usable, 1000Mbyte/sec

[Harbinger]

Gaaaah! Fixed. Thanks, Terry!

[Terry Kennedy]

Originally Posted by Harbinger

Gaaaah! Fixed. Thanks, Terry!

You missed one - "over 360MB of RAID-5 storage".

[Harbinger]

Originally Posted by Terry Kennedy

You missed one - "over 360MB of RAID-5 storage".

You know, there are some days... Thanks--got that one too.

[ben333]

I don't have much SNAP knowledge, but I greatly admire what you did to that backplane! The idea (exactly as you did it) has crossed my mind but I thought it would be a big unsucessful mess. I've dealt with my share of old P3/P4 servers and that is a GREAT way to save money and take advantage of what's infront of you. I mean, it's pretty "ghetto" on some levels but seems fine for a home server. I may have to pick up another bulky P3 box and try this out when I decide I need a home server again

[Phoenix32]

Ben, another option might be to take a Snap 4400 (they are very inexpensive) and do some modding on that (it is a P3 system requiring little power).

[Harbinger]

Originally Posted by ben333

I don't have much SNAP knowledge, but I greatly admire what you did to that backplane! The idea (exactly as you did it) has crossed my mind but I thought it would be a big unsucessful mess. I've dealt with my share of old P3/P4 servers and that is a GREAT way to save money and take advantage of what's infront of you. I mean, it's pretty "ghetto" on some levels but seems fine for a home server. I may have to pick up another bulky P3 box and try this out when I decide I need a home server again

Yep, it's definitely ghetto. But, it'll get the job done (at least for the foreseeable future). Thanks for the kind words!

[Harbinger]

Well, this server might not last me too long. It seems that my NAS software of choice, openfiler, is going to 64-bit only on the next release. My server board, the Intel SE7501WV2, apparently will not do 64-bit as it's documentation claims it will. According to someone at Intel, NONE of their 533 FSB bus Xeons will do 64-bit; that apparently started with the 800 FSB units. So now I have a quandary. The original 9550 controller I purchased through eBay turned out to be defective. I had just ordered another one and gotten it installed when I found out about the 32/64 problem. I will be able to transplant the controller and drives to another (non PCI-X) system as a 32-bit controller and still have a 64-bit system with 32-bit disk access, or I would purchase another controller and re-do the array as 64-bit. The problem will be to offload all the data to temporary storage while I do it. Oh, well...