Comparing transfer rates with different RAM amounts

[radio]

I don't think 10Mb hubs support full duplex. Anyway, the auto sense turned it off, but it's on now that I am connected to the 100Mb.


Here is what I have:

4000 series
OS: 4.0.860
HW 2.0.1
BIOS 2.0.282
4x320 in raid5

I know the extra memory was required for building the large raid set. I can see your reasoning for not needing all that much buffer if the disk has a lot - transferwise anyway. I can see it needing more for large inode tables and the like.

Originally Posted by blue68f100

Your 4000 is a version -003 or greater?
The reason I ask is that there is a HW bug in 001 and 002 models.
If you have a 1 or 2 version, a failed drive 1 or 2 will bring down your whole raid. All kinds of odd things happen. But I under stand you have tested your system which is good.

Stop scaring me!
I do not remember reading that here, and I've spent a lot of time going over this forum. I check on it though. - that would really suck. Do you mean hardware version? If that's the case, then we should post a big sticky - No sense wasting your time if the whole reason you need a raid5 is undermined.

[blue68f100]

Oop's you must have missed a few threads then. Which is very easy to do with all of the info this site has.

It is a very simple fix. In v003 and greater they changed the ide cable to cableselect. Fixed the problem. If your drives as setup using cable select and not M/S you may have the later version.

[radio]

From what I've read, changing to CS mostly solves the problem, but it can still occur.

Can I switch this in mid-raid without worrying about data loss? Also, will the snap automatically know which drive is which? (is the master on the top or bottom, and which drive goes on the end of the cable?)

I have not yet opened it up to look & see if I've got it on cable sellect, but these are obvious questions I think everyone would want to know. Actually, they should be in the WIKI so people know what they are getting into before they start. (I did break the raid once and it repaired itself successfully. )

[blue68f100]

Look at your label on the back, if it is 003 and greater you should be good to go.

I do not recommend changing it in mid stream. Phoenix32 can give you the details on what corrected the problem. He was the one who found the problem. He had both models and was able to see what was needed. The cable was a easy fix. But I do not recall if the drive needed to be change too. It was more of a communication problem, and the cable fix was very simple and worked.

[blue68f100]

I did a ftp file transfer test to my 2200. 100BaseT Full Duplex.

Write speed is 5.7MB/sec
Read speed is 8.8MB/sec

[radio]

Yup, that's about what I get for xfer speeds now.

I am not sure what you mean by 003. my hardware is listed as 2.01 from the debug command- the snap is remote right now and I don't have access to the printed labels.

I was suppose to have this running by year end 2006, and I don't really want to backup and restore all that data - of course reality doesn't care what I want.

[blue68f100]

This is a part no on the back which references to which hardware is in the unit. They start with 001 and the highest value is 005. The 2.01 referes to a hardware (pic) chip.

Like I said earlier if your drives are configured as CS you have a 003+. If you look closely at your IDE cables you will find 1 wire has been cut. This as what the change was. HW 001 & 002 were M/S configured. I have a SN where the break is I will have to see if I can find it.

[radio]

Luckly my model number contains a -004. That's good news.

Things are going along almost perfect now. The only problem is short dropouts on high demand. I am using this to store audio for a radio station, and when I put a load on the snap with another computer, I'll sometimes get two or three seconds with no input from the snap, and this results in a one second blank spot in the middle of a song or news report.

I guess I'll have to find a workaround. maybe I'll just reformat and biuld a mirror, or maybe I'll just connect the drives locally and use the other two as backup inside the snap.

It's a shame, I was so close...

[blue68f100]

The delay could come from a network problem and not necessarly be the snap.

[radio]

OK, I'm reporting back like I said I would

I finally solved the 'network like' problems. The symptoms were netwrok dropouts for a few seconds when reading audio from the disk with an internal buffer of about 2 seconds. When the snap didn't respond longer than this, the audio cut out and the radio station sounded like crap.

I am not sure which of the three checkboxes I removed did the trick, but I unchecked (in Microsoft Networking -> Advanced)

• Enable Master Browser
• Enable Opportunistic Locking
• Enable NT SMBs

This may have been a no brainer for someone that knows what's going on, but I didn't have a lot of time to read up on these things. The NT SMBs I know least about so naturally I want to blame this setting. I would do more testing, but this is now in a production envrionment, so I'm not going to mess with success.

It looks like the snap is working perfectly My only gripe is the lack of an instant power on - We will be investing is a larger UPS.

oidar

[blue68f100]

We will be investing is a larger UPS.

My APC Smart-UPS 1500VA will carry my Snap4500 1.6T, Snap2200 600gig, router, switch, a second router for 1:35. With the interface card will shut the snaps down when the battery gets low. And will restart the 4500 when the power is restored and the battery is charged backup to 30%.

[radio]

Originally Posted by radio

OK, I'm reporting back like I said I would

I finally solved the 'network like' problems. The symptoms were netwrok dropouts for a few seconds when reading audio from the disk with an internal buffer of about 2 seconds. When the snap didn't respond longer than this, the audio cut out and the radio station sounded like crap.

I am not sure which of the three checkboxes I removed did the trick, but I unchecked (in Microsoft Networking -> Advanced)

• Enable Master Browser
• Enable Opportunistic Locking
• Enable NT SMBs

Well I guess I was wrong. A day after reboot teh drop out problems are back. I would appreciate anyone else with a 4000 checking the transfers while watching a network graph - I am looking for a sustained transfer for a minute or two - and see if there are any times the snap just sends out nothing for 500ms-2000ms.

also a debug memory snapshot whould be helpful, Thanks!

oidar

[JELo]

Originally Posted by JELo

. . .

Transfered ~700MB file to and from Snap2000 and got numbers like
4.7MB/sec from Snap, 2.7MB/sec to Snap.
My V2 2000 is SnapOS 3.4.805 with 256MByte RAM,
dual 500GB Seagate 7200.10 in Raid1

Did the same with my 1100 (3.4.805, 250GB WD2500JB) and got numbers like:
4.6MB/sec from Snap, 3.8MB/sec to Snap.

My 1000 gave ~1.5MB/sec in both directions.

(For reference, my Ximeta Mini over ethernet gave
8.6MB/sec read, 8.3MB/sec write for the same.)

Are the transfer rates any better on the 2200s?

Just to complete the circle . . .
After swapping the 500GB Seagates into a 2200 and rebuilding the
Raid1 in 4.0.860 (took ~17 hours), I ran the same test again and got . . .
5.6MB/sec from Snap, 4.3MB/sec to Snap.

So to Summarize . . .
Snap1000 Read 1.5MB/s Write 1.5MB/s WD800BB Single 3.4.805
Snap1100 Read 4.6MB/s Write 3.8MB/s WD2500JB Single 3.4.805
Snap2000 Read 4.7MB/s Write 2.7MB/s ST3500630A Raid1 3.4.805
Snap2200 Read 5.6MB/s Write 4.3MB/s ST3500630A Raid1 4.0.860

All of these tests were done using ViceVersaPro to copy the file as
opposed to an FTP program, which should, as others have pointed
out, yield higher transfer rates. Whether the 4.0.860 has any influence,
I am not sure but I may rebuild the 1100 under 4.0.860 and try this
again - have to move the data off of it first.

JELo

[blue68f100]

I found that FileZillas was the faster of the ftp programs. By a significant amount.

[Phoenix32]

Okay, I don't mean to be all pissy about this subject (although people have come to expect it from me), but it seems like every month or so this subject comes up. People ask, we answer the questions, then a month later someone else asks the same thing again. It gets kind of old after a while. I hope this will answer the questions on speed once and for all. I did these tests last month for David (blue68f100) in e-mail for positive answers. This is a copy of some of that e-mail.

This applies to the SNAP 4000 and 4100. Speeds for the 2000 and 2200 will be similar and speeds for the 1000 and 1100 will be slightly less due to a slower processor. Obviously, the 1x00 and 2x00 units do not support RAID 5.


Test conditions:

My PC is a Pentium 4 Northwood 3Ghz hyperthread with 2 GB of Dual Channel 2-2-2-5 DDR with PAT on (the system is plenty snappy). I am using WinXP Pro SP2. I have a pair of WD SATA 120's in RAID 0. I am using the Intel i875 Chipset with the CSA Gigabit LAN. In case you do not know what CSA Gigabit is, it is an Intel Gigabit LAN that is connected directly to the northbridge so that it comes off the northbridge instead of the southbridge. This gives better Gigabit performance than can be had from Gigabit LAN over the southbridge PCI bus.

My PC is connected to a Gigabit Router with 4 "switched" Gigabit ports (it is not a hub like most routers are). The test SNAP 4000 is connected to an 8 port Gigabit Switch of the same brand and series as the Router. The Router and Switch are connected directly. I am not using jumbo frames, as the SNAP would not appreciate that and it would slow it down rather than speed it up. I get pretty respectable Gigabit speeds over this exact setup. All other network activity was suspended for the duration of the tests.

The SNAP is a SNAP 4000. OS v 3.4.805 with 64MB SDRAM and 4 x 30 GB Quantum Fireball LM 7200 rpm drives with 13msec access times as measured by Spinrite. The drives were freshly low level formatted with the Quantum utility, Spinrite scanned, and then freshly formatted in the SNAP 4000. The SNAP was set to factory defaults with the exception of the server name and workgroup.

I did Write and Read transfer speed checks in RAID 0 with a 2 Drive array, RAID 0 with a 4 Drive array, RAID 1, and RAID 5 with a 4 Drive array. In both cases of the RAID 0 with 2 Drive array and the RAID 1 with 2 Drives, I used Drives 1 and 3 for the array so that the drives would not be on the same IDE channel. Speeds were measured using AnalogX Netstat Live. Results are as follows. All tests were conducted to and from the exact same locations on the PC and SNAP with test files deleted between tests. The test file was a single image file of 626,258,076 bytes (597MB).

NOTE 1: The 4100 will not get faster speed with 4 IDE channels (one for each drive) rather than the 2 IDE channels used by the SNAP 4000. My experience with both 4000 and 4100 units using the same drives has indicted this speed difference is negligible. More on this later.

NOTE 2: Using faster modern drives (access times) may also speed things up slightly, but as you will see from the RAID 0 results compared to the others, I remain skeptical it would be much if anything.

NOTE 3: While I did not include the results here, I have done some testing with 64MB, 128MB, and 256MB SDRAM installed. With the 4 x 30 GB arrays, it made NO DIFFERENCE AT ALL! NONE! ZERO! ZILCH! With larger drives, this will change the picture, or if using JVM. But I have yet to see any real difference when using 4 x 30 GB drives and JVM not installed.

NOTE 4: I have also used OS v 3.4.807 and OS v 4.0.860 on both SNAP 4100 and SNAP 4000 units in similar configurations and seen no difference in speeds.

NOTE 5: I cannot explain why the RAID 1 Mirror yielded a higher Read speed than either of the Striped RAID arrays (RAID 0 and RAID 5). This should not be, but it was.


RAID 0, 2 drives, 1 and 3: Write = 4.9MB/sec, Read = 5.9MB/sec

RAID 0, 4 drives: Write = 4.9MB/sec, Read = 5.9MB/sec (the average was the same as RAID 0, 2 drives in Real Time)

RAID 1: Write = 4.5MB/sec, Read = 6.3MB/sec (I have no idea why the Read speed is higher here than in RAID 0 which should be higher)

RAID 5, 4 drives: Write = 3.5MB/sec, Read = 5.9MB/sec

Speeds were consistent and as would be predicted with the exception of the one anomaly of the RAID 1 Read speed.

See attachments...

[Phoenix32]

Now let's answer the SMB versus FTP question. The previous tests used SMB.

Here are FTP transfers, both to and from, and both a 4100 and a 4000 using the same test files.

Both the 4000 and 4100 are using the same brand and model hard disks in a 4 disk RAID 5.

The 4100 has 128MB SDRAM and the 4000 only has 64MB SDRAM.

The 4100 is using OS v4.0.860 (no JVM) and the 4000 is using OS v3.4.805 (no JVM).

The same test conditions as before except as listed above and transfers being done in FTP with SmartFTP 2.0.

As you will clearly see, FTP READ transfers are significantly faster than with SMB.


SNAP 4000 RAID 5, 4 drives: Write = 4.1MB/sec, Read = 11MB/sec

SNAP 4100 RAID 5, 4 drives: Write = 4.3MB/sec, Read = 10.5MB/sec

[Phoenix32]

Now with all of that said, keep in mind that differences of 0.1 or 0.2 MB/sec are not worth comparing. You can get variance by that much on the same equipment on different test runs.

The theoretical limit for 100baseT is 12.5MB/sec if I remember correctly, and as you can see, with FTP we are approaching that with Reads. Writes on the other hand are a different story. In almost every test I have ever run, you cannot get more than about 1/3 to 1/2 of the 100baseT bandwidth for Writes.

I have used 5400 RPM Quantum 30 GB drives, 7200 RPM Quantum 30 GB drives, 7200 RPM Seagate 250 GB drives, 7200 RPM WD drives, and a host of others. With 5400 RPM drives, you will loose some speed from what I posted here due to access times of these drives. 7200 RPM drives, old or new, will get you near the limits. If you use newer and faster 7200 RPM drives, you can get an increase of around 0.75MB/sec transfers under ideal conditions, due to access times of the drives, but as a general rule, 7200 PRM drives of any type, old or new, pretty much reach the limits you are going to get on a day in and day out usage. I am not aware of any 10000 RPM EIDE drives available, so forget that idea. Plus they would be very expensive for very little gain, if any. If you want maximum speed, then like most other things in the computer world where RAID is involved, look to the drive access times, but do not expect huge amounts of gain.

The 4100 has 4 IDE and the 4000 only has 2. It would seem reasonable that the 4100 would be slightly faster (and has been reported incorrectly as so on this forum many times). But the fact is, I have 3 SNAP 4100 units and 6 SNAP 4000 units to test with and in all cases, the 4000 and 4100 have been on par with each other (and I have done hundreds of hours of testing). If we were using Gigabit LAN, or a faster processor, or 10000 RPM drives, it might make a difference, but we aren't. With the CPU as it is, 100baseT, and 7200 RPM drives, the difference is just not there. THEY ARE THE SAME IN PERFORMANCE! Okay got it? Good!

And what about memory? It seemed reasonable, even to me, that more memory would be good and would be faster. Well guess what? Doesn't happen. I have tested with 32MB, 64MB, 128MB, and 256MB. So long as you are using the smaller drives and not using JVM, there has been very little performance gain above 64MB. 32MB is slower, but 64MB is enough. Now I am not saying there is no gain or not to do it. I am just saying not to expect a lot in performance gain with more than 64MB. Of course this is with default settings. If you want to go in and mess with RAID stripe sizes, who knows.

Now for the caveat to the memory thing. -IF- you are using larger drives (say 120 GB or larger), 64MB does take a small perfromance hit. In this case, 128MB seems to be better. Rather than start a long drawn out discussion as to why this is, I will just leave it to your own ideas. But, if you are using 120 GB drives or larger, you can get a small performance gain with 128MB of memory.

Now a caveat to the caveat. JVM installed and running changes the whole picture. When running JVM, it is suggested to use 128MB or more memory. I will go farther and say it is required, not just suggested. As an example of this, I was testing a 4000 with 64MB and JVM installed and running. Anything more than about 100MB worth of data to be transfered would get aborted. In other words, if you are using JVM, you do in fact need 128MB of memory or you will get random errors and crashes, in my opinion. I have tested a 4000 with 4 x 250 GB Seagate 7200 RPM drives with 128MB memory and JVM installed and running. I did not have any problems and did get full speeds as reported above. However, if you are going to use JVM and have large drives, I would suggest 256MB. Better safe than sorry. But do not feel you have to, 128MB did seem to be fine in each test, and I did not run into any problems with it myself. Your mileage may vary.

To wrap this up, here is the general rule(s) of thumb for SNAP 4000/4100 speeds. Assuming you are using the proper amount of memory for your configuration, and good fast access 7200 RPM drives, you should expect at or near the following speeds in FTP (SMB will be slower). The 2x00 units should be near these numbers and of course the 1x00 units will be slower due to a slower CPU (I have a 1000 also, but did not feel it was worth pursuing).

JBOD, RAID 0, and RAID 1: 6 to 6.75MB/sec Write and 10.5 to 12MB/sec Read

RAID 5: 4.0 to 4.5MB/sec Write and 10.5 to 11MB/sec Read


I sincerely hope we can now lay this question to rest. If you do not agree, hey more power to you and I got no problem with that. But, if you want to dispute these results, you better be prepared to back it up with proof and repeated controlled testing.

[shawnl]

Very nice writeup and response!

Could this be put up in the FAQ or put in a sticky post or something?

Being one of those annoying posters who brought this up again...I know If I had seen something like this ahead of time, I never would have bothered asking again.

[blue68f100]

I will be posting it in the FAQ's soon, just havn't done it yet. It will be done. Broken in to table form depending on protocol.

Phoenix32 setup is very similar to mine, our number are real close to each other. The difference is he uses etheral to monitor the port which inludes overhead values. Where I use the FileZilla reported transfer speed. And I have early data on a 2000v1 and 2200. Which he does not have.

[Phoenix32]

Originally Posted by shawnl

Very nice writeup and response!

Could this be put up in the FAQ or put in a sticky post or something?

Being one of those annoying posters who brought this up again...I know If I had seen something like this ahead of time, I never would have bothered asking again.

Thank you. And don't worry about it, I am just a grouch in general, LOL...

Hey, gotta keep up my reputation right?

[Phoenix32]

Originally Posted by blue68f100

Phoenix32 setup is very similar to mine, our number are real close to each other.

LOL, if only people knew how much we talk in e-mail eh? It is easily far more than the combined posts here on the forum even. But Shhhhhh, don't tell anyone. They all think we just know this stuff from the top of our heads.

Oh yeah, the Guardian stuff is next for us now right?

[blue68f100]

They all think we just know this stuff from the top of our heads.

That and a lot of hard work , and taking the time to test and retest.

Oh yeah, the Guardian stuff is next for us now right?

You have some catching up to do.

I wish I owned a Network version of Sandra. The free version does not support network drives. I would love to bench (speed) marks these units. Particularly the 4500's. Have no way to generate ~300mb/sec network traffic.

[JELo]

Not that the 4000 series numbers aren't interesting but for those of us who aren't running them, I thought actual comparative numbers for some of the other units might be. And I was more concerned with numbers that might reflect file serving daily use speeds as opposed to absolute maximums. FTP is clearly the best way to ship data back and forth but I wouldn't see that as a typical means of interactive data serving. For my purposes, I use the Snap as a central repository of several hundred gigabytes of GIS/Map/reference data that I can access from several different machines I work from. . . . . just my 2cents . . . JELo

[jontz]

Originally Posted by Phoenix32

Oh yeah, the Guardian stuff is next for us now right?

*sigh* I need more money

[Phoenix32]

Originally Posted by JELo

Not that the 4000 series numbers aren't interesting but for those of us who aren't running them, I thought actual comparative numbers for some of the other units might be. And I was more concerned with numbers that might reflect file serving daily use speeds as opposed to absolute maximums. FTP is clearly the best way to ship data back and forth but I wouldn't see that as a typical means of interactive data serving. For my purposes, I use the Snap as a central repository of several hundred gigabytes of GIS/Map/reference data that I can access from several different machines I work from. . . . . just my 2cents . . . JELo

Ummm Were you not paying attention?

I know that somewhere in all of that I said the numbers were representative of the 2x00 units also (just not the RAID 5 parts)? I know I said it... The 4000 and the 2000 use the same main board in some cases and the same speed CPU, which is the limiting factor here for the most part. So those numbers do apply to "those of us who aren't running them". Yup, I am pretty sure I said that somewhere. The only place they don't apply is the 1x00 units due to their slower CPU. Wait, I said that too.

Also, the majority of the numbers I gave there were using SMB, not FTP. You know, like for data tansfers for a central repository of several hundred gigabytes of GIS/Map/Reference data that a person could say access from different machines they work from. I am pretty sure I said that too.


Ya know David, sometimes I wonder why I even try.