Comparing transfer rates with different RAM amounts

[dgoodrich]

I just upgraded my V2 Snap 2000 to 256 MB ram from the original 64.

I did an informal testing, and it doesn't seem to make any difference for the transfer rate of many large (~4-6 MB each) files. I use my Snap primarily for hosting my .mp3 files, with iTunes running on any of several client machines.

To test the transfer rate, I copied a folder containing about 300 files (totalling about 1.5GB) from the Snap to my desktop PC. The lan consists of a Netgear WGR614 router and a 100mbs NIC in my desktop PC. Regardless of whether I have 64MB, 128 MB or 256 MB in the Snap, I get a transfer rate of about 4.5MB/sec.

Likewise, when I copy from the desktop machine to the snap, I get a transfer rate of about 2.9 MB/sec, regardless of how much ram is in the Snap.

The additional ram might help more with speeding up repetitive access to the same data, since more data is probably cached.

I only get about 1.3 MB/sec from the Snap to my notebook with Wireless G. Nevertheless, I have no problems hosting MP3s on the Snap for iTunes over the wireless network. In fact, I can run iTunes on my notebook, with the MP3s on the Snap, and stream the audio wirelessly to an Airport Express without any glitches. It actually kind of amazes me, it works so well. I can even store .iso files of DVDs on the Snap, and play them through the Wireless network on my notebook without any jitter or dropouts.

[blue68f100]

You must change the default raid cache size to make any difference "config raid cache size ###" in meg. Must be very careful not to allocate more than you have. "debug memory" will show how it is allocated.

Also with 128 and greater you can run JVM on some units.

oops..... giving out all my secrets.

[Phoenix32]

Are you sure that is not MB not Mb? If not, then check your network.

Here is some data, to use as a benchmark (because there are no network or HD speed bottlenecks here).

------------------------------------------------------------------------------------------------------

I did this minutes ago, after reading this thread...

This is one of my SNAP 4000 units, transfering both TO and FROM the SNAP with a Test folder of MP3 files of 2 MB to 15 MB in size and multiple (sorted) sub folders...

4 x Seagate 250 GB Drives in RAID 5 (711 GB)

128 MB SDRAM (not 256)

Factory Default memory settings, nothing changed at all (like cache sizes)...

------------------------------------------------------------------------------------------------------

Transfer rate TO the SNAP Server = 25 Mb/Sec Average (32.8 Mb PEAK)

Transfer rate FROM the SNAP Server = 46.9Mb /Sec Average (53Mb PEAK)

See those little dips in the line showing transfer? That is where the buffer/cache was reloading. This is where a larger buffer/cache could come in to getting higher average speeds.

[dgoodrich]

Yes, I am sorry, I mean MB/S (megaBYTES/sec). Mb/S (megaBITS/sec) would be about 8 times MB/S, correct? If so, this would mean you are getting about 6 MB/S from the Snap, and 3 MB/S to the Snap, and we are in the same ballpark.

I am curious about setting the cache size. This is what I show with a debug memory:

Total System Physical Memory: 268435456
File System Memory Allocated: 248796144
Memory To be Written: 0
Memory On Loan: 16608
Loan Count: 2
Heap Memory: 229203744
Memory In Use: 226426784
Objects Allocated: 26
Communication Memory Allocated: 2059232
Memory In Use: 467604
SDB Non-Volatile Memory Allocated: 65536
Memory In Use: 16222
Handles Allocated:4679
Handles In Use: 365
SDB Volatile Memory Allocated: 98304
Memory In Use: 330
Handles Allocated: 11705
Handles In Use: 12
System has been configured!

Which (if any) of these entries corresponds to the raid cache size? Any guidance on what I should set the raid cache size to?

Thanks,
Dave

[Phoenix32]

It works out to more like 10:1 rather than 8:1 I think. Yes, I know, 8 bits per Byte, but there is some kind of parity or something going on there also if I remember right (maybe one of the networking guys can clear this up). Anywise, 3 things to ask here at this point.

1) For one of you IT networking gurus out there, what is the actual limit to be expected on a 100baseT network (full duplex)? Now I am not talking SNAP, and I am not talking theory. I am talking about real world use on two systems capable of max throughput of the network. How fast can be expected on 100baseT?

2) If the cache is changed, will it stay changed, or will it go back to default if you power off the SNAP? Is it saved in the settings somewhere?

3) What about the RAID stripe itself? Can the stripe size be changed and would it help any with these larger drive sizes?

[blue68f100]

100baseT has a max 12.5 Meg/sec, theory. In real world standings I've only seen a max of ~9-10 M/s using SMB (MS drag and drop). FTP can be closer to 11 M/s. I seam to only get around 8M/s (smb).

Would not touch the raid strip size at all.

Ram usage: If you have JVM loaded it by default it will take 25% of total ram, with a min of 25meg. This is the reason for a min of 128meg. If not running JVM you can use up to 50% of your ram for raid cache.

config raid cache size (set raid cache size in MB)
threads count (set raid thread count)

The "config raid cache" will report what is being used. My 2200 in raid 1 reports 0 for raid cache. But it has only has 64meg on ram, and can not be upgraded like the 2000. If I recall bsd (unix) will only start up demons as needed. So some testing will be needed to get the best performance. This may only be available for 4x00 (raid5) capable units. Because the data array is 1 drive (mirrored).

The threads can be increased if you have a lot of users, or handling a lot of small files.

But I do not recall if it retains the setting.

Douglas may have some comments or insite on this.

[Phoenix32]

I mentioned the RAID stripe because I believe it uses 64MB stripes and most modern day RAIDs use 128MB stripes on current drives. You can go up and down from that, but 128 seems to be the most efficient in most cases.

So Douglas, what you got to say on this subject?

[dgoodrich]

Well, I tried setting the raid cache to 128MB, and it appeared to take. But it didn't change my transfer times at all. I wonder if this setting has any effect on the 2 drive (Raid 1) units.

[bigkris74]

Hey Guys,

Has anyone had any definative performance enhancments from upgrading the memory on Snap 2000s? I have just upgraded my 2000 to 256Mb and I'm sure it hasn't hurt the performance, but it's hard to tell what it HAS actually done.

I've checked in the debug menus to confirm ram upgrade, changed RAID cache to 128Mb although when viewing it, it still shows 0Kb for the RAID cache.

Are there any known settings I could change to fully utilise the extra RAM? Not running JVM or anything flash at this stage, just using it as a NAS backup for notebooks at home.


Cheers,
Kris.

[snap-tech]

not much to say other than changing memory size has no real benefit on a snap. Snap doesn't need or lets say doesn't now what to do with the extra ram. The os was not written to take advantage of larger memory sticks.

as for the striping. it is 64kb / 128 sectors.

(which equals standard 128 sectors -- 65536 bytes x 512 bytes = 128 sectors)

damn, a 128mb stripe is huge.

although it would make my life in the world of data recovery a hell of a lot easier when recovering from a failed raid. That would mean that most files would not be striped across multiple drives.

as for transfer speeds. the best thing you can do is match everything.

if you force snap to 100full you need to force the switch port(if you can) as well and as well as the pc nic. while I was still supporting snaps I wrote a couple of programs that basically would do the following:

ask you where the files were located that you wanted to copy to or from snap, as well as how many, and then would create logs of the transfer and time down to the millisecond how long took to copy each file.

I would then run the same test over and over again each time changing something like forcing pc nic and not snap and so on. Then could study all the logs to see which configuration produced the best speeds.

it all came down to matching everything to each other.

I will also tell you that I ran the same tests between to pc's and 98% of the time I did not see much difference between pc's. unless I forced on nic to 10half and the other to 100full. Then of course it made a big difference. but if one was forced to 100 full and the other set to auto, there wasn't much difference. but that same senario can produce some amazing numbers when you throw a snap in the picture.

And not all drivers are created equal. even though you may not see any changes when transfering between to pc's with different drivers loaded for the nics. You can see it sometimes when again you throw a snap in the picture.

For example:

I found big issues if you let windows 2000 supply the driver for the 3c920 nic that is built on the motherboard like in alot of dells. but if you use the driver that dell supplies for the same nic there was no issue. but again, there was no issue between the dell and any other network device. Only when doing something with the snap was the issue seen.

ok, I have had enough of this. good luck, and just sit down, take a few hours and really do some good test. Keep track/notes of all changes----- AND, I CAN NOT SAY THIS ENOUGH ---- MAKE ONLY 1 CHANGE PER TEST. OR YOU WILL NEVER KNOW WHICH CHANGE WAS THE ONE THAT MADE THE DIFFERENCE!!!!


Douglas Snap-Tech

[Phoenix32]

Originally Posted by snap-tech

damn, a 128mb stripe is huge.

Ooops, I meant 64 KB and 128 KB not MB...

[snap-tech]

LOL

That's ok Phoenix32 I was pretty sure it was a typo anyways. Thought I would just have a little fun with ya

I hope the info I posted made sense and understandable.

Douglas Snap-Tech

by the way, go ahead and ask Bluef86100 to send you the directory listing. He can explain it to you.

[snap-tech]

damn, sorry about the spelling blue68f100 on my last post. won't do that again!!!

Douglas Snap-Tech

[bigkris74]

Thanks Douglas, all sounds pretty logical to me.

Cheers.

[re3dyb0y]

Originally Posted by snap-tech

AND, I CAN NOT SAY THIS ENOUGH ---- MAKE ONLY 1 CHANGE PER TEST. OR YOU WILL NEVER KNOW WHICH CHANGE WAS THE ONE THAT MADE THE DIFFERENCE!!!!

Awww

But that takes all the fun out of it

[blue68f100]

Originally Posted by snap-tech

damn, sorry about the spelling blue68f100 on my last post. won't do that again!!!

Douglas Snap-Tech

Glad to see that I'm not the only one dislexic and can't spell.
Spelling never was a strength of mine.

[Phoenix32]

And he gave me crap for my typo...

[JELo]

So you can put more memory in a V2 2000 but it can't use it!?
The extra RAM appears to end up in Heap Memory, what is that used for?
Isn't it "debug raid" that gives the raid configuration, not "debug memory"?
Tried "config raid cache size 64" but it doesn't seem to take the
command, gives me a '-2' error

Transfered ~700MB file to and from Snap2000 and got numbers like
4.7MB/sec from Snap, 2.7MB/sec to Snap.
My V2 2000 is SnapOS 3.4.805 with 256MByte RAM,
dual 500GB Seagate 7200.10 in Raid1

Did the same with my 1100 (3.4.805, 250GB WD2500JB) and got numbers like:
4.6MB/sec from Snap, 3.8MB/sec to Snap.

My 1000 gave ~1.5MB/sec in both directions.

(For reference, my Ximeta Mini over ethernet gave 8.6MB/sec read, 8.3MB/sec write for the same.)

Are the transfer rates any better on the 2200s?

[radio]

OK, I have a memory/ transfer speed question as well.

My 4000 is transfering data really slow. I transfered 50MB in 60 seconds in last nights test.
Some will argue that I'ts because I am using disks that are "too large" in a raid 5, but everything else is working fine, and I broken and rebuild the array fine, etc.

01/16/2007 23:03:21 Command: debug memory

Total System Physical Memory: 268435456
File System Memory Allocated: 179972592
Memory To be Written: 8304
Memory On Loan: 0
Loan Count: 0
Heap Memory: 159009040
Memory In Use: 156224112
Objects Allocated: 25
Communication Memory Allocated: 2059232
Memory In Use: 467604
SDB Non-Volatile Memory Allocated: 65536
Memory In Use: 16168
Handles Allocated:4679
Handles In Use: 362
SDB Volatile Memory Allocated: 98304
Memory In Use: 330
Handles Allocated: 11705
Handles In Use: 12
System has been configured!

Perhaps the transfer speeds are totally unrelated to memory, but I have see other 'debug memory' listings and the file system memory is only 20MB lower than the total system memory. Can/should I change this?

It would be nice to see some wiki entries on these values or the debug commands in general. for example, I see these values change durring large transfers from the server:

Memory To be Written: 24912
Memory On Loan: 8304
Loan Count: 1

...but I don't know what it means or if it shows any problems.

here is more of what I know not.

cache size = 65526KB
number of worker threads = 64
current/max device total i/o count = 95/999
current/max device read i/o count = 60/130
current/max device write i/o count = 34/987
current/max work queue size = 232/1437
cache lookaside read hit ratio = 0%
cache read hit ratio = 7%

array index 0 (device 0x80060000):
stripe size = 64KB
current/max total i/o count = 331/1439
current/max read i/o count = 1/9
current/max write i/o count = 329/1437

active context = 0x25F9030
active context = 0x25F9098
active context = 0x25F9168
active context = 0x25F919C
active context = 0x25F926C



01/16/2007 23:21:27 Command: debug heap

Summary of Heap Pool Memory:

Total Physical Memory:268435456 Effective:264896464 Initial Heap:2097152 Maximum allocation size:8252
Startup Buffer:8388608 Startup required:108928944 Minimum Bio Bufs:63 Actual Bio Bufs:21672
Current Heap Allocs: 28 BioBuf Allocs: 1 Heap Dry: 0 Failed Allocs: 0 Untouched Heap:2065312
#defines: DLEVEL(0), Roundup_BigAllocs(>7168 to 8252),
Size_Queues, !Statistics, Adjust_SizeQs every 120 secs)
ASSERT_DEBUG, ASSERT_LEVEL 0

Block List: REGION Header Address Len REGION Header Address Len
Heap: 0x003E803C : 0x0685B70C - 159009008 Heap: 0x0001800C : 0x0009FFFC - 0

Free List: ADDRESS Length ADDRESS Length ADDRESS Length
0x0F6510CC - 336 0x0FD4D3CC - 2073616
Heap has been validated and no problems were found.


Summary of Queues:
QUEUE MIN BLOCKS(PER) TOTAL FREE ALLOCD TRIM
SizeQ_32 480 + 0 x 0 480 463 17 0 Valid.
SizeQ_64 640 + 0 x 0 640 607 33 0 Valid.
SizeQ_160 2048 + 0 x 0 2048 1996 52 0 Valid.
SizeQ_256 128 + 0 x 0 128 128 0 0 Valid.
SizeQ_512 256 + 0 x 0 256 238 18 0 Valid.
SizeQ_1024 64 + 0 x 0 64 57 7 0 Valid.
SizeQ Adjust Timer inactive.

Block Lists:REGON Header Address Len REGION Header Address Len
SizeQ_32 Heap: 0x0FFF87EC : 0x0FFF87FC - 30704
SizeQ_64 Heap: 0x0FFE97BC : 0x0FFE97CC - 61424
SizeQ_160 Heap: 0x0FF8978C : 0x0FF8979C - 393200
SizeQ_256 Heap: 0x0FF8075C : 0x0FF8076C - 36848
SizeQ_512 Heap: 0x0FF5E72C : 0x0FF5E73C - 139248
SizeQ_1024 Heap: 0x0FF4DEFC : 0x0FF4DF0C - 67568

Total Memory in Queues and Heap currently available: 697664

=========================
01/16/2007 23:40:32 Command: debug ffs

mounts: 1 unmounts: 0
getattrs: 879 roots: 4572
statfs: 1 fhtovps: 0
vptofhs: 0 inits: 1
lookups: 40804 opens: 161
closes: 161 strategys: 0
access: 179 reads: 322
readdirs: 0 locks: 903450
unlocks: 885305 inactives: 1326
reclaims: 663 writes: 0

Vnodes: Min: 8192 Max: 20480 Current: 8192 Free: 8164
File Descriptors: Max: 16384 Current: 15

Does anyone see something unusual here? I have no experience with snap servers other than this one.

Thanks!

[radio]

Also in the logs (from I L P 0) is this:

01/02/2007 1:56:05 46 D NET | INET: sendit: send err 39
01/02/2007 1:56:06 46 D NET | INET: sendit: send err 39
01/02/2007 2:28:51 46 D SYS | Previous msg repeating... Occurs 10 times!
01/02/2007 2:28:51 46 I SYS | System Database : SDB has been written to flash at 2007/01/02 2:28:51.
01/02/2007 2:28:52 46 D SYS | fsd: The SDB is being burned... Complete!
01/02/2007 2:28:52 46 D SYS | fsd: The SDB Shadow is being burned... Complete!
01/02/2007 2:43:48 46 D SYS | qdlScanTree found NO problems with path: /0
01/03/2007 10:19:24 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Music/Electronic/DJ/Morcheeba/Big Calm/Shoulder Holster .mp3 len_written=-1 write_len=3117 errno=6
01/03/2007 10:19:24 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Music/Electronic/DJ/Morcheeba/Big Calm/Shoulder Holster .mp3 len_written=-1 write_len=3117 errno=6
01/03/2007 10:19:24 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Music/Electronic/DJ/Morcheeba/Big Calm/Shoulder Holster .mp3 len_written=-1 write_len=3209 errno=6
01/03/2007 10:19:24 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Music/Electronic/DJ/Morcheeba/Big Calm/Shoulder Holster .mp3 len_written=-1 write_len=3209 errno=6
01/03/2007 15:55:42 46 D SYS | Previous msg repeating... Occurs 3 times!
01/03/2007 15:55:42 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Spots/IDs/Miles-Legal.wav len_written=-1 write_len=2225 errno=6
01/03/2007 15:55:42 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Spots/IDs/Miles-Legal.wav len_written=-1 write_len=2225 errno=6
01/03/2007 21:55:56 46 D SYS | Previous msg repeating... Occurs 5 times!

What would cause these errors?

[blue68f100]

Snap-tech would have to look at the logs and error codes (#6). It apears you are having write errors copying files, show multiple trys and errors. Reset the memory back to original values.

Your speed calculates out to 8.3MB/sec. If this is drag and drop, try FileZilla FTP should be slightly higher toward 10-11MB/sec.

[radio]

Originally Posted by blue68f100

Your speed calculates out to 8.3MB/sec.

Actually, my transfer speeds are about 833KB/Sec. This is on a server that used to have reasonable transfer speeds. The write_mbuf_to_file errors are leading me to believe that this may be an ethernet issue. I'll check this out further and report back.

I was hoping to get a comment from Douglas about my memory settings The ones I posted above, don't seem to take advantage of the full memory, and look quite different that those initially posted for the 2000. mostly I am curious about the low value for File System Memory Allocated

[blue68f100]

You were definately having network troubles with those speed. Try setting the port speed manually and see if it helps. Almost like they are not on the same page. I discovered playing with my gigabit switch that I was terribly slow till I turned on flow control in my nic card. Being a managed switch I'm able to check for errors and packet loss. Douglas mentioned earlier that some snap ports having problems with some switches.

[radio]

Ok, now I feel really embarrassed. It turns out that the computer I was testing from was plugged into a 10Mb hub.

I hope the lengthy debug info posted previously will help someone else for comparisons, and I am still curious about the memory usage.

It looks like I have a fully functional raid5 with 4x320Gb drives. I know I'll get no sympathy if it dies on me , but it looks like it's working fine.

[blue68f100]

I think I still have hub, but mine is a 10/100 kind. 10 base that has to be old, did it support full duplex? Well 833k is good for a 10baseT.

It's my understanding that the snaps with 128m and greater will use 1/2 (64) meg reserving the other 1/2 for JVM. The Snap OS was design to work in a small amount of ram 64meg.

Snap tech says it make NO difference. I think it probably has more impact if you are use JVM than anything else. The only way to know is play with the cache settings and run test. In the olded days when HD only had 256k of disk cache, it probably had a big impact. But with most have 8+meg probably not. Originally owers reported that it made things faster, but I don't think they ran any test to confirm.

Your 4000 is a version -003 or greater?
The reason I ask is that there is a HW bug in 001 and 002 models.
If you have a 1 or 2 version, a failed drive 1 or 2 will bring down your whole raid. All kinds of odd things happen. But I under stand you have tested your system which is good.