|
|
Snap Server / NAS / Storage Technical Goodies The Home for Snap Server Hacking, Storage and NAS info. And NAS / Snap Classifides |
Thread Tools |
12-09-2006, 07:48 PM | #1 |
Cooling Neophyte
Join Date: Jan 2006
Location: Redlands, CA
Posts: 44
|
Comparing transfer rates with different RAM amounts
I just upgraded my V2 Snap 2000 to 256 MB ram from the original 64.
I did an informal testing, and it doesn't seem to make any difference for the transfer rate of many large (~4-6 MB each) files. I use my Snap primarily for hosting my .mp3 files, with iTunes running on any of several client machines. To test the transfer rate, I copied a folder containing about 300 files (totalling about 1.5GB) from the Snap to my desktop PC. The lan consists of a Netgear WGR614 router and a 100mbs NIC in my desktop PC. Regardless of whether I have 64MB, 128 MB or 256 MB in the Snap, I get a transfer rate of about 4.5MB/sec. Likewise, when I copy from the desktop machine to the snap, I get a transfer rate of about 2.9 MB/sec, regardless of how much ram is in the Snap. The additional ram might help more with speeding up repetitive access to the same data, since more data is probably cached. I only get about 1.3 MB/sec from the Snap to my notebook with Wireless G. Nevertheless, I have no problems hosting MP3s on the Snap for iTunes over the wireless network. In fact, I can run iTunes on my notebook, with the MP3s on the Snap, and stream the audio wirelessly to an Airport Express without any glitches. It actually kind of amazes me, it works so well. I can even store .iso files of DVDs on the Snap, and play them through the Wireless network on my notebook without any jitter or dropouts. Last edited by dgoodrich; 01-04-2007 at 11:27 AM. |
12-09-2006, 08:19 PM | #2 |
Thermophile
Join Date: Jul 2005
Location: Plano, TX
Posts: 3,135
|
Re: Comparing transfer rates with different RAM amounts
You must change the default raid cache size to make any difference "config raid cache size ###" in meg. Must be very careful not to allocate more than you have. "debug memory" will show how it is allocated.
Also with 128 and greater you can run JVM on some units. oops..... giving out all my secrets.
__________________
1 Snap 4500 - 1.0T (4 x 250gig WD2500SB RE), Raid5, 1 Snap 4500 - 1.6T (4 x 400gig Seagates), Raid5, 1 Snap 4200 - 4.0T (4 x 2gig Seagates), Raid5, Using SATA converts from Andy Link to SnapOS FAQ's http://forums.procooling.com/vbb/showthread.php?t=13820 |
12-09-2006, 11:59 PM | #3 |
Thermophile
Join Date: May 2006
Location: Yakima, WA
Posts: 1,282
|
Re: Comparing transfer rates with different RAM amounts
Are you sure that is not MB not Mb? If not, then check your network.
Here is some data, to use as a benchmark (because there are no network or HD speed bottlenecks here). ------------------------------------------------------------------------------------------------------ I did this minutes ago, after reading this thread... This is one of my SNAP 4000 units, transfering both TO and FROM the SNAP with a Test folder of MP3 files of 2 MB to 15 MB in size and multiple (sorted) sub folders... 4 x Seagate 250 GB Drives in RAID 5 (711 GB) 128 MB SDRAM (not 256) Factory Default memory settings, nothing changed at all (like cache sizes)... ------------------------------------------------------------------------------------------------------ Transfer rate TO the SNAP Server = 25 Mb/Sec Average (32.8 Mb PEAK) Transfer rate FROM the SNAP Server = 46.9Mb /Sec Average (53Mb PEAK) See those little dips in the line showing transfer? That is where the buffer/cache was reloading. This is where a larger buffer/cache could come in to getting higher average speeds. |
12-10-2006, 02:46 AM | #4 |
Cooling Neophyte
Join Date: Jan 2006
Location: Redlands, CA
Posts: 44
|
Re: Comparing transfer rates with different RAM amounts
Yes, I am sorry, I mean MB/S (megaBYTES/sec). Mb/S (megaBITS/sec) would be about 8 times MB/S, correct? If so, this would mean you are getting about 6 MB/S from the Snap, and 3 MB/S to the Snap, and we are in the same ballpark.
I am curious about setting the cache size. This is what I show with a debug memory: Total System Physical Memory: 268435456Which (if any) of these entries corresponds to the raid cache size? Any guidance on what I should set the raid cache size to? Thanks, Dave |
12-10-2006, 05:45 AM | #5 |
Thermophile
Join Date: May 2006
Location: Yakima, WA
Posts: 1,282
|
Re: Comparing transfer rates with different RAM amounts
It works out to more like 10:1 rather than 8:1 I think. Yes, I know, 8 bits per Byte, but there is some kind of parity or something going on there also if I remember right (maybe one of the networking guys can clear this up). Anywise, 3 things to ask here at this point.
1) For one of you IT networking gurus out there, what is the actual limit to be expected on a 100baseT network (full duplex)? Now I am not talking SNAP, and I am not talking theory. I am talking about real world use on two systems capable of max throughput of the network. How fast can be expected on 100baseT? 2) If the cache is changed, will it stay changed, or will it go back to default if you power off the SNAP? Is it saved in the settings somewhere? 3) What about the RAID stripe itself? Can the stripe size be changed and would it help any with these larger drive sizes? |
12-10-2006, 02:01 PM | #6 | |
Thermophile
Join Date: Jul 2005
Location: Plano, TX
Posts: 3,135
|
Re: Comparing transfer rates with different RAM amounts
100baseT has a max 12.5 Meg/sec, theory. In real world standings I've only seen a max of ~9-10 M/s using SMB (MS drag and drop). FTP can be closer to 11 M/s. I seam to only get around 8M/s (smb).
Would not touch the raid strip size at all. Ram usage: If you have JVM loaded it by default it will take 25% of total ram, with a min of 25meg. This is the reason for a min of 128meg. If not running JVM you can use up to 50% of your ram for raid cache. Quote:
The threads can be increased if you have a lot of users, or handling a lot of small files. But I do not recall if it retains the setting. Douglas may have some comments or insite on this.
__________________
1 Snap 4500 - 1.0T (4 x 250gig WD2500SB RE), Raid5, 1 Snap 4500 - 1.6T (4 x 400gig Seagates), Raid5, 1 Snap 4200 - 4.0T (4 x 2gig Seagates), Raid5, Using SATA converts from Andy Link to SnapOS FAQ's http://forums.procooling.com/vbb/showthread.php?t=13820 |
|
12-10-2006, 06:50 PM | #7 |
Thermophile
Join Date: May 2006
Location: Yakima, WA
Posts: 1,282
|
Re: Comparing transfer rates with different RAM amounts
I mentioned the RAID stripe because I believe it uses 64MB stripes and most modern day RAIDs use 128MB stripes on current drives. You can go up and down from that, but 128 seems to be the most efficient in most cases.
So Douglas, what you got to say on this subject? |
12-10-2006, 09:43 PM | #8 |
Cooling Neophyte
Join Date: Jan 2006
Location: Redlands, CA
Posts: 44
|
Re: Comparing transfer rates with different RAM amounts
Well, I tried setting the raid cache to 128MB, and it appeared to take. But it didn't change my transfer times at all. I wonder if this setting has any effect on the 2 drive (Raid 1) units.
|
01-10-2007, 03:41 PM | #9 |
Cooling Neophyte
Join Date: Jan 2007
Location: Melbourne, AUS
Posts: 15
|
Re: Comparing transfer rates with different RAM amounts
Hey Guys,
Has anyone had any definative performance enhancments from upgrading the memory on Snap 2000s? I have just upgraded my 2000 to 256Mb and I'm sure it hasn't hurt the performance, but it's hard to tell what it HAS actually done. I've checked in the debug menus to confirm ram upgrade, changed RAID cache to 128Mb although when viewing it, it still shows 0Kb for the RAID cache. Are there any known settings I could change to fully utilise the extra RAM? Not running JVM or anything flash at this stage, just using it as a NAS backup for notebooks at home. Cheers, Kris.
__________________
~ Snap Server 2000 / SW v4.0.860 / HW v2.0.0 / BIOS v2.0.282 / SVR # 58xxx / RAM 256Mb / HDD 2 x 80Gb / RAID 1 ~ |
01-10-2007, 11:07 PM | #10 |
Cooling Neophyte
Join Date: Mar 2002
Location: Washington State
Posts: 54
|
Re: Comparing transfer rates with different RAM amounts
not much to say other than changing memory size has no real benefit on a snap. Snap doesn't need or lets say doesn't now what to do with the extra ram. The os was not written to take advantage of larger memory sticks.
as for the striping. it is 64kb / 128 sectors. (which equals standard 128 sectors -- 65536 bytes x 512 bytes = 128 sectors) damn, a 128mb stripe is huge. although it would make my life in the world of data recovery a hell of a lot easier when recovering from a failed raid. That would mean that most files would not be striped across multiple drives. as for transfer speeds. the best thing you can do is match everything. if you force snap to 100full you need to force the switch port(if you can) as well and as well as the pc nic. while I was still supporting snaps I wrote a couple of programs that basically would do the following: ask you where the files were located that you wanted to copy to or from snap, as well as how many, and then would create logs of the transfer and time down to the millisecond how long took to copy each file. I would then run the same test over and over again each time changing something like forcing pc nic and not snap and so on. Then could study all the logs to see which configuration produced the best speeds. it all came down to matching everything to each other. I will also tell you that I ran the same tests between to pc's and 98% of the time I did not see much difference between pc's. unless I forced on nic to 10half and the other to 100full. Then of course it made a big difference. but if one was forced to 100 full and the other set to auto, there wasn't much difference. but that same senario can produce some amazing numbers when you throw a snap in the picture. And not all drivers are created equal. even though you may not see any changes when transfering between to pc's with different drivers loaded for the nics. You can see it sometimes when again you throw a snap in the picture. For example: I found big issues if you let windows 2000 supply the driver for the 3c920 nic that is built on the motherboard like in alot of dells. but if you use the driver that dell supplies for the same nic there was no issue. but again, there was no issue between the dell and any other network device. Only when doing something with the snap was the issue seen. ok, I have had enough of this. good luck, and just sit down, take a few hours and really do some good test. Keep track/notes of all changes----- AND, I CAN NOT SAY THIS ENOUGH ---- MAKE ONLY 1 CHANGE PER TEST. OR YOU WILL NEVER KNOW WHICH CHANGE WAS THE ONE THAT MADE THE DIFFERENCE!!!! Douglas Snap-Tech |
01-11-2007, 06:15 PM | #11 | |
Thermophile
Join Date: May 2006
Location: Yakima, WA
Posts: 1,282
|
Re: Comparing transfer rates with different RAM amounts
Quote:
Ooops, I meant 64 KB and 128 KB not MB... |
|
01-11-2007, 08:42 PM | #12 |
Cooling Neophyte
Join Date: Mar 2002
Location: Washington State
Posts: 54
|
Re: Comparing transfer rates with different RAM amounts
LOL
That's ok Phoenix32 I was pretty sure it was a typo anyways. Thought I would just have a little fun with ya I hope the info I posted made sense and understandable. Douglas Snap-Tech by the way, go ahead and ask Bluef86100 to send you the directory listing. He can explain it to you. |
01-11-2007, 08:44 PM | #13 |
Cooling Neophyte
Join Date: Mar 2002
Location: Washington State
Posts: 54
|
Re: Comparing transfer rates with different RAM amounts
damn, sorry about the spelling blue68f100 on my last post. won't do that again!!!
Douglas Snap-Tech |
01-12-2007, 03:08 AM | #14 |
Cooling Neophyte
Join Date: Jan 2007
Location: Melbourne, AUS
Posts: 15
|
Re: Comparing transfer rates with different RAM amounts
Thanks Douglas, all sounds pretty logical to me.
Cheers.
__________________
~ Snap Server 2000 / SW v4.0.860 / HW v2.0.0 / BIOS v2.0.282 / SVR # 58xxx / RAM 256Mb / HDD 2 x 80Gb / RAID 1 ~ |
01-12-2007, 11:07 AM | #15 | |
Cooling Savant
Join Date: Aug 2004
Location: UK
Posts: 909
|
Re: Comparing transfer rates with different RAM amounts
Quote:
But that takes all the fun out of it
__________________
Snap Server Help Wiki - http://wiki.procooling.com/index.php/Snap_Server Snap Server 2200 v3.4.807 2x 250GB Seagate Barracuda 7200.9 w/ UNIDFC601512M Replacement Fan "Did you really think it would be that easy??" Other NAS's 1x NSLU2 w/ 512mb Corsair Flash Voyager Running Unslung 6.8b 1x NSLU2 w/ 8Gb LaCie Carte Orange Running Debian/NSLU2 Stable 4.0r0 250GB LaCie Ethernet Disk Running Windows XP Embedded |
|
01-12-2007, 11:20 AM | #16 | |
Thermophile
Join Date: Jul 2005
Location: Plano, TX
Posts: 3,135
|
Re: Comparing transfer rates with different RAM amounts
Quote:
Spelling never was a strength of mine.
__________________
1 Snap 4500 - 1.0T (4 x 250gig WD2500SB RE), Raid5, 1 Snap 4500 - 1.6T (4 x 400gig Seagates), Raid5, 1 Snap 4200 - 4.0T (4 x 2gig Seagates), Raid5, Using SATA converts from Andy Link to SnapOS FAQ's http://forums.procooling.com/vbb/showthread.php?t=13820 |
|
01-12-2007, 05:50 PM | #17 |
Thermophile
Join Date: May 2006
Location: Yakima, WA
Posts: 1,282
|
Re: Comparing transfer rates with different RAM amounts
And he gave me crap for my typo...
|
01-13-2007, 02:46 PM | #18 |
Cooling Neophyte
Join Date: Aug 2006
Location: Sheridan
Posts: 18
|
Re: Comparing transfer rates with different RAM amounts
So you can put more memory in a V2 2000 but it can't use it!?
The extra RAM appears to end up in Heap Memory, what is that used for? Isn't it "debug raid" that gives the raid configuration, not "debug memory"? Tried "config raid cache size 64" but it doesn't seem to take the command, gives me a '-2' error Transfered ~700MB file to and from Snap2000 and got numbers like 4.7MB/sec from Snap, 2.7MB/sec to Snap. My V2 2000 is SnapOS 3.4.805 with 256MByte RAM, dual 500GB Seagate 7200.10 in Raid1 Did the same with my 1100 (3.4.805, 250GB WD2500JB) and got numbers like: 4.6MB/sec from Snap, 3.8MB/sec to Snap. My 1000 gave ~1.5MB/sec in both directions. (For reference, my Ximeta Mini over ethernet gave 8.6MB/sec read, 8.3MB/sec write for the same.) Are the transfer rates any better on the 2200s? Last edited by JELo; 01-13-2007 at 04:04 PM. |
01-17-2007, 02:36 PM | #19 |
Cooling Neophyte
Join Date: Aug 2006
Location: central US
Posts: 67
|
Re: Comparing transfer rates with different RAM amounts
OK, I have a memory/ transfer speed question as well.
My 4000 is transfering data really slow. I transfered 50MB in 60 seconds in last nights test. Some will argue that I'ts because I am using disks that are "too large" in a raid 5, but everything else is working fine, and I broken and rebuild the array fine, etc. 01/16/2007 23:03:21 Command: debug memoryPerhaps the transfer speeds are totally unrelated to memory, but I have see other 'debug memory' listings and the file system memory is only 20MB lower than the total system memory. Can/should I change this? It would be nice to see some wiki entries on these values or the debug commands in general. for example, I see these values change durring large transfers from the server: Memory To be Written: 24912 ...but I don't know what it means or if it shows any problems. here is more of what I know not. cache size = 65526KB Does anyone see something unusual here? I have no experience with snap servers other than this one. Thanks! |
01-17-2007, 03:35 PM | #20 |
Cooling Neophyte
Join Date: Aug 2006
Location: central US
Posts: 67
|
Re: Comparing transfer rates with different RAM amounts
Also in the logs (from I L P 0) is this:
01/02/2007 1:56:05 46 D NET | INET: sendit: send err 39 01/02/2007 1:56:06 46 D NET | INET: sendit: send err 39 01/02/2007 2:28:51 46 D SYS | Previous msg repeating... Occurs 10 times! 01/02/2007 2:28:51 46 I SYS | System Database : SDB has been written to flash at 2007/01/02 2:28:51. 01/02/2007 2:28:52 46 D SYS | fsd: The SDB is being burned... Complete! 01/02/2007 2:28:52 46 D SYS | fsd: The SDB Shadow is being burned... Complete! 01/02/2007 2:43:48 46 D SYS | qdlScanTree found NO problems with path: /0 01/03/2007 10:19:24 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Music/Electronic/DJ/Morcheeba/Big Calm/Shoulder Holster .mp3 len_written=-1 write_len=3117 errno=6 01/03/2007 10:19:24 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Music/Electronic/DJ/Morcheeba/Big Calm/Shoulder Holster .mp3 len_written=-1 write_len=3117 errno=6 01/03/2007 10:19:24 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Music/Electronic/DJ/Morcheeba/Big Calm/Shoulder Holster .mp3 len_written=-1 write_len=3209 errno=6 01/03/2007 10:19:24 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Music/Electronic/DJ/Morcheeba/Big Calm/Shoulder Holster .mp3 len_written=-1 write_len=3209 errno=6 01/03/2007 15:55:42 46 D SYS | Previous msg repeating... Occurs 3 times! 01/03/2007 15:55:42 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Spots/IDs/Miles-Legal.wav len_written=-1 write_len=2225 errno=6 01/03/2007 15:55:42 46 D SMB | SMB : write_mbuf_to_file: file=/0/M/Spots/IDs/Miles-Legal.wav len_written=-1 write_len=2225 errno=6 01/03/2007 21:55:56 46 D SYS | Previous msg repeating... Occurs 5 times! What would cause these errors? |
01-17-2007, 05:39 PM | #21 |
Thermophile
Join Date: Jul 2005
Location: Plano, TX
Posts: 3,135
|
Re: Comparing transfer rates with different RAM amounts
Snap-tech would have to look at the logs and error codes (#6). It apears you are having write errors copying files, show multiple trys and errors. Reset the memory back to original values.
Your speed calculates out to 8.3MB/sec. If this is drag and drop, try FileZilla FTP should be slightly higher toward 10-11MB/sec.
__________________
1 Snap 4500 - 1.0T (4 x 250gig WD2500SB RE), Raid5, 1 Snap 4500 - 1.6T (4 x 400gig Seagates), Raid5, 1 Snap 4200 - 4.0T (4 x 2gig Seagates), Raid5, Using SATA converts from Andy Link to SnapOS FAQ's http://forums.procooling.com/vbb/showthread.php?t=13820 |
01-18-2007, 01:40 PM | #22 | |
Cooling Neophyte
Join Date: Aug 2006
Location: central US
Posts: 67
|
Re: Comparing transfer rates with different RAM amounts
Quote:
Actually, my transfer speeds are about 833KB/Sec. This is on a server that used to have reasonable transfer speeds. The write_mbuf_to_file errors are leading me to believe that this may be an ethernet issue. I'll check this out further and report back. I was hoping to get a comment from Douglas about my memory settings The ones I posted above, don't seem to take advantage of the full memory, and look quite different that those initially posted for the 2000. mostly I am curious about the low value for File System Memory Allocated Last edited by radio; 01-19-2007 at 01:00 AM. |
|
01-18-2007, 02:30 PM | #23 |
Thermophile
Join Date: Jul 2005
Location: Plano, TX
Posts: 3,135
|
Re: Comparing transfer rates with different RAM amounts
You were definately having network troubles with those speed. Try setting the port speed manually and see if it helps. Almost like they are not on the same page. I discovered playing with my gigabit switch that I was terribly slow till I turned on flow control in my nic card. Being a managed switch I'm able to check for errors and packet loss. Douglas mentioned earlier that some snap ports having problems with some switches.
__________________
1 Snap 4500 - 1.0T (4 x 250gig WD2500SB RE), Raid5, 1 Snap 4500 - 1.6T (4 x 400gig Seagates), Raid5, 1 Snap 4200 - 4.0T (4 x 2gig Seagates), Raid5, Using SATA converts from Andy Link to SnapOS FAQ's http://forums.procooling.com/vbb/showthread.php?t=13820 |
01-19-2007, 12:58 AM | #24 |
Cooling Neophyte
Join Date: Aug 2006
Location: central US
Posts: 67
|
Re: Comparing transfer rates with different RAM amounts
Ok, now I feel really embarrassed. It turns out that the computer I was testing from was plugged into a 10Mb hub.
I hope the lengthy debug info posted previously will help someone else for comparisons, and I am still curious about the memory usage. It looks like I have a fully functional raid5 with 4x320Gb drives. I know I'll get no sympathy if it dies on me , but it looks like it's working fine. |
01-19-2007, 07:42 AM | #25 |
Thermophile
Join Date: Jul 2005
Location: Plano, TX
Posts: 3,135
|
Re: Comparing transfer rates with different RAM amounts
I think I still have hub, but mine is a 10/100 kind. 10 base that has to be old, did it support full duplex? Well 833k is good for a 10baseT.
It's my understanding that the snaps with 128m and greater will use 1/2 (64) meg reserving the other 1/2 for JVM. The Snap OS was design to work in a small amount of ram 64meg. Snap tech says it make NO difference. I think it probably has more impact if you are use JVM than anything else. The only way to know is play with the cache settings and run test. In the olded days when HD only had 256k of disk cache, it probably had a big impact. But with most have 8+meg probably not. Originally owers reported that it made things faster, but I don't think they ran any test to confirm. Your 4000 is a version -003 or greater? The reason I ask is that there is a HW bug in 001 and 002 models. If you have a 1 or 2 version, a failed drive 1 or 2 will bring down your whole raid. All kinds of odd things happen. But I under stand you have tested your system which is good.
__________________
1 Snap 4500 - 1.0T (4 x 250gig WD2500SB RE), Raid5, 1 Snap 4500 - 1.6T (4 x 400gig Seagates), Raid5, 1 Snap 4200 - 4.0T (4 x 2gig Seagates), Raid5, Using SATA converts from Andy Link to SnapOS FAQ's http://forums.procooling.com/vbb/showthread.php?t=13820 |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|