SNAP 4000 Errors and Limits

[blue68f100]

What is your CPU doing at these times?

And what other programs are running?

How much ram does the 4000 have?

Which OS Version?

And do you have JVM active?

I have done 2-3+ gig of ftp transfer and have not got those kickups. And that's streaming at 4.5MB/sec.

Phoenix32 is the 4000 specialist, see what he has to say.

[Phoenix32]

Originally Posted by radio

I'm looking for anyone else that can confirm or deny this. All you need is a transfer of two minutes to notice the pattern in a network monitor like taskmanager or analogx netstat live.

Originally Posted by blue68f100

Phoenix32 is the 4000 specialist, see what he has to say.

Go here and look at the Netstat Live screenshots

http://forums.procooling.com/vbb/sho...t=13731&page=2

Yes, I have seen these spikes, but not anywhere near as glaring as yours are (I suspect the 320 GB drives for that). These spikes appear to be as the cache loads/unloads. If you watch the disk activity, you will see this.

Further, the spikes are almost non existant (very very small and short) when sending with FTP. Add in that it is more pronounced when receiving FTP or when using SMB and this is an indication these cache loads are being caused by CPU load (the CPU cannot keep up with the cache when under higher loading).

Reducing the cache size seems to minimize this effect and part of why I recommend to people to use 128 MB memory unless they are running JVM enough to warrant 256 MB memory.

Now with all that said, I have not seen spikes anywhere near as glaring as yours were. Again, I have to suspect the 320 GB drives as part of that equation. We recently dug up information indicating the OS does have a practical limit of 1 TB (by we, I mean David). As we have said many times, the 4 x 300/320 GB drives just do not look like a good idea in a SNAP 4000. It might "apprear" to be working properly, but in the end may cause some serious grief.

[radio]

I should mention that I maped the snap raid 5 to a windows drive. I have not tested FTP.

The graphs pictures when transfering from the snap.
No significant activity on the computer durring these snapshots.
Snap has the latest OS.
256mb RAM
No JVM

I did see those spikes on your earlier post, but never really thought about them. I suppose they are the same. It's hard to tell, but they do look regular. I assume you expect this problem to stay once I mirror the drives? - or maybe not if the cache is used more in the raid computations.

Can I set the cache size with debug? Perhpas do some more testing before I toast it. I needed more ram to build the array. FYI.

The CPU load idea does not make sense to me because this happens durring very minor transfers (23k/sec)

oidar

PS. Please see the new thread - What if the snap dies?

[blue68f100]

There are some settings for cache and page size. Use the ? from debug to get a list of debug commands.

With transfers running 4.8MB/sec the hickups are minor. I will have to load my 4100 backup with HD's and I will beable to run some test. I'm waiting for some slides to come in, so I only have to handle the unit once. SHould be here this week.

[Phoenix32]

Originally Posted by radio

I did see those spikes on your earlier post, but never really thought about them. I suppose they are the same. It's hard to tell, but they do look regular. I assume you expect this problem to stay once I mirror the drives? - or maybe not if the cache is used more in the raid computations.

I did test with RAID 0, Mirror, and Single Drive operation, but I honestly can't remember for sure what the effects were. -IF- I remember correctly, the mirror and single drive operation had no real data spikes that I remember.

Originally Posted by radio

Can I set the cache size with debug? Perhpas do some more testing before I toast it. I needed more ram to build the array. FYI.

Yes, there are some cahce settings in debug, but I don't think they stick if the unit is turned off. I could be wrong, let us know...

Originally Posted by radio

The CPU load idea does not make sense to me because this happens durring very minor transfers (23k/sec)

I don't see the spikes at all in small transfers. Even during large transfers, these spikes are less pronounced than you are showing. The spikes I have seen (on multiple SNAP 4000 units) have been very quick, meaning they only dropped the average transfer speed by at most 0.2 MB/sec. They were very momentary and not a real problem.

I have streamed music from my SNAP 4000 units (I like to listen to my music on my notebook in another room while I am working). Not once have I heard even a single glitch, even when streaming 320 BPS files.

Once again, I have to attribute this to either the 256 MB memory, the 320 GB drives, or the combination of both. CPU load does have an effect on a RAID arrarys using that CPU for XOR operations. That same CPU is what has to drive the data transfers to and from memory (or cache), as well as to and from the NIC.

[radio]

I have streamed music from my SNAP 4000 units (I like to listen to my music on my notebook in another room while I am working). Not once have I heard even a single glitch, even when streaming 320 BPS files.

Well this above all gives me hope. I would have tested the cache settings, but I'm on a tight schedule and I like to have the computer working while I'm sleeping, so reconfigures and started the restore last night. It's 1pm now, and in 5 more hours I'll be able to start the second half of the restore. The data should be back before Monday morning.

I guess I can do a few tests before starting the second restore. I'll let you guys know what I find on the mirror.

Thanks!

oidar

[blue68f100]

I wonder if he is have a network problem. Try a direct connect with a crossover cable. That will give you a clean test. Then start adding your nework gear.

Do you have the snap acting as a Master Browser? Under Networks/microsoft Advanced settings, like below.

Advanced Microsoft Networking
Enable Master Browser
Enable Opportunistic Locking
Enable NT SMBs

[radio]

I will test with a crossover cable if I can find one.

Advanced Microsoft Networking
Enable Master Browser
Enable Opportunistic Locking
Enable NT SMBs

no, no, no, and no.

I did play with these a while back and it did not seem to make much difference.


I also have Enable NetBIOS over TCP/IP set, not TCP/IP and NetBEUI

[eschw95458]

I remember seeing in either the admin guide or the faq section on snapappliance.com that you should never disable opportunistic locking, or nt smb's unless advised to do so by Snap support as they sometimes caused problems.

[radio]

01/16/2007 23:03:21 Command: debug memory

Total System Physical Memory: 268435456
File System Memory Allocated: 179972592
Memory To be Written: 4210812
Memory On Loan: 0


Ok, on those dropouts, (see previous graph) I can reproduce them. It happens when opening a program that scans a large number of MP3s to make sure they match the database. I don't know if any of them are opened. when the program stops, everything returns to normal. I do see network dips at regular intervals, but never like this (zero net traffic, often for more than a second.) Even huge file transfers will not cause the problem, but this (necessary) program does.

What I learned by studying the debug screens is that the "memory to be written" above increases until just about the time of the traffic blackout, and is then zeroed out and begins increasing again. During large transfers, this changes a little, but usually it stays under 50K, during my problem it grows to over 4500K! The eventual write must be causing the snap to stall.

I guess I'll try going back to 128M RAM to see if will clean itself up faster.

[radio]

I tried the system with 64m RAM. No luck. The "memory to be written" rose as always, and delays are just as long. I guess I'm stuck. the software must be doing something the snap just can't handle - at least for my application.

oidar

[blue68f100]

Do you have ftp turned on? Some programs uses its protocol to speed things up. Might try turning it on or off and see if it make a difference. Remember the snap uses SAMBA v2 with v4 OS. Which is not very fast. And can not handle long file names.

[radio]

Originally Posted by blue68f100

Do you have ftp turned on? Some programs uses its protocol to speed things up. Might try turning it on or off and see if it make a difference. Remember the snap uses SAMBA v2 with v4 OS. Which is not very fast. And can not handle long file names.

This is confusing. I am using windows networking and mapping drives. No FTP. it's simply a program accessing a share. FTP is off.

I am not sure what you mean by not handling long filenames. It handles long filenames just fine. I would say that 90% of the 280Gigs worth of music has filenames longer than 8 characters, many with filenames much longer.



At this point I've given up. I'll probably keep the snap around for backup, but it's useless in production. I may checkout FreeNAS in the future. It's just not worth the time and hassle anymore.

I hope the testing I did helps someone.

oidar.

So long and thanks for all the fish.

[blue68f100]

Long file names is 32 chr, a restriction in SAMBA v2. But still some chr are not allowed.

I have seen some of the activity here after doing some testing. It apears the snap has to halt net traffic then use the cpu to do xor bit calculation for writes. Mine in no were close to yours. I would wire the cpu fan to run 24/7. As the cpu heat goes up the cpu will throttle back even stall till it cools down.

Do you see these hickup during reads? should be there but less.

How may users are attached to the server? more than just 1?

[radio]

Here is a long (50+ character) filename we have on the snap:

Elegy for William Burroughs and Allen Ginsberg .mp3

it seems to work fine.

mostly the activity is reading the id3 or similar tags from audio files when the scheduling program starts. I am not sure how you can simulate reading tiny bits of large files.

As far as I know, it's durring reads.

There is only one user connected that is doing anything - sometimes other computers have the dirve.

Peace,
oidar

[blue68f100]

Adaptec Knowledge base say this:

Question : Is there a limit for Mac OS X clients on the number of characters in a filename on SnapOS ?

This information applies to the following Product(s):

- Snap Server 1000
- Snap Server 1100
- Snap Server 2000
- Snap Server 2200
- Snap Server 4000
- Snap Server 4100
- Snap Server 12000

This information applies to the following Operating System(s):

- Mac OS X
Answer : 31 characters is the limit. File names exceeding 31 cannot be saved or copied onto the SnapOS from a MAC OS X client.

and:

You must upgrade your Snap server to Snap OS 4.0854 to get support for Mac OS X.

The max path including filename is limited to 254 chr or less. This applies to windows and mac.

[radio]

I don't have a Mac.

[Metal]

I have two 4000's and a 4400 and I stream MP3's off my 4000 all the time.
What software are you running that hangs up your songs?

[radio]

This has nothing to do with streaming MP3s. streaming just shows the symptoms of the problem. I am using Station Playlist Creator (free trial come to think of it) which causes the problem when it loads. It's not even heavy traffic, but for some reason the snap can't handle it. It does something to drive up the "memory to be written" and the snap stops all data transfers while getting it back to zero. (writing?)

[eschw95458]

Just to recap
So- 4 x 250 drives = known working
4x 320 drives = ???????

Did radio (or anyone) confirm 320 drives as working with no problems?

[Phoenix32]

Originally Posted by eschw95458

Just to recap
So- 4 x 250 drives = known working
4x 320 drives = ???????

Did radio (or anyone) confirm 320 drives as working with no problems?

IMO, 4000 + 4 x 250 = A O K, 4000 + 4 x 320 = A F U

[radio]

To clarify,

The problem was not the 320GB drives, they worked fine. The problem was the snap. It can not constantly stream files under certain conditions. I tried building a mirror and got the same results. The 320GB drives do work in the snap in raid 5 like any other drives.

This is experience, not opinion.

[Phoenix32]

So let me get this right.

- Just because every single 4 x 320 GB 4000 setup we have seen or heard of has had various "problems" is just coincidence?

- David digging up the documentation on the OS it is built from, which flat out said there is a 1TB limitation, doesn't matter?

- A full formal education in analog and digital electronic engineering and 30 years of hands on tech work in the field "opinion" is worthless just because someone on a forum with a malfunctioning SNAP Server said so?


Nevermind, please don't answer...

[blue68f100]

Radio, The info Phoneinx32 say is correct.

I don't know what your problem is. We have been through this before.

Everything Phoenix32 has said about the HW in a 4000 and the 1T has been confirmed. Others have also reported problems with HD's > 250gig.

The OS has a 1T limit, due to the fact that the XFS file system is 24bit. Just because the HD Supports LBA48bit does not make it a 48bit system. The weakest link rules here.

Failure to build the raid5 array should tell you something.

You come here for help, then you disregard what we say.

Time for you to move on.

Enough said, case closed.

[radio]

Originally Posted by Phoenix32

So let me get this right.

- Just because every single 4 x 320 GB 4000 setup we have seen or heard of has had various "problems" is just coincidence?

- David digging up the documentation on the OS it is built from, which flat out said there is a 1TB limitation, doesn't matter?

- A full formal education in analog and digital electronic engineering and 30 years of hands on tech work in the field "opinion" is worthless just because someone on a forum with a malfunctioning SNAP Server said so?


Nevermind, please don't answer...

Phoenix, I am very sorry I hurt your feelings. That was not my intention.

FYI, I am a senior systems programmer, and have been in the field for over 25 years. I also have extensive electronic experience. I don't consider experience worth much if you can't use it to point to a specific reason whys someone will not work. What does matter is when something does work and you experience it. If you look at it from my point of view you can clearly see how ridiculous it sounds. I have it working. but you tell me it cant work. I guess I'll trust 'the horses mouth' experience over your opinion any day. I am sorry you don't like it and it contradicts much of what you have posted here, but there it is.


Your three bullets in order:
1) because some have had problems does not mean everyone will. (10 people with broken legs couldn’t run a marathon, therefore it's not possible) - logical fallacy. not to mention there are other who have gotten 250+ drive raids working and posted here. infact, there are very few who have tried larger drives that have posted anything here.

2) "is built from" is the key phrase here - it means nothing. you have no idea what modifications were made.

3) again, this is opinion, that is, your belief. Belief does not hold up against the reality of experience. Experience wins.