It means sometimes.
As far as i know:
A Kevlar vest, itself, is made for sharpnel and "enviormental" defence. Keeping stuff hiting you, such as rocks, branches, etc , and the ocasional piece of molten metal.
Not particulary suited for sharp and pointy objects, such as knifes and... arrows.
Its designed to spread the impact force, on a larger area.
However, there are about two types of kevlar armour. Soft and hard. The soft, mainly used, when hit by bullets, it doesnt stop them. The kevlar itself only slows down the round. It needs the ceramic or steel or even titanium plates to keep them from penetrating. Ense the several "grades" of vests. The harder ones are more efficient on stopping bullets. Even with no plates , but depends on the ammo size and power. Usually used with plates also. But heavier, bulkier.
A ceramic or titanium plated vest can stop even a 7.76mm slug. AK stuff.
A simple "soft" kevlar helmet (with no plates on the "pockets") wont stop even a 9mm bullet from medium range.
Back to arrows. They just get into (penetrate) the fabric, due to the matrix arrangement. Same as ice picks. And some guys named Anderson.
http://www.americantomahawk.com/av/kevlar.mpg
Outchy.
