My Nanni (Allah be pleased with her) was a very righteous woman, masha'Allah. One day her neighbour saw her and in a bout of frustration, shouted "what is with you? I do so much black magic on you and wish you ill, but nothing works!". Imagine that. Most of us would be frightened to our core! My Nanni? She calmly replied, "I say my five prayers and read Quran and have complete certainty that my Lord will protect me".
Let us also consider however, that if the eye of the evildoer, a heart filled with greed or envy or hate can be powerful, then what about the heart of the wali, a heart that knows Allah? How powerful is that blessed eye? Should we not then seek to place ourselves at the feet of such people, exposing ourselves to their blessed gazes?