I actually try to do the same thing. Watching people out of the corner of my eye, scanning my environment. If I hear someone walking behind me I move to a position I can defend myself, or to look behind me to see who it is. Don't think it is actually a bad thing.
Be aware of it and don't let it...