One of my pet peeves is when someone (usually a co-worker) starts small talk for no other reason than to fill the silence and obviously not caring about my answer. Once a co-worker left the room we were in, while I was still answering her question.
Also not sure if this counts as a peeve, but...