Besides from being exceptionally large/small, I think it does matter to some extent. But from the little interaction I had with girls on this subject, I take it that it's mostly about preferences, much like some guys likes big boobs and others prefers small ones but they wouldn't really care...